Extended image generation method, apparatus and device

Abstract

The application provides an extended image generation method, device and equipment, and relates to the technical field of computer and communication. The method comprises the following steps: obtaining a reference image and a reference element in the reference image; performing multi-dimensional classification on the reference element to obtain a label classification result of the reference element, the label classification result being used for determining a reference label combination of the reference element, and the reference label combination comprising reference labels of the reference element in multiple dimensions; matching the reference label combination corresponding to the label classification result with a material label combination of multiple material elements according to the dimensions to determine a target material element from the multiple material elements, the target material element satisfying a first predetermined matching condition in terms of the reference label matching degree with the reference element; wherein the material label combination of the material element is composed of material labels in at least one dimension; and generating an extended image corresponding to the reference image based on the target material element, so that the extended image can be automatically generated.

Description

Technical Field

[0001] This application relates to the fields of computer and communication technology, and in particular to an extended image generation method, apparatus and device. Background Technology

[0002] As society progresses, people have higher and higher requirements for design drawings. For the same design requirement, designers need to provide many design drawings with similar content.

[0003] In existing technologies, designers typically first design a reference image or obtain a reference image from the client, and then expand upon the reference image to create an extended image with similar content. The expansion process does not require much creativity and is mechanically repetitive, but it requires a lot of time and effort. Summary of the Invention

[0004] This application aims to provide an extended image generation method, apparatus, device, and storage medium capable of automatically generating extended images.

[0005] According to one aspect of the embodiments of this application, an extended image generation method is provided, comprising: acquiring a reference image and acquiring reference elements in the reference image; performing multi-dimensional classification on the reference elements to obtain a label classification result of the reference elements, the label classification result being used to determine a reference label combination of the reference elements, the reference label combination including reference labels of the reference elements in multiple dimensions; matching the reference label combination corresponding to the label classification result with material label combinations of multiple material elements by dimension, and determining a target material element from the multiple material elements whose reference label matching degree with the reference elements satisfies a first predetermined matching condition; wherein, the material label combination of the material elements consists of material labels in at least one dimension; and generating an extended image corresponding to the reference image based on the target material element.

[0006] According to one aspect of the embodiments of this application, an extended image generation apparatus is provided, comprising: an acquisition module configured to acquire a reference image and acquire reference elements in the reference image; a classification module configured to perform multi-dimensional classification on the reference elements to obtain a label classification result of the reference elements, the label classification result being used to determine a reference label combination of the reference elements, the reference label combination including reference labels of the reference elements in multiple dimensions; a matching module configured to match the reference label combination corresponding to the label classification result with a combination of material labels of multiple material elements by dimension, and determine a target material element from the multiple material elements whose reference label matching degree with the reference element satisfies a first predetermined matching condition; wherein the combination of material labels of the material elements consists of material labels in at least one dimension; and a generation module configured to generate an extended image corresponding to the reference image based on the target material element.

[0007] In one embodiment of this application, based on the foregoing scheme, the tag classification result includes the reference tag combination of the reference element, and the matching module is configured to: for each material element's material tag combination, compare the reference tag combination with the material tag combination in the same dimension; determine the target dimension and the corresponding number of target dimensions where the material tag combination and the reference tag combination have the same tags, wherein the reference tag matching degree between the material element and the reference element is determined by the number of target dimensions; when the number of target dimensions corresponding to the material element reaches a first predetermined number threshold, determine that the reference tag matching degree between the material element and the reference element satisfies a first predetermined matching condition; based on the number of target dimensions corresponding to the material tag combination of each material element among the plurality of material elements, determine the material element whose number of target dimensions reaches the first predetermined number threshold as the target material element.

[0008] In one embodiment of this application, based on the foregoing scheme, the matching module is further configured to: semantically expand each reference tag in the reference tag combination to obtain an expanded tag combination of the reference element, wherein the expanded tag combination is composed of expanded tags in the multiple dimensions; match the expanded tag combination with the material tag combination by dimension, and determine the expanded material element from the multiple material elements whose expanded tag matching degree with the reference element satisfies a second predetermined matching condition; the generation module is configured to: generate an expanded image corresponding to the reference image based on the target material element and the expanded material element.

[0009] In one embodiment of this application, based on the foregoing scheme, the matching module is configured to: obtain word vectors corresponding to the text of each reference tag; calculate the distance between the word vectors corresponding to the text of each reference tag and the word vectors corresponding to the text of the candidate tag; and select extended tags from the candidate tags whose distance is within a set range for combination to obtain the extended tag combination.

[0010] In one embodiment of this application, based on the aforementioned scheme, the tag classification result includes the reference tag probability distribution of the reference element in multiple dimensions, the reference tag probability distribution includes the reference tag probability of the reference element corresponding to multiple dimension tags in each dimension, and the reference tag is the dimension tag with the highest reference tag probability in each dimension; the matching module is configured to: obtain the reference tag probability distribution of the reference tag combination in each dimension; obtain the material tag probability distribution of the material tag combination in each dimension, the material tag probability distribution corresponding to each dimension includes the material tag probability of the material element corresponding to multiple dimension tags in each dimension, and the material tag is the dimension tag with the highest material tag probability in each dimension; match the reference tag probability distribution and the material tag probability distribution by dimension, and determine the target material element from the multiple material elements whose reference tag matching degree with the reference element satisfies the first predetermined matching condition.

[0011] In one embodiment of this application, based on the aforementioned scheme, the matching module is configured to: calculate a first difference between the probability of a reference tag and the probability of a material tag corresponding to the same dimension tag in the same dimension; determine a first tag difference degree between the reference tag combination and the material tag combination corresponding to each tag in each dimension based on the first tag difference degree in each dimension; determine a first element difference degree between the reference element and the material element based on the first tag difference degree corresponding to each dimension tag in each dimension, wherein the reference tag matching degree between the material element and the reference element is determined by the first element difference degree; when the first element difference degree corresponding to the material element reaches a first difference degree threshold, determine that the reference tag matching degree between the material element and the reference element satisfies a first predetermined matching condition; and determine the material element whose first element difference degree is lower than the first difference threshold as the target material element based on the first element difference degree corresponding to each material element among the plurality of material elements.

[0012] In one embodiment of this application, based on the aforementioned scheme, the matching module is configured to: sum the first tag differences corresponding to each dimension tag in each dimension to obtain the first dimension differences between the reference element and the material element in each dimension; and sum the first dimension differences corresponding to each dimension of the material element as the first element differences between the reference element and the material element.

[0013] In one embodiment of this application, based on the foregoing scheme, the generation module is configured to: replace the reference element in the reference image with the target material element to obtain the extended image.

[0014] According to one aspect of the embodiments of this application, a computer program storage medium is provided, which stores computer program instructions that, when executed by a computer, cause the computer to perform any of the methods described above.

[0015] According to one aspect of the embodiments of this application, an electronic device is provided, including: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the method as described in any of the preceding claims.

[0016] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative embodiments described above.

[0017] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0018] In some embodiments of this application, the technical solutions involve acquiring a reference image and obtaining reference elements within the reference image; classifying the reference elements in multiple dimensions to obtain label classification results for the reference elements, whereby the label classification results are used to determine reference label combinations for the reference elements, and the reference label combinations include reference labels for the reference elements in multiple dimensions; matching the reference label combinations corresponding to the label classification results with the material label combinations of multiple material elements by dimension, and determining a target material element from the multiple material elements whose reference label matching degree with the reference element satisfies a first predetermined matching condition; wherein, the material label combination of the material element consists of material labels in at least one dimension; and generating an extended image corresponding to the reference image based on the target material element. Thus, an extended image of the reference image can be automatically generated, and since the extended image corresponding to the reference image is generated by matching multi-dimensionally similar material elements by dimension, the generated extended image is also similar to the reference image in multiple dimensions, thereby ensuring image extension quality and greatly improving image output efficiency. In a practical application, such as in the design field, this application can automatically convert one design drawing into multiple design drawings, enriching the design drawings while greatly reducing the involvement of designers and lowering labor costs; at the same time, it can also categorize and consolidate designers' design rules and specifications through multi-dimensional classification, which not only improves the efficiency of design output, but also ensures high-quality images that conform to design rules and specifications.

[0019] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application.

[0021] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0022] Figure 2A A schematic diagram of a data sharing system to which an embodiment of the present application can be applied is shown;

[0023] Figure 2B A schematic diagram of a blockchain to which one embodiment of this application may be applied is shown;

[0024] Figure 2C This diagram illustrates the generation of a new block in a blockchain to which one embodiment of this application can be applied;

[0025] Figure 3 A flowchart illustrating an extended image generation method according to an embodiment of this application is shown schematically;

[0026] Figure 4A A schematic diagram of a reference image according to an embodiment of this application is shown;

[0027] Figure 4B This illustration schematically shows the application according to the present application. Figure 4A A schematic diagram of the extended image obtained as a reference image;

[0028] Figure 5 A flowchart illustrating an extended image generation method according to an embodiment of this application is shown schematically;

[0029] Figure 6 This illustration schematically shows a CNN algorithm classification process according to an embodiment of the present application;

[0030] Figure 7 A flowchart illustrating an extended image generation method according to an embodiment of this application is shown schematically;

[0031] Figure 8 A block diagram of an extended image generation apparatus according to one embodiment of this application is shown schematically;

[0032] Figure 9 This is a hardware diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0033] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0034] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0035] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] Figure 1 A schematic diagram of an exemplary system architecture 100 to which the technical solutions of the embodiments of this application can be applied is shown.

[0038] like Figure 1 As shown, system architecture 100 may include client 101, network 102, and server 103. Network 102 is used as a medium to provide a communication link between client 101 and server 103. Network 102 may include various connection types, such as wired communication links, wireless communication links, etc., which are not limited herein.

[0039] It should be understood that Figure 1 The number of clients 101, networks 102, and servers 103 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 101, networks 102, and servers 103. For example, server 103 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Client 101 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these.

[0040] In one embodiment of this application, server 103 acquires a reference image and reference elements within the reference image; performs multi-dimensional classification on the reference elements to obtain label classification results for the reference elements, the label classification results being used to determine reference label combinations for the reference elements, the reference label combinations including reference labels for the reference elements in multiple dimensions; matches the reference label combinations corresponding to the label classification results with material label combinations of multiple material elements by dimension, and determines a target material element from the multiple material elements whose reference label matching degree with the reference element satisfies a first predetermined matching condition; wherein, the material label combination of the material element consists of material labels in at least one dimension; and generates an extended image corresponding to the reference image based on the target material element, which can automatically generate the extended image.

[0041] It should be noted that the extended image generation method provided in this application embodiment is generally executed by server 103, and correspondingly, the extended image generation device is generally located in server 103. However, in other embodiments of this application, client 101 may also have similar functions to server 103, thereby executing the extended image generation method provided in this application embodiment.

[0042] Figure 2A A schematic diagram of an exemplary data sharing system 200 to which the technical solutions of embodiments of the present invention can be applied is shown.

[0043] See Figure 2A The data sharing system 200 shown refers to a system for data sharing between nodes. Each node 201, during normal operation, can receive input information and maintain shared data within the data sharing system 200 based on the received input information. To ensure information interoperability within the data sharing system 200, information connections can exist between each node 201, allowing information transmission between nodes. For example, when any node 201 in the data sharing system 200 receives input information, other nodes 201 in the data sharing system 200 obtain the input information according to a consensus algorithm and store it as data in the shared data, ensuring that the data stored on all nodes 201 in the data sharing system 200 is consistent.

[0044] Each node 201 in the data sharing system 200 has a corresponding node identifier. Each node 201 can also store the node identifiers of other nodes 201 in the data sharing system 200, so that the generated block can be broadcast to other nodes 201 in the data sharing system 200 based on their node identifiers. Each node 201 can maintain a node identifier list as shown in the table below, storing the node name and node identifier in this list. The node identifier can be an IP (Internet Protocol) address or any other information that can be used to identify the node; Table 1 only uses IP addresses as an example.

[0045] Node Name Node identifier Node 1 117.114.151.174 Node 2 117.116.189.145 … … Node N 119.123.789.258

[0046] Table 1

[0047] Figure 2B A schematic diagram of a blockchain that can be applied to one embodiment of this application is shown.

[0048] Each node in the data-sharing system 200 stores the same blockchain. A blockchain consists of multiple blocks; see [link to relevant documentation]. Figure 2B A blockchain consists of multiple blocks. The genesis block includes a block header and a block body. The block header stores input information feature values, version number, timestamp, and difficulty value, while the block body stores the input information. The next block after the genesis block takes the genesis block as its parent block. The next block also includes a block header and a block body. The block header stores the input information feature values ​​of the current block, the block header feature values ​​of the parent block, version number, timestamp, and difficulty value, and so on. This ensures that the block data stored in each block is related to the block data stored in the parent block, guaranteeing the security of the input information in the blocks.

[0049] Figure 2C A schematic diagram of new block generation in a blockchain that can be applied to one embodiment of this application is shown.

[0050] When generating the individual blocks in the blockchain, see Figure 2C When a node in the blockchain receives input information, it verifies the input information. After verification, it stores the input information in a memory pool and updates its hash tree used to record the input information. Then, it updates the timestamp to the time the input information was received and tries different random numbers multiple times to calculate the feature value, ensuring that the calculated feature value satisfies the following formula:

[0051] SHA256(SHA256(version+prev_hash+merkle_root+ntime+nbits+x))<TARGET

[0052] Wherein, SH256 is the feature value algorithm used to calculate the feature value; version (version number) is the version information of the relevant block protocol in the blockchain; prev_hsh is the block header feature value of the parent block of the current block; merkle_root is the feature value of the input information; ntime is the update time of the update timestamp; nbits is the current difficulty, which is a fixed value for a period of time and is determined again after exceeding the fixed time period; x is a random number; TRGET is the feature value threshold, which can be determined based on nbits.

[0053] Thus, when a random number satisfying the above formula is calculated, the information can be stored accordingly, generating a block header and a block body to obtain the current block. Subsequently, the node where the blockchain resides sends the newly generated block to the other nodes 201 in the data sharing system 200 according to the node identifiers of the other nodes 201 in the data sharing system 200. The other nodes 201 verify the newly generated block and add it to their stored blockchain after verification.

[0054] In one embodiment of this application, the input information stored in the data sharing system 200 can be material elements or candidate tags used to select extended tags. Using blockchain-based processing of the extended image generation method of this application can improve the accuracy of the acquired data, thereby improving the accuracy of the extended image generation.

[0055] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0056] Figure 3 The flowchart illustrating an embodiment of an extended image generation method according to this application is shown in the schematic diagram. The execution entity of the extended image generation method may be a server, for example, a... Figure 1 Server 103 is shown in the image.

[0057] Reference Figure 3 As shown, the extended image generation method includes at least steps S310 to S340, which are described in detail below:

[0058] In step S310, a reference image is obtained, and reference elements in the reference image are obtained.

[0059] In one embodiment of this application, the reference image may be an image uploaded by client 101. The reference image may contain one or more reference elements, which may be text, graphics, pictures, or tables, etc., for example... Figure 4A A schematic diagram of a reference image according to an embodiment of this application is shown. Figure 4A Reference elements can be Figure 4A The options include 401A hot dogs, 402A pizzas, 403A donuts, 404A drinks, 405A text, or 406A backgrounds.

[0060] In one embodiment of this application, a reference image and reference elements in the reference image can be obtained from a design source file. The design source file can be a source file designed using design software such as Photoshop or Sketch, which contains the elements of each layer and their position, size and other information.

[0061] In one embodiment of this application, the reference element may be all elements contained in the reference image. In other embodiments of this application, the reference element may be selected from the elements contained in the reference image, and may be selected based on user instructions or based on preset rules.

[0062] In step S320, the reference element is classified in multiple dimensions to obtain the label classification result of the reference element. The label classification result is used to determine the reference label combination of the reference element. The reference label combination includes the reference label of the reference element in multiple dimensions.

[0063] In one embodiment of this application, multiple dimensions may include image quality, color vibrancy, style, emotion, technique, and object dimensions. Specifically, the image quality dimension may include labels such as high-definition, medium-definition, and low-definition; the color vibrancy dimension may include labels such as highly vibrant, moderately vibrant, and lowly vibrant; the style dimension may include labels such as modern, classical, popular, minimalist, and illustration; the emotion dimension may include labels such as passionate, dark, and cute; the technique dimension may include labels such as watercolor, oil painting, and traditional Chinese painting; and the object dimension labels can be set according to the actual content expressed by the element, such as hamburger, fries, donut, and hot dog.

[0064] In this embodiment, since the classification of reference elements takes into account not only image dimensions such as image quality and color vibrancy, but also content dimensions such as style, emotion, technique, and object, the reference elements can be interpreted from more dimensions, and the target material elements can be found more accurately.

[0065] In one embodiment of this application, the label classification result may include a combination of reference labels for reference elements. The feature maps of the reference elements may be convolved using multiple different convolution kernels to obtain multiple convolutional images; the multiple convolutional images may be combined to obtain a combined feature map; based on the combined feature map, the reference labels of the reference elements in each dimension may be determined, and the reference labels of the reference elements in each dimension may be combined to obtain a combination of reference labels for the reference elements.

[0066] In one embodiment of this application, a neural network model can be used to determine the reference label of a reference element in multiple dimensions. The neural network model can be a convolutional neural network (CNN), such as a MobileNet, a ResNet, or an Inception neural network.

[0067] In one embodiment of this application, after obtaining the labels of the reference elements output by the neural network model in each dimension, the labels output by the neural network model can be deduplicated, and the deduplicated labels output by the neural network model can be used as reference labels.

[0068] In one embodiment of this application, before using the neural network model, a training image set can be used to train the neural network model, where the training images in the training image set correspond to known training labels in each dimension. The training images are input into the neural network model to obtain the predicted labels output by the neural network model. If the predicted label corresponding to the same training image is inconsistent with the training label, the neural network model is adjusted until the predicted label corresponding to the same training image is consistent with the training label.

[0069] In one embodiment of this application, the tag classification result includes the reference tag probability distribution of the reference element across multiple dimensions. The reference tag probability distribution includes the reference tag probability of the reference element corresponding to multiple dimension tags in each dimension. For example, the reference tag probability distribution of the reference element in the technique dimension is: Modern 80%, Classical 10%, Illustration 10%. The reference tag probability of the reference element corresponding to the Modern dimension tag in the technique dimension is 80%, the reference tag probability of the reference element corresponding to the Classical dimension tag in the technique dimension is 10%, and the reference tag probability of the reference element corresponding to the Illustration dimension tag in the technique dimension is 10%. The reference tag probability distribution of the reference element in the emotion dimension is: Passionate 90%. The probability of a reference element corresponding to the "Dark" dimension tag in the emotion dimension is 90%, the probability of a reference element corresponding to the "Dark" dimension tag in the emotion dimension is 8%, and the probability of a reference element corresponding to the "Cute" dimension tag in the emotion dimension is 2%. The probability distribution of reference elements in the painting type dimension is: watercolor 75%, oil painting 20%, traditional Chinese painting 5%, etc. The probability of a reference element corresponding to the watercolor dimension tag in the conversation type dimension is 75%, the probability of a reference element corresponding to the oil painting dimension tag in the conversation type dimension is 20%, and the probability of a reference element corresponding to the traditional Chinese painting dimension tag in the conversation type dimension is 5%.

[0070] In one embodiment of this application, the reference label probability distribution of a reference element in multiple dimensions can be determined by a neural network model.

[0071] In one embodiment of this application, the dimension label with the highest reference label probability among all dimensions can be used as the reference label.

[0072] In one embodiment of this application, the reference label probability distributions of reference elements in various dimensions can be combined as a reference label combination of reference elements. For example, the reference label combination of reference elements can be [(Modern 80%, Classical 10%, Illustration 10%) (Hot-blooded 90%, Dark 8%, Cute 2%) (Watercolor 75%, Oil Painting 20%, Traditional Chinese Painting 5%)].

[0073] In one embodiment of this application, the reference label probabilities of the reference elements corresponding to the reference labels can be combined as the reference label combination of the reference elements. For example, the reference label combination of the reference elements can be (Modern 80%, Hot-blooded 90%, Watercolor 75%).

[0074] In one embodiment of this application, the reference element may have one or more reference labels in each dimension, and the reference label may be selected from the dimension labels according to the reference label probability of the reference element corresponding to each dimension label.

[0075] Continue to refer to Figure 3 In step S330, the reference tag combination corresponding to the tag classification result is matched with the material tag combination of multiple material elements by dimension, and the target material element whose reference tag matching degree with the reference element meets the first predetermined matching condition is determined from the multiple material elements; wherein, the material tag combination of the material element is composed of material tags in at least one dimension.

[0076] In one embodiment of this application, the material element corresponding to the material tag combination that is completely identical to the reference tag combination can be used as the target material element.

[0077] In one embodiment of this application, a reference tag combination and a material tag combination can be compared to obtain the number of identical tags in the reference tag combination and the material tag combination. The material element with the most identical tags is taken as the target material element, or the material element with the number of identical tags reaching a set ratio is taken as the target material element.

[0078] In one embodiment of this application, for each material element's material tag combination, the tags in the same dimension of the reference tag combination and the material tag combination can be compared; the target dimension and the corresponding number of target dimensions with the same tags as the material tag combination and the reference tag combination can be determined; based on the number of target dimensions corresponding to the material tag combination of each material element, the material elements whose number of target dimensions reaches a first predetermined threshold are determined as target material elements.

[0079] In one embodiment of this application, the combination of material tags can be obtained by processing material elements through a neural network model.

[0080] In one embodiment of this application, the probability distribution of reference tags for reference tag combinations in multiple dimensions can be obtained; the probability distribution of material tags for material tag combinations in multiple dimensions can be obtained, wherein the probability distribution of material tags for each dimension includes the probability of material tags for material elements in each dimension corresponding to multiple dimension tags, and the material tag is the dimension tag with the highest material tag probability in each dimension; the probability distribution of reference tags and the probability distribution of material tags are matched according to the dimensions, and the target material element whose reference tag matching degree with the reference element meets the first predetermined matching condition is determined from multiple material elements.

[0081] In one embodiment of this application, a first difference between the probability of a reference tag and the probability of a material tag corresponding to the same dimension tag can be calculated. Based on the first difference, a first tag difference degree is determined between the reference tag combination and the material tag combination corresponding to each tag in each dimension. Based on the first tag difference degree corresponding to each dimension tag in each dimension, a first element difference degree is determined between the reference element and the material element. Based on the material elements whose first element difference degree is lower than the first difference degree threshold, a target material element is determined. The probability distribution of the reference tag and the probability distribution of the material tag corresponding to the same dimension tag refer to the probability distribution of the reference tag corresponding to the same tag and the probability distribution of the material tag corresponding to the same tag when the reference tag and the material tag are the same.

[0082] In one embodiment of this application, the material element with the lowest first element difference degree and the first element difference degree can be used as the target material element.

[0083] In one embodiment of this application, a target material element can be selected from material elements whose first element difference is lower than a first difference threshold.

[0084] In one embodiment of this application, the absolute value of the first difference between the reference tag probability and the material tag probability corresponding to multiple dimension tags in the same dimension can be used to obtain the first tag difference degree between the reference tag combination and the material tag combination for each tag in each dimension.

[0085] In other embodiments of this application, the first difference between the reference tag probability and the material tag probability corresponding to multiple dimension tags in the same dimension can be squared to obtain the first tag difference degree between the reference tag combination and the material tag combination for each tag in each dimension.

[0086] In other embodiments of this application, the absolute values ​​of the first differences between the reference tag probabilities and material tag probabilities corresponding to multiple dimension tags in the same dimension can be taken and then averaged to obtain the first tag difference degree between the reference tag combination and the material tag combination for each tag in each dimension.

[0087] In one embodiment of this application, the first label difference degree corresponding to multiple dimension labels in each dimension can be summed to obtain the first dimension difference degree of the reference element and the material element corresponding to each dimension; the first dimension difference degree of multiple dimensions can be summed to obtain the first element difference degree of the reference element and the material element.

[0088] In one embodiment of this application, the probability distribution of material tags can be obtained by processing material elements through a neural network model.

[0089] In one embodiment of this application, the first-dimensional difference degree of each dimension corresponding to the material element can be weighted and summed to obtain the first-dimensional difference degree between the reference element and the material element, wherein the weight can be set corresponding to the dimension.

[0090] In one embodiment of this application, reference tags and material tags can be obtained not only through machine learning and other means, but also directly input by the designer during the database entry process. The probability that a reference element corresponds to a manually input reference tag in the human input dimension can be 100%. Similarly, the probability that a material element corresponds to a manually input material tag in the human input dimension can be 100%.

[0091] Continue to refer to Figure 3 In step S350, an extended image corresponding to the reference image is generated based on the target material elements.

[0092] In one embodiment of this application, the reference element can be replaced with the target material element to obtain an extended image.

[0093] In one embodiment of this application, if a material element that meets the first predetermined matching condition cannot be found, or if a material element that meets the first predetermined matching condition fluctuates within a set error range, such as exceeding 10% of the first predetermined quantity threshold, it means that the target material element cannot be found, and the reference element is not replaced.

[0094] In one embodiment of this application, the matching degree of reference tags between material elements and reference elements is determined by the number of target dimensions; when the number of target dimensions corresponding to a material element reaches a first predetermined threshold, it is determined that the matching degree of reference tags between the material element and the reference element satisfies a first predetermined matching condition.

[0095] In one embodiment of this application, different extended images can be generated by using different elements in a reference image as reference elements.

[0096] In one embodiment of this application, Figure 4B This illustration schematically shows the application according to the present application. Figure 4A An illustration of the extended image obtained as a reference image, such as... Figure 4BAs shown, 401A hot dog can be replaced with 401B sandwich, 402A pizza can be replaced with 402B chicken leg, 403A donut can be replaced with 403B meat roll, and 404A beverage can be replaced with 404B coffee. In other embodiments of this application, the light color in the background 406A, such as yellow-green, can be replaced with the dark color in the background 406B, such as orange.

[0097] exist Figure 3 In this embodiment, a reference image is acquired, and reference elements in the reference image are acquired; the reference elements are classified in multiple dimensions to obtain the label classification results of the reference elements. The label classification results are used to determine the reference label combination of the reference elements. The reference label combination includes the reference labels of the reference elements in multiple dimensions; the reference label combination corresponding to the label classification results is matched with the material label combination of multiple material elements by dimension, and a target material element whose reference label matching degree with the reference element satisfies the first predetermined matching condition is determined from the multiple material elements; wherein, the material label combination of the material element is composed of material labels in at least one dimension; an extended image corresponding to the reference image is generated based on the target material element, and the extended image can be automatically generated.

[0098] exist Figure 3 In the embodiment, before generating the extended image corresponding to the reference image based on the target material element in step S350, the semantic extension of each reference tag in the reference tag combination can be performed to obtain an extended tag combination for representing the semantic features of the reference element. The extended tag combination is composed of extended tags in multiple dimensions. The extended tag combination is matched with the material tag combination by dimension, and the target material element whose extended tag matching degree with the reference element satisfies the second predetermined matching condition is determined from multiple material elements.

[0099] In one embodiment of this application, word vectors corresponding to the text of each reference tag can be obtained; the distance between the word vectors corresponding to the text of each reference tag and the word vectors corresponding to the text of the candidate tag can be calculated; and extended tags that are within a set distance from the candidate tags can be selected and combined to obtain an extended tag combination.

[0100] In one embodiment of this application, a neural network language model can be used to determine the word vectors corresponding to the text of the reference label and the word vectors corresponding to the text of the candidate label.

[0101] In one embodiment of this application, the cosine distance or Euclidean distance between the word vector corresponding to the text of the reference label and the word vector corresponding to the text of the candidate label can be calculated as the distance between the word vector corresponding to the text of the reference label and the word vector corresponding to the text of the candidate label.

[0102] In one embodiment of this application, after semantically expanding each reference tag in the reference tag combination to obtain expanded tags, one or more expanded tags corresponding to the reference tags in each dimension can be selected to form an expanded tag combination.

[0103] In one embodiment of this application, the extended label may be the same as the reference label.

[0104] In one embodiment of this application, extended tags can be obtained by expanding the glyphs of each reference tag.

[0105] In one embodiment of this application, when expanding reference labels, reference labels can be filtered according to the probability distribution of reference labels, and only reference labels whose probability distribution reaches a set probability threshold can be expanded.

[0106] In one embodiment of this application, the vocabulary can be expanded using manually input reference tags, such as "fruit." Word vectors and other techniques can be used to expand this vocabulary; for example, "fruit" can be expanded to include related words like "watermelon," "apple," and so on. This technique first requires a word vector library, such as the Tencent_AILab_ChineseEmbedding dataset. A word vector library can obtain related words by calculating the similarity between words, such as using the cosine similarity algorithm, or it can be obtained by loading the word vector library into existing Natural Language Processing (NLP) frameworks such as Gensim and Annoy. Similar words have quantifiable distances. These distances can be normalized; for example, the distance between "hamburger" and "cola" is 0.8, and between "hamburger" and "fries" is 0.75.

[0107] In one embodiment of this application, for each material element's material tag combination, the tags in the same dimension of the extended tag combination and the material tag combination can be compared; the extended dimensions with the same tags as the reference tag combination and the corresponding number of extended dimensions can be determined; based on the number of extended dimensions corresponding to the material tag combination of each material element, the material element whose target dimension number reaches a second predetermined threshold is determined as the target material element.

[0108] In one embodiment of this application, the probability distribution of extended tag combinations in multiple dimensions can be obtained; the probability distribution of material tag combinations in multiple dimensions can be obtained, wherein the probability distribution of material tag combinations in multiple dimensions includes the probability of material tags corresponding to multiple dimension tags in each dimension, and the material tag is the dimension tag with the highest material tag probability in each dimension; the probability distribution of extended tag combinations and the probability distribution of material tags are matched according to dimensions, and extended material elements that meet the second predetermined matching condition with the extended tag matching degree of extended elements are determined from multiple material elements; and an extended image corresponding to the reference image is generated based on the target material element and the extended material element.

[0109] In one embodiment of this application, a second difference between the probability of extended tags and the probability of material tags corresponding to the same dimension tags can be calculated. Based on the second difference, the second tag difference degree between the extended tag combination and the material tag combination corresponding to each tag in each dimension can be determined. Based on the second tag difference degree corresponding to each dimension tags in each dimension, the second element difference degree between the extended element and the material element can be determined. Based on the material elements whose second element difference degree is lower than the second difference degree threshold, the target material element can be determined. Here, the distribution probability of extended tags and the distribution probability of material tags corresponding to the same dimension tags refer to the distribution probability of extended tags corresponding to the same tags and the distribution probability of material tags corresponding to the same tags when extended tags and material tags are the same.

[0110] In one embodiment of this application, the material element with the second element difference degree lower than the second difference degree threshold and the lowest second element difference degree can be used as the target material element.

[0111] In one embodiment of this application, a target material element can be selected from material elements whose second element difference is lower than a second difference threshold.

[0112] In one embodiment of this application, the absolute value of the second difference between the probabilities of extended tags and material tags corresponding to multiple dimension tags in the same dimension can be used to obtain the second tag difference degree between the extended tag combination and the material tag combination for each tag in each dimension.

[0113] In other embodiments of this application, the second difference between the probabilities of extended tags and material tags corresponding to multiple dimension tags in the same dimension can be squared to obtain the second tag difference degree between the extended tag combination and the material tag combination for each tag in each dimension.

[0114] In other embodiments of this application, the absolute values ​​of the second differences between the probabilities of extended tags and material tags corresponding to multiple dimension tags in the same dimension can be taken and then averaged to obtain the second tag difference degree between the extended tag combination and the material tag combination for each tag in each dimension.

[0115] In one embodiment of this application, the second label difference degree corresponding to multiple dimension labels in each dimension can be summed to obtain the second dimension difference degree between the extended element and the material element in each dimension; the second dimension difference degree of multiple dimensions can be summed to obtain the second element difference degree between the extended element and the material element.

[0116] In one embodiment of this application, the second-dimensional difference degree of each dimension corresponding to the material element can be weighted and summed to obtain the second-dimensional difference degree between the extended element and the material element, wherein the weight can be set corresponding to the dimension.

[0117] In one embodiment of this application, the reference tag probability distribution of the reference tag corresponding to the extended tag can be used as the extended tag probability distribution of the extended tag.

[0118] In one embodiment of this application, the probability distribution of the expanded tag combination in each dimension can be determined based on the probability distribution of the reference tag corresponding to the expanded tag, and the distance between the word vector of the reference tag corresponding to the expanded tag and the word vector of the expanded tag. Specifically, the expanded tag restoration probability, which represents the similarity between the reference tag corresponding to the expanded tag and the expanded tag, can be determined based on the vector difference between the word vector of the reference tag corresponding to the expanded tag and the word vector of the expanded tag. The product of the probability of the reference tag corresponding to the expanded tag and the restored probability of the expanded tag is calculated to obtain the expanded tag probability distribution.

[0119] In one embodiment of this application, a predefined correspondence between vector difference degree and extended label restoration probability can be established, thereby determining the extended label restoration probability corresponding to vector difference based on the correspondence.

[0120] In one embodiment of this application, the vector distance between the word vectors of the reference tag and the word vectors of the extended tag can be calculated as the vector difference degree.

[0121] Figure 5The flowchart illustrating an embodiment of the extended image generation method according to this application is shown. The main process is divided into: 1) reading the element information of the design source file; 2) classifying the element images in multiple dimensions using an algorithm; 3) matching and replacing the element images with materials consistent with each dimension from a pre-processed material library; and 4) outputting the replaced extended image. The auxiliary process includes: 1) classifying the material images in multiple dimensions using a machine learning algorithm; and 2) labeling the classified images according to the type of each dimension and storing them in the material library. The steps are described below. The execution entity of this extended image generation method can be a server, for example, a... Figure 1 Server 103 is shown in the image.

[0122] Reference Figure 5 As shown, the extended image generation method includes at least steps S510 to S580, which are described in detail below:

[0123] In step S510, the design drawing is obtained;

[0124] In step S520, the design drawing element information is obtained;

[0125] In step S530, the CNN algorithm is used to classify the design graph elements according to a predetermined dimension;

[0126] In step S540, the source image is acquired;

[0127] In step S550, the CNN algorithm is used to classify the materials in the material library according to the designer's predetermined dimensions, and the dimension labels are stored.

[0128] In step S560, the material library is obtained.

[0129] In step S570, material elements with the same category dimension label are matched from the material library and replaced;

[0130] In step S580, the extended graph is output.

[0131] exist Figure 5 In this embodiment, by using the design drawing as a reference image, the design drawing elements as reference elements, and the materials in the material library as material elements, the target material element corresponding to the design drawing element is searched from the material library, and the output extended image is the extended image. Step S540 can be executed before or after step S530; this application does not impose any restrictions on this.

[0132] In one embodiment of this application, in step S530, when using a CNN algorithm to classify the design drawing elements according to a predetermined dimension, the input to the CNN algorithm classifier is the image itself. Other information about the image, such as its dimensions on the design drawing, can also be added for a more accurate classification.

[0133] In one embodiment of this application, the layers of the design source file, the design elements on the layers, and other relevant information such as size and position can be obtained through corresponding plugins or by reading and parsing the file using code.

[0134] exist Figure 5 In this embodiment, matching of source images can be achieved by matching tags across various dimensions. For example, if the reference elements in the design drawing are categorized as (modern, action-packed, watercolor) across various dimensions, then the source image library is searched for images with the closest probability differences for each tag across each dimension. If source images for (modern, action-packed, watercolor) exist in the source image library, they can be selected; otherwise, (modern, action-packed, oil painting) can be used as a substitute, selecting based on the number of consistent dimensions, the more consistent the better. Alternatively, matching can also be based on the probability distribution of reference tags for each element across various dimensions, which is more precise. For example, if the source image library contains multiple source images for (modern, action-packed, watercolor), then the probability distribution of reference tags for each tag can be compared. The probability distribution on the original design is (Modern 80%, Classical 10%, Illustration 10%), (Action 90%, Dark 8%, Cute 2%), (Watercolor 75%, Oil Painting 20%, Traditional Chinese Painting 5%). Therefore, we can find image resources from the resource library that have the most consistent probability distribution for each dimension. Consistency is calculated by summing the absolute values ​​of the differences between the probabilities to obtain the element difference degree. Then, we select the image resource element with the smallest error value as the target image resource element. For example, the image resources in the resource library are (Modern 70%, Classical 20%, Illustration 10%), (Action 80%, Dark 18%, Cute 2%), (Watercolor 80%, Oil Painting 15%, Traditional Chinese Painting 5%). The element dissimilarity is then: (|80%-70%|+|10%-20%|+|10%-10%|)+(|90%-80%|+|8%-18%|+|2%-2%|)+(|75%-80%|+|20%-15%|+|5%-10%|)=0.55. There are many algorithms for calculating element dissimilarity; for example, the absolute value distance can be replaced by the sum of squares.

[0135] In one embodiment of this application, in step S550, material tags and material images are stored together in the system to form a material library. Material tags can be specific (e.g., classical, watercolor, passionate), or they can be a combination of material tags and a probability distribution of those tags, with the latter offering higher accuracy. After performing the aforementioned reasonable preprocessing on the material library, it can be effectively and fully utilized in future design drawings, improving the utilization rate of design materials.

[0136] Figure 6 This illustration schematically depicts a CNN algorithm classification process according to an embodiment of this application. The execution entity of this CNN algorithm classification method can be a server, for example, a... Figure 1 Server 103 is shown in the image.

[0137] exist Figure 6 In this process, designers categorize images according to predetermined dimensions. Example images for each category are pre-trained on the network to further train it, aiming to categorize images according to the designer's defined style dimensions. The pre-trained network includes a MobileNet with pre-trained weights, a pooling layer (Polling), and a classification layer (Softmax). A massive amount of source material is input into the MobileNet with pre-trained weights. The output of the MobileNet is then input into the pooling layer, and the structure of the pooling layer is input into the classification layer to obtain a classification probability distribution. This distribution serves as the probability distribution of the source material labels, allowing the machine to automatically classify the massive amount of source images using the implemented algorithm classifier. For example, a source image might be classified as classical style in the style dimension, watercolor in the technique dimension, and passionate in the emotion dimension. Besides obtaining a definitive classification, each dimension can also yield a probability distribution. For instance, in the style dimension, 80% of the machine might infer classical style, 10% as pop style, and 10% as minimalist style, with similar distributions for other dimensions.

[0138] exist Figure 6 In the embodiments, this technical solution achieves the following: based on the machine learning classification algorithm, according to the designer's pre-defined multiple design dimensions and the classification of each dimension, the reference elements on the design drawing are analyzed, and images with the same design dimension are matched from the material library for replacement, thereby automatically generating more extended drawings.

[0139] In one embodiment of this application, a CNN can be used to classify reference elements. An algorithm is used to analyze the design elements: the algorithm used is consistent with the algorithm used to build the design material library in the auxiliary process. The background and reference elements of the design are analyzed to determine their reference labels in various dimensions. The results can be reference labels or the probability distribution of reference labels among various reference labels. The dimensions and labels can be set as meticulously as possible, and can vary depending on different needs. For example, if all the extended images only need to be anime-related, then only the dimensions and classifications need to be set based on anime.

[0140] Figure 7 The flowchart illustrating an embodiment of an extended image generation method according to this application is shown in the schematic diagram. The execution entity of the extended image generation method may be a server, for example, a... Figure 1 Server 103 is shown in the image.

[0141] Reference Figure 7 As shown, this extended image generation method is compared to Figure 6 The extended image generation method in the embodiment adds steps S710 to S740, which are described in detail below:

[0142] After reading the design drawing element information, in step S710, the content of the design drawing elements is identified;

[0143] In step S720, word vector technology is used to expand the vocabulary of the identified content;

[0144] In step S730, the content of the design drawing elements is identified, and content tags are generated and stored in the database;

[0145] In step S740, the content with the same classification dimension is matched from the material library and replaced with content that has expanded vocabulary to output an expanded graph.

[0146] exist Figure 7 In the embodiments, by using the design drawing as a reference image, the design drawing elements as reference elements, the identified content words forming content tags as reference tags, and the extended words as extended tags, the content words in the content dimension can be expanded, resulting in more extended images with similar content.

[0147] The following describes an apparatus embodiment of this application, which can be used to execute the extended image generation method in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the extended image generation method described above.

[0148] Figure 8 A block diagram of an extended image generation apparatus according to an embodiment of this application is shown schematically.

[0149] Reference Figure 8 As shown, an extended image generation apparatus 800 according to an embodiment of this application includes an acquisition module 801, a classification module 802, a matching module 803, and a generation module 804.

[0150] According to one aspect of the embodiments of this application, based on the foregoing scheme, the acquisition module 801 is configured to acquire a reference image and acquire reference elements in the reference image; the classification module 802 is configured to perform multi-dimensional classification on the reference elements to obtain the label classification results of the reference elements, the label classification results are used to determine the reference label combination of the reference elements, the reference label combination includes the reference labels of the reference elements in multiple dimensions; the matching module 803 is configured to match the reference label combination corresponding to the label classification results with the material label combination of multiple material elements by dimension, and determine the target material element from the multiple material elements whose reference label matching degree with the reference element satisfies the first predetermined matching condition; wherein, the material label combination of the material element is composed of material labels in at least one dimension; the generation module 804 is configured to generate an extended image corresponding to the reference image based on the target material element.

[0151] In one embodiment of this application, based on the aforementioned scheme, the tag classification result includes a reference tag combination of reference elements. The matching module 803 is configured to: for each material element's material tag combination, compare the tags of the reference tag combination and the material tag combination in the same dimension; determine the target dimension and the corresponding number of target dimensions where the material tag combination and the reference tag combination have the same tags; wherein, the reference tag matching degree between the material element and the reference element is determined by the number of target dimensions; when the number of target dimensions corresponding to the material element reaches a first predetermined number threshold, determine that the reference tag matching degree between the material element and the reference element satisfies the first predetermined matching condition; based on the number of target dimensions corresponding to the material tag combinations of each material element among multiple material elements, determine the material element whose number of target dimensions reaches the first predetermined number threshold as the target material element.

[0152] In one embodiment of this application, based on the foregoing scheme, the matching module 803 is further configured to: semantically expand each reference tag in the reference tag combination to obtain an expanded tag combination of the reference element, wherein the expanded tag combination is composed of expanded tags in multiple dimensions; match the expanded tag combination with the material tag combination by dimension, and determine the expanded material element from multiple material elements whose expanded tag matching degree with the reference element satisfies the second predetermined matching condition; the generation module 804 is configured to: generate an expanded image corresponding to the reference image based on the target material element and the expanded material element.

[0153] In one embodiment of this application, based on the aforementioned scheme, the matching module 803 is configured to: obtain the word vectors corresponding to the text of each reference tag; calculate the distance between the word vectors corresponding to the text of each reference tag and the word vectors corresponding to the text of the candidate tag; and select extended tags from the candidate tags whose distance is within a set range for combination to obtain an extended tag combination.

[0154] In one embodiment of this application, based on the aforementioned scheme, the tag classification result includes the reference tag probability distribution of the reference element in multiple dimensions. The reference tag probability distribution includes the reference tag probability of the reference element corresponding to multiple dimension tags in each dimension, and the reference tag is the dimension tag with the highest reference tag probability in each dimension. The matching module 803 is configured to: obtain the reference tag probability distribution of the reference tag combination in multiple dimensions; obtain the material tag probability distribution of the material tag combination in multiple dimensions, where the material tag probability distribution corresponding to each dimension includes the material tag probability of the material element corresponding to multiple dimension tags in each dimension, and the material tag is the dimension tag with the highest material tag probability in each dimension; match the reference tag probability distribution and the material tag probability distribution by dimension, and determine the target material element from multiple material elements whose reference tag matching degree with the reference element meets the first predetermined matching condition.

[0155] In one embodiment of this application, based on the aforementioned scheme, the matching module 803 is configured to: calculate a first difference between the probability of a reference tag and the probability of a material tag corresponding to the same dimension tag in the same dimension; determine the first tag difference degree between the reference tag combination and the material tag combination corresponding to each tag in each dimension based on the first tag difference degree; determine the first element difference degree between the reference element and the material element based on the first tag difference degree corresponding to each dimension tag in each dimension, wherein the reference tag matching degree between the material element and the reference element is determined by the first element difference degree; when the first element difference degree corresponding to the material element reaches the first difference degree threshold, determine that the reference tag matching degree between the material element and the reference element satisfies the first predetermined matching condition; and determine the material element whose first element difference degree is lower than the first difference degree threshold as the target material element based on the first element difference degree corresponding to each material element among multiple material elements.

[0156] In one embodiment of this application, based on the aforementioned scheme, the matching module 803 is configured to: sum the first label differences corresponding to multiple dimension labels in each dimension to obtain the first dimension differences between the reference element and the material element in each dimension; and sum the first dimension differences of the material element in each dimension as the first element differences between the reference element and the material element.

[0157] In one embodiment of this application, based on the aforementioned scheme, the generation module 804 is configured to: replace the reference elements in the reference image with the target material elements to obtain the extended image.

[0158] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0159] The following reference Figure 9 To describe an electronic device 90 according to this embodiment of the present application. Figure 9 The electronic device 90 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0160] like Figure 9 As shown, the electronic device 90 is presented in the form of a general-purpose computing device. The components of the electronic device 90 may include, but are not limited to: at least one processing unit 91, at least one storage unit 92, a bus 93 connecting different system components (including storage unit 92 and processing unit 91), and a display unit 94.

[0161] The storage unit stores program code that can be executed by the processing unit 91, causing the processing unit 91 to perform the steps described in the "Embodiment Methods" section above according to various exemplary embodiments of this application.

[0162] Storage unit 92 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 921 and / or a cache memory unit 922, and may further include a read-only memory unit (ROM) 923.

[0163] Storage unit 92 may also include a program / utility 924 having a set (at least one) of program modules 925, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0164] Bus 93 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0165] Electronic device 90 can also communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 90, and / or with any device that enables electronic device 90 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 95. Furthermore, electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 96. As shown, network adapter 96 communicates with other modules of electronic device 90 via bus 93. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 90, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0166] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of this application.

[0167] According to one embodiment of this application, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above in this specification is stored. In some possible implementations, various aspects of this application may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of this application described in the "Exemplary Methods" section of this specification.

[0168] According to one embodiment of this application, the program product for implementing the above method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0169] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0170] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0171] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0172] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0173] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0174] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An extended image generation method, characterized in that, include: Obtain a reference image, and obtain reference elements in the reference image; The reference element is classified in multiple dimensions to obtain the label classification result of the reference element. The label classification result includes the reference label probability distribution of the reference element in multiple dimensions. The reference label probability distribution includes the reference label probability of the reference element corresponding to multiple dimension labels in each dimension. The reference label is the dimension label with the highest reference label probability in each dimension. The reference tag combination corresponding to the tag classification result is matched with the material tag combination of multiple material elements by dimension to determine the target material element from the multiple material elements; Generate an extended image corresponding to the reference image based on the target material elements; Specifically, the reference tag combination corresponding to the tag classification result is matched with the material tag combination of multiple material elements by dimension to determine the target material element from the multiple material elements, including: Obtain the probability distribution of material tag combinations for each of multiple material elements across multiple dimensions. The probability distribution of material tag includes the material tag probability of each material element corresponding to multiple dimension tags in each dimension, where the material tag is the dimension tag with the highest material tag probability in each dimension. The material tag combination of each material element consists of material tags in at least one dimension. Calculate the first difference between the reference tag probability and the material tag probability corresponding to the same dimension tag in the same dimension, and determine the first tag difference degree between the reference tag combination and the material tag combination in each dimension based on the first difference; Based on the first tag difference degree, a first element difference degree is determined between the reference element and the material element. Based on the first element difference degree corresponding to each material element among the plurality of material elements, the material elements with a first element difference degree lower than the first difference degree threshold are determined as the target material elements.

2. The extended image generation method according to claim 1, characterized in that, The tag classification result includes a combination of reference tags for the reference elements; the step of matching the combination of reference tags corresponding to the tag classification result with the combination of material tags for multiple material elements by dimension, and determining the target material element from the multiple material elements, includes: For each material element's material tag combination, compare the reference tag combination with the material tag combination in the same dimension; determine the target dimension and the corresponding number of target dimensions where the material tag combination and the reference tag combination have the same tags; Based on the number of target dimensions corresponding to the combination of material tags of each of the multiple material elements, the material elements whose number of target dimensions reaches a first predetermined threshold are determined as the target material elements.

3. The extended image generation method according to claim 2, characterized in that, Before generating the extended image corresponding to the reference image based on the target material element, the method further includes: semantically expanding each reference tag in the reference tag combination to obtain an extended tag combination of the reference element, wherein the extended tag combination is composed of extended tags in the multiple dimensions; matching the extended tag combination with the material tag combination by dimension, and determining from the multiple material elements an extended material element whose extended tag matching degree with the reference element satisfies a second predetermined matching condition; The step of generating an extended image corresponding to the reference image based on the target material elements includes: An extended image corresponding to the reference image is generated based on the target material element and the extended material element.

4. The extended image generation method according to claim 3, characterized in that, The step of semantically expanding each reference label in the reference label combination to obtain an expanded label combination representing the semantic features of the reference element includes: Obtain the word vectors corresponding to the text of each reference tag; Calculate the distance between the word vector corresponding to the text of each reference tag and the word vector corresponding to the text of the candidate tag; Extended tags that are within the set distance range are selected from the candidate tags and combined to obtain the extended tag combination.

5. The extended image generation method according to claim 1, characterized in that, The step of determining the first element difference between the reference element and the material element based on the first tag difference includes: The difference between the first label corresponding to each dimension label in each dimension is summed to obtain the difference between the first dimension of the reference element and the material element in each dimension. The summation of the first dimension differences of each dimension corresponding to the material element is used as the first element difference between the reference element and the material element.

6. The extended image generation method according to claim 1, characterized in that, The step of generating an extended image corresponding to the reference image based on the target material elements includes: The reference element in the reference image is replaced with the target material element to obtain the extended image.

7. An extended image generation apparatus, characterized in that, include: The acquisition module is configured to acquire a reference image and acquire reference elements in the reference image; The classification module is configured to perform multi-dimensional classification on the reference element to obtain the label classification result of the reference element. The label classification result includes the reference label probability distribution of the reference element in multiple dimensions. The reference label probability distribution includes the reference label probability of the reference element corresponding to multiple dimension labels in each dimension. The reference label is the dimension label with the highest reference label probability in each dimension. The matching module is configured to match the reference tag combination corresponding to the tag classification result with the material tag combination of multiple material elements by dimension, and determine the target material element from the multiple material elements; The generation module is configured to generate an extended image corresponding to the reference image based on the target material elements; The matching module is configured to: obtain the probability distribution of material tag combinations of multiple material elements in multiple dimensions, wherein the material tag probability distribution includes the material tag probability of the material element corresponding to multiple dimension tags in each dimension, and the material tag is the dimension tag with the highest material tag probability in each dimension; the material tag combination of the material element is composed of material tags in at least one dimension; Calculate the first difference between the reference tag probability and the material tag probability corresponding to the same dimension tag in the same dimension, and determine the first tag difference degree between the reference tag combination and the material tag combination in each dimension based on the first difference; Based on the first tag difference degree, a first element difference degree is determined between the reference element and the material element. Based on the first element difference degree corresponding to each material element among the plurality of material elements, the material elements with a first element difference degree lower than the first difference degree threshold are determined as the target material elements.

8. A computer device, characterized in that, include: Memory, which stores computer-readable instructions; A processor reads computer-readable instructions stored in memory to perform the method described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, wherein a processor of a computer device reads from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method of any one of claims 1 to 6.

Images