Image generation method and system based on information consistency generative network

Abstract

The invention discloses an image generation method and system based on an information consistency generative network, and the method comprises the steps: mapping an initial image to a feature space, and extracting discrimination features of the mapped image; performing detail enhancement on the image and generating coding features in a feature space; aligning the discrimination features and the coding features in the feature space, and converting the aligned features to obtain a first vector representation; converting the process description into second vector representation, and generating a simulation graph by the generative network by taking the first vector representation and the second vector representation as input; the embroidery process effect with rich details and layering sense can be output, and the sense of reality is better; design details such as textures, colors and styles of a design drawing can be fully kept, and the fidelity is higher; and based on the text semantic information of embroidery, more embroidery effect pictures can be output, and the diversity is stronger.

Description

Technical Field

[0001] The present invention relates to the technical field of process simulation graph processing, and in particular to an image generation method and system based on an information consistency generation network. Background Art

[0002] Embroidery is widely used in fashion, home decoration, gifts and art. It can appear on clothing, shoes, bags, bedding, wall decorations and other products, adding a unique aesthetic to them. Although embroidery has a long history and wide application in the textile industry, it still faces some difficulties and challenges. For example, hand embroidery takes a long time to complete complex patterns and details. This makes the delivery cycle of customized embroidery products long and cannot meet the needs of some consumers for fast delivery. In addition, the hand embroidery process requires skilled skills and experience, and requires a lot of manual participation, which increases production costs. Although machine embroidery can improve production efficiency, the accuracy and flexibility of the machine are still limited when dealing with complex patterns and details. This may result in some complex and delicate embroidery designs that cannot be fully realized by machines. Although machine embroidery can be customized, the embroidery effect of the machine may appear relatively mechanical and standardized compared to hand embroidery. For consumers who pursue uniqueness and personalization, machine embroidery may not fully meet the needs. In particular, the embroidery patterns generated in the prior art are not real and realistic enough.

[0003] For example, an "embroidery style configuration system and device, embroidery style configuration method and sewing machine" disclosed in Chinese patent literature, with publication number: CN105821589B, discloses a shooting unit that includes an embroidery frame and an embroidery cloth in a shooting range for shooting, a captured image analysis unit that analyzes the embroidery frame information obtained by an embroidery frame information acquisition unit so that the embroidery frame in the captured image corresponds to the embroidery frame, and an embroidery style acquisition unit that acquires information related to a predetermined embroidery style for embroidery; however, this solution cannot specifically identify personalized customization of embroidery details. Summary of the invention

[0004] In order to solve the problem in the prior art that the fine recognition effect of embroidery pattern details is poor, resulting in unrealistic generated patterns, the present invention provides an embroidery generation process and system based on an information consistency generation network, which can make full use of the global features and detail information of the design drawing to generate a more realistic and lifelike embroidery process effect.

[0005] In order to achieve the above object, the present invention provides the following technical solutions: A pattern generation method based on an information consistency generation network, comprising: Map the initial image to the feature space and extract discriminant features from the mapped image; Enhance the image details and generate encoding features in feature space; Aligning the discriminant features and the encoding features in the feature space, and converting the aligned features to obtain a first vector representation; The process description is converted into a second vector representation, and the generative network generates a simulation image with the first vector representation and the second vector representation as input. First, the consistency information is extracted. The consistency information includes the overall discriminant features and the detailed encoding features. By aligning the feature information obtained by the two-part operation, the generative network can make full use of the global features and detail information of the design drawing to generate a more realistic and more restored embroidery process effect. After obtaining the consistency information of the embroidery process, the guide vector representation of different embroidery processes is obtained. The guide vector guides the consistency information of the embroidery process to generate an embroidery design simulation image that conforms to the specified embroidery process in the generative network; the personalized and customized embroidery design is simulated by combining image generation technology and text description, which enriches the diversity of embroidery technology and meets the needs of different types of embroidery styles. It is especially convenient for the accurate generation of unconventional features in customized personalized embroidery products.

[0006] Preferably, the alignment of the discriminant feature and the coding feature includes determining the similarity vector of the discriminant feature and the coding feature, aligning the similarity vector of the coding feature with the similarity vector of the discriminant feature, and using the aligned similarity vector as an alignment matrix to coordinately align the discriminant feature and the coding feature. The method includes respectively obtaining the similarity vectors of the discriminant feature and the coding feature, wherein the similarity vector of the discriminant feature is the similarity vector of each location in the initial image as a whole, and the similarity vector of the coding feature is the similarity vector of the image details, aligning the similarity vector of the coding feature with the similarity vector of the corresponding position in the discriminant feature, and fitting the discriminant feature and the coding feature corresponding to the similarity vector after aligning the similarity vector, and obtaining the feature vector representing the embroidery image after fitting. The structure and pattern elements of the initial image of the design drawing are captured through the similarity vector and converted into the specific expression of the embroidery craft, including line fineness, color matching and texture effect; by aligning them with the vectors, the global features and detail information of the design drawing can be effectively integrated, so that the generative network can make full use of these feature information; ultimately, the overall structure of the design drawing is accurately captured, and it is coordinated with the detail elements to accurately present the exquisite patterns and layered effects required for embroidery works.

[0007] Preferably, when converting the process description into a second vector representation, the process type in the process description is identified, and different process types are transcoded into a second vector representation in a corresponding feature space. This includes performing neural network learning on different embroidery processes to obtain the characteristic values ​​of the stitches and colors in each embroidery process, transcoding the characteristic values ​​to obtain a second vector representation, and the second vector representation determines the level type, stitch style, and color style of different embroidery processes; transcoding includes identifying customized process characteristics in the process type, including customized process characteristics and characteristics of pre-stored process types, transcoding the above characteristics into a vector representation, thereby obtaining a second vector representation of the corresponding feature space. The pattern diversity of the embroidery generation process can be further enriched through the multiple characteristic values ​​of different embroidery processes, meeting the needs of different types of embroidery styles, and facilitating the provision of a more personalized and customized embroidery design experience.

[0008] Preferably, the transcoding includes converting different embroidery techniques into vector representation relationships in a feature space, and determining the vector representation relationships according to type parameters of different embroidery techniques. It includes obtaining a process text, identifying feature words about the process in the process text, extracting feature values ​​corresponding to the feature words, using the feature values ​​specific to each process as type parameters, and determining the vector representation relationship together with the customized features. In this way, the network can be guided to generate a simulation design pattern that meets the specified embroidery process according to the text description provided by the user.

[0009] As a preferred method, an embroidery process library is established, in which a text description of multiple embroidery is provided, and the text description of each embroidery process includes embroidery features and embroidery process features. The embroidery process library includes different types of embroidery processes such as Suzhou embroidery, Guangdong embroidery, Beijing embroidery, silk embroidery, and thick thread embroidery, and each embroidery process includes the needle method, color, pattern style and other parameters. Therefore, based on these inputs, the embroidery design pattern that meets the given embroidery process style is learned and generated to meet the needs of different types of embroidery styles.

[0010] Preferably, a machine learning model is used to map the two-dimensional embroidery image to a feature space, and the machine learning model includes a self-supervised model. The self-supervised model uses the structure and information of the image data itself for training, and performs representation prediction and weight allocation, the representation prediction obtains the image representation, and the weight allocation determines the representation mapping. The representation mapping is the conversion relationship of multiple image representations after the image representation is adjusted by weight. By training according to the design drawing itself, a more accurate output result is obtained, which facilitates the accurate global pattern representation.

[0011] Preferably, when extracting the discriminant features, a multi-layer neural network is established as a conversion space, the conversion space is connected to the feature space, and the feature space is in the generation network. The multi-layer neural network uses the discriminant features as input vectors, and recognizes the discriminant features in layers, generates multi-layer feature vectors, and sends the multi-layer feature vectors to the generation network for image generation. In the generation network, the multi-layer feature vectors form a layered image model, and the generation network generates layers at each layer of the layered image model, that is, multiple layers are generated according to the discriminant features to represent the image.

[0012] Preferably, the image detail enhancement includes using high-pass filtering to enhance the details, suppressing the smooth area and background in the image, identifying the edges and details and enhancing them in the high-frequency area, so that the high-frequency part of the image is more prominent and clear.

[0013] An image generation system based on an information consistency generation network is applicable to an image generation method based on an information consistency generation network described in any embodiment, comprising an image encoder, the image encoder extracts consistency information; the image encoder is connected to a generation network, the generation network is connected to a text encoder, the text encoder encodes the text; the generation network is connected to a decoder, the decoder decodes the data in the generation network into an embroidery pattern. It can quickly generate an embroidery pattern with obvious details, diverse features and more realistic.

[0014] An electronic device, comprising: one or more processors; a memory for storing processor-executable instructions; The processor implements the method described in any embodiment by running the executable instructions.

[0015] The present invention has the following advantages: (1) For a given design drawing, a generative network is used to quickly output embroidery effects with rich details and layering, which has a better sense of reality; (2) The design details of the design drawing, such as texture, color, style, etc., can be fully preserved, with higher fidelity; (3) Based on the text semantic information of embroidery, more embroidery effect pictures can be output with greater diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described below are merely exemplary. A person skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0017] Figure 1 It is a schematic diagram of the process of the present invention.

[0018] Figure 2 It is a schematic diagram of the method steps of the present invention. DETAILED DESCRIPTION

[0019] The following is an explanation of the implementation mode of the present invention by means of specific embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.

[0020] like Figure 1-2 As shown, in a preferred embodiment, the present invention discloses an image generation method based on an information consistency generation network, comprising: Map the initial image to the feature space and extract discriminant features from the mapped image; Enhance the image details and generate encoding features in feature space; Aligning the discriminant features and the encoding features in the feature space, and converting the aligned features to obtain a first vector representation; The process description is converted into a second vector representation, and the generative network generates a simulation image with the first vector representation and the second vector representation as input. First, the consistency information is extracted. The consistency information includes the overall discriminant features and the detailed encoding features. By aligning the feature information obtained by the two-part operation, the generative network can make full use of the global features and detail information of the design drawing to generate a more realistic and more restored embroidery process effect. After obtaining the consistency information of the embroidery process, the guide vector representation of different embroidery processes is obtained. The guide vector guides the consistency information of the embroidery process to generate an embroidery design pattern that conforms to the specified embroidery process in the generative network; it realizes the design of personalized and customized embroidery designs by combining image generation technology and text description, enriching the diversity of embroidery technology, meeting the needs of different types of embroidery styles, and especially facilitating the accurate generation of unconventional features in customized personalized embroidery products.

[0021] When used, after obtaining the consistency information of the embroidery image, the consistency information of the image is optimized through the second vector representation to generate an image, thereby obtaining a simulation image of the initial image. By combining text description and image generation technology, this method can further enrich the diversity of embroidery crafts, meet the needs of users for different types of embroidery styles, and provide a more personalized and customized embroidery design experience.

[0022] In other embodiments, the initial image includes a design drawing, and the simulation image is a production drawing of an embroidery process, a weaving process or other processes.

[0023] In other embodiments, the alignment of the discriminant feature and the coding feature includes determining the similarity vector of the discriminant feature and the coding feature, aligning the similarity vector of the coding feature with the similarity vector of the discriminant feature, and using the aligned similarity vector as an alignment matrix to coordinately align the discriminant feature and the coding feature. The method includes respectively obtaining the similarity vectors of the discriminant feature and the coding feature, the similarity vector of the discriminant feature is the similarity vector of each location in the entire embroidery pattern, and the similarity vector of the coding feature is the similarity vector of the pattern details, aligning the similarity vector of the coding feature with the similarity vector of the corresponding position in the discriminant feature, and fitting the discriminant feature and the coding feature corresponding to the similarity vector after aligning the similarity vector, and obtaining the feature vector representing the embroidery image after fitting. The structure and pattern elements of the initial pattern of the design drawing are captured by the similarity vector and converted into the specific expression of the embroidery craft, including line fineness, color matching and texture effect; by aligning them by vectors, the global features and detail information of the design drawing can be effectively integrated, so that the generation network can make full use of these feature information; ultimately, the overall structure of the design drawing is accurately captured, and it is coordinated with the detail elements to accurately present the exquisite patterns and layered effects required for embroidery works.

[0024] In other embodiments, when converting the process description into a second vector representation, the process type in the process description is identified, and different process types are transcoded into a second vector representation in a corresponding feature space. This includes performing neural network learning on different embroidery processes to obtain the characteristic values ​​of the stitches and colors in each embroidery process, transcoding the characteristic values ​​to obtain a second vector representation, and the second vector representation determines the level type, stitch style, and color style of different embroidery processes; transcoding includes identifying customized process characteristics in the process type, including customized process characteristics and characteristics of the pre-stored process type, transcoding the above characteristics into a vector representation, thereby obtaining a second vector representation of the corresponding feature space. The pattern diversity of the embroidery generation process can be further enriched through the multiple characteristic values ​​of different embroidery processes, meeting the needs of different types of embroidery styles, and facilitating the provision of a more personalized and customized embroidery design experience.

[0025] In other embodiments, the process description is determined using a text encoder, and the text encoder also includes process types including different processes.

[0026] In other embodiments, the transcoding includes converting different embroidery processes into vector representation relationships in a feature space, and determining the vector representation relationship according to type parameters of different embroidery processes. It includes obtaining a process text, identifying feature words about the process in the process text, extracting feature values ​​corresponding to the feature words, using the feature values ​​specific to each process as type parameters, and determining the vector representation relationship together with the customized features. In this way, the network can be guided to generate a simulation design pattern that meets the specified embroidery process according to the text description provided by the user.

[0027] In other embodiments, different processes are converted into vector representation relationships in a feature space, and the processes include weaving, embroidery and other processes.

[0028] In other embodiments, an embroidery process library is established, and a text description of multiple embroidery processes is provided in the embroidery process library, and the text description of each embroidery process includes embroidery features and embroidery process features. The embroidery process library includes different types of embroidery processes such as Suzhou embroidery, Guangdong embroidery, Beijing embroidery, silk embroidery, and thick thread embroidery, and each embroidery process includes parameters such as needlework, color, and pattern style. Therefore, based on these inputs, embroidery design patterns that meet the given embroidery process style are learned and generated to meet the needs of different types of embroidery styles.

[0029] In other embodiments, a process library is included. When in use, relevant text descriptions of different processes are obtained through the process library, and targeted parameters are determined through the text descriptions.

[0030] In other embodiments, a two-dimensional embroidery image is mapped to a feature space using a machine learning model, wherein the machine learning model includes a self-supervised model. The self-supervised model uses the image data's own structure and information for training, and performs representation prediction and weight assignment, wherein the representation prediction obtains the image representation, and the weight assignment determines the representation mapping. The representation mapping is a transformation relationship of multiple image representations after the image representation is adjusted by weight. By training according to the design drawing itself, a more accurate output result is obtained, thereby facilitating the acquisition of an accurate global pattern representation.

[0031] In other embodiments, the machine learning model includes one or more of other machine learning methods such as metric learning, weak supervision, and semi-supervision.

[0032] In other embodiments, when extracting discriminant features, a multi-layer neural network is established as a transformation space, the transformation space is connected to the feature space, and the feature space is in the generation network.

[0033] In other embodiments, the multi-layer neural network uses the discriminant feature as an input vector, and recognizes the discriminant feature in layers, generates multi-layer feature vectors, and sends the multi-layer feature vectors to the generative network for image generation. In the generative network, the multi-layer feature vectors form a hierarchical image model, and the generative network generates layers at each layer of the hierarchical image model, that is, generates multiple layers according to the discriminant feature to represent the image.

[0034] In other embodiments, the detail enhancement of the image includes using high-pass filtering to enhance the details, suppressing the smooth area and background in the image, identifying the edges and details and enhancing them, so that the high-frequency part of the image is more prominent and clear.

[0035] In another embodiment, detail enhancement recognition is performed by means of normal, depth map, edge detection, etc.

[0036] In another embodiment, an image generation system based on an information consistency generation network is included, which is applicable to an image generation method based on an information consistency generation network described in any embodiment, including an image encoder, which extracts consistency information; the image encoder is connected to a generation network, which is connected to a text encoder, which encodes text; the generation network is connected to a decoder, which decodes the data in the generation network into an embroidery pattern. It can quickly generate an embroidery pattern with obvious details, diverse features and more realistic.

[0037] In another embodiment, a detail enhancement module is included, and the detail enhancement module performs detail enhancement and recognition through normals, depth maps, edge detection, etc.

[0038] In another embodiment, a discriminative feature extraction module is included, and the discriminative feature extraction module performs feature recognition through other machine learning methods such as metric learning, weak supervision, semi-supervision, etc.

[0039] In another embodiment, given a 2D design drawing, first, the consistency information of the image is extracted, which mainly includes two parts: one part is, 1. First, the design drawing is processed using an image encoder, mainly using a self-supervised model to map the image to a feature representation in the feature space; the self-supervised model is trained by utilizing the data’s own structure and information without the need for manually annotated labels, and can learn meaningful global feature representations from the data.

[0040] 2. Next, the discriminative features are extracted from the feature representation output by the self-supervised model. Specifically, an MLP network is used to process the feature information to convert it into a feature space suitable for the generative network.

[0041] The feature representation obtained by this operation effectively retains the visual information of the input design drawing.

[0042] The other part is, 1. First, enhance the details of the image, mainly by using image high-pass filtering technology. The function of the image high-pass filter is to enhance the high-frequency part of the image (that is, the details and edges), while suppressing the low-frequency part (that is, the smooth area and background in the image). The high-pass filter removes the low-frequency components in the image, making the details in the image more prominent and clear.

[0043] 2. The enhanced image is processed by an image encoder to encode the detail features into the feature representation space to guide the generative network to control the details of the embroidery process. The feature representation obtained by this operation effectively retains the detail information of the input design.

[0044] When in use, first, a text encoder is trained to convert a given embroidery craft text description into a matching feature space representation to achieve the correspondence between text and image. In the input stage of the generative network, users can provide text descriptions of embroidery crafts, such as "Suzhou embroidery" or "Guangdong embroidery". Through the text encoder, these descriptions can be converted into vector representations consistent with image features. Then, the image features and text features are used as inputs to the generative network at the same time. The generative network will learn and generate embroidery design patterns that meet the given embroidery craft style based on these inputs. In this way, the generative network can be guided according to the text description provided by the user, and then an embroidery design pattern that meets the specified embroidery craft can be generated.

[0045] In other embodiments, an electronic device is provided, comprising: one or more processors; a memory for storing processor-executable instructions; The processor implements the method described in any of the above embodiments by running the executable instructions.

[0046] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. An image generation method based on an information consistency generation network, characterized in that: include: Map the initial image to the feature space and extract discriminant features from the mapped image; Enhance the image details and generate encoding features in feature space; Aligning the discriminant features and the encoding features in the feature space, and converting the aligned features to obtain a first vector representation; The process description is converted into a second vector representation, and a generation network generates a simulation graph using the first vector representation and the second vector representation as input.

2. The image generation method based on information consistency generation network according to claim 1, characterized in that: The alignment discrimination features and encoding features include: Determine the similarity vector of the discriminant feature and the encoding feature, align the similarity vector of the encoding feature with the similarity vector of the discriminant feature, and use the aligned similarity vector as an alignment matrix to align the coordinates of the discriminant feature and the encoding feature.

3. The image generation method based on information consistency generation network according to claim 1 or 2, characterized in that: When converting the process description into the second vector representation, the process type in the process description is identified, and different process types are transcoded into the second vector representation in the corresponding feature space.

4. The image generation method based on information consistency generation network according to claim 3 is characterized in that: The transcoding includes converting different processes into vector representation relationships in a feature space, and determining the vector representation relationships according to type parameters of the different processes.

5. The image generation method based on information consistency generation network according to claim 4, characterized in that: The process includes establishing a process library, in which text descriptions of various processes are provided, and the text description of each process includes pattern features and process features.

6. The image generation method based on information consistency generation network according to claim 1, characterized in that: It includes mapping a two-dimensional embroidery image to a feature space using a machine learning model, wherein the machine learning model includes a self-supervised model.

7. The image generation method based on information consistency generation network according to claim 6, characterized in that: When extracting the discriminant features, a multi-layer neural network is established as a conversion space, the conversion space is connected to the feature space, and the feature space is in the generation network.

8. The image generation method based on information consistency generation network according to claim 1, characterized in that: The detail enhancement of the image includes using high-pass filtering to enhance the details, suppressing the smooth area and background in the image, and enhancing the high-frequency area, identifying edges and details.

9. An image generation system based on an information consistency generation network, applicable to an image generation method based on an information consistency generation network as claimed in any one of claims 1 to 8, characterized in that: It includes an image encoder, which extracts consistency information; the image encoder is connected to a generation network, the generation network is connected to a text encoder, and the text encoder encodes the text; the generation network is connected to a decoder, and the decoder decodes the data in the generation network into a simulation image.

10. An electronic device comprising: one or more processors; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

Images