Sketch-based image conversion method, device, equipment and storage medium
By extracting the feature of the sketch and determining the target image type, and using the conversion model to convert the picture, the problem of inaccurate sketch conversion is solved, efficient and accurate picture conversion effect is achieved, and user experience is improved.
Patent Information
- Application Number
- CN202210319444.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-29
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-03-29
AI Technical Summary
The prior art cannot accurately convert pictures based on sketches, resulting in inaccurate conversion results and cannot meet the visual needs of users.
By obtaining the user's sketch and sketch conversion instructions, determine the target image type, and extract the sketch feature, including lines, colors and shape recognition, image conversion based on the target features and types, use the target conversion model to perform matching ranking, and obtain the most Match the conversion picture.
It realizes the accuracy and efficiency of sketch conversion, improves the user experience, and ensures that the conversion results are consistent with the user's intentions.
Smart Images

Figure CN114757816B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a sketch-based picture conversion method, device, equipment and storage medium. Background Art
[0002] Sketching is one of the most natural and flexible ways for humans to express and convey information. With the increasing popularity of touch-screen devices in recent years, sketching on touch screens with just a few strokes to express human visual needs has become an important way of human-computer interaction. Since sketches are highly abstract and sketches drawn by different people may vary greatly in accuracy and style, the sketch-to-image translation problem still faces huge challenges.
[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention
[0004] The main purpose of the present invention is to provide a sketch-based image conversion method, device, equipment and storage medium, aiming to solve the technical problem that the prior art cannot convert accurate images based on sketch-based images.
[0005] To achieve the above object, the present invention provides a sketch-based image conversion method, the sketch-based image conversion method comprising:
[0006] Obtain the user's sketch to be converted and sketch conversion instructions;
[0007] Determine the target image type according to the sketch conversion instruction;
[0008] Performing feature extraction on the sketch to be converted to obtain target conversion features of the sketch to be converted;
[0009] Perform image conversion according to the target image type and the target conversion feature to obtain a target conversion image.
[0010] Optionally, determining the target image type according to the sketch conversion instruction includes:
[0011] Determining a target label according to the sketch conversion instruction;
[0012] Searching for the image type corresponding to the target tag in a tag type mapping table;
[0013] The target image type is determined according to the image type corresponding to the target tag.
[0014] Optionally, the extracting features of the sketch to be converted to obtain target conversion features of the sketch to be converted includes:
[0015] Performing line recognition on the sketch to be converted to obtain line features of the sketch to be converted;
[0016] Performing color recognition on the sketch to be converted to obtain color features of the sketch to be converted;
[0017] Performing basic shape recognition on the sketch to be converted to obtain shape features of the sketch to be converted;
[0018] The target conversion features of the sketch to be converted are determined according to the line features, color features and shape features.
[0019] Optionally, performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image includes:
[0020] Determining a target conversion model according to the target image type;
[0021] Inputting the target conversion feature into the target conversion model to perform image conversion, and obtaining an initial conversion image and its corresponding image matching degree;
[0022] Ranking the matching degrees of the pictures to obtain a ranking result;
[0023] The target image matching degree ranked first in the ranking results is obtained, and the initial conversion image corresponding to the target image matching degree is used as the target conversion image.
[0024] Optionally, determining a target conversion model according to the target picture type includes:
[0025] Get each sample image type and the corresponding sample conversion image;
[0026] Determine a sample conversion sketch corresponding to the sample conversion picture according to the sample conversion picture;
[0027] Performing model training on the initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketches to obtain sample conversion models corresponding to each type of the sample pictures;
[0028] A target conversion model is determined according to the target picture type, the sample picture type, and the sample conversion models corresponding to each of the sample picture types.
[0029] Optionally, the performing model training on the initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketches to obtain sample conversion models corresponding to each of the sample picture types includes:
[0030] Extracting features from the sample conversion sketch to obtain training line features, training color features, and training shape features;
[0031] The training line features, training color features, training shape features, sample conversion images, and sample matching degrees corresponding to the sample conversion images are input into the initial conversion model for model training to obtain sample conversion models corresponding to each of the sample image types.
[0032] Optionally, after performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image, the method further includes:
[0033] Sending the target conversion image to the user, and receiving the evaluation result fed back by the user;
[0034] When the evaluation result is not a preset evaluation result, the step of returning to the step of performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image.
[0035] In addition, to achieve the above-mentioned purpose, the present invention also proposes a sketch-based image conversion device, the sketch-based image conversion device comprising:
[0036] An acquisition module, used to acquire the user's sketch to be converted and the sketch conversion instruction;
[0037] A determination module, used to determine the target image type according to the sketch conversion instruction;
[0038] An extraction module, used for extracting features of the sketch to be converted to obtain target conversion features of the sketch to be converted;
[0039] The conversion module is used to perform image conversion according to the target image type and the target conversion feature to obtain a target conversion image.
[0040] In addition, to achieve the above-mentioned purpose, the present invention also proposes a sketch-based image conversion device, which includes: a memory, a processor, and a sketch-based image conversion program stored in the memory and runnable on the processor, and the sketch-based image conversion program is configured to implement the sketch-based image conversion method described above.
[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a sketch-based image conversion program is stored. When the sketch-based image conversion program is executed by a processor, the sketch-based image conversion method as described above is implemented.
[0042] The present invention obtains the user's sketch to be converted and sketch conversion instructions; determines the target image type according to the sketch conversion instructions; extracts features of the sketch to be converted to obtain the target conversion features of the sketch to be converted; and performs image conversion according to the target image type and the target conversion features to obtain a target conversion image. In the above manner, accurate target conversion features and target image types are determined through the user's sketch to be converted and sketch conversion instructions, and image conversion is performed based on the target image type and target conversion features, thereby ensuring the accuracy and efficiency of image conversion and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a structural schematic diagram of a sketch-based picture conversion device in a hardware operating environment involved in an embodiment of the present invention;
[0044] Figure 2 It is a flowchart of the first embodiment of the sketch-based picture conversion method of the present invention;
[0045] Figure 3 It is a flowchart diagram of a second embodiment of the sketch-based picture conversion method of the present invention;
[0046] Figure 4 It is a structural block diagram of the first embodiment of the sketch-based picture conversion device of the present invention.
[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0049] Reference Figure 1 , Figure 1 The schematic diagram of the structure of a sketch-based image conversion device in the hardware operating environment involved in the embodiment of the present invention is shown.
[0050] like Figure 1As shown, the sketch-based image conversion device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0051] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the sketch-based picture conversion device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0052] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a sketch-based image conversion program.
[0053] exist Figure 1 In the sketch-based image conversion device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the sketch-based image conversion device of the present invention can be set in the sketch-based image conversion device, and the sketch-based image conversion device calls the sketch-based image conversion program stored in the memory 1005 through the processor 1001, and executes the sketch-based image conversion method provided by the embodiment of the present invention.
[0054] The embodiment of the present invention provides a method for converting pictures based on sketches. Figure 2 , Figure 2 The figure is a flowchart of a first embodiment of a sketch-based image conversion method of the present invention.
[0055] The sketch-based image conversion method includes the following steps:
[0056] Step S10: Obtain the user's sketch to be converted and the sketch conversion instruction.
[0057] It should be noted that the execution subject of this embodiment is a terminal device, and a sketch-based image conversion system is installed on the terminal device. When the terminal device receives the sketch to be converted and the sketch conversion instruction sent by the user, the sketch to be converted and the sketch conversion instruction are sent to the sketch-based image conversion system. The sketch-based image conversion system determines the target image type according to the sketch conversion instruction, and extracts features of the sketch to be converted to obtain the target conversion features of the sketch to be converted, and finally converts the image according to the target conversion features and the target image type to obtain the target conversion image corresponding to the sketch to be converted. In this embodiment, the terminal device can be a computer, a mobile phone or other intelligent terminal device, and this embodiment does not limit this.
[0058] It is understood that the sketch to be converted refers to an image drawn by the user manually to express the user's basic intentions and concepts. The sketch conversion instruction refers to a conversion instruction sent by the user for the sketch to be converted. After receiving the sketch conversion instruction, the image conversion process is performed based on the sketch conversion instruction and the sketch to be converted.
[0059] Step S20: Determine the target image type according to the sketch conversion instruction.
[0060] It should be noted that the target image type refers to the image type of the sketch to be converted drawn by the user. For example, if the image type of the sketch to be converted drawn by the user is a person image type, then the target image type is a person image type.
[0061] It can be understood that in order to obtain the accurate target image type according to the sketch conversion instruction, thereby ensuring the accuracy of subsequent image conversion, further, the target image type is determined according to the sketch conversion instruction, including: determining the target label according to the sketch conversion instruction; searching the image type corresponding to the target label in the label type mapping table; and determining the target image type according to the image type corresponding to the target label.
[0062] In a specific implementation, the sketch conversion instruction includes but is not limited to the target label of the type corresponding to the sketch to be converted. The label type mapping table refers to a table that stores the mapping relationship between image types and labels after labeling the image types in advance. For example, the image types are labeled by letters, and the label corresponding to the image type of the elderly is A, the label corresponding to the image type of the building is B, and the label corresponding to the image type of the dog is C.
[0063] It should be noted that the target label in the sketch conversion instruction is extracted, the image type corresponding to the target label is searched in the label type mapping table, and the target image type is determined according to the image type corresponding to the target label. For example, if the target label in the sketch conversion instruction is A, and the image type corresponding to the target label A is searched in the label type mapping table as the image type of the elderly, then the target image type determined according to the sketch conversion instruction is the elderly.
[0064] Step S30: extracting features of the sketch to be converted to obtain target conversion features of the sketch to be converted.
[0065] It should be noted that the target conversion feature refers to the feature extracted from the sketch to be converted for image conversion. The target conversion feature includes but is not limited to the line feature, color feature and shape feature of the sketch to be converted. After obtaining the sketch to be converted, feature extraction is performed on the sketch to be converted to obtain the target conversion feature of the sketch to be converted.
[0066] It can be understood that in order to extract accurate target conversion features based on the sketch to be converted, further, the feature extraction of the sketch to be converted to obtain the target conversion features of the sketch to be converted includes: performing line recognition on the sketch to be converted to obtain the line features of the sketch to be converted; performing color recognition on the sketch to be converted to obtain the color features of the sketch to be converted; performing basic shape recognition on the sketch to be converted to obtain the shape features of the sketch to be converted; and determining the target conversion features of the sketch to be converted based on the line features, color features and shape features.
[0067] In a specific implementation, after obtaining the sketch to be converted, line recognition is performed on the sketch to be converted to obtain all lines in the sketch to be converted and the contours corresponding to the lines, and the line features of the sketch to be converted are obtained based on the lines and the contours corresponding to the lines.
[0068] It should be noted that color recognition is performed on the sketch to be converted, and color feature information related to color in the sketch to be converted is extracted, so as to obtain the color features of the sketch to be converted.
[0069] It is understandable that the basic shape recognition is performed on the sketch to be converted, so as to obtain the basic shape in the sketch to be converted, and finally obtain the shape features of the sketch to be converted. The basic shape includes but is not limited to a series of basic shapes such as rectangle, circle, ellipse and parallelogram.
[0070] In the specific implementation, the line features, color features and shape features of the sketch to be identified are extracted to determine
[0071] Step S40: performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image.
[0072] It should be noted that after obtaining the target image type and the target conversion feature, the sketch-based image conversion system can perform image simulation conversion based on the target image type and the target conversion feature, thereby converting the target image. The target conversion image refers to the image that the user wants to express and convey according to the user's sketch to be converted. The target conversion image can be an image that has appeared in the real world or an image that has never appeared in the real world, and this embodiment does not limit this.
[0073] It can be understood that in order to make the target conversion picture meet the user requirements and further improve the user satisfaction, further, after the picture is converted according to the target picture type and the target conversion characteristics to obtain the target conversion picture, it also includes: sending the target conversion picture to the user and receiving the evaluation result fed back by the user; when the evaluation result is not the preset evaluation result, returning to execute the step of converting the picture according to the target picture type and the target conversion characteristics to obtain the target conversion picture.
[0074] In the specific implementation, the preset evaluation result refers to the target conversion image that matches the image corresponding to the sketch to be converted. After obtaining the target conversion image, the target conversion image is sent to the user's terminal for image display, and the user's evaluation result for the target conversion image is received. When the evaluation result is the preset evaluation result, the target conversion image is directly output. When the evaluation result is that the target conversion image does not match the image corresponding to the sketch to be converted, the step of returning to execute image conversion according to the target image type and target conversion characteristics to obtain the target conversion image is returned.
[0075] This embodiment obtains the user's sketch to be converted and sketch conversion instructions; determines the target image type according to the sketch conversion instructions; extracts features of the sketch to be converted to obtain the target conversion features of the sketch to be converted; performs image conversion according to the target image type and the target conversion features to obtain the target conversion image. In the above manner, accurate target conversion features and target image types are determined through the user's sketch to be converted and sketch conversion instructions, and image conversion is performed based on the target image type and target conversion features, thereby ensuring the accuracy and efficiency of image conversion and improving the user experience.
[0076] refer to Figure 3 , Figure 3 This is a flow chart of a second embodiment of a sketch-based image conversion method of the present invention.
[0077] Based on the first embodiment, step S40 in the sketch-based picture conversion method of this embodiment includes:
[0078] Step S41: determining a target conversion model according to the target image type.
[0079] It should be noted that the target conversion model refers to a model obtained after deep learning training of sample image types and sample conversion sketches corresponding to the sample image types. The target conversion model can convert images based on target conversion features, and output conversion images corresponding to multiple sketches to be converted and the image matching degree corresponding to each conversion image. Since different image types correspond to different sample conversion models, after obtaining the target image type, it is necessary to determine the corresponding target conversion model according to the target image type in the sample conversion model.
[0080] It can be understood that in order to obtain an accurate target conversion model, thereby ensuring that the converted target conversion picture highly matches the user needs, further, each sample picture type and the corresponding sample conversion picture are obtained; the sample conversion sketch corresponding to the sample conversion picture is determined according to the sample conversion picture; the initial conversion model is trained according to the sample conversion picture and the corresponding sample conversion sketch to obtain the sample conversion model corresponding to each of the sample picture types; the target conversion model is determined according to the target picture type, the sample picture type and the sample conversion model corresponding to each of the sample picture types.
[0081] In a specific implementation, each sample picture type refers to a plurality of picture types stored in a preset database, for example, an elderly picture type, a building picture type, a dog picture type, a child picture type, etc. It may also be other picture types, which is not limited in this embodiment.
[0082] It should be noted that sample conversion pictures corresponding to each sample picture type are stored in the preset database. After obtaining the sample conversion pictures corresponding to the sample picture types, the sample conversion sketches corresponding to each sample conversion picture are obtained in the preset database based on each sample conversion picture. The sample conversion sketch is a sample sketch drawn by the user for model training. One sample conversion sketch corresponds to multiple sample conversion pictures. Each sample conversion picture is annotated with a sample matching graph evaluated by the user who drew the sample conversion sketch. The sample matching graph is used to reflect the degree of matching between the sample conversion picture and the sample conversion sketch.
[0083] It is understandable that after obtaining the sample conversion sketch and the sample conversion picture, model training is performed based on the sample conversion sketch and the sample conversion picture to obtain the sample conversion model corresponding to each sample picture type. According to the target picture type and the sample picture type are matched, after finding the sample picture type that is the same as the target picture type, the sample conversion model corresponding to the sample picture type is obtained, and the sample conversion model corresponding to the sample picture type is used as the target conversion model.
[0084] In a specific implementation, in order to ensure the accuracy of model training, further, the initial conversion model is trained according to the sample conversion picture and the corresponding sample conversion sketch to obtain a sample conversion model corresponding to each of the sample picture types, including: extracting features from the sample conversion sketch to obtain training line features, training color features and training shape features; inputting the training line features, training color features, training shape features, sample conversion pictures and the sample matching degrees corresponding to the sample conversion pictures into the initial conversion model for model training to obtain sample conversion models corresponding to each of the sample picture types.
[0085] It should be noted that, feature extraction is performed on the sample conversion sketch to obtain sample training features of the sample conversion sketch. The sample training features include training line features, training color features and training shape features. The training line features, training color features, training shape features, sample conversion images and the sample matching degrees corresponding to the sample conversion images are input into the initial conversion model for model training, thereby obtaining sample conversion models corresponding to each sample image type.
[0086] Step S42: input the target conversion feature into the target conversion model to perform image conversion, and obtain an initial conversion image and its corresponding image matching degree.
[0087] It should be noted that after obtaining the target conversion features of the sketch to be converted, the target conversion features are input into the target conversion model for image conversion, so as to obtain multiple initial conversion images corresponding to the sketch to be converted, and the image matching degree between each initial conversion image and the sketch to be converted is output.
[0088] Step S43: rank the image matching degrees to obtain a ranking result.
[0089] It should be noted that after obtaining the image matching degrees, the image matching degrees are ranked to obtain the ranking results of the image matching degrees corresponding to the initial conversion images.
[0090] Step S44: obtaining the target image matching degree that ranks first in the ranking results, and taking the initial conversion image corresponding to the target image matching degree as the target conversion image.
[0091] It should be noted that after obtaining the ranking results, the first-ranked image matching degree is obtained, the first-ranked image matching degree is used as the target image matching degree, and the initial conversion image corresponding to the target image matching degree is used as the target conversion image for image output.
[0092] This embodiment determines the target conversion model according to the target image type; inputs the target conversion features into the target conversion model to perform image conversion, obtains the initial conversion image and its corresponding image matching degree; ranks the image matching degrees to obtain ranking results; obtains the target image matching degree ranked first in the ranking results, and uses the initial conversion image corresponding to the target image matching degree as the target conversion image. In the above manner, image conversion is performed based on the target conversion model and target conversion features corresponding to the target image type, and the final target conversion image is obtained based on the image matching degree, which can improve the accuracy of image conversion and customer satisfaction.
[0093] In addition, refer to Figure 4 The embodiment of the present invention further provides a sketch-based image conversion device, the sketch-based image conversion device comprising:
[0094] The acquisition module 10 is used to acquire the user's sketch to be converted and the sketch conversion instruction.
[0095] The determination module 20 is used to determine the target image type according to the sketch conversion instruction.
[0096] The extraction module 30 is used to extract features of the sketch to be converted to obtain target conversion features of the sketch to be converted.
[0097] The conversion module 40 is used to perform image conversion according to the target image type and the target conversion feature to obtain a target conversion image.
[0098] This embodiment obtains the user's sketch to be converted and sketch conversion instructions; determines the target image type according to the sketch conversion instructions; extracts features of the sketch to be converted to obtain the target conversion features of the sketch to be converted; performs image conversion according to the target image type and the target conversion features to obtain the target conversion image. In the above manner, accurate target conversion features and target image types are determined through the user's sketch to be converted and sketch conversion instructions, and image conversion is performed based on the target image type and target conversion features, thereby ensuring the accuracy and efficiency of image conversion and improving the user experience.
[0099] In one embodiment, the determination module 20 is further used to determine a target tag according to the sketch conversion instruction;
[0100] Searching for the image type corresponding to the target tag in a tag type mapping table;
[0101] The target image type is determined according to the image type corresponding to the target tag.
[0102] In one embodiment, the extraction module 30 is further used to perform line recognition on the sketch to be converted to obtain line features of the sketch to be converted;
[0103] Performing color recognition on the sketch to be converted to obtain color features of the sketch to be converted;
[0104] Performing basic shape recognition on the sketch to be converted to obtain shape features of the sketch to be converted;
[0105] The target conversion features of the sketch to be converted are determined according to the line features, color features and shape features.
[0106] In one embodiment, the conversion module 40 is further used to determine a target conversion model according to the target image type;
[0107] Inputting the target conversion feature into the target conversion model to perform image conversion, and obtaining an initial conversion image and its corresponding image matching degree;
[0108] Ranking the matching degrees of the pictures to obtain a ranking result;
[0109] The target image matching degree ranked first in the ranking results is obtained, and the initial conversion image corresponding to the target image matching degree is used as the target conversion image.
[0110] In one embodiment, the conversion module 40 is further used to obtain each sample image type and a corresponding sample conversion image;
[0111] Determine a sample conversion sketch corresponding to the sample conversion picture according to the sample conversion picture;
[0112] Performing model training on the initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketches to obtain sample conversion models corresponding to each type of the sample pictures;
[0113] A target conversion model is determined according to the target picture type, the sample picture type, and the sample conversion models corresponding to each of the sample picture types.
[0114] In one embodiment, the conversion module 40 is further used to extract features from the sample conversion sketch to obtain training line features, training color features and training shape features;
[0115] The training line features, training color features, training shape features, sample conversion images, and sample matching degrees corresponding to the sample conversion images are input into the initial conversion model for model training to obtain sample conversion models corresponding to each of the sample image types.
[0116] In one embodiment, the conversion module 40 is further used to send the target conversion image to the user and receive the evaluation result fed back by the user;
[0117] When the evaluation result is not a preset evaluation result, the step of returning to the step of performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image.
[0118] Since the present device adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.
[0119] In addition, an embodiment of the present invention further proposes a storage medium, on which a sketch-based image conversion program is stored. When the sketch-based image conversion program is executed by a processor, the steps of the sketch-based image conversion method described above are implemented.
[0120] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.
[0121] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.
[0122] In addition, for technical details not fully described in this embodiment, reference can be made to the sketch-based image conversion method provided in any embodiment of the present invention, and will not be repeated here.
[0123] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0124] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0126] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A sketch-based image conversion method, characterized in that: The sketch-based image conversion method includes: Obtain the user's sketch to be converted and sketch conversion instructions; Determine the target image type according to the sketch conversion instruction; Performing feature extraction on the sketch to be converted to obtain target conversion features of the sketch to be converted; Performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image; Wherein, determining the target image type according to the sketch conversion instruction includes: Determining a target label according to the sketch conversion instruction; Searching for the image type corresponding to the target tag in a tag type mapping table; Determine the target image type according to the image type corresponding to the target label; The step of extracting features of the sketch to be converted to obtain target conversion features of the sketch to be converted includes: Performing line recognition on the sketch to be converted to obtain line features of the sketch to be converted; Performing color recognition on the sketch to be converted to obtain color features of the sketch to be converted; Performing basic shape recognition on the sketch to be converted to obtain shape features of the sketch to be converted; The target conversion features of the sketch to be converted are determined according to the line features, color features and shape features.
2. The sketch-based image conversion method according to claim 1, wherein: The performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image includes: Determining a target conversion model according to the target image type; Inputting the target conversion feature into the target conversion model to perform image conversion, and obtaining an initial conversion image and its corresponding image matching degree; Ranking the matching degrees of the pictures to obtain a ranking result; The target image matching degree ranked first in the ranking results is obtained, and the initial conversion image corresponding to the target image matching degree is used as the target conversion image.
3. The sketch-based image conversion method according to claim 2, wherein: The determining the target conversion model according to the target picture type includes: Get each sample image type and the corresponding sample conversion image; Determine a sample conversion sketch corresponding to the sample conversion picture according to the sample conversion picture; Performing model training on the initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketches to obtain sample conversion models corresponding to each type of the sample pictures; A target conversion model is determined according to the target picture type, the sample picture type, and the sample conversion models corresponding to each of the sample picture types.
4. The sketch-based image conversion method according to claim 3, wherein: The initial conversion model is trained according to the sample conversion pictures and the corresponding sample conversion sketches to obtain the sample conversion models corresponding to the sample picture types, including: Extracting features from the sample conversion sketch to obtain training line features, training color features, and training shape features; The training line features, training color features, training shape features, sample conversion images, and sample matching degrees corresponding to the sample conversion images are input into the initial conversion model for model training to obtain sample conversion models corresponding to each of the sample image types.
5. The sketch-based image conversion method according to any one of claims 1 to 4, characterized in that: After performing image conversion according to the target image type and the target conversion feature to obtain the target conversion image, the method further includes: Sending the target conversion image to the user, and receiving the evaluation result fed back by the user; When the evaluation result is not a preset evaluation result, the step of returning to the step of performing image conversion according to the target image type and the target conversion feature to obtain a target conversion image.
6. A sketch-based image conversion device, characterized in that: The sketch-based picture conversion device comprises: An acquisition module, used to acquire the user's sketch to be converted and the sketch conversion instruction; A determination module, used to determine the target image type according to the sketch conversion instruction; An extraction module, used for extracting features of the sketch to be converted to obtain target conversion features of the sketch to be converted; A conversion module, used to perform image conversion according to the target image type and the target conversion feature to obtain a target conversion image; The determination module is further used to determine the target tag according to the sketch conversion instruction; and search the image type corresponding to the target tag in the tag type mapping table; Determine the target image type according to the image type corresponding to the target label; The extraction module is further used to perform line recognition on the sketch to be converted to obtain line features of the sketch to be converted; Performing color recognition on the sketch to be converted to obtain color features of the sketch to be converted; Performing basic shape recognition on the sketch to be converted to obtain shape features of the sketch to be converted; The target conversion features of the sketch to be converted are determined according to the line features, color features and shape features.
7. A sketch-based image conversion device, characterized in that: The device comprises: a memory, a processor, and a sketch-based image conversion program stored in the memory and executable on the processor, wherein the sketch-based image conversion program is configured to implement the sketch-based image conversion method as described in any one of claims 1 to 5.
8. A storage medium, characterized in that: The storage medium stores a sketch-based image conversion program, and when the sketch-based image conversion program is executed by the processor, the sketch-based image conversion method according to any one of claims 1 to 5 is implemented.
