Image processing method and related device

Abstract

The present disclosure provides an image processing method and a related device. The method comprises: acquiring a text inputted by a user; on the basis of the text, determining at least one keyword; on the basis of the at least one keyword, determining at least one piece of gradient color information; and on the basis of the at least one piece of gradient color information, determining a placeholder image having a gradient color effect.

Description

Image processing method and related equipment

[0001] This application claims priority to the Chinese invention patent application entitled “Image Processing Methods and Related Equipment” and application number 202311268339.5 filed on September 27, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of computer technology, and in particular to an image processing method and related equipment. Background Art

[0003] In the field of image generation, images can be automatically generated from user-provided text. However, because image generation models are typically large and the image generation process is lengthy, placeholder images are often pre-displayed in the corresponding locations. Once the image is generated, the generated image replaces the placeholder image.

[0004] However, the inventors of the present disclosure have discovered that the placeholder images generated in the related art have a single style and a poor user experience.

[0005] Summary of the Invention

[0006] The present disclosure proposes an image processing method and related devices to solve or partially solve the above problems.

[0007] In a first aspect of the present disclosure, an image processing method is provided, comprising: obtaining text input by a user; determining at least one keyword based on the text; determining at least one gradient color information based on the at least one keyword; and determining a placeholder image with a gradient color effect based on the at least one gradient color information.

[0008] In a second aspect of the present disclosure, an image processing device is provided, comprising: an acquisition module configured to acquire text input by a user; a first determination module configured to determine at least one keyword based on the text; a second determination module configured to determine at least one gradient color information based on the at least one keyword; and a third determination module configured to determine a placeholder image having a gradient color effect based on the at least one gradient color information.

[0009] In a third aspect of the present disclosure, a computer device is provided, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to the first aspect.

[0010] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors are caused to execute the method described in the first aspect.

[0011] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] FIG1A shows a schematic diagram of an exemplary system provided by an embodiment of the present disclosure.

[0014] FIG1B shows a schematic diagram of an exemplary page.

[0015] FIG2A shows a schematic flow chart of an exemplary method provided by an embodiment of the present disclosure.

[0016] FIG2B shows a flowchart of another exemplary method provided by an embodiment of the present disclosure.

[0017] FIG2C shows a flowchart of an exemplary method for generating a placeholder image according to an embodiment of the present disclosure.

[0018] FIG3A shows a schematic diagram of an exemplary page according to an embodiment of the present disclosure.

[0019] FIG3B shows a schematic diagram of another exemplary page according to an embodiment of the present disclosure.

[0020] FIG. 3C shows a schematic diagram of an exemplary container according to an embodiment of the present disclosure.

[0021] FIG4 shows a schematic diagram of another exemplary page according to an embodiment of the present disclosure.

[0022] FIG5 shows a schematic diagram of the hardware structure of an exemplary computer device provided by an embodiment of the present disclosure.

[0023] FIG6 shows a schematic diagram of an exemplary device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0025] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0026] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0027] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the disclosed technical solution based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] FIG1A shows a schematic diagram of an exemplary system 100 provided by an embodiment of the present disclosure.

[0031] As shown in FIG1A , system 100 may include a terminal device 102, a terminal device 104, a server 106, and a database server 108. A medium (e.g., a network) providing a communication link may be provided between the terminal device 102, the terminal device 104, the server 106, and the database server 108. The network may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0032] Various applications (APPs) can be installed on the terminal device 104, such as image processing applications, video conferencing applications, reading applications, video applications, social applications, payment applications, web browsers and instant messaging tools, etc. These applications can all be used to display the delivered information.

[0033] The terminal devices 102 and 104 herein can be either hardware or software. When the terminal devices 102 and 104 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, e-book readers, MP3 players, laptop computers (Laptops), and desktop computers (PCs), etc. When the terminal devices 102 and 104 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitations are given here.

[0034] Server 106 may be a server that provides various services, such as a backend server that supports various applications displayed on terminal devices 102 and 104. Database server 108 may also be a database server that provides various services. It is understood that if server 106 can implement the relevant functions of database server 108, database server 108 may not be provided in system 100.

[0035] The server 106 and database server 108 herein can also be hardware or software. When they are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When they are software, they can be implemented as multiple software programs or software modules (for example, to provide distributed services), or as a single software program or software module. No specific limitations are imposed herein.

[0036] It should be noted that the image processing method provided in the embodiments of the present disclosure can be executed by terminal device 102 and / or terminal device 104. It should be understood that the number of terminal devices, users, servers, and database servers in FIG1A is merely illustrative. Any number of terminal devices, users, servers, and database servers may be provided as required.

[0037] In an exemplary scenario, server 106 can provide an image generation service. For example, user 112 can open an image processing application on terminal device 104, enter a text on a page, and then send the text to server 106. Server 106 can generate an image based on the text and return the image to terminal device 104 for display.

[0038] In some cases, since the image generation process takes a long time, a placeholder image can usually be displayed first to alleviate the user's anxiety while waiting for the image to be generated.

[0039] FIG. 1B shows a schematic diagram of an exemplary page 120 .

[0040] As shown in Figure 1B, a placeholder image 122 is displayed on the page 120. Optionally, the placeholder image 122 may be generated by an image processing application in the terminal device 104, or may be pre-generated by the server 106 and then sent to the terminal device 104 for display.

[0041] As shown in FIG. 1B , in the related art, the placeholder image 122 is usually a default image with a single and fixed style, which cannot bring a good visual experience to the user.

[0042] In view of this, an embodiment of the present disclosure provides an image processing method, which can determine gradient color information based on the text input by the user, and then determine a placeholder image with a gradient color effect based on the gradient color information. In this way, the placeholder image can be more in line with the artistic conception of the input text, and the user experience is better.

[0043] FIG2A illustrates a flow chart of an exemplary method 200 provided in an embodiment of the present disclosure. Method 200 can be used to process an image. Optionally, method 200 can be implemented by terminal devices 102 and 104 of FIG1A , or by server 106 of FIG1A . The following description uses the implementation of method 200 by terminal device 104 as an example.

[0044] As shown in FIG. 2A , the method 200 may further include the following steps.

[0045] In step 202, the text input by the user is obtained.

[0046] FIG3A shows a schematic diagram of an exemplary page 300 according to an embodiment of the present disclosure.

[0047] As shown in FIG. 3A , as an optional embodiment, after the user 112 opens an image processing application in the terminal device 104 , a page 300 may be entered, and an input field 302 for inputting text may be displayed on the page 300 .

[0048] The user 112 may input a text in the input field 302 to serve as text for subsequently determining the placeholder image and the first image.

[0049] In some embodiments, after user 112 enters text in input field 302, terminal device 104 can retrieve the corresponding text from input field 302. As an alternative embodiment, when user 112 clicks "Generate Now" button 304, terminal device 104 begins to retrieve text from input field 302. It will be appreciated that there are various ways to retrieve text, and the description here is merely illustrative and should not be used to limit the scope of protection of this disclosure.

[0050] After acquiring the text input by the user 112 , in step 204 , the terminal device 104 may determine at least one keyword based on the text.

[0051] Optionally, the keywords may be directly extracted from the text, or may be generated by extracting features from the text, and the specific method is not limited.

[0052] In some embodiments, in order to enable the keyword to better reflect the meaning of the text itself, the at least one keyword may be obtained by performing word segmentation processing on the text.

[0053] The word segmentation method may be different depending on the text input by the user 112 .

[0054] For example, if the text input by user 112 is "withered vines, old trees, dusky crows, small bridges, flowing water, people's homes," word segmentation can be directly implemented based on the symbols that user 112 has inserted into the text, without requiring a complex processing procedure. This eliminates the complex steps of word segmentation and improves processing efficiency. As an optional embodiment, when terminal device 104 implements word segmentation in this manner, a prompt such as "Please use symbols to separate keywords" or similar can be displayed on one side of input field 302 or within input field 302, thereby prompting user 112 to use symbols to separate keywords for text input.

[0055] For another example, the text input by user 112 is "When will the bright moon appear? I raise my cup to ask the blue sky", then the text can be split into multiple keywords by using text word segmentation or text semantic segmentation.

[0056] FIG2B shows a flowchart of another exemplary method provided by an embodiment of the present disclosure.

[0057] As shown in FIG2B , after the text input by the user 112 is segmented using text word sense segmentation or text semantic segmentation, multiple keywords can be obtained.

[0058] In some embodiments, the text can be input into a pre-trained text processing model to output the at least one keyword. Optionally, the text processing model can be implemented using various models in the field of natural language processing (NLP) and can be trained using various corpora in the field of natural language processing. The specific training method is not limited.

[0059] Because natural language processing models are typically large, the computing and storage resources of terminal device 104 may not be able to process and store such text processing models. Therefore, as an alternative embodiment, terminal device 104 can send the text to server 106. The text processing model running on server 106 will perform word segmentation and obtain the at least one keyword, which is then returned to terminal device 104. It will be appreciated that in some cases, when the text processing model itself is sufficiently small, it can also run on terminal device 104.

[0060] After obtaining the at least one keyword, in step 206 , the terminal device 104 may determine at least one piece of gradient color information based on the at least one keyword. The at least one piece of gradient color information may be used to determine a placeholder image with a gradient color effect.

[0061] As an optional embodiment, the at least one gradient color information may be associated with the keyword. When a keyword is obtained, the gradient color information associated with the keyword can be searched accordingly.

[0062] Understandably, this processing approach requires a massive database to store various keywords and their associated gradient color information. Therefore, to simplify, expedite, and efficiently process, in some embodiments, a pre-trained parameter generation model can be utilized to determine at least one gradient color information based on the at least one keyword. Specifically, the at least one keyword can be input into the pre-trained parameter generation model, which then outputs the at least one gradient color information.

[0063] As shown in FIG2B , as an optional embodiment, the parameter generation model can be implemented using a neural network model. Optionally, the neural network model can be a relatively small neural network model. For example, the neural network model includes three layers, an input layer, a hidden layer, and an output layer. In this way, the pre-trained parameter generation model can be run in the terminal device 104. After the terminal device 104 obtains the at least one keyword, the parameter generation model can be called to obtain the at least one gradient color information without going through the server 106 to obtain the at least one gradient color information. At the same time, since the parameter generation model is small in size, the gradient color information can be generated quickly, and then the placeholder image can be quickly determined based on the gradient color information, so that the placeholder operation can be completed faster, thereby improving the user experience.

[0064] In some embodiments, user 110 (eg, a developer of an image processing application) may utilize system 100 to train the parameter generation model.

[0065] FIG3B shows a schematic diagram of an exemplary page 320 according to an embodiment of the present disclosure.

[0066] As shown in FIG3B , a page 320 may be displayed on the terminal device 102, and an input field 322 for entering text may be displayed on the page 320. The user 110 may enter any text in the input field 322 as a training sample, and then click the "Generate Now" button 324. The terminal device 102 may upload the text to the server 106. After receiving the text uploaded by the terminal device 102, the server 106 may process the text into at least one keyword (the processing method may refer to the previous embodiment), and then input the obtained keyword into the initialized neural network model to output corresponding gradient color information, and then return the gradient color information to the terminal device 102.

[0067] After obtaining the gradient color information returned by the server 106 , the terminal device 102 may generate an image 326 according to the gradient color information, as shown in FIG3B .

[0068] According to the gradient effect presented by image 326, user 110 can adjust the parameters. As shown in Figure 3B, user 110 can click parameter setting button 328 to open the parameter adjustment window (not shown in the figure), in which model parameter options and gradient parameter options can be displayed. User 110 can adjust the parameters of the neural network model by adjusting the model parameters corresponding to the model parameter options, and can adjust the gradient parameters by adjusting the gradient parameters corresponding to the gradient parameter options. After adjusting the parameters, user 110 can click the generate immediately button 324 again to upload the adjusted parameters to server 106, and then server 106 continues to train the neural network model based on the adjusted parameters until the gradient effect presented by image 326 meets expectations. Then, user 110 can re-enter new text and continue to train the neural network model until the model meets the set requirements.

[0069] In some embodiments, user 110 can click the random word button 330 to generate a random word, and then use the neural network model to generate corresponding gradient color parameters, and then generate a corresponding image based on the gradient color parameters. By determining whether the image meets expectations, it can be determined whether the neural network model is trained. As an optional embodiment, the random word button 330 can also be used to randomly generate some text as training samples, thereby improving the robustness and generalization ability of the model.

[0070] After the neural network model is trained, the parameter generation model can be obtained. In some embodiments, the server 106 can process the parameter generation model into a software development kit (SDK) for integration into an image processing application, so that the terminal device 104 running the application can directly call the model to generate gradient color information.

[0071] In some embodiments, the at least one gradient color information includes: at least one of a gradient starting point, a gradient direction, a length of gradient color processing along the gradient direction, a starting point color, an ending point color, and a gradient midpoint.

[0072] Among them, the gradient starting point can be the gradient color parameter <side-or-corner>, which describes the starting point of the gradient line. The gradient direction can be the gradient color parameter <angle>, which specifies the direction (or angle) of the gradient using an angle value, which increases clockwise. The length of the gradient along the gradient direction can be the gradient parameter <length>This parameter is used to indicate the length of the gradient process or the length of the gradient line, which expresses the distance size of the gradient. The starting color can be a gradient color parameter <color>, which represents a color in the standard RGB color space (SRGB color space). The end color can be a gradient color parameter <linear-color-stop>, which is a parameter <color>value, followed by an optional end position (either a percentage or a position along the gradient axis or line) <length>). The gradient midpoint can be the gradient color parameter <color-hint>, this parameter represents the color transition point and length. The color transition point is an interpolation hint that defines how the gradient between adjacent colors is performed, and the length defines at which point between two colors the gradient should stop. The color should reach the midpoint of the color transition; if this parameter is omitted, the midpoint of the color transition is the midpoint between the two color stops.

[0073] In this way, through the above 6 gradient color parameters, an image with a gradient color effect can be generated.

[0074] In step 208 , a placeholder image with a gradient color effect is determined according to the at least one gradient color information.

[0075] In some embodiments, the specific value of the gradient color information can be pre-associated with the corresponding placeholder image. In this way, after obtaining the gradient color information, the associated placeholder image can be found. In other embodiments, a placeholder image with a gradient color effect can be generated based on the gradient color information, thereby better reflecting the gradient color effect corresponding to the text.

[0076] In some embodiments, the at least one gradient color information used to generate the placeholder image includes a gradient starting point, a starting color, and an ending color. The gradient starting point can be a gradient color parameter. <side-or-corner>, this parameter describes the starting point position of the gradient line. The starting color and the ending color can both be gradient color parameters <color>, this parameter represents a color in the standard RGB color space (SRGB color space).

[0077] After knowing the starting point, starting color, and ending color of the gradient, you can generate a placeholder image with a gradient color effect.

[0078] In some embodiments, as shown in FIG2C , step 208 of determining a placeholder image with a gradient color effect based on the at least one gradient color information may further include the following steps:

[0079] In step 2082, the gradient direction is determined according to the gradient starting point.

[0080] Optionally, for linear gradients, the gradient starting point can be the gradient color parameter <side-or-corner>For example, "toright" means that the starting point is at the far left. Accordingly, the gradient direction can be determined to be from left to right.

[0081] It can be understood that "toleft", "totop", "tobottom", "totopright", "totopleft", "tobottomright", and "tobottomleft" respectively refer to starting points at the far right, bottom, top, lower left, lower right, upper left, and upper right. Accordingly, their gradient directions can be determined to be from right to left, from bottom to top, from top to bottom, from lower left to upper right, from lower right to upper left, from upper left to lower right, and from upper right to lower left. In addition, when the gradient starting point is not specified, the default gradient direction can be from top to bottom.

[0082] It should be noted that the gradient direction can also use the gradient color parameter <angle>In some embodiments, when a gradient starting point is not specified (for example, the at least one gradient color information does not include a gradient starting point), the gradient direction can be determined by specifying an angle, with a gradient direction from top to bottom being 0 degrees. As the angle increases, the gradient direction rotates clockwise. For example, when the specified angle is 45 degrees, the gradient direction is from the upper right to the lower left.

[0083] In step 2084, a color difference is determined based on the starting color and the ending color.

[0084] Optionally, both the starting color and the ending color can be gradient color parameters. <color>This parameter represents a color in the standard RGB color space (SRGB color space). For example, if the starting color is red, the corresponding RGB value is 255,0,0. If the ending color is blue, the corresponding RGB value is 0,0,255. In this way, the RGB difference (i.e., the color difference) can be calculated based on the RGB values ​​of red and blue to be -255,0,255.

[0085] In step 2086 , the length of the gradient color processing along the gradient direction is determined according to the size of the container corresponding to the placeholder image and the gradient direction.

[0086] FIG3C shows a schematic diagram of an exemplary container 340 according to an embodiment of the present disclosure.

[0087] As shown in FIG. 3C , the container 340 corresponding to the placeholder image may be an area having the same size as the placeholder image, and the area is used to be filled with the corresponding gradient color effect.

[0088] For a gradient direction determined by a gradient starting point, the length of the gradient color process along the gradient direction can be the side length or diagonal length of the container 340. For example, when the gradient direction is from top to bottom or from bottom to top, the length of the gradient color process along the gradient direction is the longitudinal length of the container 340. When the gradient direction is from left to right or from right to left, the length of the gradient color process along the gradient direction is the transverse length of the container 340. When the gradient direction is from bottom left to top right, from bottom right to top left, from top left to bottom right, or from top right to bottom left, the length of the gradient color process along the gradient direction is the diagonal length of the container 340.

[0089] As previously mentioned, in some embodiments, the gradient direction can also be determined by angle. As shown in FIG3C , assume that the direction indicated by the dashed line is the gradient direction determined by angle, and that the dashed line passes through the center of container 340. Next, draw two straight lines perpendicular to the dashed line, with the space between the two parallel lines just enough to completely cover container 340. The length L that the two straight lines intersect on the dashed line is the length of the gradient color process along the gradient direction.

[0090] In step 2088, the color corresponding to each unit distance along the gradient direction is determined based on the color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction.

[0091] The gradient unit ratio indicates how many times the color changes from the gradient's starting point to its end point. For example, a unit ratio of 1% means 100 color changes from the gradient's starting point (0%) to its end point (100%). To calculate this, divide the color difference into 100 equal parts to get the unit color difference. Also, divide the gradient length along the gradient's direction into 100 equal parts to get the unit distance.

[0092] For example, if the unit ratio is 1%, the starting color is red, and the ending color is blue, the color difference is -255, 0, 255. Accordingly, the unit color difference is -2.55, 0, 2.55. Therefore, the color at the 1% position is 252, 0, 3, the color at the 2% position is 250, 0, 5, the color at the 3% position is 247, 0, 8, and so on. Accordingly, the color from the starting point (0%) to the 1% position is red, the color from the 1% to the 2% position is 252, 0, 3, the color from the 2% to the 3% position is 250, 0, 5, the color from the 3% to the 4% position is 247, 0, 8, and so on, until the color at the end of the gradient (100%) is blue.

[0093] It is understandable that the number of equally divided colors and lengths may be different depending on the unit ratio. The specific unit ratio may be set or defaulted, and no specific limitation is imposed here.

[0094] In step 2090 , gradient color processing is performed on the container according to the gradient direction, the starting color, the ending color, and the color corresponding to each unit distance along the gradient direction to obtain the placeholder image with the gradient color effect.

[0095] In this step, the container 340 may be filled along the gradient direction according to the colors corresponding to each unit distance along the gradient direction and the starting color and the ending color obtained previously, thereby obtaining a placeholder image with a gradient color effect.

[0096] It can be seen that when the gradient starting point, starting color and ending color are different, the artistic conception embodied by the generated placeholder image with a gradient color effect may be different. By determining at least one gradient color information based on the at least one keyword, when the gradient color information is different, placeholder images with a gradient color effect that express different artistic conceptions can be generated, thereby improving the user experience.

[0097] In some embodiments, the at least one gradient color information may further include an intermediate color; optionally, the intermediate color may also be a gradient color parameter <color>For example, the middle color is green, and its RGB value is 0,255,0.

[0098] The step 2084 of determining the color difference based on the starting color and the end color may further include: determining a first color difference based on the starting color and the intermediate color; and determining a second color difference based on the intermediate color and the end color.

[0099] Specifically, for example, the starting color is red, the middle color is green, and the end color is blue, then the first color difference is -255, 255, 0, and the second color difference is 0, -255, 255.

[0100] The step 2088 of determining the color corresponding to each unit distance along the gradient direction based on the color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction may further include: determining the color corresponding to each unit distance along the gradient direction based on the first color difference, the second color difference, the position corresponding to the intermediate color, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction.

[0101] Specifically, assuming the gradient unit ratio is 1%, the starting color needs to change 50 times to the middle color, and the middle color needs to change 50 times to the end color. Therefore, the first color difference can be divided into 50 equal parts, and the corresponding first unit color difference is -5.1, 5.1, 0; similarly, the second color difference can be divided into 50 equal parts, and the corresponding second unit color difference is 0, -5.1, 5.1.

[0102] So, the color at the 1% position can be 250, 5, 0, the color at the 2% position can be 245, 10, 0, the color at the 3% position can be 240, 15, 0, and so on, until the color at the 50% position is green. Furthermore, the color at the 51% position can be 0, 250, 5, the color at the 52% position can be 0, 245, 10, the color at the 3% position can be 0, 240, 15, and so on, until the color at the 100% position is blue. Accordingly, the color from the gradient start point (0%) to the 1% position is red, the color from the 1% position to the 2% position is 250, 5, 0, the color from the 2% position to the 3% position is 245, 10, 0, the color from the 3% position to the 4% position is 240, 15, 0, and so on, until the color at the 50% position is green. Next, the color of the area from 51% to 52% is 0,250,5, the color of the area from 52% to 53% is 0,245,10, the color of the area from 3% to 4% is 0,240,15, and so on, until the color of the 100% position is blue.

[0103] In this way, with the addition of the gradient color information of the middle color, a placeholder image is generated that achieves a gradient color effect between the three colors, which can further enrich the artistic conception that the placeholder image can embody, thereby improving the user experience.

[0104] In some embodiments, the at least one gradient color information may further include a gradient midpoint, which may be a gradient color parameter. <color-hint>, this parameter represents the color transition point, that is, which position is used as the transition point in the gradient process. In the case where there is no gradient color information such as the gradient midpoint, the turning point of the gradient process (the position of the middle color) defaults to the 50% position. When the gradient color information of the gradient midpoint is not the specified 50% position, for example, the 40% position is specified, the first color difference can be divided into 40 equal parts of the first unit color difference, and correspondingly, the second color difference can be divided into 60 equal parts of the second unit color difference, and the color corresponding to each unit distance along the gradient direction can be determined according to similar steps as mentioned above, which will not be repeated here.

[0105] In this embodiment, the gradient midpoint is one of the gradient color information, which is generated based on the at least one keyword. The position of the gradient midpoint can be different depending on the artistic conception of the text input by the user (for example, not at the midpoint of the gradient length but at any position on the length), so that a placeholder image with a gradient color effect that is more in line with the artistic conception of the text can be generated, thereby improving the user experience.

[0106] In some embodiments, there can be more than one intermediate color, and the number of gradient midpoints can be increased accordingly, thereby achieving a placeholder image with a gradient color effect that can reflect a richer textual image. The specific implementation method can be referred to the previous embodiment and will not be repeated here.

[0107] It can be understood that in addition to the gradient starting point, starting color, ending color, intermediate color, and gradient midpoint, the gradient direction, gradient length, etc. in the gradient color information can also change the textual artistic conception that can be expressed by the gradient color effect of the placeholder image. By outputting this information according to keywords and then generating a placeholder image based on this information, the corresponding gradient color effect can be used to well reflect the artistic conception that can be expressed by the text entered by the user, thereby improving the user experience.

[0108] It should be noted that none of the gradient information—gradient starting point, gradient direction, gradient length along the gradient direction, starting color, ending color, intermediate color, and gradient midpoint—is essential. When the generated gradient information includes only one or more of these, the remaining gradient information can be obtained using default values ​​or random generation. It's understood that the more of this gradient information is generated based on text, the better the placeholder image will convey the text's artistic conception.

[0109] Furthermore, it should be noted that the aforementioned embodiment describes the linear gradient color processing of a container as an example. It is understood that the gradient color processing can also be radial gradient color processing, since radial gradient color processing involves drawing concentric circles from the center of the container outward. Therefore, the required gradient color information may not be as much as in the aforementioned embodiment. For example, there may be no gradient starting point or gradient direction. The at least one gradient color information may only include the starting color and the ending color to achieve the gradient color processing of the container, thereby obtaining a placeholder image with a gradient color effect. Of course, it is understood that for radial gradient color processing, placeholder images with different gradient color effects can also be achieved based on the intermediate color and the gradient midpoint, which will not be described in detail here.

[0110] In some embodiments, the method 200 further includes: displaying the placeholder image (eg, placeholder images 306A-306D) at a target location (eg, preview area 308 of page 300) of a target page (eg, page 300 of FIG. 3A).

[0111] As shown in FIG3A , the number of placeholder images can be more than one, and the number of placeholder images can vary depending on the amount of text content. For example, a threshold for the number of keywords can be pre-set (e.g., 2). Thus, based on the number of keywords determined from the text, a group of adjacent keywords is formed, and a set of gradient color information is generated to create a placeholder image. When the number of keywords exceeds the threshold, a number of placeholder images corresponding to the multiple of the threshold can be generated based on the multiple of the number of keywords.

[0112] In some embodiments, the text input by the user 112 in the page 300 may be used to generate a first image associated with the text; the method 200 further includes: generating the first image according to the text.

[0113] As an optional embodiment, generating the first image according to the text may further include: generating the first image according to the at least one keyword. Similarly, the number of first images generated may be more than one, and the number of first images may be different depending on the amount of text content. For example, a threshold value for the number of keywords may be pre-set (e.g., 2), so that according to the number of keywords determined from the text, a number of adjacent keywords are grouped together to generate a first image. When the number of keywords exceeds the threshold, according to the relationship between the number of keywords and the multiple of the threshold, the number of first images corresponding to the multiple may be generated. It can be understood that at this time, the first image and the placeholder image are one-to-one corresponding, and the gradient color effect presented by the placeholder image also corresponds to the first image, which has a better visual effect.

[0114] In some embodiments, generating the first image based on the text, or generating the first image based on the at least one keyword, may further include: inputting the text and / or the at least one keyword into a pre-trained image generation model and outputting the first image.

[0115] Optionally, the image generation model can be a convolutional neural network model based on deep learning. The model is large and can have a very deep model structure, so that it can learn more information from the text and generate a more delicate first image that is more consistent with the meaning of the text.

[0116] As mentioned above, since the image generation model is large (the deep learning model is usually several times, dozens of times, or even more times the size of a simple neural network model), the process of generating the first image takes a long time. Therefore, it can be understood that the step of generating the first image and the aforementioned step of generating the placeholder image can be performed simultaneously, thereby improving processing efficiency.

[0117] In some embodiments, the image generation model can be running in the server 106. Therefore, the terminal device 104 can send the text entered by the user 112 in the page 300 to the server 106, and the image generation model running in the server 106 generates the first image and returns it to the terminal device 104.

[0118] In some embodiments, in response to the first image being generated, the placeholder image in the target page is replaced with the first image.

[0119] FIG4 shows a schematic diagram of an exemplary page 400 according to an embodiment of the present disclosure.

[0120] As shown in FIG. 4 , when the first image has been generated, the terminal device 104 may display the generated first image (eg, images 402A to 402D) on the page 400 to replace the placeholder image.

[0121] In some embodiments, as shown in FIG4 , page 400 may further include a re-edit button 404. After triggering button 404, page 400 may display an input field (not shown), and user 112 may then re-edit the text in the input field and then click re-generate button 406 to regenerate a new first image. It will be appreciated that in some embodiments, a new placeholder image may be generated using the new text to replace the previously generated first image, thereby placing user 112 in a state of waiting for a new image, making the entire operation process more consistent.

[0122] As can be seen from the above examples, the image processing methods provided by the embodiments of the present disclosure can intelligently and automatically generate semantically consistent placeholder images, providing users with a good visual experience. In some embodiments, based on actual generated placeholder images, the relevant parameters for generating placeholder images can be repeatedly trained to make the gradient-colored placeholder images increasingly consistent with the user's semantics.

[0123] In some application scenarios, when the user enters text, semantically correct gradient color parameters can be trained and generated in real time. When the user clicks the "Generate Now" button, the generated gradient color image can be displayed on the screen in real time, reducing the user's anxiety while waiting for the generation process.

[0124] It should be noted that the method of the embodiments of the present disclosure can be performed by a single device, such as a computer or server. The method of the embodiments of the present disclosure can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0125] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] The present disclosure also provides a computer device for implementing the aforementioned method 200. Figure 5 illustrates a hardware schematic diagram of an exemplary computer device 500 provided in the present disclosure. Computer device 500 can be used to implement server 106 in Figure 1A , or terminal devices 102 and 104 in Figure 1A . In some scenarios, computer device 500 can also be used to implement database server 108 in Figure 1A .

[0127] 5 , a computer device 500 may include a processor 502, a memory 504, a network module 506, a peripheral interface 508, and a bus 510. The processor 502, the memory 504, the network module 506, and the peripheral interface 508 are connected to each other in communication within the computer device 500 via the bus 510.

[0128] The processor 502 may be a central processing unit (CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device (PLD), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or one or more integrated circuits. The processor 502 may be used to perform functions related to the technology described in this disclosure. In some embodiments, the processor 502 may also include multiple processors integrated into a single logical component. For example, as shown in FIG5 , the processor 502 may include multiple processors 502 a, 502 b, and 502 c.

[0129] The memory 504 can be configured to store data (e.g., instructions, computer codes, etc.). As shown in Figure 5, the data stored in the memory 504 can include program instructions (e.g., program instructions for implementing the method 200 of the embodiment of the present disclosure) and data to be processed (e.g., the memory can store configuration files of other modules, etc.). The processor 502 can also access the program instructions and data stored in the memory 504, and execute the program instructions to operate on the data to be processed. The memory 504 can include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 504 can include a random access memory (RAM), a read-only memory (ROM), an optical disc, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0130] The network interface 506 can be configured to provide the computer device 500 with communication with other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC)), a cellular network, the Internet, or a combination thereof. It will be understood that the type of network is not limited to the specific examples above.

[0131] The peripheral interface 508 can be configured to connect the computer device 500 to one or more peripheral devices to enable information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touchpad, a touch screen, a microphone, and various sensors, as well as output devices such as a display, a speaker, a vibrator, and an indicator light.

[0132] The bus 510 may be configured to transmit information between various components of the computer device 500 (e.g., the processor 502, the memory 504, the network interface 506, and the peripheral interface 508), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0133] It should be noted that although the architecture of the computer device 500 shown above only shows the processor 502, memory 504, network interface 506, peripheral interface 508, and bus 510, in a specific implementation, the architecture of the computer device 500 may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the architecture of the computer device 500 may also include only the components necessary to implement the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0134] The present disclosure also provides an image processing apparatus. FIG6 shows a schematic diagram of an exemplary apparatus 600 provided by the present disclosure. As shown in FIG6 , the apparatus 600 can be used to implement the method 200 and can further include the following modules.

[0135] The acquisition module 602 is configured to obtain text input by the user; the first determination module 604 is configured to determine at least one keyword based on the text; the second determination module 606 is configured to determine at least one gradient color information based on the at least one keyword; and the third determination module 608 is configured to determine a placeholder image with a gradient color effect based on the at least one gradient color information.

[0136] In some embodiments, the apparatus further comprises a display module configured to display the placeholder image at a target location on a target page. In some embodiments, the text is used to generate a first image associated with the text; the apparatus further comprises a generation module configured to generate the first image based on the text; and the display module is configured to replace the placeholder image on the target page with the first image in response to the first image being generated.

[0137] In some embodiments, the first determination module 604 is configured to perform word segmentation processing on the text to obtain the at least one keyword.

[0138] In some embodiments, the first determination module 604 is configured to: input the text into a pre-trained text processing model, and output the at least one keyword.

[0139] In some embodiments, the second determination module 606 is configured to: input the at least one keyword into a pre-trained parameter generation model, and output the at least one gradient color information.

[0140] In some embodiments, the at least one gradient color information includes at least one of a gradient starting point, a gradient direction, a length of gradient color processing along the gradient direction, a starting point color, an ending point color, and a gradient midpoint.

[0141] In some embodiments, the at least one gradient color information includes a gradient starting point, a starting color, and an ending color; the third determination module 608 is configured to: determine the gradient direction based on the gradient starting point; determine the color difference based on the starting color and the ending color; determine the length of the gradient color processing along the gradient direction based on the size of the container corresponding to the placeholder image and the gradient direction; determine the color corresponding to each unit distance along the gradient direction based on the color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction; perform gradient color processing on the container based on the gradient direction, the starting color, the ending color, and the color corresponding to each unit distance along the gradient direction to obtain the placeholder image with a gradient color effect.

[0142] In some embodiments, the at least one gradient color information also includes an intermediate color; the third determination module 608 is configured to: determine a first color difference based on the starting color and the intermediate color; determine a second color difference based on the intermediate color and the end color; determine the color corresponding to each unit distance along the gradient direction based on the first color difference, the second color difference, the unit ratio of the gradient and the length of the gradient color processing along the gradient direction.

[0143] In some embodiments, the generating module is configured to generate the first image according to the at least one keyword.

[0144] In some embodiments, the generation module is configured to: input the text and / or the at least one keyword into a pre-trained image generation model, and output the first image.

[0145] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0146] The apparatus of the above embodiment is used to implement the corresponding method 200 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0147] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute method 400 as described in any of the above embodiments.

[0148] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0149] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method 200 described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0150] Based on the same inventive concept, corresponding to any of the above-described embodiments of method 200, the present disclosure further provides a computer program product comprising a computer program. In some embodiments, the computer program is executable by one or more processors to cause the processors to perform method 200. For each step in each embodiment of method 200, the processor that executes the corresponding step may be a member of the corresponding execution entity.

[0151] The computer program product of the above embodiment is used to enable a processor to execute the method 200 described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0152] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0153] In addition, to simplify the description and discussion, and so as not to obscure the embodiments of the present disclosure, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, devices may be shown in the form of block diagrams to avoid obscuring the embodiments of the present disclosure, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the purview of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0154] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0155] The embodiments of the present disclosure are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure. < / color> < / color> < / angle> < / color> < / length> < / color> < / color> < / length> < / angle>

Claims

1. An image processing method, comprising: Get the text entered by the user; Determining at least one keyword based on the text; Determining at least one gradient color information according to the at least one keyword; as well as A placeholder image with a gradient color effect is determined according to the at least one gradient color information.

2. The method of claim 1, wherein: The method further comprises: The placeholder image is displayed at a target location on the target page.

3. The method of claim 2, wherein: The text is used to generate a first image associated with the text; the method further comprising: generating the first image according to the text; and In response to the first image being generated, the placeholder image in the target page is replaced with the first image.

4. The method of claim 1, wherein: According to the text, at least one keyword is determined, including: The text is segmented using a pre-trained text processing model to obtain the at least one keyword.

5. The method of claim 1, wherein: Determining at least one gradient color information according to the at least one keyword includes: The at least one keyword is input into a pre-trained parameter generation model, and the at least one gradient color information is output.

6. The method according to claim 1 or 5, wherein: The at least one gradient color information includes: at least one of a gradient starting point, a gradient direction, a length of gradient color processing along the gradient direction, a starting point color, an end point color, an intermediate color, and a gradient midpoint.

7. The method of claim 6, wherein: The at least one gradient color information includes a gradient starting point, a starting point color, and an ending point color; Determining a placeholder image with a gradient color effect according to the at least one gradient color information includes: Determine the gradient direction according to the gradient starting point; Determine a color difference according to the starting color and the end color; Determining a length of gradient color processing along the gradient direction according to a size of a container corresponding to the placeholder image and the gradient direction; Determine the color corresponding to each unit distance along the gradient direction according to the color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction; and According to the gradient direction, the starting color, the ending color, and the color corresponding to each unit distance along the gradient direction, the container is subjected to gradient color processing to obtain the placeholder image with the gradient color effect.

8. The method of claim 7, wherein: The at least one gradient color information also includes an intermediate color; Determining a color difference according to the starting color and the end color includes: determining a first color difference according to the starting color and the intermediate color; determining a second color difference according to the intermediate color and the end color; and Determine the color corresponding to each unit distance along the gradient direction according to the color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction, including: determine the color corresponding to each unit distance along the gradient direction according to the first color difference, the second color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction.

9. An image processing device, comprising: The acquisition module is configured to: acquire the text input by the user; A first determination module is configured to: determine at least one keyword according to the text; A second determination module is configured to: determine at least one gradient color information according to the at least one keyword; The third determining module is configured to determine a placeholder image with a gradient color effect according to the at least one gradient color information.

10. A computer device comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to any one of claims 1-8.

11. A non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images