Image processing method and related equipment

Abstract

The invention provides an image processing method and related equipment. The method comprises the following steps: acquiring a text input by a user; determining at least one keyword according to the text; determining at least one piece of gradient color information according to the at least one keyword; and according to the at least one piece of gradient color information, determining a placeholder image with a gradient color effect.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing method and related equipment. Background Art

[0002] In the field of image generation technology, images can be automatically generated using text provided by users. However, since image generation models are usually large and the image generation process is long, placeholder images are usually used to pre-display them at the corresponding locations. After the image is generated, the generated image is used to replace the placeholder image.

[0003] However, the inventors of the present disclosure have discovered that the placeholder images generated in the related art have a single style and a poor user experience. Summary of the invention

[0004] The present disclosure proposes an image processing method and related devices to solve or partially solve the above problems.

[0005] In a first aspect, the present disclosure provides an image processing method, comprising:

[0006] Get the text entered by the user;

[0007] Determining at least one keyword based on the text;

[0008] Determining at least one gradient color information according to the at least one keyword;

[0009] A placeholder image with a gradient color effect is determined according to the at least one gradient color information.

[0010] In a second aspect of the present disclosure, there is provided an image processing device, comprising:

[0011] The acquisition module is configured to: acquire the text input by the user;

[0012] A first determination module is configured to: determine at least one keyword according to the text;

[0013] A second determination module is configured to: determine at least one gradient color information according to the at least one keyword;

[0014] The third determining module is configured to determine a placeholder image with a gradient color effect according to the at least one gradient color information.

[0015] In a third aspect of the present disclosure, a computer device is provided, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method described in the first aspect.

[0016] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors execute the method described in the first aspect.

[0017] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect.

[0018] The image processing method and related devices provided by the embodiments of the present disclosure can determine gradient color information based on the text input by the user, and then determine a placeholder image with a gradient color effect based on the gradient color information. In this way, the placeholder image can be more in line with the artistic conception of the input text and the user experience is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1A A schematic diagram showing an exemplary system provided by an embodiment of the present disclosure is shown.

[0021] Figure 1B A schematic diagram of an exemplary page is shown.

[0022] Figure 2A A flowchart of an exemplary method provided by an embodiment of the present disclosure is shown.

[0023] Figure 2B A flowchart of another exemplary method provided by an embodiment of the present disclosure is shown.

[0024] Figure 2C A flowchart of an exemplary method for generating a placeholder image according to an embodiment of the present disclosure is shown.

[0025] Figure 3A A schematic diagram of an exemplary page according to an embodiment of the present disclosure is shown.

[0026] Figure 3B A schematic diagram showing another exemplary page according to an embodiment of the present disclosure is shown.

[0027] Figure 3C A schematic diagram of an exemplary container according to an embodiment of the present disclosure is shown.

[0028] Figure 4 A schematic diagram showing another exemplary page according to an embodiment of the present disclosure is shown.

[0029] Figure 5 A schematic diagram of the hardware structure of an exemplary computer device provided in an embodiment of the present disclosure is shown.

[0030] Figure 6 A schematic diagram of an exemplary device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0032] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0033] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0034] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0035] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0037] Figure 1A A schematic diagram of an exemplary system 100 provided by an embodiment of the present disclosure is shown.

[0038] like Figure 1A As shown, the system 100 may include a terminal device 102, a terminal device 104, a server 106, and a database server 108. A medium (e.g., a network) providing a communication link may be included between the terminal device 102, the terminal device 104 and the server 106 and the database server 108. The network may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0039] Various applications (APPs) may be installed on the terminal device 104, such as image processing applications, video conferencing applications, reading applications, video applications, social applications, payment applications, web browsers, and instant messaging tools, etc. These applications may be used to display the delivered information.

[0040] The terminal device 102 and the terminal device 104 here can be hardware or software. When the terminal device 102 and the terminal device 104 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, e-book readers, MP3 players, laptop computers (Laptops) and desktop computers (PCs), etc. When the terminal device 102 and the terminal device 104 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0041] The server 106 may be a server that provides various services, such as a background server that provides support for various applications displayed on the terminal devices 102 and 104. The database server 108 may also be a database server that provides various services. It is understood that if the server 106 can implement the relevant functions of the database server 108, the database server 108 may not be set in the system 100.

[0042] Here, the server 106 and the database server 108 can also be hardware or software. When they are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When they are software, they can be implemented as multiple software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitation is made here.

[0043] It should be noted that the image processing method provided in the embodiment of the present disclosure may be executed by the terminal device 102 and / or the terminal device 104. It should be understood that Figure 1A The number of terminal devices, users, servers and database servers in the embodiment is only for illustration. Any number of terminal devices, users, servers and database servers may be provided as required.

[0044] In an exemplary scenario, the server 106 can provide an image generation service. For example, the user 112 can open an image processing application in the terminal device 104, enter a paragraph of text in the page, and then send the paragraph of text to the server 106. The server 106 can generate an image based on the paragraph of text and return the image to the terminal device 104 for display.

[0045] In some cases, since the image generation process takes a long time, a placeholder image can usually be displayed first to relieve the user's anxiety while waiting for the image to be generated.

[0046] Figure 1B A schematic diagram of an exemplary page 120 is shown.

[0047] like Figure 1B As shown, a placeholder image 122 is displayed in the page 120. Optionally, the placeholder image 122 may be generated by an image processing application in the terminal device 104, or may be generated in advance by the server 106 and then sent to the terminal device 104 for display.

[0048] like Figure 1B As shown, in the related art, the placeholder image 122 is usually a default image, which has a single and fixed style and cannot bring a good visual experience to the user.

[0049] In view of this, an embodiment of the present disclosure provides an image processing method, which can determine gradient color information based on text input by a user, and then determine a placeholder image with a gradient color effect based on the gradient color information. In this way, the placeholder image can be more in line with the artistic conception of the input text and the user experience can be better.

[0050] Figure 2AFIG. 2 is a flow chart of an exemplary method 200 provided in an embodiment of the present disclosure. The method 200 may be used to process an image. Optionally, the method 200 may be performed by Figure 1A The terminal devices 102 and 104 can also be implemented by Figure 1A The method 200 is implemented by the server 106. The following describes the method 200 by taking the terminal device 104 as an example.

[0051] like Figure 2A As shown, the method 200 may further include the following steps.

[0052] In step 202, the text input by the user is obtained.

[0053] Figure 3A A schematic diagram of an exemplary page 300 according to an embodiment of the present disclosure is shown.

[0054] like Figure 3A As shown, as an optional embodiment, after the user 112 opens the image processing application in the terminal device 104 , the page 300 may be entered, and an input field 302 for inputting text may be displayed in the page 300 .

[0055] The user 112 may input a piece of text in the input field 302 to serve as text for subsequently determining the placeholder image and the first image.

[0056] In some embodiments, after the user 112 enters text in the input field 302, the terminal device 104 can obtain the corresponding text from the input field 302. As an optional embodiment, when the user 112 clicks the Generate Now button 304, the terminal device 104 starts to obtain text from the input field 302. It can be understood that there are many ways to obtain text, which are only exemplarily described here and should not be used to limit the scope of protection of the present disclosure.

[0057] After acquiring the text input by the user 112 , in step 204 , the terminal device 104 may determine at least one keyword based on the text.

[0058] Optionally, the keywords may be directly extracted from the text, or may be generated by extracting features from the text, and the specific method is not limited.

[0059] In some embodiments, in order to make the keywords better reflect the meaning of the text itself, the at least one keyword can be obtained by performing word segmentation processing on the text.

[0060] The word segmentation method may be different depending on the text input by the user 112 .

[0061] For example, if the text input by user 112 is "withered vines, old trees, crows, small bridges, flowing water, people", the word segmentation process can be directly implemented based on the symbols that user 112 has inserted into the text without complicated processing, thereby eliminating the complicated steps of word segmentation processing and improving processing efficiency. As an optional embodiment, when terminal device 104 implements word segmentation in this way, "Please use symbols to separate keywords" or similar prompts can be displayed on one side of input bar 302 or in input bar 302, thereby prompting user 112 to use symbols to separate keywords for text input.

[0062] For another example, the text input by user 112 is "When will the bright moon appear? I raise my cup to ask the blue sky", then the text can be split into multiple keywords by using text word segmentation or text semantic segmentation.

[0063] Figure 2B A flowchart of another exemplary method provided by an embodiment of the present disclosure is shown.

[0064] like Figure 2B As shown, after the text input by the user 112 is segmented by using text word segmentation or text semantic segmentation, multiple keywords can be obtained.

[0065] In some embodiments, the text may be input into a pre-trained text processing model to output the at least one keyword. Optionally, the text processing model may be implemented using various models in the field of natural language processing (NLP), and may be trained using various corpora in the field of natural language processing, and the specific training method is not limited.

[0066] Since the model of natural language processing is usually large, the computing resources and storage resources of the terminal device 104 may not be able to process and store such a text processing model. Therefore, as an optional embodiment, the terminal device 104 can send the text to the server 106, and the text processing model running in the server 106 performs word segmentation processing and obtains the at least one keyword, and then returns the at least one keyword to the terminal device 104. It can be understood that in some cases, when the text processing model itself is small enough, it can also run in the terminal device 104.

[0067] After obtaining the at least one keyword, in step 206, the terminal device 104 may determine at least one gradient color information according to the at least one keyword, and the at least one gradient color information may be used to determine a placeholder image with a gradient color effect.

[0068] As an optional embodiment, the at least one gradient color information may be associated with the keyword. When a certain keyword is obtained, the gradient color information associated with the keyword may be searched accordingly.

[0069] It is understandable that such a processing method requires a huge database to store various keywords and their associated gradient color information. Therefore, in order to simplify, speed up and efficiently process, in some embodiments, a pre-trained parameter generation model can be used to determine at least one gradient color information according to the at least one keyword. Specifically, the at least one keyword can be input into the pre-trained parameter generation model, and the at least one gradient color information can be output.

[0070] like Figure 2B As shown, as an optional embodiment, the parameter generation model can be implemented using a neural network model. Optionally, the neural network model can be a relatively small neural network model. For example, the neural network model includes three layers, an input layer, a hidden layer, and an output layer. In this way, the pre-trained parameter generation model can be run in the terminal device 104. After the terminal device 104 obtains the at least one keyword, the parameter generation model can be called to obtain the at least one gradient color information without obtaining the at least one gradient color information through the server 106. At the same time, since the parameter generation model is small in size, the gradient color information can be generated quickly, and then the placeholder image can be quickly determined based on the gradient color information, so that the placeholder operation can be completed faster, thereby improving the user experience.

[0071] In some embodiments, user 110 (eg, a developer of an image processing application) may utilize system 100 to train the parameter generation model.

[0072] Figure 3B A schematic diagram of an exemplary page 320 according to an embodiment of the present disclosure is shown.

[0073] like Figure 3B As shown, the terminal device 102 may display a page 320, and an input bar 322 for inputting text may be displayed in the page 320. The user 110 may input any text in the input bar 322 as a training sample, and then click the Generate Now button 324, and the terminal device 102 may upload the text to the server 106. After receiving the text uploaded by the terminal device 102, the server 106 may process the text into at least one keyword (the processing method may refer to the previous embodiment), and then input the obtained keyword into the initialized neural network model to output corresponding gradient color information, and return the gradient color information to the terminal device 102.

[0074] After acquiring the gradient color information returned by the server 106, the terminal device 102 can generate an image 326 according to the gradient color information, such as Figure 3B shown.

[0075] According to the gradient color effect presented by the image 326, the user 110 can adjust the parameters. Figure 3B As shown, the user 110 can click the parameter setting button 328 to open the parameter adjustment window (not shown in the figure), which can display the model parameter options and the gradient color parameter options. The user 110 can adjust the parameters of the neural network model by adjusting the model parameters corresponding to the model parameter options, and can adjust the gradient color parameters by adjusting the gradient color parameters corresponding to the gradient color parameter options. After adjusting the parameters, the user 110 can click the generate immediately button 324 again to upload the adjusted parameters to the server 106, and then the server 106 continues to train the neural network model based on the adjusted parameters until the gradient color effect presented by the image 326 meets the expectations. Then, the user 110 can re-enter new text and continue to train the neural network model until the model meets the set requirements.

[0076] In some embodiments, the user 110 can click the random word button 330 to generate a random word, and then use the neural network model to generate corresponding gradient color parameters, and then generate a corresponding image based on the gradient color parameters. By determining whether the image meets expectations, it can be determined whether the neural network model is trained. As an optional embodiment, the random word button 330 can also be used to randomly generate some text as training samples, thereby improving the robustness and generalization ability of the model.

[0077] After the neural network model training is completed, the parameter generation model can be obtained. In some embodiments, the server 106 can process the parameter generation model into a software development kit (SDK) to be integrated into an image processing application, so that the terminal device 104 running the application can directly call the model to generate gradient color information.

[0078] In some embodiments, the at least one gradient color information includes: at least one of a gradient starting point, a gradient direction, a length of gradient color processing along the gradient direction, a starting point color, an end point color, and a gradient midpoint.

[0079] The gradient starting point can be the gradient color parameter <side-or-corner>, which describes the starting point of the gradient line. The gradient direction can be a gradient color parameter <angle>, which specifies the direction (or angle) of the gradient with an angle value, which increases clockwise. The length of the gradient along the gradient direction can be the gradient parameter <length>This parameter is used to indicate the length of the gradient process or the length of the gradient line, expressing the distance size of the gradient. The starting color can be a gradient color parameter <color>, which represents a color in the standard RGB color space (SRGB color space). The end color can be a gradient color parameter <linear-color-stop>, which is a parameter <color>value, followed by an optional end position (either a percentage value or a position along the gradient axis or line) <length>). The gradient midpoint can be a gradient color parameter <color-hint>, this parameter represents the color transition point and length. The color transition point is an interpolation hint that defines how the gradient is performed between adjacent colors. The length defines at which point between two colors the gradient stops. The color should reach the midpoint of the color transition. If this parameter is omitted, the midpoint of the color transition is the midpoint between the two color stops.

[0080] In this way, through the above 6 gradient color parameters, an image with gradient color effect can be generated.

[0081] In step 208, a placeholder image with a gradient color effect is determined according to the at least one gradient color information.

[0082] In some embodiments, the specific value of the gradient color information can be pre-associated with the corresponding placeholder image, so that after the gradient color information is obtained, the associated placeholder image can be found. In other embodiments, a placeholder image with a gradient color effect can be generated based on the gradient color information, so as to better reflect the gradient color effect corresponding to the text.

[0083] In some embodiments, the at least one gradient color information used to generate the placeholder image includes a gradient starting point, a starting point color, and an ending point color. The gradient starting point may be a gradient color parameter. <side-or-corner>, which describes the starting point of the gradient line. Both the starting color and the ending color can be gradient color parameters <color>, this parameter represents a color in a standard RGB color space (SRGB color space).

[0084] After knowing the starting point, starting color and ending color of the gradient, a placeholder image with a gradient color effect can be generated accordingly.

[0085] In some embodiments, Figure 2C As shown, step 208 of determining a placeholder image with a gradient color effect according to the at least one gradient color information may further include the following steps:

[0086] In step 2082, the gradient direction is determined according to the gradient starting point.

[0087] Optionally, for linear gradients, the gradient start point can be the gradient color parameter <side-or-corner>For example, "to right" means that the starting point is at the far left, and accordingly, the gradient direction can be determined to be from left to right.

[0088] It can be understood that "to left", "to top", "to bottom", "to top right", "to top left", "to bottom right", and "to bottom left" refer to the starting point being at the rightmost, bottom, top, bottom left, bottom right, top left, and top right, respectively. Accordingly, their gradient directions can be determined to be from right to left, from bottom to top, from top to bottom, from bottom left to top right, from bottom right to top left, from top left to bottom right, and from top right to bottom left, respectively. In addition, when the gradient starting point is not specified, the default gradient direction can be from top to bottom.

[0089] It should be noted that the gradient direction can also use the gradient color parameter <angle>In some embodiments, when the gradient starting point is not specified (for example, the at least one gradient color information does not include the gradient starting point), the gradient direction can be determined by specifying an angle, with the gradient direction from top to bottom being 0 degrees, and the gradient direction rotating in a clockwise direction as the angle increases. For example, when the specified angle is 45 degrees, the gradient direction is from the upper right to the lower left.

[0090] In step 2084, a color difference is determined based on the starting color and the ending color.

[0091] Optionally, both the start color and the end color can be gradient color parameters <color>, this parameter represents a color in a standard RGB color space (SRGB color space). For example, if the starting color is red, the corresponding RGB value is 255,0,0, and if the ending color is blue, the corresponding RGB value is 0,0,255. Thus, the RGB difference (i.e., color difference) can be calculated based on the RGB values ​​of red and blue to be -255,0,255.

[0092] In step 2086, the length of the gradient color processing along the gradient direction is determined according to the size of the container corresponding to the placeholder image and the gradient direction.

[0093] Figure 3C A schematic diagram of an exemplary container 340 is shown in accordance with an embodiment of the present disclosure.

[0094] like Figure 3C As shown, the container 340 corresponding to the placeholder image may be an area with the same size as the placeholder image, and the area is used to fill the corresponding gradient color effect.

[0095] For the gradient direction determined by the gradient starting point, the length of the gradient color processing along the gradient direction can be the side length or diagonal of the container 340. For example, when the gradient direction is from top to bottom or from bottom to top, the length of the gradient color processing along the gradient direction is the longitudinal side length of the container 340, when the gradient direction is from left to right or from right to left, the length of the gradient color processing along the gradient direction is the transverse side length of the container 340, and when the gradient direction is from lower left to upper right, from lower right to upper left, from upper left to lower right, or from upper right to lower left, the length of the gradient color processing along the gradient direction is the diagonal length of the container 340.

[0096] As mentioned above, in some embodiments, the gradient direction may also be determined based on an angle. Figure 3C As shown, it is assumed that the direction indicated by the dotted line is the gradient direction determined according to the angle, and the dotted line passes through the center of the container 340. Then, two straight lines are drawn perpendicular to the dotted line, and the space between the two parallel straight lines can just completely cover the container 340. At this time, the length L intercepted by the two straight lines on the dotted line is the length of the gradient color processing along the gradient direction.

[0097] In step 2088, the color corresponding to each unit distance along the gradient direction is determined according to the color difference, the unit ratio of the gradient and the length of the gradient color processing along the gradient direction.

[0098] The unit ratio of the gradient indicates how many times the color needs to change from the gradient start point to the gradient end point. For example, when the unit ratio is 1%, a total of 100 changes are required from the gradient start point (0%) to the gradient end point (100%). In this way, the color difference needs to be divided into 100 equal parts to obtain the unit color difference, and the length of the gradient color processing along the gradient direction is divided into 100 equal parts to obtain the unit distance.

[0099] For example, when the unit ratio is 1% and the starting color is red and the ending color is blue, the color difference is -255,0,255, and the corresponding unit color difference is -2.55,0,2.55. Therefore, the color corresponding to the 1% position can be 252,0,3, the color corresponding to the 2% position can be 250,0,5, the color corresponding to the 3% position can be 247,0,8, and so on. Accordingly, the color corresponding to the area from the starting point (0%) to the 1% position is red, the color corresponding to the area from the 1% position to the 2% position is 252,0,3, the color corresponding to the area from the 2% position to the 3% position is 250,0,5, the color corresponding to the area from the 3% position to the 4% position is 247,0,8, and so on, until the color of the end point (100%) of the gradient is blue.

[0100] It can be understood that the number of equal divisions of color and length may be different depending on the unit ratio. The specific unit ratio may be set or adopted as a default setting, and no specific limitation is made here.

[0101] In step 2090, the container is subjected to gradient color processing according to the gradient direction, the starting color, the ending color, and the color corresponding to each unit distance along the gradient direction to obtain the placeholder image with the gradient color effect.

[0102] In this step, the container 340 may be filled along the gradient direction according to the color corresponding to each unit distance along the gradient direction and the starting color and the ending color calculated above, thereby obtaining a placeholder image with a gradient color effect.

[0103] It can be known that when the gradient starting point, starting color and ending color are different, the artistic conception embodied by the generated placeholder image with gradient color effect may be different. By determining at least one gradient color information according to the at least one keyword, when the gradient color information is different, a placeholder image with a gradient color effect expressing different artistic conceptions can be generated, thereby improving the user experience.

[0104] In some embodiments, the at least one gradient color information may also include an intermediate color; optionally, the intermediate color may also be a gradient color parameter. <color>For example, the middle color is green, and its RGB value is 0,255,0.

[0105] The step 2084 of determining the color difference according to the starting color and the end color may further include: determining a first color difference according to the starting color and the intermediate color; and determining a second color difference according to the intermediate color and the end color.

[0106] Specifically, for example, the starting color is red, the middle color is green, and the end color is blue, then the first color difference is -255, 255, 0, and the second color difference is 0, -255, 255.

[0107] The step 2088 of determining the color corresponding to each unit distance along the gradient direction according to the color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction may further include: determining the color corresponding to each unit distance along the gradient direction according to the first color difference, the second color difference, the position corresponding to the intermediate color, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction.

[0108] Specifically, assuming that the unit ratio of the gradient is 1%, the color needs to change 50 times from the starting color to the middle color, and 50 times from the middle color to the end color. Therefore, the first color difference can be divided into 50 equal parts, and the corresponding first unit color difference is -5.1, 5.1, 0; similarly, the second color difference can be divided into 50 equal parts, and the corresponding second unit color difference is 0, -5.1, 5.1.

[0109] Therefore, the color corresponding to the 1% position can be 250,5,0, the color corresponding to the 2% position can be 245,10,0, the color corresponding to the 3% position can be 240,15,0, and so on, until the color corresponding to the 50% position is green; further, the color corresponding to the 51% position can be 0,250,5, the color corresponding to the 52% position can be 0,245,10, the color corresponding to the 3% position can be 0,240,15, and so on, until the color corresponding to the 100% position is blue. Correspondingly, the color of the area from the gradient starting point (0%) to the 1% position is red, the color of the area from the 1% position to the 2% position is 250,5,0, the color of the area from the 2% position to the 3% position is 245,10,0, the color of the area from the 3% position to the 4% position is 240,15,0, and so on, until the corresponding color at the 50% position is green, then, the color of the area from the 51% position to the 52% position is 0,250,5, the color of the area from the 52% position to the 53% position is 0,245,10, the color of the area from the 3% position to the 4% position is 0,240,15, and so on, until the corresponding color at the 100% position is blue.

[0110] In this way, with the addition of the gradient color information of the middle color, a placeholder image is generated that achieves a gradient color effect between three colors, which can further enrich the artistic conception that the placeholder image can embody, thereby improving the user experience.

[0111] In some embodiments, the at least one gradient color information may further include a gradient midpoint, which may be a gradient color parameter. <color-hint>, this parameter indicates the color transition point, that is, which position is used as the transition point in the gradient process. In the case where there is no gradient color information of the gradient midpoint, the turning point of the gradient process (the position of the middle color) defaults to the 50% position, and when the gradient color information of the gradient midpoint is not the specified 50% position, for example, the 40% position is specified, the first color difference can be divided into 40 first unit color differences, and correspondingly, the second color difference can be divided into 60 second unit color differences, and the color corresponding to each unit distance along the gradient direction can be determined according to the aforementioned similar steps, which will not be repeated here.

[0112] In this embodiment, the gradient midpoint is one of the gradient color information, which is generated based on the at least one keyword. The position of the gradient midpoint may be different depending on the artistic conception of the text input by the user (for example, not at the midpoint of the gradient length but at any position on the length), thereby generating a placeholder image with a gradient color effect that is more in line with the artistic conception of the text, thereby improving the user experience.

[0113] In some embodiments, there may be more than one intermediate color, and the gradient midpoints may be increased accordingly, so as to realize a placeholder image with a gradient color effect that can reflect a richer text conception. The specific implementation method can refer to the previous embodiment, and will not be repeated here.

[0114] It can be understood that in addition to the gradient starting point, starting color, ending color, intermediate color, and gradient midpoint, the gradient direction, gradient length, etc. in the gradient color information can also change the textual artistic conception that can be expressed by the gradient color effect of the placeholder image. By outputting this information according to keywords and then generating a placeholder image based on this information, the corresponding gradient color effect can be used to well reflect the artistic conception that can be expressed by the text input by the user, thereby improving the user experience.

[0115] It should be noted that the gradient color information such as the gradient starting point, gradient direction, length of gradient color processing along the gradient direction, starting color, end color, intermediate color, and gradient midpoint are not indispensable. When the generated gradient color information includes only one or several of them, the remaining gradient color information can be obtained by using default values ​​or random generation. It can be understood that the more information in these gradient color information is generated based on text, the better the effect of the placeholder image in reflecting the artistic conception of the text.

[0116] In addition, it should be noted that the aforementioned embodiment describes the linear gradient color processing of the container as an example. It can be understood that the gradient color processing can also be radial gradient color processing. Since the radial gradient color processing is to draw concentric circles from the center of the container to the outside, the required gradient color information may not be as much as the aforementioned embodiment. For example, there may be no gradient starting point or gradient direction. The at least one gradient color information may only include the starting color and the end point color to achieve the gradient color processing of the container, thereby obtaining a placeholder image with a gradient color effect. Of course, it can be understood that for radial gradient color processing, placeholder images with different gradient color effects can also be achieved based on the intermediate color and the gradient midpoint, which will not be repeated here.

[0117] In some embodiments, the method 200 further includes: on the target page (eg, Figure 3A The placeholder images (eg, placeholder images 306A-306D) are displayed at a target location (eg, preview area 308 of page 300) of the page 300.

[0118] like Figure 3A As shown, the number of placeholder images can be more than one, and the number of placeholder images can be different according to the amount of text content. For example, a threshold value of the number of keywords can be preset (e.g., 2), so that according to the number of keywords determined from the text, a group of adjacent keywords is formed, and a group of gradient color information is generated to generate a placeholder image. When the number of keywords exceeds the threshold, according to the multiple relationship between the number of keywords and the threshold, the number of placeholder images corresponding to the multiple can be generated.

[0119] In some embodiments, the text input by the user 112 in the page 300 can be used to generate a first image associated with the text; the method 200 further includes: generating the first image according to the text.

[0120] As an optional embodiment, generating the first image according to the text may further include: generating the first image according to the at least one keyword. Similarly, the number of first images generated may be more than one, and the number of first images may be different depending on the amount of text content. For example, a threshold value for the number of keywords may be preset (e.g., 2), so that according to the number of keywords determined from the text, a number of adjacent keywords are grouped together to generate a first image, and when the number of keywords exceeds the threshold, according to the relationship between the number of keywords and the multiple of the threshold, the number of first images corresponding to the multiple may be generated. It can be understood that at this time, the first image corresponds one-to-one to the placeholder image, and the gradient color effect presented by the placeholder image also corresponds to the first image, and the visual effect is better.

[0121] In some embodiments, generating the first image based on the text, or generating the first image based on the at least one keyword may further include: inputting the text and / or the at least one keyword into a pre-trained image generation model, and outputting the first image.

[0122] Optionally, the image generation model can be a convolutional neural network model based on deep learning, which is larger and can have a very deep model structure, so that it can learn more information from the text and generate a more delicate first image that is more consistent with the meaning of the text.

[0123] As mentioned above, since the image generation model is large (the size of a deep learning model is usually several times, dozens of times, or even more times that of a simple neural network model), the process of generating the first image takes a long time. Therefore, it can be understood that the step of generating the first image and the step of generating the placeholder image mentioned above can be performed simultaneously, thereby improving processing efficiency.

[0124] In some embodiments, the image generation model may be running in the server 106, so the terminal device 104 may send the text entered by the user 112 in the page 300 to the server 106, and the image generation model running in the server 106 may generate the first image and then return it to the terminal device 104.

[0125] In some embodiments, in response to the first image being generated, the placeholder image in the target page is replaced with the first image.

[0126] Figure 4 A schematic diagram of an exemplary page 400 according to an embodiment of the present disclosure is shown.

[0127] like Figure 4 As shown, when the first image has been generated, the terminal device 104 may display the generated first image (eg, images 402A-402D) in the page 400 to replace the placeholder image.

[0128] In some embodiments, Figure 4 As shown, the page 400 may also include a re-edit button 404. After the button 404 is triggered, the page 400 may display an input field (not shown in the figure), and then the user 112 may re-edit the text in the input field, and then click the re-generate button 406 to regenerate a new first image. It can be understood that in some embodiments, a new placeholder image can be generated using new text to replace the previously generated first image, so that the user 112 enters a state of waiting for a new image, making the entire operation process more coherent.

[0129] From the above embodiments, it can be seen that the image processing method provided by the embodiments of the present disclosure can intelligently and automatically generate semantically consistent placeholder images, giving users a good visual experience. In some embodiments, based on the actually generated placeholder images, the relevant parameters for generating placeholder images can be repeatedly trained to make the gradient color placeholder images more and more consistent with the user's semantics.

[0130] In some application scenarios, when the user enters text, semantically correct gradient color parameters can be trained and generated in real time. When the user clicks the "Generate Now" button, the generated gradient color image can be displayed on the screen in real time, reducing the user's anxiety while waiting for the generation process.

[0131] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0132] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0133] The embodiment of the present disclosure also provides a computer device for implementing the above method 200. Figure 5 FIG. 5 is a schematic diagram showing the hardware structure of an exemplary computer device 500 provided in an embodiment of the present disclosure. The computer device 500 can be used to implement Figure 1A The server 106 can also be used to implement Figure 1A In some scenarios, the computer device 500 can also be used to implement Figure 1A The database server 108.

[0134] like Figure 5 As shown, the computer device 500 may include: a processor 502, a memory 504, a network module 506, a peripheral interface 508 and a bus 510. The processor 502, the memory 504, the network module 506 and the peripheral interface 508 are connected to each other in communication within the computer device 500 through the bus 510.

[0135] Processor 502 may be a central processing unit (CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. Processor 502 may be used to perform functions related to the technology described in this disclosure. In some embodiments, processor 502 may also include multiple processors integrated into a single logical component. For example, Figure 5 As shown, processor 502 may include a plurality of processors 502a, 502b, and 502c.

[0136] The memory 504 may be configured to store data (eg, instructions, computer code, etc.). Figure 5 As shown, the data stored in the memory 504 may include program instructions (e.g., program instructions for implementing the method 200 of the embodiment of the present disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). The processor 502 may also access the program instructions and data stored in the memory 504, and execute the program instructions to operate on the data to be processed. The memory 504 may include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 504 may include a random access memory (RAM), a read-only memory (ROM), an optical disk, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0137] The network interface 506 can be configured to provide the computer device 500 with communication with other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC)), a cellular network, the Internet, or a combination thereof. It is understood that the type of network is not limited to the above specific examples.

[0138] The peripheral interface 508 can be configured to connect the computer device 500 to one or more peripheral devices to achieve information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touch pad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0139] The bus 510 may be configured to transmit information between various components of the computer device 500 (e.g., the processor 502, the memory 504, the network interface 506, and the peripheral interface 508), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0140] It should be noted that, although the architecture of the above-mentioned computer device 500 only shows the processor 502, the memory 504, the network interface 506, the peripheral interface 508 and the bus 510, in the specific implementation process, the architecture of the computer device 500 may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the architecture of the above-mentioned computer device 500 may also only include the components necessary for implementing the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0141] The embodiment of the present disclosure also provides an image processing device. Figure 6 FIG. 6 is a schematic diagram of an exemplary device 600 provided by an embodiment of the present disclosure. Figure 6 As shown, the device 600 can be used to implement the method 200 and can further include the following modules.

[0142] The acquisition module 602 is configured to: acquire text input by the user;

[0143] The first determination module 604 is configured to: determine at least one keyword according to the text;

[0144] The second determination module 606 is configured to: determine at least one gradient color information according to the at least one keyword;

[0145] The third determining module 608 is configured to determine a placeholder image with a gradient color effect according to the at least one gradient color information.

[0146] In some embodiments, the apparatus further comprises:

[0147] The display module is configured to: display the placeholder image at a target position of a target page.

[0148] In some embodiments, the text is used to generate a first image associated with the text; the apparatus further comprises a generating module configured to: generate the first image according to the text;

[0149] The display module is configured to: in response to the first image being generated, replace the placeholder image in the target page with the first image.

[0150] In some embodiments, the first determination module 604 is configured to: perform word segmentation processing on the text to obtain the at least one keyword.

[0151] In some embodiments, the first determination module 604 is configured to: input the text into a pre-trained text processing model, and output the at least one keyword.

[0152] In some embodiments, the second determination module 606 is configured to: input the at least one keyword into a pre-trained parameter generation model, and output the at least one gradient color information.

[0153] In some embodiments, the at least one gradient color information includes: at least one of a gradient starting point, a gradient direction, a length of gradient color processing along the gradient direction, a starting point color, an end point color, and a gradient midpoint.

[0154] In some embodiments, the at least one gradient color information includes a gradient starting point, a starting point color, and an ending point color;

[0155] The third determination module 608 is configured to:

[0156] Determine the gradient direction according to the gradient starting point;

[0157] Determine a color difference according to the starting color and the end color;

[0158] Determining a length of gradient color processing along the gradient direction according to a size of a container corresponding to the placeholder image and the gradient direction;

[0159] Determine the color corresponding to each unit distance along the gradient direction according to the color difference, the unit ratio of the gradient and the length of the gradient color processing along the gradient direction;

[0160] According to the gradient direction, the starting color, the ending color, and the color corresponding to each unit distance along the gradient direction, the container is subjected to gradient color processing to obtain the placeholder image with the gradient color effect.

[0161] In some embodiments, the at least one gradient color information further includes an intermediate color;

[0162] The third determination module 608 is configured to:

[0163] Determine a first color difference value according to the starting color and the intermediate color;

[0164] Determine a second color difference value according to the intermediate color and the end point color;

[0165] The color corresponding to each unit distance along the gradient direction is determined according to the first color difference, the second color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction.

[0166] In some embodiments, the generating module is configured to: generate the first image according to the at least one keyword.

[0167] In some embodiments, the generation module is configured to: input the text and / or the at least one keyword into a pre-trained image generation model, and output the first image.

[0168] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0169] The device of the above embodiment is used to implement the corresponding method 200 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0170] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute method 400 described in any of the above embodiments.

[0171] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0172] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method 200 described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0173] Based on the same inventive concept, corresponding to the method 200 in any of the above embodiments, the present disclosure further provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors so that the processors execute the method 200. Corresponding to the execution subject corresponding to each step in each embodiment of the method 200, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0174] The computer program product of the above embodiment is used to enable the processor to execute the method 200 described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0175] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0176] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0177] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0178] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure. < / color> < / color> < / angle> < / color> < / length> < / color> < / color> < / length> < / angle>

Claims

1. An image processing method, comprising: Get the text entered by the user; Determining at least one keyword based on the text; Determining at least one gradient color information according to the at least one keyword; A placeholder image with a gradient color effect is determined according to the at least one gradient color information.

2. The method of claim 1, wherein: The method further comprises: The placeholder image is displayed at a target location on the target page.

3. The method of claim 2, wherein: The text is used to generate a first image associated with the text; the method further comprising: Generate the first image according to the text; In response to the first image being generated, the placeholder image in the target page is replaced with the first image.

4. The method of claim 1, wherein: According to the text, at least one keyword is determined, including: The text is segmented using a pre-trained text processing model to obtain the at least one keyword.

5. The method of claim 1, wherein: Determining at least one gradient color information according to the at least one keyword includes: The at least one keyword is input into a pre-trained parameter generation model, and the at least one gradient color information is output.

6. The method according to claim 1 or 5, wherein: The at least one gradient color information includes: at least one of a gradient starting point, a gradient direction, a length of gradient color processing along the gradient direction, a starting point color, an end point color, an intermediate color, and a gradient midpoint.

7. The method of claim 6, wherein: The at least one gradient color information includes a gradient starting point, a starting point color, and an ending point color; Determining a placeholder image with a gradient color effect according to the at least one gradient color information includes: Determine the gradient direction according to the gradient starting point; Determine a color difference according to the starting color and the end color; Determining a length of gradient color processing along the gradient direction according to a size of a container corresponding to the placeholder image and the gradient direction; Determine the color corresponding to each unit distance along the gradient direction according to the color difference, the unit ratio of the gradient and the length of the gradient color processing along the gradient direction; According to the gradient direction, the starting color, the ending color, and the color corresponding to each unit distance along the gradient direction, the container is subjected to gradient color processing to obtain the placeholder image with the gradient color effect.

8. The method of claim 7, wherein: The at least one gradient color information also includes an intermediate color; Determining a color difference according to the starting color and the end color, including: determining a first color difference according to the starting color and the intermediate color; determining a second color difference according to the intermediate color and the end color; Determine the color corresponding to each unit distance along the gradient direction according to the color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction, including: determine the color corresponding to each unit distance along the gradient direction according to the first color difference, the second color difference, the unit ratio of the gradient, and the length of the gradient color processing along the gradient direction.

9. An image processing device, comprising: The acquisition module is configured to: acquire the text input by the user; A first determination module is configured to: determine at least one keyword according to the text; A second determination module is configured to: determine at least one gradient color information according to the at least one keyword; The third determining module is configured to determine a placeholder image with a gradient color effect according to the at least one gradient color information.

10. A computer device comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to any one of claims 1-8.

11. A non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method according to any one of claims 1 to 8.

Images