Method, device, electronic device and medium for matching materials to images

Abstract

A method, device, electronic device and medium for material matching of images, wherein a material matching request is received. The target image selected by the user and information of M material images in a predetermined material library are obtained, where M≥2 and M is a positive integer. Based on the information of the M material images, the target image and the M material images are respectively synthesized to generate a synthetic image corresponding to N material images, where 1≤N≤M and N is a positive integer. A list of synthetic images to be displayed is determined based on the similarity between the synthetic image and the target image, for the user to select.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and more specifically, to a method, device, electronic device and medium for performing material matching on a picture. Background Art

[0002] With the development of mobile networks, the application of image editing is becoming more and more widespread. When editing images, it is often necessary to add some embellishments to the original image, such as superimposing some text, widgets and other material content. In related technologies, the image and the available material content are first displayed, and the user selects the corresponding widget or text content according to preference, and then the material selected by the user is superimposed on the original image for preview. When the user feels it is appropriate, the image is superimposed and merged to form a new image.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the prior art: the existing material matching method relies entirely on manual selection in terms of selecting material content, which is relatively inefficient; if the material library has a large amount of content, the efficiency of selecting the appropriate material is very low in this case, and the corresponding cost is also high. Moreover, the efficiency of superimposing a single material with the original image is low. If the material is not suitable, it is necessary to select new materials and repeat the superposition process. Summary of the invention

[0004] In view of this, the present disclosure provides a method, device, electronic device and medium for performing material matching on a picture.

[0005] The first aspect of the present disclosure provides a method for material matching for images. In the above method, a material matching request is received. The target image selected by the user and the information of M material images in the predetermined material library are obtained, where M≥2 and M is a positive integer. Based on the information of the M material images, the target image is synthesized with the M material images respectively to generate a synthetic image corresponding to N material images, where 1≤N≤M and N is a positive integer. A list of synthetic images to be displayed is determined based on the similarity between the synthetic image and the target image for selection by the user.

[0006] According to an embodiment of the present disclosure, the method further includes: determining a predetermined material library. The method further includes: determining a first pending material library according to a popularity ranking of the number of uses of material images in the initial material library within a predetermined time range. The method further includes: determining a second pending material library according to a month-on-month growth rate of the number of uses of material images in the initial material library within two consecutive statistical periods. The method further includes: determining a predetermined material library according to the first pending material library and the second pending material library.

[0007] According to an embodiment of the present disclosure, if a user selects a specific composite image from the list of composite images to be displayed, the number of times the material image corresponding to the specific composite image is used is increased by 1. The first pending material library and the second pending material library are updated according to the change in the number of times the material image corresponding to the specific composite image is used to update the predetermined material library. After receiving a new target image selected by the user and a material matching request next time, the material image in the updated predetermined material library is synthesized with the new target image.

[0008] According to an embodiment of the present disclosure, the method further includes: if the user does not select any of the images in the list of composite images to be displayed, then determining the remaining material library from the initial material library, the remaining material library being a material library composed of the remaining materials obtained after the initial material library removes all the material images corresponding to the composite images in the list of composite images to be displayed. The method further includes: obtaining information about the material images in the remaining material library. The method further includes: based on the information about the material images in the remaining material library, synthesizing the target image with the material images in the remaining material library to generate an alternative composite image. The method further includes: determining a list of alternative composite images to be displayed based on the similarity between the alternative composite images and the target image, for the user to select.

[0009] According to an embodiment of the present disclosure, the information of the material picture includes: preset position information and size information, and the preset position information includes: the horizontal position ratio T% and the vertical position ratio U% of a specific position point of the material picture relative to the picture to be synthesized.

[0010] According to an embodiment of the present disclosure, the position coordinates (a0, b0) of a specific position point in a picture to be synthesized with a size of r×s satisfy the following relationship: a0=r×T%, b0=s×U%.

[0011] According to an embodiment of the present disclosure, the above-mentioned method of synthesizing the target image selected by the user with the M material images based on the information of the M material images to generate a synthesized image corresponding to the N material images includes: determining the position coordinates of the M material images in the target image according to the preset position information of the M material images and the size of the target image. Determining whether the outer edges of the M material images are located within the target image according to the position coordinates, the size information of the M material images, and the size of the target image. If the outer edge of the material image is located within the target image, the material image is overlaid on the target image at the corresponding position coordinates to generate a synthesized image corresponding to the N material images.

[0012] According to another embodiment of the present disclosure, based on the information of the M material pictures, the target picture selected by the user is synthesized with the M material pictures respectively to generate a synthesized picture corresponding to the N material pictures, including: determining the position coordinates of the M material pictures in the target picture according to the preset position information of the M material pictures and the size of the target picture. Determine the reduction ratio of the M material pictures according to the position coordinates, the size information of the M material pictures and the size of the target picture so that the outer edges of the M material pictures are all located within the target picture. Cover the M material pictures on the target picture at the corresponding position coordinates to generate a synthesized picture corresponding to the M material pictures.

[0013] According to an embodiment of the present disclosure, a list of composite pictures to be displayed is determined based on the similarity between the composite pictures and the target pictures for selection by the user, including: determining whether the similarity between the composite pictures and the target pictures falls within a threshold interval. The composite pictures whose similarity falls within the threshold interval are used as the composite pictures to be displayed for selection by the user. The list of composite pictures to be displayed is generated based on all the composite pictures to be displayed.

[0014] The second aspect of the present disclosure provides a device for material matching of images. The above-mentioned device includes: a matching request receiving module, a target image and material image information acquisition module, a picture synthesis module and a synthetic image list determination module. The matching request receiving module is used to receive a material matching request. The target image and material image information acquisition module is used to obtain the information of the target image selected by the user and M material images in a predetermined material library, M≥2, and M is a positive integer. The picture synthesis module is used to synthesize the target image with the M material images respectively based on the information of the M material images to generate a synthetic image corresponding to the N material images, 1≤N≤M, and N is a positive integer. The synthetic image list determination module is used to determine the list of synthetic images to be displayed according to the similarity between the synthetic image and the target image, for the user to select.

[0015] According to an embodiment of the present disclosure, the above-mentioned device further includes: a predetermined material library determining module, which is used to determine the predetermined material library.

[0016] According to an embodiment of the present disclosure, the above-mentioned device also includes: a material image usage statistics module, which is used to count the number of times the material image corresponding to the specific composite image is used when the user selects a specific composite image from the list of composite images to be displayed.

[0017] According to an embodiment of the present disclosure, the above-mentioned predetermined material library determination module is further used to update the predetermined material library according to the change in the number of times the material picture corresponding to the specific composite picture is used.

[0018] According to an embodiment of the present disclosure, the device further includes: a remaining material library determination module, which is used to determine the remaining material library from the initial material library when the user does not select any picture in the list of composite pictures to be displayed. The remaining material library is a material library composed of the remaining materials obtained after the initial material library removes all material pictures corresponding to the composite pictures in the list of composite pictures to be displayed.

[0019] According to an embodiment of the present disclosure, the target image and material image information acquisition module is also used to acquire information about material images in the remaining material library. The image synthesis module is also used to synthesize the target image with the material images in the remaining material library based on the information about the material images in the remaining material library to generate an alternative synthetic image. The synthetic image list determination module is also used to determine a list of alternative synthetic images to be displayed based on the similarity between the alternative synthetic images and the target image, so as to provide a selection for the user.

[0020] The third aspect of the present disclosure provides an electronic device. The electronic device includes: one or more processors; and a storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described above.

[0021] A fourth aspect of the present disclosure provides a computer-readable storage medium. The computer-readable storage medium stores executable instructions, which, when executed by a processor, enable the processor to implement any of the methods described above.

[0022] According to the embodiments of the present disclosure, a target image and a material image in a predetermined material library are synthesized, and a list of synthesized images to be displayed for user selection is determined based on the similarity between the synthesized image and the target image. The user does not need to manually synthesize and adjust the target image and the material image, and a plurality of options of synthesized images are directly provided to the user. Moreover, based on the similarity between the synthesized image and the target image, a list of synthesized images to be selected is provided to the user, and the synthesized images are screened to remove situations where the material image is too small relative to the target image so that the similarity is too high, and situations where the material image is too large and occludes the target image too much so that the similarity is too low, thereby providing the user with a more suitable choice of synthesized images, at least partially solving the problem of low efficiency in the existing manual addition of materials to the original image. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0024] Figure 1 The system architecture of the method for material matching for pictures according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 2 A flowchart of a method for performing material matching on a picture according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 3 A flowchart of a method for performing material matching on a picture according to another embodiment of the present disclosure is schematically shown;

[0027] Figure 4 Schematically shows a detailed implementation flow chart of operation S10 according to an embodiment of the present disclosure;

[0028] Figure 5 A flowchart of a method for performing material matching on a picture according to another embodiment of the present disclosure is schematically shown;

[0029] Figure 6 A flowchart of a method for performing material matching on a picture according to another embodiment of the present disclosure is schematically shown;

[0030] Figure 7 The detailed implementation flow chart of operation S13 according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 8 The implementation scenario of operation S13 according to an embodiment of the present disclosure is schematically shown, wherein (a) to (c) illustrate the situation where the outer edge of the material image is outside the target image, and (d) to (f) illustrate the situation where the outer edge of the material image is inside the target image.

[0032] Fig. 9 Schematically shows a detailed implementation flow chart of operation S13 according to another embodiment of the present disclosure;

[0033] Fig.10 Schematically illustrating implementation scenarios of operation S13 according to another embodiment of the present disclosure, wherein (a) is a material picture, (b) is a scenario of determining the position coordinates of the material picture in the target picture, (c) is a scenario in which the material picture is placed at the corresponding position coordinates of the target picture and the outer edge of the material picture is located outside the target picture, (d) is a scenario in which the material picture is reduced so that the outer edge of the material picture is located within the target picture, and (e) is a scenario in which the reduced material picture is combined with the target picture to obtain a combined picture;

[0034] Fig.11 Schematically shows a detailed implementation flow chart of operation S14 according to an embodiment of the present disclosure;

[0035] Fig.12 The structural block diagram of the device for performing material matching for a picture according to an embodiment of the present disclosure is schematically shown;

[0036] Fig.13 A structural block diagram of an apparatus for performing material matching on a picture according to another embodiment of the present disclosure is schematically shown; and

[0037] Fig.14 The structural block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0038] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0039] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0040] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0041] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0042] In the related art, in the editing operation of a picture, the picture and the available material content are usually displayed first. The material content includes widgets or text content. These materials are all in the format of pictures. The user selects the corresponding widget or text content according to his or her preference, and then the material selected by the user is superimposed on the original picture for preview. When the user feels it is appropriate, the images are superimposed and merged to form a new image.

[0043] The existing material matching method relies entirely on manual selection in terms of material content selection, which is relatively inefficient. If the material library has a large amount of content, the efficiency of selecting the right material is very low, and the corresponding cost is also high. Moreover, superimposing a single material with the original image is inefficient. If the material is not suitable, new materials need to be screened and repeated superimposed. In addition, when editing these images, it is necessary to best match the existing material images to avoid covering the original core content of the image when superimposing the images.

[0044] In view of this, the embodiments of the present disclosure provide a method, device, electronic device and medium for material matching for images. In the above method for material matching for images, a material matching request is received. The target image selected by the user and the information of M material images in the predetermined material library are obtained, where M≥2 and M is a positive integer. Based on the information of the M material images, the target image is synthesized with the M material images respectively to generate a synthetic image corresponding to the N material images, where 1≤N≤M and N is a positive integer. According to the similarity between the synthetic image and the target image, a list of synthetic images to be displayed is determined for the user to select.

[0045] The above method does not require the user to manually synthesize and adjust the target image and the material image, but directly provides the user with multiple options for synthesized images. It also provides the user with a list of synthesized images to be selected based on the similarity between the synthesized image and the target image, thereby providing the user with a more suitable choice of synthesized images, and at least partially solves the existing problem of inefficiency caused by manual addition of materials to the original image.

[0046] Figure 1 The system architecture of the method for material matching for pictures according to an embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0047] Reference Figure 1As shown, the system architecture 1 according to this embodiment may include a terminal device 10, a network 11 and a server 12. The network 11 is used to provide a medium for a communication link between the terminal device 10 and the server 12. The network 11 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0048] The user can use the terminal device 10 to interact with the server 12 through the network 11 to receive or send pictures and related information of the pictures, etc. The terminal device 10 can be installed with various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0049] The terminal device 10 may be any electronic device having a display screen and supporting web browsing, for example, Figure 1 The illustrated smart phone 101, tablet computer 102 or laptop computer 103 may also be a desktop computer, a smart watch or other electronic devices.

[0050] The server 12 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal device 10. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0051] It should be noted that the method for matching materials for pictures provided in the embodiment of the present disclosure can generally be executed by the terminal 10, or by the server 12 that can communicate with the terminal. Accordingly, the device for matching materials for pictures provided in the embodiment of the present disclosure can generally be set in the terminal 10 or the server 12. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0052] A first exemplary embodiment of the present disclosure provides a method for performing material matching on a picture.

[0053] Figure 2 The flowchart of the method for performing material matching for a picture according to an embodiment of the present disclosure is schematically shown.

[0054] Reference Figure 2 As shown, the method for performing material matching on a picture according to an embodiment of the present disclosure includes the following operations: S11, S12, S13 and S14.

[0055] In operation S11, a material matching request is received.

[0056] In operation S12, information of a target picture selected by a user and M material pictures in a predetermined material library is obtained, where M≥2 and M is a positive integer.

[0057] In operation S13, the target picture is synthesized with the M material pictures based on the information of the M material pictures to generate a synthesized picture corresponding to the N material pictures, where 1≤N≤M, and N is a positive integer.

[0058] In operation S14, a list of synthesized pictures to be displayed is determined according to the similarity between the synthesized pictures and the target pictures, for selection by the user.

[0059] In the above operation S11, the operation of receiving the material matching request can be performed by the terminal device, and the user clicks the material matching button on the terminal device, which is deemed that the user has initiated the material matching request. The above operation can also be performed by the server, and after the user powers on the terminal device and clicks the material matching button, the terminal device sends the material matching request to the server based on the network, and the server receives the material matching request.

[0060] In the above operation S12, the operations of obtaining the target picture selected by the user and obtaining the information of M material pictures in the predetermined material library can be performed step by step.

[0061] In the present disclosure, the material pictures can be pictures of various forms, such as pendant pictures, text pictures or other forms of pictures, and the content of the pictures can also be various, such as cute pets, scenery, food or architecture.

[0062] In one scenario, after the user clicks the material matching button on the terminal device, the display interface of the terminal device may prompt the user to select a picture from a given picture library. After the user selects a specific picture, the selected picture is used as the target picture, and the material picture is added on the basis of the target picture. In the above operation S12, the operation of obtaining the target picture selected by the user may be performed by the terminal device. After the user selects the target picture, the terminal device obtains the target picture. The operation of obtaining the target picture selected by the user may also be performed by the server. After the user selects a target picture on the terminal, the server communicates with the terminal device through the network to obtain the target picture selected by the user. The number M of material pictures in the predetermined material library may be multiple, for example, 10 to 100, or even more.

[0063] The predetermined material library can be stored in a server or a terminal device. The terminal device or the server can obtain the information of the M material pictures in the predetermined material library. Here, the material pictures are not directly obtained, but the information of the material pictures, which will be used in the subsequent picture synthesis operation.

[0064] In the above operation S13, the operation of synthesizing the target image with the M material images respectively based on the information of the M material images can be performed by the terminal device or the server. By synthesizing the target image and the material images based on the information of the material images, in most cases, there will be material images that do not meet the conditions for synthesizing with the target image, so the number N of material images that can generate the synthesized image is N<M; in some cases, all the material images meet the conditions for synthesizing with the target image, in this case, the number N of material images that can generate the synthesized image is N=M.

[0065] The above operation S14 can be performed by a terminal device or a server. By determining a list of composite images to be displayed based on the similarity between the composite image and the target image for the user to choose, the user is saved from the operation of manually adding materials one by one to manually synthesize with the target image, and the composite images to be displayed are screened based on the similarity, providing the user with suitable composite images for selection.

[0066] The method of material matching for images in the present embodiment synthesizes the target image and the material image in the predetermined material library, and determines a list of synthesized images to be displayed for the user to choose from according to the similarity between the synthesized image and the target image. The user does not need to manually synthesize and adjust the target image and the material image, and a plurality of synthesized image options are directly provided to the user. In addition, a list of synthesized images to be selected is provided to the user based on the similarity between the synthesized image and the target image, and the synthesized images are screened to remove situations where the material image is too small relative to the target image so that the similarity is too high, and situations where the material image is too large and occludes the target image too much so that the similarity is too low, thereby providing the user with a more suitable choice of synthesized images, and at least partially solving the problem of low efficiency of the existing manual addition of materials to the original image.

[0067] Figure 3 The flowchart of a method for performing material matching on a picture according to another embodiment of the present disclosure is schematically shown.

[0068] Reference Figure 3 As shown, according to another embodiment of the present disclosure, the above method for performing material matching on a picture, in addition to operations S11 to S14, further includes operation S10: determining a predetermined material library.

[0069] Figure 4 Schematically shows a detailed implementation flow chart of operation S10 according to an embodiment of the present disclosure;

[0070] Reference Figure 4 As shown, the above operation S10 of determining the predetermined material library includes the following sub-operations: S101, S102 and S103.

[0071] In sub-operation S101, a first pending material library is determined according to a popularity ranking of usage times of material pictures in an initial material library within a predetermined time range.

[0072] In sub-operation S102, a second pending material library is determined according to a month-on-month growth rate of usage times of material pictures in the initial material library in two consecutive statistical periods.

[0073] In sub-operation S103, a predetermined material library is determined according to the first to-be-determined material library and the second to-be-determined material library.

[0074] In the above sub-operations S101 and S102, the predetermined time range and statistical period can be adjusted according to actual needs, for example, one week, 10 days, half a month, one month, one quarter or one year.

[0075] The initial material library may be a material library in a terminal device or a server, and the initial material library has a certain amount of materials, for example, materials in the thousands to millions.

[0076] In sub-operation S101, there are many implementation methods for determining the first pending material library according to the usage frequency ranking of the material pictures in the initial material library within a predetermined time range. For example, the material pictures in the initial material library whose usage frequency ranking is within a predetermined ranking within the predetermined time range can be used as the material pictures of the first pending material library, for example, the material pictures whose usage frequency ranking is within the top 50 within a month in the initial material library can be used as the material pictures of the first pending material library. The material pictures in the initial material library whose usage frequency ranking is within a predetermined proportion within the predetermined time range can also be used as the material pictures of the first pending material library, for example, the material pictures whose usage frequency ranking is in the top 10% within a week in the initial material library can be used as the material pictures of the first pending material library.

[0077] In sub-operation S102, there are multiple implementation methods for determining the second pending material library based on the month-on-month growth rate of the number of times the material pictures in the initial material library are used in two consecutive statistical periods. For example, the material pictures in the initial material library whose month-on-month growth rate of the number of times used in two consecutive statistical periods is higher than a set value can be used as the material pictures in the second pending material library, for example, the material pictures in the initial material library whose month-on-month growth rate of the number of times used in two consecutive statistical periods is higher than 60% can be used as the material pictures in the second pending material library. The material pictures in the initial material library whose month-on-month growth rate of the number of times used in two consecutive statistical periods ranks within a predetermined ranking can also be used as the material pictures in the second pending material library, for example, the material pictures in the initial material library whose month-on-month growth rate of the number of times used in two consecutive statistical periods ranks within the top 20 can be used as the material pictures in the second pending material library.

[0078] In sub-operation S103, determining the predetermined material library according to the first pending material library and the second pending material library includes: removing duplicates and merging the first pending material library and the second pending material library (removing duplicate contents and merging) to obtain a material library as the predetermined material library. In other embodiments, the intersection of the first pending material library and the second pending material library may be used as the predetermined material library. Alternatively, one of the first pending material library and the second pending material library is used as the predetermined material library.

[0079] Figure 5 The flowchart of a method for performing material matching on a picture according to yet another embodiment of the present disclosure is schematically shown.

[0080] According to another embodiment of the present disclosure, referring to Figure 5 As shown in the contents without dashed line boxes, the above method includes the following operations in addition to operations S10 to S14: S15a, S16a and S17a.

[0081] In operation S15a, if the user selects a specific composite picture from the list of composite pictures to be displayed, the number of times the material picture corresponding to the specific composite picture is used is increased by 1.

[0082] In operation S16a, the first pending material library and the second pending material library are updated according to the change in the number of times the material picture corresponding to the specific composite picture is used, so as to update the predetermined material library.

[0083] In operation S17a, after receiving a new target picture selected by the user and a material matching request next time, the updated material picture in the predetermined material library is synthesized with the new target picture.

[0084] In this embodiment, the predetermined material library is dynamically updated by counting the number of times the material images are used according to the selection results of the user for the synthesized image, so that the user can be provided with material images that meet the user's needs as much as possible for synthesis under each material matching request.

[0085] Figure 6 The following schematically shows a flow chart of a method for performing material matching on a picture according to yet another embodiment of the present disclosure.

[0086] In one scenario, the user does not select any of the options in the list of composite images to be displayed. In this case, the materials are matched in the manner described in the following embodiment.

[0087] According to yet another embodiment of the present disclosure, referring to Figure 6 As shown in the content without the dotted box, the above method includes the following operations in addition to operations S11 to S14: S15b, S16b, S17b and S18b.

[0088] In operation S15b, if the user does not select any picture in the list of composite pictures to be displayed, a remaining material library is determined from the initial material library. The remaining material library is a material library composed of remaining materials obtained after the initial material library removes all material pictures corresponding to the composite pictures in the list of composite pictures to be displayed.

[0089] In operation S16b, information of material pictures in the remaining material library is obtained.

[0090] In operation S17b, the target picture is synthesized with the material pictures in the remaining material library based on the information of the material pictures in the remaining material library to generate a candidate synthesized picture.

[0091] In operation S18b, a list of candidate composite pictures to be displayed is determined according to the similarity between the candidate composite pictures and the target picture, for selection by the user.

[0092] The various embodiments described above can be integrated into one embodiment, or they can be separate embodiments described above. In the case of integration into one embodiment, when the user selects a specific composite image in the list of composite images to be displayed, the predetermined material library can be dynamically updated so that the material image in the predetermined material library in the next material matching request can reflect the popularity and month-on-month growth rate of the material image, and provide the user with a more suitable composite image that is closer to the user's needs. In the case that the user does not select any of the images in the list of composite images to be displayed, a list of candidate composite images to be displayed is further provided for the user to select, which can meet the user's needs as much as possible. Since the operation of the composite image is performed by the terminal device or the server, the speed and efficiency of the automated processing are very high, which saves the user from the manual operation of the editing process. The user only needs to select a target image and can select from the list of composite images to be displayed. If all the composite images in the list of composite images to be displayed are not satisfactory, they can be further selected in the list of candidate composite images to be displayed to find a satisfactory composite image. In addition, the screening based on similarity can achieve a better match between the target image and the material image, avoiding the original core content of the image being covered when the image is superimposed.

[0093] Combine the following Figure 7-Figure 11 The implementation process of operations S13 and S14 will be described in detail.

[0094] Figure 7 The detailed implementation flowchart of operation S13 according to an embodiment of the present disclosure is schematically shown. Figure 8 The implementation scenario of operation S13 according to an embodiment of the present disclosure is schematically shown, wherein (a) to (c) illustrate the situation where the outer edge of the material image is outside the target image, and (d) to (f) illustrate the situation where the outer edge of the material image is inside the target image.

[0095] According to the embodiments of the present disclosure, referring to Figure 7 As shown, the operation S13 of synthesizing the target image selected by the user with the M material images respectively based on the information of the M material images to generate a synthesized image corresponding to the N material images includes the following sub-operations: S131a, S132a and S133a.

[0096] In sub-operation S131a, position coordinates of the M material pictures within the target picture are determined according to the preset position information of the M material pictures and the size of the target picture.

[0097] In sub-operation S132a, it is determined whether the outer edges of the M material pictures are located within the target picture according to the position coordinates, the size information of the M material pictures, and the size of the target picture.

[0098] In sub-operation S133a, if the outer edge of the material picture is located within the target picture, the material picture is overlaid on the target picture at the corresponding position coordinates to generate a composite picture corresponding to the N material pictures.

[0099] According to the embodiments of the present disclosure, referring to Figure 8 As shown in (a) and (d), material pictures 31 and 32 are shown respectively, and the dotted boxes show the length×width dimensions of the material pictures. The information of the material pictures includes: preset position information and size information.

[0100] The preset position information includes: a horizontal position ratio T% and a vertical position ratio U% of a specific position point of the material image relative to the image to be synthesized, and the ratio value of the horizontal position ratio and the vertical position ratio is between 0 and 1. Figure 8 In (a) and (d), the center point is used to indicate the specific position point of the material picture 31 and the material picture 32. In other embodiments, the specific position point may be other position point besides the center point.

[0101] Figure 8 (b) and (e) respectively illustrate the situation of determining the position coordinates of the material picture 31 / material picture 32 in the target picture 200 according to the preset position information of the material picture 31 / material picture 32 and the size of the target picture 200, and the curved arrows illustrate the process of determining the position coordinates of the specific position point of the material picture on the target picture. The position coordinates (a0, b0) of the specific position point in the picture to be synthesized with a size of r (horizontal size) × s (vertical size) satisfy the following relationship: a0 = r × T%, b0 = s × U%.

[0102] By determining the position coordinates of the material image on the target image, the material image can be positioned on the target image. Further, the material image is placed on the target image and the outer edge of the material image is determined to be within the target image based on the size information of the material image. Figure 8 (c) illustrates a case where the outer edge of the material image 31 is outside the target image 200. Figure 8 (f) illustrates a case where the outer edge of the material image 32 is located within the target image 200 .

[0103] When the outer edge of the material picture 32 is located within the target picture 200, the material picture 32 is overlaid on the target picture 200 at the corresponding position coordinates to generate a composite picture 22 corresponding to the material picture 32, such as Figure 8 As shown in (f).

[0104] Fig. 9 The detailed implementation flowchart of operation S13 according to another embodiment of the present disclosure is schematically shown. Fig.10 The schematic diagram shows an implementation scenario of operation S13 according to another embodiment of the present disclosure, wherein (a) is a material picture, (b) is a scenario for determining the position coordinates of the material picture in the target picture, (c) is a scenario in which the material picture is placed at the corresponding position coordinates of the target picture and the outer edge of the material picture is outside the target picture, (d) is a scenario in which the material picture is reduced so that the outer edge of the material picture is within the target picture, and (e) is a scenario in which the reduced material picture is combined with the target picture to obtain a combined picture.

[0105] According to another embodiment of the present disclosure, referring to Fig. 9 As shown, the operation S13 of synthesizing the target picture selected by the user with the M material pictures respectively based on the information of the M material pictures to generate a synthesized picture corresponding to the N material pictures includes the following sub-operations: S131b, S132b and S133b.

[0106] In sub-operation S131b, position coordinates of the M material pictures within the target picture are determined according to the preset position information of the M material pictures and the size of the target picture.

[0107] In sub-operation S132b, the reduction ratios of the M material pictures are determined according to the position coordinates, the size information of the M material pictures, and the size of the target picture, so that the outer edges of the M material pictures are all located within the target picture.

[0108] In sub-operation S133b, the reduced M material pictures are overlaid on the target picture at corresponding position coordinates to generate a composite picture corresponding to the M material pictures.

[0109] Reference Fig.10As shown in (a), the material picture 31 is shown, and the dotted frame shows the length×width size of the material picture. The information of the material picture includes: preset position information and size information.

[0110] Fig.10 (b) shows the situation of determining the position coordinates of the material image 31 in the target image 200 according to the preset position information of the material image 31 and the size of the target image 200, and the curved arrow shows the process of determining the position coordinates of the specific position point of the material image on the target image. The position coordinates (a0, b0) of the specific position point in the image to be synthesized with a size of r×s satisfy the following relationship: a0=r×T%, b0=s×U%.

[0111] By determining the position coordinates of the material image on the target image, the material image can be positioned on the target image. Further, the material image is placed on the target image and the outer edge of the material image is determined to be within the target image based on the size information of the material image. Fig.10 (c) illustrates a case where the outer edge of the material image 31 is outside the target image 200. In this case, the material image 31 needs to be reduced so that the outer edge of the material image 31 is within the target image. Fig.10 The reduction ratio can be set according to actual needs. Generally speaking, the reduction ratio is to reduce the material image whose outer edge has exceeded the target image to a suitable preset value that can enter the interior of the target image 200. It is not necessary to reduce it too small, so as to avoid the subsequent similarity calculation due to excessive reduction, which will affect the material image from entering the list of composite images to be displayed.

[0112] In the case where the outer edge of the original material image 31 is located within the target image, there is no need to reduce it, and the reduction ratio in this case is 1.

[0113] Reference Fig.10 As shown in (e), the reduced material image is overlaid on the target image at the corresponding position coordinates. Since all the material images can generate a composite image after being reduced, a composite image corresponding to M material images can be generated, which corresponds to the situation of N=M.

[0114] Fig.11 The detailed implementation flow chart of operation S14 according to the embodiment of the present disclosure is schematically shown.

[0115] According to the embodiments of the present disclosure, referring to Fig.11 As shown, the operation S14 of determining the list of synthesized pictures to be displayed according to the similarity between the synthesized pictures and the target pictures includes the following sub-operations: S141, S142 and S143.

[0116] In sub-operation S141, it is determined whether the similarity between the synthesized picture and the target picture falls within a threshold interval.

[0117] In sub-operation S142, the synthesized pictures whose similarities fall within the threshold range are used as synthesized pictures to be displayed for selection by the user.

[0118] The threshold interval can be adjusted according to actual needs. For example, the threshold interval is set to 80% to 90%.

[0119] In sub-operation S143, a list of composite images to be displayed is generated based on all the composite images to be displayed. The list of composite images to be displayed can be displayed in order of similarity, and the list of composite images to be displayed can have multiple pages for display, for example, 5 to 10 images can be placed on one page for display.

[0120] A second exemplary embodiment of the present disclosure provides a device for performing material matching on a picture.

[0121] Fig.12 The structural block diagram of a device for performing material matching for a picture according to an embodiment of the present disclosure is schematically shown.

[0122] Reference Fig.12 As shown, the above-mentioned device 4 for performing material matching on pictures includes: a matching request receiving module 41, a target picture and material picture information obtaining module 42, a picture synthesis module 43 and a synthesis picture list determining module 44.

[0123] The matching request receiving module 41 is used to receive a material matching request.

[0124] The target picture and material picture information acquisition module 42 is used to acquire information of the target picture selected by the user and M material pictures in a predetermined material library, where M≥2 and M is a positive integer.

[0125] The picture synthesis module 43 is used to synthesize the target picture with the M material pictures respectively based on the information of the M material pictures to generate a synthesized picture corresponding to the N material pictures, 1≤N≤M, and N is a positive integer. The above-mentioned picture synthesis module 43 may have a sub-functional module for implementing sub-operations S131a, S132a, and S133a, and / or, a sub-functional module for implementing sub-operations S131b, S132b, and S133b.

[0126] The composite picture list determination module 44 is used to determine a composite picture list to be displayed according to the similarity between the composite picture and the target picture for the user to select. The composite picture list determination module 44 may have sub-function modules for implementing sub-operations S141 to S143.

[0127] Fig.13 The structural block diagram of a device for performing material matching for a picture according to another embodiment of the present disclosure is schematically shown.

[0128] Reference Fig.13 As shown, in another embodiment, the above-mentioned device 4 includes, in addition to the above-mentioned matching request receiving module 41, target picture and material picture information obtaining module 42, picture synthesis module 43 and synthesis picture list determination module 44, further includes: a predetermined material library determination module 40. The predetermined material library determination module 40 is used to determine the predetermined material library.

[0129] According to the embodiments of the present disclosure, referring to Fig.13 As shown, the above-mentioned device 4 also includes: a material picture usage statistics module 45.

[0130] The material picture usage statistics module 45 is used to count the usage times of the material picture corresponding to the specific composite picture when the user selects the specific composite picture from the list of composite pictures to be displayed.

[0131] According to an embodiment of the present disclosure, the above-mentioned predetermined material library determination module 40 is further used to update the predetermined material library according to the change in the number of times the material picture corresponding to the specific composite picture is used.

[0132] According to the embodiments of the present disclosure, referring to Fig.13 As shown, the above device 4 further includes: a remaining material library determining module 46 .

[0133] The remaining material library determination module 46 is used to determine the remaining material library from the initial material library when the user does not select any picture in the list of composite pictures to be displayed. The above-mentioned remaining material library is a material library composed of the remaining materials obtained after the initial material library removes all material pictures corresponding to the composite pictures in the list of composite pictures to be displayed.

[0134] According to an embodiment of the present disclosure, the target picture and material picture information acquisition module 42 is also used to acquire information of material pictures in the remaining material library.

[0135] The above-mentioned picture synthesis module 43 is also used to synthesize the target picture with the material pictures in the remaining material library based on the information of the material pictures in the remaining material library to generate an alternative synthesized picture.

[0136] The above-mentioned composite picture list determination module 44 is also used to determine a list of candidate composite pictures to be displayed according to the similarity between the candidate composite pictures and the target picture, so as to provide selection for the user.

[0137] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0138] For example, any multiple of the matching request receiving module 41, the target image and material image information acquisition module 42, the image synthesis module 43, and the synthetic image list determination module 44 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the matching request receiving module 41, the target image and material image information acquisition module 42, the image synthesis module 43, and the synthetic image list determination module 44 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the matching request receiving module 41, the target image and material image information acquisition module 42, the image synthesis module 43 and the synthesized image list determination module 44 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0139] A third exemplary embodiment of the present disclosure provides an electronic device. The electronic device includes: one or more processors; and a storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described above.

[0140] Fig.14 The structural block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown.

[0141] Reference Fig.14 As shown, the electronic device 5 according to the embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0142] In RAM 503, various programs and data required for the operation of electronic device 5 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0143] According to an embodiment of the present disclosure, the electronic device 5 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 5 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a local area network (LAN) card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0144] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0145] A fourth exemplary embodiment of the present disclosure provides a computer-readable storage medium, wherein executable instructions are stored on the computer-readable storage medium, and when the instructions are executed by a processor, the processor implements any of the methods described above.

[0146] The computer-readable storage medium may be included in the device / apparatus / system described in the above embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0147] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0148] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0149] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0150] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for matching materials to an image, comprising: Receiving a material matching request; Obtain information of the target image selected by the user and M material images in the predetermined material library, where M≥2 and M is a positive integer; Based on the information of the M material pictures, the target picture and the M material pictures are respectively synthesized to generate a synthesized picture corresponding to the N material pictures, 1≤N≤M, N is a positive integer, wherein the synthesis is based on the position coordinates of the material pictures in the target picture, the size information of the material pictures and the size of the target picture; as well as The list of synthesized images to be displayed is determined based on the similarity between the synthesized image and the target image, so that the user can choose.

2. The method according to claim 1, further comprising: Determining a predetermined material library, wherein determining the predetermined material library comprises: Determine the first pending material library according to the popularity ranking of the number of times the material images in the initial material library are used within a predetermined time range; Determine the second pending material library according to the month-on-month growth rate of the number of times the material images in the initial material library are used in two consecutive statistical periods; and A predetermined material library is determined according to the first pending material library and the second pending material library.

3. The method according to claim 2, wherein: Also includes: If the user selects a specific composite image from the list of composite images to be displayed, the number of times the material image corresponding to the specific composite image is used is increased by 1; updating the first pending material library and the second pending material library according to the change in the number of times the material picture corresponding to the specific composite picture is used, so as to update the predetermined material library; as well as After receiving a new target picture selected by the user and a material matching request next time, the updated material picture in the predetermined material library is synthesized with the new target picture.

4. The method according to claim 1, wherein: Also includes: If the user does not select any picture in the list of composite pictures to be displayed, a remaining material library is determined from the initial material library, wherein the remaining material library is a material library composed of remaining materials obtained after the initial material library removes all material pictures corresponding to the composite pictures in the list of composite pictures to be displayed; Obtaining information of material pictures in the remaining material library; Based on the information of the material pictures in the remaining material library, the target picture is synthesized with the material pictures in the remaining material library to generate an alternative synthesized picture; as well as A list of candidate composite pictures to be displayed is determined according to the similarity between the candidate composite pictures and the target picture, for the user to select.

5. The method according to claim 1, wherein: The information of the material picture includes: preset position information and size information, and the preset position information includes: the horizontal position ratio T% and the vertical position ratio U% of the specific position point of the material picture relative to the picture to be synthesized; The synthesizing the target image selected by the user and the M material images respectively based on the information of the M material images to generate a synthesized image corresponding to the N material images includes: Determining the position coordinates of the M material pictures within the target picture according to the preset position information of the M material pictures and the size of the target picture; Determining whether the outer edges of the M material pictures are located within the target picture according to the position coordinates, the size information of the M material pictures, and the size of the target picture; and If the outer edge of the material picture is located within the target picture, the material picture is overlaid on the target picture at the corresponding position coordinates to generate a composite picture corresponding to the N material pictures.

6. The method according to claim 1, wherein: The information of the material picture includes: preset position information and size information, and the preset position information includes: the horizontal position ratio T% and the vertical position ratio U% of the specific position point of the material picture relative to the picture to be synthesized; The synthesizing the target image selected by the user and the M material images respectively based on the information of the M material images to generate a synthesized image corresponding to the N material images includes: Determining the position coordinates of the M material pictures within the target picture according to the preset position information of the M material pictures and the size of the target picture; Determining reduction ratios of the M material pictures according to the position coordinates, size information of the M material pictures, and the size of the target picture, so that outer edges of the M material pictures are all located within the target picture; and Overlay the M material pictures on the target picture at corresponding position coordinates to generate a composite picture corresponding to the M material pictures.

7. The method according to claim 1, wherein: The step of determining a list of synthesized images to be displayed according to the similarity between the synthesized image and the target image for selection by the user includes: Determine whether the similarity between the synthesized image and the target image falls within a threshold range; The composite images whose similarity falls within the threshold range are used as composite images to be displayed for selection by the user; and A list of composite images to be displayed is generated based on all composite images to be displayed.

8. A device for matching materials to images, comprising: A matching request receiving module, used for receiving a material matching request; The target image and material image information acquisition module is used to acquire the information of the target image selected by the user and M material images in the predetermined material library, where M≥2 and M is a positive integer; a picture synthesis module, configured to synthesize the target picture with the M material pictures respectively based on information of the M material pictures, so as to generate a synthesized picture corresponding to the N material pictures, 1≤N≤M, N being a positive integer, wherein the synthesis is performed based on the position coordinates of the material pictures in the target picture, the size information of the material pictures and the size of the target picture; and The composite image list determination module is used to determine the composite image list to be displayed according to the similarity between the composite image and the target image, so as to provide selection for the user.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 7.

Images