Image processing method and device, computer device, and storage medium

Abstract

Embodiments of the present application disclose an image processing method and device, computer equipment and a storage medium. The image processing method comprises: displaying multimedia data; in response to a requirement of adding a border to the multimedia data, obtaining a target border, the target border being a pattern type border generated according to a pattern of the multimedia data, or the target border being a map type border generated according to a theme of the multimedia data; and adding the target border to the multimedia data. The present application can enrich the way of adding a border to an image and ensure the decoration effect of the border on the image.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] In image publishing scenarios, an increasing number of users are using borders as decorations to enhance the images to be published and add visual interest. The specific process for using borders is as follows: after the user selects the image to be published, the terminal device randomly selects one from multiple preset borders as the decorative border, and then uses the selected border to decorate the image to be published.

[0003] Randomly selecting borders by the terminal device results in a limited way of adding borders to images; furthermore, a randomly selected border that does not match the image to be published will also affect the decorative effect of the border on the image. Summary of the Invention

[0004] This application provides an image processing method, apparatus, computer device, storage medium, and computer program product, which can enrich the ways to add borders to images and ensure the decorative effect of borders on images.

[0005] One embodiment of this application provides an image processing method, including:

[0006] Display multimedia data;

[0007] In response to the need to add a border to the multimedia data, a target border is obtained, which is either a patterned border generated based on the pattern of the multimedia data, or a textured border generated based on the theme of the multimedia data.

[0008] Add the target border to the multimedia data.

[0009] One embodiment of this application provides an image processing apparatus, including:

[0010] The display module is used to display multimedia data;

[0011] The acquisition module is used to acquire a target border in response to the requirement to add a border to the multimedia data. The target border is a patterned border generated based on the pattern of the multimedia data, or the target border is a textured border generated based on the theme of the multimedia data.

[0012] An add module is used to add the target border to the multimedia data.

[0013] One aspect of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the methods described in the above embodiments.

[0014] One aspect of this application provides a computer storage medium storing a computer program, which includes program instructions. When the program instructions are executed by a processor, they perform the methods described in the above embodiments.

[0015] One aspect of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. When the computer instructions are executed by the processor of a computer device, the methods described in the above embodiments are performed.

[0016] When it is necessary to add borders to multimedia data, the terminal device will automatically generate borders based on the patterns or themes of the multimedia data, and then add borders to the multimedia data, which can enrich the ways in which the terminal device adds borders to images. Automatic border generation does not require the preparation of multiple alternative borders in advance, which can improve the diversity of user content at low cost. Furthermore, the generated borders are generated based on the patterns or themes of the multimedia data, which can make the generated borders coordinated with the multimedia data, ensure the decorative effect of the borders on the multimedia data, and make the border adding process intelligent. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a system architecture diagram of image processing provided by an embodiment of the present invention;

[0019] Figures 2a-2e This is a schematic diagram of an image processing scenario provided by an example of this application;

[0020] Figure 3 This is a schematic diagram of an image processing embodiment provided in this application;

[0021] Figures 4a-4d This is a schematic diagram illustrating the addition of a patterned border according to an embodiment of this application;

[0022] Figures 5a-5dThis is a schematic diagram illustrating the addition of a textured border according to an embodiment of this application;

[0023] Figures 6a-6d This is a schematic diagram illustrating the addition of a textured border according to an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of an image processing flow provided in an embodiment of this application;

[0025] Figures 8a-8c This is a schematic diagram illustrating the addition of a patterned border according to an embodiment of this application;

[0026] Figure 9 This is a schematic diagram illustrating a process for adding a patterned border according to an embodiment of this application;

[0027] Figure 10 This is a schematic diagram of a patterned border synthesis device provided in an embodiment of this application;

[0028] Figure 11 This is a schematic diagram of an image processing flow provided in an embodiment of this application;

[0029] Figure 12 This is a schematic diagram illustrating a process for adding a textured border according to an embodiment of this application;

[0030] Figure 13 This is a schematic diagram of a texture-type border compositing device provided in an embodiment of this application;

[0031] Figure 14 This is a schematic diagram of a texture-type border compositing device provided in an embodiment of this application;

[0032] Figure 15 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0033] Figure 16 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0035] Please see Figure 1This is a system architecture diagram for image processing provided in an embodiment of the present invention. Server 10f establishes a connection with a cluster of terminal devices via switch 10e and communication bus 10d. The cluster of terminal devices may include: terminal device 10a, terminal device 10b, ..., terminal device 10c. Taking terminal device 10a as an example, terminal device 10a displays multimedia data for which a border needs to be added. When a request to add a border to the multimedia data is received, terminal device 10a obtains the target border and adds the target border to the multimedia data. The target border is either a pattern-type border generated in real-time by terminal device 10a based on the pattern of the multimedia data, or a texture-type border generated in real-time based on the theme of the multimedia data.

[0036] Of course, the step of generating the target border can also be performed by the server 10f. That is, the server 10f generates the target border based on the pattern of the multimedia data or the theme of the multimedia data, and then the server 10f sends the target border to the terminal device 10a, and then the terminal device 10a adds the target border to the multimedia data.

[0037] Terminal equipment can also be referred to as terminal, user equipment (UE), access terminal, user unit, mobile device, user terminal, wireless communication equipment, user agent, or user device. Terminal equipment can be smart home appliances, handheld devices with wireless communication capabilities (such as smartphones and tablets), computing devices (such as personal computers (PCs), in-vehicle terminals, smart voice interaction devices, wearable devices, or other smart devices, but is not limited to these.

[0038] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0039] This application can be applied to various multimedia data editing scenarios (e.g., image editing scenarios, video editing scenarios, etc.). By generating exclusive borders for different multimedia data, it enhances the matching effect of thousands of images and styles, improves the decorative effect of borders, and increases the fun of the multimedia data editing process.

[0040] The following are Figures 2a-2e For example, this section explains in detail how to generate custom borders for different multimedia data.

[0041] Please see Figures 2a-2e This is a schematic diagram of an image processing scenario provided in this application. For example... Figure 2a As shown, Figure 2a The middle page is an image selection page, where users can choose images to which borders will be added. For example... Figure 2b As shown, after selecting an image (the currently selected image is a soccer jersey), the user can click the "Next" button on the page. After clicking the "Next" button, the following will be displayed. Figure 2c The page shown displays the image selected by the user and two editing tabs: an "Add Text" tab for adding text to the image and an "Add Border" tab for adding a border to the image (e.g., ...). Figure 2c (As shown).

[0042] When a user clicks the "Add Border" tab, the terminal device adds a border to the image selected by the user. Figure 2d As can be seen, the image contains a vertical stripe pattern, and the border also features a vertical stripe pattern. In other words, the border is generated based on the pattern contained in the image, and the border's pattern is similar to or the same as the pattern in the image. Generating custom border patterns by using different patterns contained in different images ensures consistency between the border and the image, increasing the decorative effect of the border and enhancing the fun of the image editing process.

[0043] Furthermore, not only are the border patterns similar to or the same as the image patterns, but the border patterns also match the image patterns in terms of color tone, making the borders and images more consistent overall.

[0044] If the user is not satisfied with the currently added vertical stripe border, they can change the border. For example... Figure 2d As shown, when a user clicks the "Replace" button on the page, a new border is added to the image (e.g., ...). Figure 2e (As shown). From Figure 2e As you can see, the new border's background doesn't contain any patterns, but it carries three textures (two soccer ball textures and one sun texture). Since the image content is a soccer jersey, the three textures are related to the "soccer" theme; in other words, the border is generated based on the image's theme. By generating custom texture borders for different images containing different themes, the border and image theme are matched perfectly, enriching the ways to add borders to images and enhancing the matching effect of "one thousand images, one style."

[0045] Furthermore, not only do the textures carried by the border match the image theme, but the background color of the border can also match the color of the image, making the texture border and the image more consistent overall.

[0046] The specific process of displaying multimedia data (such as the jersey image in the above embodiment), obtaining the target border (such as the vertical striped pattern border or the soccer ball texture border in the above embodiment), and adding the target border to the multimedia data can be found in the following description. Figures 3-14 Corresponding implementation examples.

[0047] Please see Figure 3 This is a schematic diagram of image processing provided in an embodiment of this application. Since this application involves user-side visualization, the following steps are described with a terminal device (e.g., smartphone, tablet, smart bracelet, etc.) as the execution subject. Image processing may include the following steps:

[0048] Step S101: Display multimedia data.

[0049] Specifically, the terminal device displays multiple multimedia data to be selected in the multimedia data selection area of ​​the display interface. The multimedia data can be images, music, animations, videos, etc.

[0050] Users can select the multimedia data to which a border will be added from these multiple multimedia data options (as described above). Figures 2a-2e (As shown in the jersey image in the corresponding embodiment), the terminal device displays the selected multimedia data in the image display area.

[0051] Step S102: In response to the need to add a border to the multimedia data, a target border is obtained. The target border is a patterned border generated based on the pattern of the multimedia data, or a textured border generated based on the theme of the multimedia data.

[0052] Specifically, the terminal device has a border type selection control on the display interface (as mentioned above). Figures 2a-2e (As shown in the "Add Border" label in the corresponding embodiment), when the border type selection control is triggered by the user, the terminal device receives a request to add a border to the multimedia data. In response to this request, the terminal device generates a corresponding border (referred to as the target border, as described above) based on the pattern in the multimedia data. Figures 2a-2e In the corresponding embodiment, the border is a vertical stripe pattern; or, the terminal device generates a corresponding target border based on the theme of the multimedia data (as described above). Figures 2a-2e In the corresponding embodiment, the texture border carries both a soccer ball texture and a sun texture.

[0053] If the target border is generated based on a pattern from the multimedia data, then the target border is a patterned border; if the target border is generated based on a theme from the multimedia data, then the target border is a textured border. Patterned borders and textured borders are distinguished by their shape; patterned borders have a background pattern, while textured borders carry a texture.

[0054] The subject matter of multimedia data can include both biometrics and semantics. Biometrics can include one or more of the following: species (human, cat, dog, etc.), facial features (eyes, nose), expression, face shape, and hairstyle. The semantics of multimedia data can include both the directly expressed semantics and the indirectly expressed semantics. Directly expressed semantics refer to the core content of the multimedia data, while indirectly expressed semantics refer to its extended meanings. For example, if the multimedia data is an image of a puppy running on a lawn, the subject matter could include: dog, lawn, vitality, exercise, animal welfare, etc.

[0055] Border type selection controls can include pattern selection controls and texture selection controls. If the user triggers a pattern selection control, the generated target border will be a pattern border; conversely, if the user triggers a texture selection control, the generated target border will be a texture border.

[0056] Pattern-based selection controls also include descriptive text for pattern-based borders, and texture-based selection controls also include descriptive text for texture-based borders. When a user triggers a pattern-based selection control or a texture-based selection control, they can roughly understand which type of border the control will generate by looking at the descriptive text on the control.

[0057] For example, the descriptive characters on pattern-type selection controls can be: checkered border, striped border, houndstooth border, etc.; the descriptive characters on sticker-type selection controls can be: EMOJI border, emoticon sticker border, etc., where EMOJI refers to visual emotional symbols.

[0058] In addition to receiving a request to add a border to multimedia data by having the user trigger a border type selection control, the terminal device can also receive a request to add a border to the multimedia data by detecting whether a preset operation exists in the multimedia data display interface. If a preset operation exists, the terminal device can also receive a request to add a border to the multimedia data. Preset operations can be: long press, five-finger pinch, etc.

[0059] In addition to receiving a request to add a border to multimedia data by having the user trigger a border type selection control or by having the user perform a preset operation on the display interface, the terminal device can also detect whether it is currently in border-adding mode. If it is in border-adding mode, the terminal device will receive a request to add a border to the multimedia data. For example, once the user selects the multimedia data to be bordered and enters border-adding mode, the terminal device can receive a request to add a border to the multimedia data in real time.

[0060] Step S103: Add the target border to the multimedia data.

[0061] Specifically, the terminal device adds a target border to the multimedia data, that is, displays the multimedia data and the target border together on the display interface.

[0062] Optionally, to enhance the display effect of the target border, the terminal device can determine the special effects display method (called the target special effects display method) based on the pattern or theme of the multimedia data. Subsequently, the terminal device can add a target border to the multimedia data according to the target special effects display method.

[0063] For example, you can add a target border to multimedia data using a slow-entry effect or a fan-shaped expansion effect.

[0064] Optionally, if the user is not satisfied with the target border, they can replace it with another border to decorate the multimedia data. The specific process is as follows:

[0065] The multimedia data display interface also includes a replacement control (as described above). Figures 2a-2e (Replace button in the corresponding embodiment) When the user triggers the replacement control, the terminal device receives and responds to the request to replace the target border. It randomly selects one border from p-1 borders (excluding the target border) as the replacement border, where p is a positive integer, and further, p is an integer greater than or equal to 2. It can be seen that both the target border and the replacement border are one of the p borders, but they are two different borders. Any border among the p borders is either a pattern-type border generated based on the pattern of the multimedia data, or a texture-type border generated based on the theme of the multimedia data.

[0066] Of course, besides randomly selecting a replacement border from p-1 borders, the replacement border can also be selected based on whether the target border is a patterned border or a textured border. If the target border is a patterned border, the selected replacement border will also be a patterned border; conversely, if the target border is a textured border, the selected replacement border will also be a textured border. In short, the border type of the target border and the border type of the replacement border must be the same.

[0067] After the terminal device obtains the replacement border, it replaces the target border added for the multimedia data with the obtained replacement border. Of course, the target border can also be replaced with the replacement border according to certain special effects display methods.

[0068] Optionally, as mentioned above, the target border can be obtained after the user triggers the pattern selection control. In this case, each of the p borders can be a pattern border generated based on the pattern of the multimedia data. Conversely, the target border can also be obtained after the user triggers the image selection control. In this case, each of the p borders can be an image border generated based on the main image of the multimedia data.

[0069] In addition to the "one-click random border replacement" described above, users can also choose the border they want to replace themselves. The specific process is as follows:

[0070] The multimedia data display interface also has p border content selection controls. Each border content selection control corresponds to one border, and the icon of the border content selection control matches (or is consistent with) the corresponding border. This allows the user to preview the border's style, color, and other border details by looking at the icon of the border content selection control before triggering it.

[0071] When a user triggers one of the p border content selection controls (referred to as the target border content selection control), the terminal device receives and responds to the request to replace the target border, obtains the border corresponding to the target border content selection control, and uses the obtained border as the replacement border. Then, the terminal device replaces the target border added for multimedia data with the obtained replacement border. Alternatively, the target border can be replaced with the replacement border according to certain special effects display methods.

[0072] Please see Figures 4a-4d This is a schematic diagram of adding a patterned border according to an embodiment of this application, such as... Figure 4a As shown, the user has selected the image to which the border will be added and can click the "Next" button to display... Figure 4b The page shown. In Figure 4b The page displays an image to be bordered, along with a "Striped Border" button 40a. This "Striped Border" button corresponds to the pattern selection control in this application, and the four characters "Striped Border" correspond to the descriptive characters on the pattern selection control, allowing users to roughly understand the shape of the border. Furthermore, the descriptive characters on the pattern selection control describe a patterned border of the target pattern type. After the user clicks the "Striped Border" button 40a, the following is displayed: Figure 4c The page shown. From Figure 4c As can be seen, because the image contains vertical stripes, the border added to the image is also a patterned border with a vertical stripe pattern. Furthermore, since the main color of the image is black, the generated patterned border with vertical stripes is also black, which makes the border coordinate with the overall image.

[0073] In addition to displaying the image and the border added to it, the page also includes three border content selection controls (namely: border content selection control 40b, border content selection control 40c, and border content selection control 40d). From Figure 4c As can be seen, the icon of each border content selection control matches the corresponding border, and the border currently being added to the image is the border corresponding to border content selection control 40b. Since the user wants a striped border, all three replaceable borders have striped patterns, and each replaceable border is black. The user can understand the specific style of the border by looking at the icon of the border content selection control. When the user clicks border content selection control 40d, as shown... Figure 4d As shown, the patterned border of the original vertical stripe pattern added to the image is adjusted to a patterned border of horizontal stripes.

[0074] Please see Figures 5a-5d This is a schematic diagram of adding a textured border according to an embodiment of this application, such as... Figure 5a As shown, the user has selected the image to which the border will be added and can click the "Next" button to display... Figure 5b The page shown. In Figure 5b The page displays an image to be bordered, along with a "Patch Border" button 50a. This "Patch Border" button 50a corresponds to the patch selection control in this application, and the four characters "Patch Border" correspond to the descriptive characters on the patch selection control, allowing users to roughly understand the shape of the border through the descriptive characters. After the user clicks the "Patch Border" button 50a, the following is displayed: Figure 5c The page shown. From Figure 5c As can be seen, since the image's subject includes dogs, the border added to the image carries puppy and heart textures that match the "dog" theme. Furthermore, because the image's main color contains gray, the background area of ​​the generated texture-based border is also gray.

[0075] In addition to displaying the image and the border added to it, the page also includes three border content selection controls (50b, 50c, and 50d) and a "Replace" button. Figure 5cAs can be seen, the icons of each border content selection control match (or are identical to) the corresponding borders, and the border currently being added to the image is the border corresponding to border content selection control 50b. Users can understand the specific style of the border by looking at the icons of the border content selection controls. When the user clicks the "Replace" button, the terminal device will adjust the original image border with puppy and heart stickers to a border with footprint and heart stickers. Of course, the user can also click border content selection control 50d, which will also adjust the image border with puppy and heart stickers to a border with footprint and heart stickers.

[0076] Please see Figures 6a-6d This is a schematic diagram of adding a textured border according to an embodiment of this application, such as... Figure 6a As shown, the user has selected the image to which the border will be added and can click the "Next" button to display... Figure 6b The page shown. In Figure 6b The page displays an image to be bordered, along with a "Apply Border" button. After the user clicks the "Apply Border" button, the following is displayed: Figure 6c The page shown. From Figure 6c As can be seen, since the image's subject contains biometric features—a boy, smiling—the texture carried by the border added to the image is a smiling emoji texture that matches the subject "smiling." Furthermore, since the image's main color contains gray, the background area of ​​the generated texture-based border is also gray.

[0077] In addition to displaying the image and the border added to it, the page also includes two border content selection controls (border content selection control 60b and border content selection control 60c) and a "Replace" button. Figure 6c As you can see, the icon of each border content selection control matches the corresponding border (matching here means similar or identical), and the border currently being added to the image is the border corresponding to border content selection control 60b. Users can better understand the specific style of the border by looking at the icons of the border content selection controls. When the user clicks the "Replace" button, the original border with a smiley face sticker will be changed to a border with a boy sticker (the boy sticker matches the image theme "boy"). Of course, the user can also click border content selection control 60c, which will also change the border with a smiley face sticker to a border with a boy sticker.

[0078] As mentioned above, by triggering a pattern-type selection control or a texture-type selection control, a target border of corresponding shape can be generated, which can enrich the ways of generating borders; furthermore, users can receive the request to add borders to multimedia data through various operations, which can increase the application scenarios for adding borders; this application also provides two ways to replace borders, which can increase the fun of the border adding process for users and enhance the stickiness of users using borders.

[0079] Please see Figure 7 , Figure 7 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. This embodiment mainly illustrates how to generate a patterned border. The image processing includes the following steps:

[0080] Step S201: Display multimedia data.

[0081] The specific process of step S201 can be found above. Figure 3 The corresponding step S101 in the embodiment.

[0082] Step S202: In response to the requirement to add a border to the multimedia data, target detection is performed on the multimedia data to obtain the target region, and the image of the target region is extracted from the multimedia data to obtain the target region image.

[0083] Specifically, when the pattern selection control is triggered, the terminal device responds to the request to add a patterned border to the multimedia data by performing object detection on the multimedia data to obtain the target region. The purpose of object detection is to find the pattern in the multimedia data. If the multimedia data is an image, then the target region is the area in the image that contains the pattern; if the multimedia data is video, then the target region is the area in the video frame that contains the pattern. The terminal device extracts the image containing the target region from the multimedia data and uses the extracted image as the target region image.

[0084] Step S203: Select a target pattern from k preset patterns according to the pattern style of the target area image.

[0085] Specifically, the terminal device detects whether the pattern in the target area image belongs to the target pattern type. The target pattern type can be a checkered pattern, a striped pattern, a maple leaf pattern, a houndstooth pattern, etc. If the pattern in the target area image belongs to the target pattern type, the terminal device segments the target area image, and the segmented image is called a segmentation unit image. The similarity between the segmentation unit image and each of the k preset patterns is calculated, and the m patterns with the highest similarity are all taken as target patterns, where m is less than or equal to k, and each preset pattern belongs to the target pattern type.

[0086] The reason for segmenting the target region image is that, generally speaking, the patterns in an image are composed of multiple identical patterns stacked together. Segmenting the image is to extract the basic patterns from the target region image in order to calculate the similarity between the basic patterns and k preset patterns, which can improve the accuracy of the calculated similarity.

[0087] It should be noted that the preset pattern is a vector graphic (i.e., the preset pattern is a grayscale pattern without color information, but the segmentation unit image contains color information). Therefore, the similarity between the segmentation unit image and the preset pattern refers to the similarity between the pattern style of the segmentation unit image and the pattern style of the preset pattern.

[0088] If the pattern in the target area image does not belong to the target pattern type, the terminal device can randomly select m patterns from k preset patterns as the target patterns.

[0089] In general, the target pattern is selected from k preset patterns based on the pattern pattern of the target region image (or the pattern pattern of the segmented unit image).

[0090] When m=1, the pattern that is most similar to the pattern of the target area image is selected from k preset patterns and used as the target pattern.

[0091] Step S204: Extract the theme color of the multimedia data, fill the target pattern with color according to the theme color of the multimedia data to obtain n borders, and select the target border from the n borders.

[0092] Specifically, after acquiring the target pattern, the terminal device extracts the theme color of the multimedia data. The theme color can be either a global theme color or a pattern theme color. A global theme color refers to the color that occupies a large proportion of the entire multimedia data, while a pattern theme color refers to the color that occupies a large proportion of the pattern within the multimedia data. For example, if the pattern in the target area image belongs to the target pattern type, the theme color is the pattern theme color, meaning the color that occupies a large proportion of the pattern in the multimedia data; if the pattern in the target area image does not belong to the target pattern type, the theme color is the global theme color, meaning the color that occupies a large proportion of the multimedia data. The terminal device fills the target pattern with the theme color of the multimedia data and combines the filled target patterns to obtain n background materials. Since the final generated borders need to be combined with the multimedia data for display, it is also necessary to determine the display area of ​​the multimedia data in the background materials and set a floating layer in this area to obtain n borders. These n borders are generated based on the pattern in the multimedia data. The display area in the border materials can be located in the middle, top, bottom, or any position between the bottom and bottom positions.

[0093] The extracted multimedia data can contain one or more theme colors. It can fill the target pattern with all theme colors, or with only some theme colors, or by filling the target pattern with combinations of theme colors. For example, if the extracted theme colors are red, black, and green, you can fill the target pattern with all three colors, any two colors (there are three possible combinations), or each color can be filled separately.

[0094] Optionally, the number of theme colors to fill the target pattern can be determined based on the colors required for that target pattern. For example, if the target pattern requires two colors to fill, and the number of extracted theme colors is three, then two theme colors can be randomly selected from the three theme colors (there are three possible combinations) to fill the target pattern. If the number of target patterns is one, and there are three color combinations, then the target pattern filled with three color combinations can be obtained. Of course, the pattern style of these three target patterns filled with color combinations is the same, only the colors are different.

[0095] After the terminal device acquires n borders, it selects a target border from the n borders.

[0096] If multiple target patterns are selected based on the pattern style of the target region image, the borders generated by these multiple target patterns can be sorted according to their similarity (of course, the borders generated by target patterns with greater similarity will be sorted higher), and the border that is ranked first will be used as the target border. Subsequently, when it is necessary to replace the target border, the replacement border can also be determined according to the sorting, for example, the border that is ranked second can be used as the replacement border, and so on.

[0097] If there is only one target pattern and multiple theme colors, multiple borders (with the same pattern style but different colors) can be generated based on the combination of multiple theme colors. Therefore, the multiple borders can be sorted according to the proportion of these multiple theme colors in the multimedia data, and then the target border can be selected and the border can be replaced based on the sorting result.

[0098] Please see Figures 8a-8c This is a schematic diagram illustrating the addition of a patterned border according to an embodiment of this application. Figures 8a-8cAs shown, the image currently used to add a border is a puppy image, and the user has selected the "Striped Border" button (i.e., a pattern selection control). Since the puppy image's pattern does not belong to the target pattern type (the target pattern type is striped), the terminal device randomly selects the target pattern from k preset patterns (all k preset patterns belong to the striped type) and fills it with color to generate the border. For example... Figure 8c As shown, an image and a striped border added to the image are displayed in the middle of the page. At the bottom of the page, three border content selection controls are also displayed. Through these three border content selection controls, we can see that the randomly selected target pattern is: vertical stripe pattern, diagonal stripe pattern, and horizontal stripe pattern. Since the main color of the image is gray, the randomly selected target pattern is also filled with gray.

[0099] Optionally, the preceding description outlines how to select a target pattern from k preset patterns to generate a border. Besides selecting the target pattern from preset patterns, the target pattern can also be generated in real-time based on the patterns in the multimedia data, eliminating the need for upfront investment in creating preset patterns. The specific process is as follows:

[0100] Similarly, target detection is performed on the multimedia data, and the target region containing the image is extracted. The image containing the target region is extracted from the multimedia data and used as the target region image. The target pattern is extracted from the target region image. Generally, patterns in an image are composed of multiple identical patterns stacked together. The extracted target pattern can be considered as the basic pattern in the target region image. The target pattern can be extracted using machine learning algorithms or based on the brightness differences between pixels in the target region image. The theme color of the multimedia data is also extracted. The terminal device fills the target pattern with the theme color of the multimedia data and then combines the filled target patterns to obtain n background materials. Since the final generated borders need to be combined with the multimedia data for display, the display area of ​​the multimedia data needs to be determined in the background materials. A floating layer is set in this area, resulting in n borders. These n borders are generated based on the patterns in the multimedia data. The display area in the border materials can be located in the middle, top, bottom, or any position between the bottom and bottom positions.

[0101] After the terminal device acquires n borders, it selects a target border from the n borders.

[0102] If there is more than one target pattern generated, the borders generated by these target patterns can be sorted according to the proportion of the target pattern in the multimedia data (of course, the larger the proportion of the target pattern in the multimedia data, the higher the ranking of the border generated by that target pattern), and the border ranked first will be used as the target border. Subsequently, when it is necessary to replace the target border, the replacement border can also be determined according to the ranking, for example, the border ranked second can be used as the replacement border, and so on.

[0103] If there is only one target pattern but multiple theme colors, the terminal device can generate multiple borders (with the same pattern style but different colors) based on the combination of multiple theme colors. Therefore, the multiple borders can be sorted according to the proportion of these multiple theme colors in the multimedia data, and then the target border can be selected and the border can be replaced based on the sorting result.

[0104] Step S205: Add the target border to the multimedia data.

[0105] The specific process of step S205 can be found above. Figure 3 The corresponding step S103 in the embodiment.

[0106] Please see Figure 9 , Figure 9 This is a flowchart illustrating the process of adding a patterned border according to an embodiment of this application. Adding a patterned border mainly involves three process devices: an image subject recognition and segmentation device, an image theme color recognition device, and a patterned border synthesis device. The specific process is as follows:

[0107] In step S301, the user uploads any original image on the front end and selects the smart JK border function. The original image is then sent back to the back end.

[0108] In step S302, the image subject recognition and segmentation device performs preprocessing on the image, such as noise reduction and enhancement.

[0109] In step S303, the image subject recognition and segmentation device segments the preprocessed image to distinguish between the foreground and background of the image.

[0110] In step S304, the image subject recognition and segmentation device extracts features from the segmented image to segment the main body region of the image.

[0111] Step S305: Based on the classification decision, further refine the segmentation of the region image containing the pattern.

[0112] Among these, machine learning can be used to optimize the discrimination rules for classification decisions.

[0113] Step S306: Determine whether a JK grid handle exists in the region image. If it exists, proceed to steps S307-S308, S310, S312-S313, S315-S320. If it does not exist, proceed to steps S309, S311-S312, S314-S320.

[0114] JK plaid is a grid pattern on JK clothing.

[0115] Step S307: Compare the similarity between the region image and the existing grid handle material, and select the grid handle material with the highest similarity for output.

[0116] In step S308, the theme color recognition device extracts the theme color from the grid handle region in the region image.

[0117] In step S309, the theme color recognition device extracts the theme color based on the total number of pixels in the image.

[0118] The theme colors are sorted by the percentage of pixels with that color value in the entire image. They can be deduplicated based on the hue (e.g., if multiple blue color values ​​are extracted, and their transparency ranges are similar, only the one with the highest percentage is taken as the representative color of that hue). The number of theme colors extracted can be customized, but is generally no less than 5.

[0119] Step S310: Call the grid pattern material output in step S307, and randomly fill it with the theme color extracted from the grid pattern area of ​​the original image, and combine to generate multiple sets of borders (the number of sets can be customized, and it can also be combined with other grid patterns in the library).

[0120] Step S311: Randomly select grid handle material, randomly fill it with theme color extracted from the whole image, and combine to generate multiple sets of borders (the number of sets can be customized).

[0121] Step S312: Store the generated border.

[0122] Step S313: Use the most matching grid material plus the border generated by randomly filling the theme color as the default border.

[0123] Step S314: Randomly select a set of borders from the multiple sets of candidate borders output in step S311 as the default border.

[0124] Step S315: The front end draws a single-layer View and places the selected default border.

[0125] Step S316: Place the original image in the image position defined by the border for secondary layout.

[0126] Step S317: The front-end preview area displays the application result of the border.

[0127] In step S318, the user can click the "Change Pattern" button, and the front end sends a command to the back end.

[0128] Step S319: After receiving the command, the backend retrieves other stored borders and sends them back to the frontend.

[0129] Step S320: Display the returned borders in the border panel, and reapply them to the image after the user makes a second selection.

[0130] Please see Figure 10 , Figure 10 This is a schematic diagram of a patterned border compositing device provided in an embodiment of this application. The execution process of the patterned border compositing device involves the following steps:

[0131] First, the target area containing the pattern in the image is obtained through an image subject recognition and segmentation device, and the theme colors of the area are extracted, generally no less than 5 theme colors. After obtaining the target area containing the pattern, its pattern is extracted and segmented into the smallest unit. This unit is then matched with pattern materials in an existing image library based on similarity, and the pattern material with the highest matching degree is selected for use. The pattern material with the highest matching degree is used. Each pattern material has pre-made color-filling spaces. The extracted theme colors are randomly filled into the color-filling areas of the pattern material to produce patterns with different color schemes. The filled patterns are then tiled and combined to form a border background (i.e., the background material in this application). Finally, foreground decorations are added on the background, and the image display overlay where the original image is located is defined to generate a complete border.

[0132] As mentioned above, the border pattern is selected from preset patterns or generated in real time based on the pattern patterns in the multimedia data, ensuring consistency between the border pattern and the multimedia data pattern. Furthermore, the border color matches the theme color of the multimedia data, ensuring consistency between the overall color tone of the border and the overall color tone of the multimedia data. Ensuring consistency between the border and the multimedia data in both pattern style and color enhances the decorative effect of the border on the multimedia data.

[0133] Please see Figure 11 , Figure 11 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. This embodiment mainly illustrates how to generate a texture-type border. The image processing includes the following steps:

[0134] Step S401: Display multimedia data.

[0135] The specific process of step S401 can be found above. Figure 3 The corresponding step S101 in the embodiment.

[0136] Step S402: In response to the need to add a border to the multimedia data, a target texture is determined from the texture set according to the theme of the multimedia data.

[0137] Specifically, when the texture selection control is triggered, the terminal device responds with a request to add texture borders to the multimedia data. It can identify the theme of the multimedia data based on a trained theme recognition model and obtain a texture set. If a texture in the texture set matches the theme of the multimedia data, it can be used as the target texture. If no texture matches the theme, the theme color of the multimedia data can be extracted. Here, the theme color refers to the color that occupies a large proportion in the multimedia data. A texture matching the theme color of the multimedia data is selected from the texture set and used as the target texture. Since there may be multiple theme colors for both the multimedia data and the textures, a texture with exactly the same theme color can be used as the target texture; alternatively, a texture matching at least one theme color of the multimedia data can be used as the target texture, meaning there is an overlap between the theme colors of the target texture and the theme colors of the multimedia data.

[0138] Step S403: Extract the theme color of the multimedia data, fill the border template with the target texture and theme color to obtain n borders, and select the target border from the n borders.

[0139] Specifically, the terminal device extracts the theme color of the multimedia data and obtains a border template. The border template determines the number and position of textures, etc. The target texture is filled into the texture area of ​​the border model, and the theme color of the multimedia data is filled into the background area of ​​the border model, resulting in n borders. It should be noted that since the number of texture areas in the border template may be greater than the number of target textures, the target textures can be reused, meaning the same target texture can be used in multiple texture areas. If the number of texture areas in the border template is less than the number of target textures, the terminal device can fill only a portion of the target textures into the texture area of ​​the border model.

[0140] Since multimedia data may have multiple theme colors, in this case, each theme color can be filled separately into the border template to obtain a sticker. Figure 1 This can result in multiple borders with different background colors; or when the number of target textures is greater than the number of texture areas of the border template, there can be a combination of multiple target textures, which can also result in multiple borders.

[0141] The terminal device can sort these multiple borders according to certain rules. For example, based on the proportion of the theme color in the multimedia data, the n borders generated corresponding to the theme color can be sorted; or, based on the degree of matching between the target texture and the theme of the multimedia data, the n borders generated corresponding to the target texture can be sorted. Of course, the higher the proportion of the theme color, the higher the ranking; the higher the degree of matching, the higher the ranking. The terminal device uses the border that ranks first as the target border.

[0142] When the theme of multimedia data includes biometric features, these features can be one or more of the following: biological species (human, cat, dog, etc.), facial features (eyes, nose), expression, face shape, and hairstyle. The corresponding target texture will then be a texture that matches these biometric features. For example, if the theme of the multimedia data includes "human," then the target texture could be a humanoid texture; if the theme of the multimedia data includes "smiling," then the target texture could be a smiling expression texture, etc.

[0143] Optionally, as long as the theme of the multimedia data contains emoticons, the matching target image will only contain emoticon images. In this way, the resulting image border will be the emoticon image border, and the emoticon type of the target image will be the same as the emoticon type contained in the theme of the multimedia data.

[0144] Step S404: Add the target border to the multimedia data.

[0145] The specific process of step S404 can be found above. Figure 3 The corresponding step S103 in the embodiment.

[0146] Please see Figure 12 , Figure 12 This is a flowchart illustrating the process of adding a textured border according to an embodiment of this application. Adding a textured border mainly involves three process devices: an image subject recognition and segmentation device, an image theme color recognition device, and a textured border synthesis device. The specific process is as follows:

[0147] In step S501, the user uploads any original image on the front end and selects the EMOJI border function. The original image is then sent back to the back end.

[0148] In step S502, the image subject recognition and segmentation device performs preprocessing on the image, such as noise reduction and enhancement.

[0149] In step S503, the image subject recognition and segmentation device segments the preprocessed image to distinguish the foreground and background of the image.

[0150] In step S504, the image subject recognition and segmentation device extracts features from the segmented image to segment the main body region of the image.

[0151] In step S505, the image subject recognition and segmentation device identifies the expression and subject content of the image based on classification decisions and the subject region.

[0152] Among them, the discrimination rules for classification decisions can be optimized through machine learning, and the main content can refer to the semantics of the image. Here, the expression and the main content can correspond to the theme in this application.

[0153] In step S506, the image subject recognition and segmentation device determines whether the expression in the image matches at least one expression image in the expression image set. If yes, then steps S507, S511, and S514-521 are executed; otherwise, step S508 is executed.

[0154] Step S507: Extract facial expressions from the image and sort the extracted facial expressions.

[0155] Step S508: Determine whether the main content of the image matches at least one non-expression image in the set of non-expression images. If yes, proceed to steps S509, S512, S514-521; otherwise, proceed to steps S510, S513-521.

[0156] Step S509: Sort the extracted main content.

[0157] In step S510, the image theme color recognition device extracts the theme color based on the total number of pixels in the image.

[0158] The theme colors are sorted by the percentage of pixels with that color value in the entire image. They can be deduplicated based on the hue (e.g., if multiple blue color values ​​are extracted, and their transparency ranges are similar, only the one with the highest percentage is taken as the representative color of that hue). The number of theme colors extracted can be customized, but is generally no less than 5.

[0159] In step S511, the sticker-type border synthesis device retrieves corresponding Emoji materials (Emoji materials are stickers) according to the number of Emojis that the border template can hold, and randomly inserts them into the border template to generate different combinations; if the number of recognized Emojis is less than the number of Emojis that can be held, the process is repeated.

[0160] In step S512, the texture-based border compositing device retrieves corresponding non-expression Emoji materials (i.e., non-expression stickers) based on the number of Emojis the border template can hold, ranked according to the proportion of the main content, and randomly inserts them into the border template to generate different combinations. If the identified main content is less than the number of Emojis that can be held, the process is repeated.

[0161] In step S513, the texture-based border compositing device calls up Emoji materials (including emoji expressions and non-emoji expressions) that match the theme color based on the number of Emojis that the border template can hold, and randomly inserts them into the border template to generate different combinations. If the recognized color tone is less than the number of Emojis that can be held, the application is repeated.

[0162] Step S514: Store the generated border.

[0163] Step S515: Randomly select a set of borders from the multiple sets of candidate borders output in step S514 as the default border.

[0164] Step S516: The front end draws a single-layer View and places the selected default border.

[0165] Step S517: Place the original image in the image position defined by the border for secondary layout.

[0166] Step S518: The front-end preview area displays the application result of the border.

[0167] In step S519, the user can click the "Change Pattern" button, and the front end sends a command to the back end.

[0168] In step S520, after receiving the command, the backend retrieves the other stored borders and sends them back to the frontend.

[0169] Step S521: Display the returned borders in the border panel, and reapply them to the image after the user makes a second selection.

[0170] Please see Figure 13 , Figure 13 This is a schematic diagram of a texture-based border compositing device provided in an embodiment of this application. This embodiment mainly describes how the texture-based border compositing device generates a texture-based border when a texture matching the expression or main content of an image exists. The execution process involves the following steps:

[0171] First, the facial expressions and main content of the image are identified, and then sorted according to their proportions within the image. Next, an existing emoji border template is randomly selected. Each emoji border template contains a varying number of sticker slots (the template determines the number, position, and original image display position of the emojis within the border). When the number of sticker slots in the template exceeds the number of facial expressions or main content identified in the image, emojis matching the facial expressions or main content are randomly inserted into the template's sticker slots, repeating this process until the template is full. When the number of sticker slots in the template is less than the number of facial expressions or main content identified (n), the top n facial expressions or main content are selected from the sorted results, and their corresponding emojis are randomly inserted until the template is full. This process generates different combinations.

[0172] Please see Figure 14 This is a schematic diagram of a texture-based border compositing device provided in an embodiment of this application. This embodiment mainly describes how the texture-based border compositing device generates a texture-based border when there is no texture matching the expression or main content of the image. The execution process involves the following steps:

[0173] When no matching texture is found in the texture set that matches the image's expression or main content, the image's theme color can be used as a fallback. After extracting the image's theme color ( Figure 14 The image's theme colors are blue, red, and white. The system searches for Emoji elements in the texture set that match the image's theme colors. Matching here means that the Emoji element's color belongs to the image's theme colors. The found Emoji elements are then randomly inserted into the texture slots of the border template to generate different combinations.

[0174] For further details, please see Figure 15 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Figure 15 As shown, the image processing device 1 can be applied to the above-mentioned... Figures 3-14 The terminal device in the corresponding embodiment. Specifically, the image processing device 1 may be a computer program (including program code) running on a computer device, for example, the image processing device 1 may be an application software; the image processing device 1 may be used to execute the corresponding steps in the method provided in the embodiments of this application.

[0175] The image processing device 1 may include a display module 11, an acquisition module 12, and an addition module 13.

[0176] Display module 11 is used to display multimedia data;

[0177] The acquisition module 12 is used to acquire a target border in response to the need to add a border to the multimedia data. The target border is a patterned border generated based on the pattern of the multimedia data, or the target border is a textured border generated based on the theme of the multimedia data.

[0178] Add module 13 to add the target border to the multimedia data.

[0179] In one possible implementation, the multimedia data display interface includes a border type selection control. When the border type selection control is triggered, a request to add a border to the multimedia data is received; or...

[0180] When a preset operation is detected in the display interface of the multimedia data, a request to add a border to the multimedia data is received; or,

[0181] When it is detected that the multimedia data is in border-adding mode, a request to add a border to the multimedia data is received.

[0182] In one possible implementation, the border type selection control includes a pattern selection control and a texture selection control;

[0183] If the target border is obtained after triggering a pattern selection control, then the target border is a pattern border.

[0184] If the target border is obtained after triggering a texture-type selection control, then the target border is a texture-type border;

[0185] The pattern border and the sticker border are distinguished by the shape of the border. The pattern selection control contains the descriptive characters of the pattern border, and the sticker selection control contains the descriptive characters of the sticker border.

[0186] In one possible implementation, when the target border is a patterned border, the pattern color of the target border is generated based on the theme color of the multimedia data;

[0187] The pattern of the target border is selected from k preset patterns based on the pattern of the multimedia data; or,

[0188] The pattern of the target border is extracted from the pattern of the multimedia data.

[0189] In one possible implementation, when acquiring the target border, the acquisition module 12 is specifically used for:

[0190] Target detection is performed on the multimedia data to obtain the target region, and the image of the target region is extracted from the multimedia data to obtain the target region image;

[0191] Select a target pattern from the k preset patterns based on the pattern style of the target area image;

[0192] Extract the theme color of the multimedia data, and fill the target pattern with color according to the theme color of the multimedia data to obtain n borders, where n is a positive integer;

[0193] Select the target border from n borders.

[0194] In one possible implementation, all k preset patterns belong to the target pattern type;

[0195] When the acquisition module 12 selects a target pattern from the k preset patterns based on the pattern style of the target region image, it is specifically used for:

[0196] If the pattern of the target area image does not belong to the target pattern type, then a target pattern is randomly selected from the k preset patterns;

[0197] If the pattern of the target region image belongs to the target pattern type, the target region image is segmented to obtain segmentation unit images. The similarity between the segmentation unit images and each preset pattern is obtained, and the m patterns with the highest similarity are taken as target patterns, where m is a positive integer and m≤k.

[0198] In one possible implementation, when acquiring the target border, the acquisition module 12 is specifically used for:

[0199] Target detection is performed on the multimedia data to obtain the target region, and the image of the target region is extracted from the multimedia data to obtain the target region image;

[0200] Extract the target pattern from the pattern of the target area image, extract the theme color of the multimedia data, and fill the target pattern with color according to the theme color of the multimedia data to obtain n borders, where n is a positive integer;

[0201] Select the target border from n borders.

[0202] In one possible implementation, when the target border is a texture-type border, the background color of the target border is generated based on the theme color of the multimedia data;

[0203] The target border carries a texture, which is selected from a texture set based on the theme of the multimedia data.

[0204] In one possible implementation, the acquisition module 12, when used to acquire the target border, is specifically used for:

[0205] The target texture is determined from the texture set based on the theme of the multimedia data;

[0206] Extract the theme color from the multimedia data, and fill the border template with the target texture and theme color to obtain n borders, where n is a positive integer;

[0207] Select the target border from n borders.

[0208] In one possible implementation, when the acquisition module 12 determines the target texture from the texture set based on the theme of the multimedia data, it is specifically used for:

[0209] If there is a texture in the texture set that matches the theme of the multimedia data, then the matching texture will be used as the target texture.

[0210] If no texture in the texture set matches the theme of the multimedia data, then a texture with the same theme color as the multimedia data is selected from the texture set, and the selected texture is used as the target texture.

[0211] In one possible implementation, when the subject of the multimedia data contains biometric features, the target border carries a texture that matches the biometric features, which include one or more of the following: species, facial features, expression, face shape, and hairstyle.

[0212] In one possible implementation, the device 1 may further include a replacement module 14.

[0213] Replacement module 14 is used to obtain a replacement border in response to the need to replace the target border;

[0214] The replacement module 14 is further configured to replace the target border added for the multimedia data with the replacement border;

[0215] Wherein, the replacement border and the target border are both one of p borders, the replacement border and the target border are different, and any border is a patterned border generated based on the pattern of the multimedia data, or a textured border generated based on the theme of the multimedia data, where p is a positive integer.

[0216] In one possible implementation, the multimedia data display interface also includes a replacement control;

[0217] When the replacement control is triggered, a request to replace the target border is received;

[0218] When the replacement module 14 is used to obtain the replacement border, it is specifically used for:

[0219] The replacement border is randomly selected from p-1 borders excluding the target border.

[0220] In one possible implementation, the multimedia data display interface is further provided with p border content selection controls, one border content selection control corresponding to one border.

[0221] When the target border content selection control is triggered, a request to replace the target border is received; the target border content selection control is one of p border content selection controls.

[0222] When the replacement module 14 is used to obtain the replacement border, it is specifically used for:

[0223] The border corresponding to the target border content selection control is used as the replacement border.

[0224] In one possible implementation, the adding module 13, when adding the target border to the multimedia data, is specifically used for:

[0225] Add a target border to the multimedia data according to the target effect display method, wherein the target effect display method is determined based on the pattern or theme of the multimedia data.

[0226] According to an embodiment of the present invention, Figures 3-14 Each step involved in the method shown can be performed by... Figure 15 The image processing apparatus shown is executed by various modules within it. For example, Figure 3 Steps S101-S103 shown Figure 7 Steps S201-S205 are shown. Figure 9 Steps S302-S316 shown in the figure Figure 11 Steps S401-S404 shown in the figure Figure 12 Steps 502-S521 shown can be respectively generated by Figure 15 The display module 11, acquisition module 12, and addition module 13 shown are used to perform this; for example, Figure 9 Steps S318 and S320 shown, and Figure 12 Steps S519 and S521 shown can be derived from... Figure 15 The replacement module 14 shown is used to perform this.

[0227] Further, please see Figure 16 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figures 3-14 In the corresponding embodiment, the terminal device can be a computer device 1000. For example... Figure 16 As shown, computer device 1000 may include a user interface 1002, a processor 1004, an encoder 1006, and a memory 1008. A signal receiver 1016 is used to receive or transmit data via a cellular interface 1010, a Wi-Fi interface 1012, ..., or an NFC interface 1014. The encoder 1006 encodes the received data into a data format that can be processed by a computer. The memory 1008 stores a computer program, and the processor 1004 is configured to execute the steps in any of the above method embodiments via the computer program. The memory 1008 may include volatile memory (e.g., dynamic random access memory DRAM) and may also include non-volatile memory (e.g., one-time programmable read-only memory OTPROM). In some instances, the memory 1008 may further include memory remotely located relative to the processor 1004, which can be connected to the computer device 1000 via a network. The user interface 1002 may include a keyboard 1018 and a display 1020.

[0228] exist Figure 16 In the computer device 1000 shown, the processor 1004 can be used to call computer programs stored in the memory 1008 to achieve:

[0229] Display multimedia data;

[0230] In response to the need to add a border to the multimedia data, a target border is obtained, which is either a patterned border generated based on the pattern of the multimedia data, or a textured border generated based on the theme of the multimedia data.

[0231] Add the target border to the multimedia data.

[0232] In one embodiment, the multimedia data display interface includes a border type selection control. When the border type selection control is triggered, a request to add a border to the multimedia data is received; or...

[0233] When a preset operation is detected in the display interface of the multimedia data, a request to add a border to the multimedia data is received; or,

[0234] When it is detected that the multimedia data is in border-adding mode, a request to add a border to the multimedia data is received.

[0235] In one embodiment, the border type selection control includes a pattern selection control and a texture selection control;

[0236] If the target border is obtained after triggering a pattern selection control, then the target border is a pattern border.

[0237] If the target border is obtained after triggering a texture-type selection control, then the target border is a texture-type border;

[0238] The pattern border and the sticker border are distinguished by the shape of the border. The pattern selection control contains the descriptive characters of the pattern border, and the sticker selection control contains the descriptive characters of the sticker border.

[0239] In one embodiment, when the target border is a patterned border, the pattern color of the target border is generated based on the theme color of the multimedia data;

[0240] The pattern of the target border is selected from k preset patterns based on the pattern of the multimedia data; or,

[0241] The pattern of the target border is extracted from the pattern of the multimedia data.

[0242] In one embodiment, when the processor 1004 acquires the target bounding box, it specifically performs the following steps:

[0243] Target detection is performed on the multimedia data to obtain the target region, and the image of the target region is extracted from the multimedia data to obtain the target region image;

[0244] Select a target pattern from the k preset patterns based on the pattern style of the target area image;

[0245] Extract the theme color of the multimedia data, and fill the target pattern with color according to the theme color of the multimedia data to obtain n borders, where n is a positive integer;

[0246] Select the target border from n borders.

[0247] In one embodiment, all k preset patterns belong to the target pattern type;

[0248] When processor 1004 selects a target pattern from the k preset patterns based on the pattern style of the target region image, it specifically performs the following steps:

[0249] If the pattern of the target area image does not belong to the target pattern type, then a target pattern is randomly selected from the k preset patterns;

[0250] If the pattern of the target region image belongs to the target pattern type, the target region image is segmented to obtain segmentation unit images. The similarity between the segmentation unit images and each preset pattern is obtained, and the m patterns with the highest similarity are taken as target patterns, where m is a positive integer and m≤k.

[0251] In one embodiment, when the processor 1004 acquires the target bounding box, it specifically performs the following steps:

[0252] Target detection is performed on the multimedia data to obtain the target region, and the image of the target region is extracted from the multimedia data to obtain the target region image;

[0253] Extract the target pattern from the pattern of the target area image, extract the theme color of the multimedia data, and fill the target pattern with color according to the theme color of the multimedia data to obtain n borders, where n is a positive integer;

[0254] Select the target border from n borders.

[0255] In one embodiment, when the target border is a texture-type border, the background color of the target border is generated based on the theme color of the multimedia data;

[0256] The target border carries a texture, which is selected from a texture set based on the theme of the multimedia data.

[0257] In one embodiment, when the processor 1004 acquires the target bounding box, it specifically performs the following steps:

[0258] The target texture is determined from the texture set based on the theme of the multimedia data;

[0259] Extract the theme color from the multimedia data, and fill the border template with the target texture and theme color to obtain n borders, where n is a positive integer;

[0260] Select the target border from n borders.

[0261] In one embodiment, when the processor 1004 determines a target texture from the texture set based on the theme of the multimedia data, it specifically performs the following steps:

[0262] If there is a texture in the texture set that matches the theme of the multimedia data, then the matching texture will be used as the target texture.

[0263] If no texture in the texture set matches the theme of the multimedia data, a texture matching the theme color of the multimedia data is selected from the texture set, and the selected texture is used as the target texture. In one embodiment, when the theme of the multimedia data contains biometric features, the texture carried by the target border is a texture matching the biometric features, which include one or more of the following: species, facial features, expression, face shape, and hairstyle.

[0264] In one embodiment, the processor 1004 further performs the following steps:

[0265] In response to the need to replace the target border, obtain the replacement border;

[0266] Replace the target border added to the multimedia data with the replacement border;

[0267] Wherein, the replacement border and the target border are both one of p borders, the replacement border and the target border are different, and any border is a patterned border generated based on the pattern of the multimedia data, or a textured border generated based on the theme of the multimedia data, where p is a positive integer.

[0268] In one embodiment, the multimedia data display interface also includes a replacement control;

[0269] When the replacement control is triggered, a request to replace the target border is received;

[0270] When processor 1004 executes the process of obtaining the replacement border, it specifically performs the following steps:

[0271] The replacement border is randomly selected from p-1 borders excluding the target border.

[0272] In one embodiment, the multimedia data display interface is further provided with p border content selection controls, one border content selection control corresponding to one border;

[0273] When the target border content selection control is triggered, a request to replace the target border is received; the target border content selection control is one of p border content selection controls.

[0274] When processor 1004 executes the process of obtaining the replacement border, it specifically performs the following steps:

[0275] The border corresponding to the target border content selection control is used as the replacement border.

[0276] In one embodiment, when the processor 1004 adds the target border to the multimedia data, it specifically performs the following steps:

[0277] Add a target border to the multimedia data according to the target effect display method, wherein the target effect display method is determined based on the pattern or theme of the multimedia data.

[0278] User actions that meet the trigger conditions refer to situations where the user moves the terminal device, causing a change in the position of the terminal device, or the user moves the container image, causing a change in the display area of ​​the container image.

[0279] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figures 3-14 The description of the image processing method in the corresponding embodiments can also be performed as described above. Figure 15 The description of the image processing apparatus 1 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0280] Furthermore, it should be noted that this application embodiment also provides a computer storage medium, which stores a computer program executed by the aforementioned image processing device 1. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figures 3-14 The description of the image processing method in the corresponding embodiments is already provided and will not be repeated here. Similarly, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the embodiments of the computer storage medium involved in this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed and executed on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network. Multiple computer devices distributed across multiple locations and interconnected via a communication network can be combined to form a blockchain network.

[0281] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figures 3 to 14 The methods described in the corresponding embodiments are therefore not repeated here.

[0282] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0283] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An image processing method, comprising: Display multimedia data; In response to the need to add a border to the multimedia data, a target border is obtained. The target border is a patterned border generated based on the pattern of the multimedia data. The pattern color of the target border is generated based on the theme color of the multimedia data. The pattern style of the target border is selected from k preset patterns based on the pattern of the multimedia data, where k is a positive integer. Alternatively, the pattern of the target border may be extracted from the pattern of the multimedia data; Add the target border to the multimedia data; Wherein, if the target border pattern style is selected from k preset patterns based on the pattern of the multimedia data, then obtaining the target border includes: performing target detection on the multimedia data to obtain a target region; extracting the image of the target region from the multimedia data to obtain a target region image; selecting a target pattern from k preset patterns based on the pattern style of the target region image, where k is a positive integer; extracting the theme color of the multimedia data; performing color filling processing on the target pattern according to the theme color of the multimedia data to obtain n borders, where n is a positive integer; and selecting the target border from the n borders. If the pattern of the target border is extracted from the pattern of the multimedia data, then obtaining the target border includes: performing target detection on the multimedia data to obtain a target region; extracting the image of the target region from the multimedia data to obtain a target region image; extracting the target pattern from the pattern of the target region image and extracting the theme color of the multimedia data; performing color filling processing on the target pattern according to the theme color of the multimedia data to obtain n borders, where n is a positive integer; and selecting the target border from the n borders.

2. The method according to claim 1, characterized in that, The multimedia data display interface includes a border type selection control. When the border type selection control is triggered, a request to add a border to the multimedia data is received; or... When a preset operation is detected in the display interface of the multimedia data, a request to add a border to the multimedia data is received; or, When it is detected that the multimedia data is in border-adding mode, a request to add a border to the multimedia data is received.

3. The method according to claim 2, characterized in that, The border type selection control includes pattern selection control and sticker selection control; If the target border is obtained after triggering a pattern selection control, then the target border is a pattern border. If the target border is obtained after triggering a texture-type selection control, then the target border is a texture-type border; The pattern border and the sticker border are distinguished by the shape of the border. The pattern selection control contains the descriptive characters of the pattern border, and the sticker selection control contains the descriptive characters of the sticker border.

4. The method according to claim 1, characterized in that, All k preset patterns belong to the target pattern type; The step of selecting a target pattern from the k preset patterns based on the pattern style of the target region image includes: If the pattern of the target area image does not belong to the target pattern type, then a target pattern is randomly selected from the k preset patterns; If the pattern of the target region image belongs to the target pattern type, the target region image is segmented to obtain segmentation unit images. The similarity between the segmentation unit images and each preset pattern is obtained, and the m patterns with the highest similarity are taken as target patterns, where m is a positive integer and m≤k.

5. The method according to claim 1, characterized in that, The method further includes: In response to the need to replace the target border, obtain the replacement border; Replace the target border added to the multimedia data with the replacement border; Wherein, the replacement border and the target border are both one of p borders, the replacement border and the target border are different, and any border is a patterned border generated based on the pattern of the multimedia data, or a textured border generated based on the theme of the multimedia data, where p is a positive integer.

6. The method according to claim 5, characterized in that, The multimedia data display interface also includes a replacement control; When the replacement control is triggered, a request to replace the target border is received; The process of obtaining the replacement border includes: The replacement border is randomly selected from p-1 borders excluding the target border.

7. The method according to claim 5, characterized in that, The multimedia data display interface also includes p border content selection controls, with one border content selection control corresponding to one border. When the target border content selection control is triggered, a request to replace the target border is received; The target border content selection control is one of p border content selection controls. Then obtain the replacement border, including: The border corresponding to the target border content selection control is used as the replacement border.

8. The method according to claim 1, characterized in that, Adding the target border to the multimedia data includes: Add a target border to the multimedia data according to the target effect display method, wherein the target effect display method is determined based on the pattern or theme of the multimedia data.

9. An image processing apparatus, characterized in that, include: The display module is used to display multimedia data; The acquisition module is used to acquire a target border in response to the requirement of adding a border to the multimedia data. The target border is a patterned border generated based on the pattern of the multimedia data. The pattern color of the target border is generated based on the theme color of the multimedia data. The pattern style of the target border is selected from k preset patterns based on the pattern of the multimedia data, where k is a positive integer. Alternatively, the pattern of the target border may be extracted from the pattern of the multimedia data; An add module is used to add the target border to the multimedia data; Wherein, if the target border pattern style is selected from k preset patterns based on the pattern of the multimedia data, then the acquisition module, when used to acquire the target border, specifically performs the following: performing target detection on the multimedia data to obtain a target region; extracting the image of the target region from the multimedia data to obtain a target region image; selecting a target pattern from the k preset patterns based on the pattern style of the target region image, where k is a positive integer; extracting the theme color of the multimedia data; performing color filling processing on the target pattern according to the theme color of the multimedia data to obtain n borders, where n is a positive integer; and selecting the target border from the n borders. If the pattern of the target border is extracted from the pattern of the multimedia data, the acquisition module for acquiring the target border includes: performing target detection on the multimedia data to obtain a target region; extracting the image of the target region from the multimedia data to obtain a target region image; extracting the target pattern from the pattern of the target region image and extracting the theme color of the multimedia data; performing color filling processing on the target pattern according to the theme color of the multimedia data to obtain n borders, where n is a positive integer; and selecting the target border from the n borders.

10. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1-8.

11. A computer storage medium, characterized in that, The computer storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, cause a computer device having the processor to perform the steps of the method according to any one of claims 1-8.

12. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-8.

Images