Image processing method, mobile terminal and computer readable storage medium

Abstract

The invention discloses an image processing method, a mobile terminal and a computer readable storage medium. The image processing method is applied to a first mobile terminal, and comprises the following steps: acquiring a first picture containing a target main body, and extracting a target main body area from the first picture along the edge of the target main body to form a second picture; and according to the main body color of the target main body on the second picture, and / or according to a scene to be displayed by the second picture, determining background information of a background region of the second picture. By implementing the method and the device, the flexibility of picture display is enhanced, and the method and the device can be suitable for image display in different application scenes.

Description

Technical Field

[0001] This disclosure generally relates to the field of computer vision. More specifically, this disclosure relates to an image processing method, apparatus, mobile terminal, and storage medium. Background Art

[0002] Image display technology is a core component of modern information transmission and interaction, and is widely used in fields such as consumer electronics, medical imaging, advertising media, virtual reality (VR), and augmented reality (AR). With the rapid development of image display technology, users' demands for image display clarity, interactivity, and personalization are increasing. Traditional image display methods are usually designed for specific scenarios and cannot flexibly adapt to the needs of other scenarios, making it difficult to meet the requirements of diverse application scenarios. For example, as Figure 1 shown, Figure 1 the picture (rectangular picture) in the upper left corner is only suitable for display in the photo album and cannot be well integrated into the desktop, while Figure 1 the picture (cartoon ornament) in the lower right corner can be well integrated into the desktop, but the photo album generally has only a black background and is not suitable for display in the photo album. In view of this, there is an urgent need to provide an image processing method, apparatus, mobile terminal, and storage medium to adapt to image display in different application scenarios. Summary of the Invention

[0003] To at least solve one or more of the above-mentioned technical problems, this disclosure proposes an image processing method, mobile terminal, and computer-readable storage medium in multiple aspects.

[0004] In a first aspect, this disclosure provides an image processing method, which is applied to a first mobile terminal. The method includes: obtaining a first picture including a target subject, and extracting the target subject area from the first picture along the edge of the target subject to form a second picture; determining background information of the background area of the second picture according to the subject color of the target subject on the second picture, and / or according to the scene where the second picture is to be displayed.

[0005] In some embodiments, if the scene where the second picture is to be displayed is the photo album of the first mobile terminal, the background information includes the background area color and the background area ratio; the method further includes: determining the background area color according to the subject color of the target subject on the second picture, and determining the background area ratio according to the ratio of the target subject on the second picture.

[0006] In some embodiments, if the scene to be shown in the second picture is a widget on the desktop of the first mobile terminal, and there is a wallpaper set on the desktop, the background information includes the color of the background area, and the color of the background area is set to have a light-transmitting effect; so that the wallpaper is presented in the background area.

[0007] In some embodiments, the album further includes multiple third pictures. The second picture is shown adjacent to the first picture in the album, or replaces the first picture and is displayed at the position where the first picture is located; the display size of the second picture corresponds to the album zoom instruction, and the second pictures are arranged and displayed according to a first preset correspondence; the display size of the third picture corresponds to the album zoom instruction, and the third pictures are arranged and displayed according to a second preset correspondence.

[0008] In some embodiments, in the album, for each row where a second picture is located, the number of second pictures displayed side by side in that row is less than or equal to a first set number.

[0009] In some embodiments, a background outline is formed according to the background area ratio, so that the background outline is separated from the edge area of the picture unit configured on the album.

[0010] In some embodiments, before extracting the target subject area from the first picture along the edge of the target subject to form a second picture, the method further includes one or more of the following: determining whether the edge of the target subject in the first picture is clear; if the edge of the target subject in the first picture is clear, extracting the target subject area from the first picture along the edge of the target subject to form a second picture; determining whether the target subject in the first picture is covered; if the target subject in the first picture is not covered, extracting the target subject area from the first picture along the edge of the target subject to form a second picture; determining whether the number of target subjects in the first picture is less than or equal to a second set number; if the number of target subjects in the first picture is less than or equal to the second set number, extracting one or all of the target subject areas together from the first picture along the edge of the target subject to form a second picture.

[0011] In some embodiments, the background information includes: the background area color; determining the background area color according to the main body color of the target main body in the second picture, including: determining the candidate color of the background area based on the main body color of the target main body; if the absolute difference between the candidate color and the main body edge color of the target main body in the second picture is greater than or equal to the set color difference threshold, using the candidate color as the background area color; if the absolute difference between the candidate color and the main body edge color data of the target main body in the second picture is less than the set color difference threshold, adjusting the candidate color until the absolute difference between the adjusted candidate color and the main body edge color of the target main body in the second picture is greater than or equal to the set color difference threshold, and using the adjusted candidate color as the background area color.

[0012] In some embodiments, if the target main body belongs to a preset category, the method further includes: inputting the second picture into a trained dynamic effect generation model to obtain the dynamic effect image of the target main body; and displaying the dynamic effect image of the target main body according to the scene to be shown in the second picture.

[0013] In some embodiments, after obtaining the dynamic effect image of the target main body, the method further includes: generating interaction content in response to an interaction action of the user with the dynamic effect image of the target main body; and sending the interaction content to a second mobile terminal for display.

[0014] In some embodiments, the first mobile terminal includes: a dot matrix display screen formed by lamp beads arranged in an array; the method further includes: performing array segmentation on the second picture, and each lamp bead maps at least one segmentation area of the second picture; determining the display parameters of the lamp beads mapped by each segmentation area to obtain the second picture in dot matrix form; and displaying the second picture in dot matrix form on the dot matrix display screen.

[0015] In a second aspect, the present disclosure provides a mobile terminal, which is the first mobile terminal and includes: a processor configured to execute program instructions; and a memory configured to store the program instructions, when the program instructions are loaded and executed by the processor, the processor is caused to execute the image processing method as described in the first aspect or any optional embodiment of the first aspect.

[0016] In a third aspect, the present disclosure provides a computer-readable storage medium, in which program instructions are stored, when the program instructions are loaded and executed by a processor, the processor is caused to execute the image processing method as described in the first aspect or any optional embodiment of the first aspect.

[0017] An image processing method, a mobile terminal, and a computer-readable storage medium are provided as above. In the embodiments of the present disclosure, the target subject area is extracted from the first picture to form a second picture, and the background information of the background area of the second picture is automatically generated according to the scene to be shown in the second picture and / or the subject color of the target subject in the second picture, enhancing the flexibility of picture display and being applicable to image display in different application scenarios. For example, when the scene to be shown in the second picture is an album, generating the background information of the background area of the second picture according to the scene to be shown in the second picture and / or the subject color of the target subject in the second picture can make the target subject more prominent. Description of the Drawings

[0018] By referring to the drawings and reading the detailed description below, the above and other objects, features, and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, where:

[0019] Figure 1 Shows a specific example diagram of image display in some embodiments of the present disclosure;

[0020] Figure 2 Shows an exemplary flowchart of an image processing method in some embodiments of the present disclosure;

[0021] Figure 3 Shows a specific example diagram of a first picture in some embodiments of the present disclosure;

[0022] Figure 4 Shows a specific example diagram of a second picture in some embodiments of the present disclosure;

[0023] Figure 5A Shows a specific example diagram of image processing in some embodiments of the present disclosure;

[0024] Figure 5B Shows another specific example diagram of image processing in some embodiments of the present disclosure;

[0025] Figure 6 Shows a specific example diagram of an album in some embodiments of the present disclosure;

[0026] Figure 7 Shows a specific example diagram of a dot matrix display screen in some embodiments of the present disclosure;

[0027] Figure 8 Shows an exemplary structural block diagram of a mobile terminal in some embodiments of the present disclosure. Detailed Description of the Embodiments

[0028] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.

[0029] It should be understood that the terms "comprising" and "including" as used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0030] It should also be understood that the terms used in the specification of the present disclosure are merely for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used in the specification and claims of the present disclosure, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms. It should be further understood that the term "and / or" as used in the specification and claims of the present disclosure refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0031] As used in this specification and the claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" according to the context.

[0032] The specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0033] Figure 2 An exemplary flowchart of an image processing method 200 according to some embodiments of the present disclosure is shown.

[0034] The image processing method 200 is applied to a first mobile terminal. That is, in the embodiments of the present disclosure, pictures are processed and displayed by the first mobile terminal. Specifically, the first mobile terminal may be a smart phone, a tablet computer, etc. As Figure 2As shown, the image processing method 200 includes: Step S210: Obtain a first picture including a target subject, and extract the target subject area from the first picture along the edge of the target subject to form a second picture; Step S220: Determine the background information of the background area of the second picture according to the main color of the target subject in the second picture and / or according to the scene to be shown in the second picture.

[0035] Exemplarily, in addition to the target subject, the above first picture may further include other objects in the environment where the target subject is located. The target subject may be a person, an animal, or other objects that the user wishes to separate from the background, and it needs to be a complete object. For example, it cannot be a partially occluded object or a cut-off object. It should be noted that the target subject cannot be a human face. The embodiments of the present disclosure do not specifically limit the target subject, and it can be determined according to the actual situation. In the embodiments of the present disclosure, the first picture including the target subject may be obtained by the camera module of the first mobile terminal shooting the target subject, or may be obtained from a social networking platform (such as Xiaohongshu, etc.). The embodiments of the present disclosure do not specifically limit this.

[0036] Figure 3 Shows a specific example diagram of the first picture of some embodiments of the present disclosure. As Figure 3 shown, Figure 3 The shown first picture includes, in addition to the ornament (i.e., the target subject), other objects such as a computer, a charging cable, and so on.

[0037] Exemplarily, in the above step S210, the first picture is obtained, and the target subject area is extracted from the first picture along the edge of the target subject to form a second picture. Here, the second picture is a picture that only includes the target subject, and its background can be transparent or blank. As Figure 4 shown, Figure 4 The shown second picture only includes the target subject (i.e., the ornament). In the embodiments of the present disclosure, there are many methods for extracting the target subject area from the first picture. For example, conventional image processing methods can be used for extraction, and for another example, a neural network model can be used for extraction. The embodiments of the present disclosure do not specifically limit this.

[0038] The following is an example description of the specific method for extracting the target object area from the first picture along the edge of the target object to form the second picture. If the extraction is performed using a conventional image processing method, a conventional edge detection method can be used to detect the edge of the target object in the first picture, and then the target object area can be extracted from the first picture along the edge of the target object to form the second picture. If the extraction is performed using a neural network model, the first picture is output to a trained semantic segmentation model, and the semantic segmentation model performs semantic segmentation on the target object area in the first picture and outputs the target object area (i.e., the second picture).

[0039] Exemplarily, the scene to be shown in the second picture in step S220 above can be the album of the first mobile terminal or the widget on the desktop of the first mobile terminal, etc. Here, the widget refers to a kind of micro application view that can be embedded in other applications (such as the home screen) and receive regular updates. It is usually used to quickly display the key information or functions of the application on the home screen, and users can directly interact without opening the application. In a specific embodiment, the scene to be shown in the second picture above can be the default (for example, the default scene to be shown is the album). After obtaining the second picture, the background information of the background area of the second picture can be determined based on the album scene. When the user wants to show the second picture in other scenes (such as the desktop widget), the scene to be shown in the second picture can be switched from the default scene to other scenes in response to the user's selection operation, and then the background information of the background area of the second picture can be determined based on the other scene.

[0040] The background information of the above background area can include the background area color and / or the background area ratio, etc. Among them, the background area color can be transparent, or a color obtained through calculation and analysis, or a user-defined color; the background area ratio can be determined according to the ratio of the target object, or can be a fixed one set in advance. The embodiments of the present disclosure do not make specific limitations on the above background area color and background area ratio.

[0041] In the embodiments of the present disclosure, when determining the background information, it can be determined based on the main color of the target object on the second picture, or based on the scene to be shown in the second picture, or based on the main color of the target object on the second picture and the scene to be shown in the second picture. The embodiments of the present disclosure do not make specific limitations on this, and it can be determined according to the actual situation.

[0042] As a specific embodiment, if the background information of the background area is determined only based on the main color of the target subject in the second picture, the background information may include the background area color. When specifically determining the background area color, a color that is not similar to the color of the target subject area may be randomly selected, or a color that is similar but different from the color of the target subject area may be randomly selected, or the background area color may be determined based on the color of the target subject area. The embodiments of the present disclosure do not make specific limitations on this.

[0043] As Figure 5A shown, Figure 5A The leftmost picture is the first picture. In the first picture, the red maple leaf is the target subject. In specific implementation, first locate the maple leaf in the first picture, and the obtained Figure 5A In the middle picture, the blue area is the target subject area. The background area color determined based on the main color of the target subject can be seen in the Figure 5A light red color in the rightmost picture in

[0044] As Figure 5B shown, Figure 5B The leftmost picture is the first picture. In the first picture, the black vehicle is the target subject. In specific implementation, first locate the vehicle in the first picture, and the obtained Figure 5B In the middle picture, the blue area is the target subject area. The background area color determined based on the main color of the target subject can be seen in the Figure 5B light black color in the rightmost picture in

[0045] As for other specific implementation manners of determining the background information of the background area of the second picture based on the main color of the target subject in the second picture, and / or according to the scene to be shown in the second picture, and how to specifically determine the background area color based on the target subject area color, the following embodiments give examples and will not be elaborated here for the time being.

[0046] The embodiments of the present disclosure determine the background area color through the main color of the target subject, making the whole picture more harmonious. Through proportion and graphic display (circular), a simple and clear visual effect similar to an "icon" is formed, which is convenient for users to identify and use.

[0047] The embodiments of the present disclosure extract the target subject area from the first picture to form the second picture, and automatically generate the background information of the background area of the second picture according to the scene to be shown in the second picture and / or the main color of the target subject in the second picture, enhancing the flexibility of picture display and being applicable to image display in different application scenarios.

[0048] It should be noted that after determining the background information of the background area of the target subject, the background on the first picture can be directly replaced with the determined background information to obtain the final picture, or a color layer (i.e., the above background information) can be overlaid on the background on the first picture to form the final picture. The embodiments of the present disclosure do not make specific limitations on this.

[0049] The following describes the automatic generation of the background information of the background area of the second picture according to the scene to be shown in the second picture and / or the main color of the target subject on the second picture:

[0050] As a specific embodiment, if the scene to be shown in the second picture is the album of the first mobile terminal, the background information includes the background area color and the background area ratio; the above image processing method 200 further includes: determining the background area color according to the main color of the target subject on the second picture, and determining the background area ratio according to the ratio of the target subject on the second picture.

[0051] In the embodiments of the present disclosure, determining the background area color according to the main color of the target subject on the second picture may specifically be: determining the candidate color of the background area based on the main color of the target subject; if the absolute difference between the candidate color and the main edge color of the target subject on the second picture is greater than or equal to the set color difference threshold, then taking the candidate color as the background area color; if the absolute difference between the candidate color and the main edge color data of the target subject on the second picture is less than the set color difference threshold, then adjusting the candidate color until the absolute difference between the adjusted candidate color and the main edge color of the target subject on the second picture is greater than or equal to the set color difference threshold, and taking the adjusted candidate color as the background area color.

[0052] Exemplarily, determining the candidate color of the background area based on the main color of the target subject may specifically be: taking the average value of the main color of the target subject as the candidate color, or taking the extreme value of the target subject color as the candidate color. Of course, there may also be other determination methods. The embodiments of the present disclosure do not make specific limitations on this. The embodiments of the present disclosure only describe taking the average value of the main color of the target subject as the candidate color as an example.

[0053] In the embodiments of the present disclosure, after obtaining the above candidate color, the candidate color can be directly taken as the background area color; it is also possible to compare the candidate color with the main edge color of the target subject on the second picture, and determine the background area color according to the comparison result, so that the background area color is clearly distinguishable from the target subject color. For example, when the candidate color is quite different from the main edge color of the target subject on the second picture, the candidate color can be directly taken as the background area color. When the difference is small, the candidate color is adjusted until the adjusted candidate color is quite different from the background area color, and the adjusted candidate color is taken as the background area color.

[0054] Specifically, if the absolute difference between the candidate color and the color of the main body edge of the target main body in the second picture is greater than or equal to the set color difference threshold, it is considered that the candidate color is quite different from the color of the main body edge of the target main body in the second picture, and the background and the target main body can be clearly distinguished. Then, the candidate color can be used as the background area color. If the absolute difference between the candidate color and the color data of the main body edge of the target main body in the second picture is less than the set color difference threshold, it is considered that the candidate color is less different from the color of the main body edge of the target main body in the second picture, and the candidate color is adjusted. Here, the adjustment can be to increase or decrease, and the embodiments of the present disclosure do not make specific limitations on this. Until the absolute difference between the adjusted candidate color and the color of the main body edge of the target main body in the second picture is greater than or equal to the above-mentioned set color difference threshold, the adjusted candidate color is used as the background area color. The set color difference threshold here can be set in advance, and for example, it can be 1.5, 1.8, etc.

[0055] The embodiments of the present disclosure determine the background area color through the main body color of the target main body, so that the tone of the background area color and the color of the target main body is unified on the finally formed picture, and the target main body can be highlighted.

[0056] In the embodiments of the present disclosure, determining the background area ratio according to the ratio of the target main body in the second picture can be specifically as follows: First, identify the ratio of the target main body in the second picture (i.e., the length-width ratio), and then determine the background area ratio according to the ratio of the target main body. Here, an approximate integer ratio can be determined according to the ratio of the target main body, and then this integer ratio is used as the background area ratio. For example, if the ratio of the target main body is 1.6:1, then the determined approximate integer ratio is 2:1, and the background area ratio is 2:1.

[0057] As an optional embodiment of the present disclosure, the background ratio can be preset in advance, such as 2:1 and 1:1. Then, the ratio of the identified target main body is compared with the preset background ratio, and the preset ratio that is closer is used as the background area ratio. For example, the background area ratio of the vertical and positive target main bodies can be determined as 1:1, and the background area ratio of the horizontal target main body can be determined as 2:1.

[0058] Also as Figure 5A shown, the ratio of the target main body (maple leaf) is approximately 1:1. Therefore, Figure 5A the background area ratio in Figure 5B is 1:1. Also as Figure 5B shown, the ratio of the target main body (vehicle) is approximately 2:1. Therefore,

[0059] As an alternative embodiment of the present disclosure, in the case where the first mobile terminal is a smart phone, that is, the second picture is displayed on the smart phone. Since the aspect ratio of the smart phone is small, it can accommodate taller target objects and achieve a relatively unique display effect (i.e., an object spanning several lines vertically), which can give the user a visual mark. Therefore, the above background ratio can also be set to a smaller value, such as 1:4, 1:5, etc. For example, when a user travels to Dubai and takes a first picture containing the Burj Khalifa, the Burj Khalifa area is cut out to form the second picture. This second picture spans 5 lines vertically in the photo album and is similar to a "landmark" in the thumbnail view of the photo album. When the user wants to find the relevant photos, they can easily find the batch of photos taken at that time.

[0060] Through the above settings in the embodiments of the present disclosure, when the user travels to a new place, while taking pictures of local iconic objects, they may also take pictures of other landscapes and people. At this time node, the user will have a lot of photos, and these photos can form an associative memory with the "landmark", that is, finding the "landmark" can easily find other pictures.

[0061] As another specific embodiment, if the scene to be shown in the second picture is a widget on the desktop of the first mobile terminal, and there is a wallpaper set on the desktop, the background information includes the color of the background area, and the color of the background area is set to have a light-transmitting effect; so that the wallpaper is presented in the background area.

[0062] Exemplarily, in the embodiments of the present disclosure, the above light-transmitting effect can be completely transparent or semi-transparent, and the present disclosure does not make specific limitations on light transmission.

[0063] In the embodiments of the present disclosure, if the scene to be shown in the second picture is a widget on the desktop of the first mobile terminal, the color of the background area is set to have a light-transmitting effect, so that the wallpaper of the desktop can be displayed through the background area, thus keeping consistent with the overall style of the scene to be shown in the second picture.

[0064] As an alternative embodiment of the present disclosure, if the scene to be shown in the second picture is a widget on the desktop of the first mobile terminal, the position of the second picture on the desktop can also be combined with the desktop background. For example, also as Figure 1 shown, Figure 1 The main body (the second picture) in the lower right corner is located on the branch of the desktop background, forming the impression of "the main body is on the tree", which can blend into the background, eliminating the sense of boundary of the "photo" and forming a unique visual effect.

[0065] It should be noted that if the scene to be displayed in the second picture is a widget on the desktop of the first mobile terminal, the background area color can also be determined based on the main color of the target subject, and the background information can also include the background area ratio, which can also be determined based on the ratio of the target subject; or if the scene to be displayed in the second picture is the photo album of the first mobile terminal, the background area color can also be transparent; the present disclosure embodiment does not make specific limitations on this, and other reasonable implementation methods are also within the scope of protection of this disclosure.

[0066] As an optional embodiment of the present disclosure, if the scene to be displayed by the second picture is the album of the first mobile terminal, then after the target main area (i.e., the second picture) is cut out from the first picture, it can be displayed in the album. Specifically, in order to more clearly show the relationship between the second picture and the first picture, the second picture can be displayed adjacent to the first picture in the album, or directly replace the first picture to be displayed at the position where the first picture is located, or displayed behind the last picture in the album. The present disclosure embodiment does not specifically limit the display position of the second picture in the album. The adjacent here can refer to adjacent to the left and right, adjacent to the top and bottom, etc., and the present disclosure embodiment does not specifically limit this.

[0067] Figure 6 A specific example diagram of an album of some embodiments of the present disclosure is shown, Figure 6 As shown, Figure 6 The pictures in the first row of circular positions, the pictures in the second row of circular positions of the runway, and the pictures in the penultimate row of circular positions are all second pictures. Figure 6 The picture at the third rectangular position from left to right in the first row can be the original picture (i.e., the first picture) corresponding to the second picture in the circular position behind it, and the picture at the first rectangular position from left to right in the first row can be the original picture (i.e., the first picture) of the pictures in the circular position of the runway in the second row.

[0068] In the disclosed embodiment, the album also includes multiple third images. The third images here are images that have not been cut out, and may include photographed images, images downloaded from social platforms, etc. This disclosed embodiment does not specifically limit this. The display frame of the second image corresponds to the album zoom instruction, and the second images are arranged and displayed according to the first preset correspondence relationship; the display frame of the third image corresponds to the album zoom instruction, and the third images are arranged and displayed according to the second preset correspondence relationship.

[0069] Exemplarily, in the embodiments of the present disclosure, in order to ensure that the second picture cut out can still be clearly and obviously displayed when the album is in thumbnail view, the scaling correspondence relationship between the second picture and the album scaling instruction (i.e., the above-mentioned first preset correspondence relationship) is different from the scaling correspondence relationship between the third picture and the album scaling instruction (i.e., the above-mentioned second preset correspondence relationship). For example, the above-mentioned first preset correspondence relationship is that the second picture corresponds to 2 scaling levels, and the second preset correspondence relationship is that the third picture corresponds to 4 scaling levels. The above-mentioned first preset correspondence relationship and second preset correspondence relationship are both examples and are not used to limit the present disclosure.

[0070] As an optional embodiment of the present disclosure, in the album, for each row where the second picture is located, the number of second pictures displayed side by side in that row is less than or equal to the first set number.

[0071] Exemplarily, in the embodiments of the present disclosure, the above-mentioned first set number is set in advance, and it can be, for example, 1 or 2. The above-mentioned first set number in the embodiments of the present disclosure is not specifically limited, as long as it is ensured that it is less than the number of pictures set for each row of the album itself.

[0072] In the embodiments of the present disclosure, by setting the display number of the second pictures in each row where the second picture is located in the album to be less than or equal to the first set number, space is provided for the scaling of the second pictures.

[0073] As an optional embodiment of the present disclosure, in order to better display the second pictures in the album, at least every preset number (for example, 6) of the first pictures in the album can be subjected to the steps of extracting the second pictures and displaying them. That is to say, if the first picture in the album is subjected to the steps of extracting the second pictures and displaying them, then at least the 7th first picture can be subjected to the steps of extracting the second pictures and displaying them.

[0074] As an optional embodiment of the present disclosure, if multiple first pictures are collected for a certain target subject during image acquisition, then when performing the steps of extracting the second pictures and displaying them for the first pictures containing the target subject, one clear first picture can be selected from the multiple collected first pictures to perform the steps of extracting the second pictures and displaying them, rather than performing the steps of extracting the second pictures and displaying them for each first picture. This ensures that in a scenario like an album with multiple photos, it will not become crowded and overly occupy visual and memory space.

[0075] As an optional embodiment of the present disclosure, a background contour is formed according to the background area ratio, so that the background contour is separated from the edge area of the picture unit configured on the album.

[0076] Exemplarily, in the embodiments of the present disclosure, also as Figure 6As shown, the picture units configured on the photo album are rectangular. In order to separate the background contour from the edge area of the picture units configured on the photo album, the background contour of the above-mentioned background area can be set to be arc-shaped. For example, when the ratio of the background area is 1:1, the background contour is circular; when the ratio of the background area is 2:1, the background contour is oval-shaped.

[0077] In the embodiment of the present disclosure, by setting the background contour of the background area to be arc-shaped, the background contour is separated from the edge area of the picture units configured on the photo album, and the second picture can be more prominent in the photo album.

[0078] As an optional embodiment of the present disclosure, before extracting the target object area from the first picture along the edge of the target object to form the second picture, the above image processing method 200 further includes one or more of the following: determining whether the edge of the target object in the first picture is clear; if the edge of the target object in the first picture is clear, then extracting the target object area from the first picture along the edge of the target object to form the second picture; determining whether the target object in the first picture is covered; if the target object in the first picture is not covered, then extracting the target object area from the first picture along the edge of the target object to form the second picture; determining whether the number of target objects in the first picture is less than or equal to the second set number; if the number of target objects in the first picture is less than or equal to the second set number, then extracting one or all of the target object areas together from the first picture along the edge of the target object to form the second picture.

[0079] Exemplarily, in this embodiment, if the edge of the target object in the first picture is continuous, it is considered that the edge of the target object is clear. Specifically, an edge detection algorithm (for example, Sobel operator detection algorithm, Canny edge detection algorithm, etc.) can be used to determine whether the edge of the target object in the first picture is clear. As a specific embodiment, if the Canny edge detection algorithm is used to determine whether the edge of the target object in the first picture is clear, a clear edge will be manifested as a continuous line. As another specific embodiment, if the Sobel operator detection algorithm is used to determine whether the edge of the target object in the first picture is clear, the edge is detected by calculating the gradient of the target object area, and the greater the gradient, the clearer the edge.

[0080] In this embodiment, if the edge of the target object in the first picture is clear, the target object area is extracted from the first picture along the edge of the target object to form a second picture. If the edge of the target object in the first picture is not clear, the cropping operation may not be performed, or the user may be prompted to use some image enhancement techniques (such as edge sharpening, etc.) to improve the edge sharpness. After enhancing the first picture, it is determined again whether the edge of the target object in the first picture is clear. If it is clear, the target object area can be extracted from the first picture along the edge of the target object to form a second picture. If it is still not clear, the first picture is not processed.

[0081] Exemplarily, in the embodiment of the present disclosure, if the target object in the first picture is not covered, the target object area is extracted from the first picture along the edge of the target object to form a second picture. If the target object in the first picture is covered, the first picture is not processed.

[0082] Exemplarily, in the embodiment of the present disclosure, the above-mentioned second set quantity may be any reasonable value, such as 2. The embodiment of the present disclosure does not make specific limitations on this. If the number of target objects in the first picture is less than or equal to the second set quantity, one or all of the target object areas are extracted from the first picture along the edge of the target object to form a second picture. If the number of target objects in the first picture is greater than the second set quantity, the first picture is not processed.

[0083] For example, if a first picture includes 2 coffee cups, one of the coffee cup areas can be extracted as the second picture, or the 2 coffee cup areas can be extracted as a whole as the second picture.

[0084] In the embodiment of the present disclosure, before cropping, it is first determined whether the edge of the target object on the first picture is clear, whether the target object is covered, and whether the number of target objects in the first picture is less than or equal to the second set quantity. Only when the edge of the target object is clear, the target object is not covered, and the number of target objects is less than or equal to the second set quantity will the subsequent cropping be performed, so that the edge of the target object can be more accurately recognized during cropping, reducing errors.

[0085] As an optional embodiment of the present disclosure, if the target object belongs to a preset category, the above-mentioned image processing method 200 further includes: inputting the second picture into a trained dynamic effect generation model to obtain a dynamic effect image of the target object; and displaying the dynamic effect image of the target object according to the scene to be shown in the second picture.

[0086] Exemplarily, the above preset categories may be animals, cartoon characters, etc., and the embodiments of the present disclosure do not specifically limit the preset categories. If the target subject belongs to the above preset categories, the second picture is input into the trained dynamic effect generation model to obtain the dynamic effect image of the target subject. For example, if the target subject is a cat, the second picture is input into the dynamic effect generation model to obtain a dynamic picture of the cat (for example, the cat wagging its tail or blinking, etc.).

[0087] In the embodiments of the present disclosure, according to the scene to be shown in the second picture, the specific display of the dynamic effect image of the target subject may be: if the scene to be shown in the second picture is a widget, the dynamic effect image of the target subject is shown on the widget; if the scene to be shown in the second picture is an album, the dynamic effect image of the target subject is shown in the album. As for the position of the dynamic effect image of the target subject in the album, for example, it is shown adjacent to the second picture in the album, or replaces the second picture and is shown at the position where the second picture is located, or directly shown behind the last picture in the album, etc.

[0088] As an optional embodiment of the present disclosure, after obtaining the dynamic effect image of the target subject, the dynamic effect image of the target subject may be sent to another mobile terminal (denoted as the second mobile terminal) for display.

[0089] As an optional embodiment of the present disclosure, after obtaining the dynamic effect image of the target subject, the above image processing method 200 further includes: generating interaction content in response to an interaction action of the user with the dynamic effect image of the target subject; sending the interaction content to the second mobile terminal for display.

[0090] Exemplarily, in the embodiments of the present disclosure, after obtaining the dynamic effect image of the target subject, the user can also interact with the dynamic effect image to generate interaction content. Specifically, the user interacts with the dynamic effect image on the display screen of the first mobile terminal. For example, touching the head of the little cat, the generated interaction content can be the reaction of the little cat to this interaction content, such as the little cat nodding its head.

[0091] In the embodiments of the present disclosure, after obtaining the interaction content, the interaction content can also be sent to the second mobile terminal for display. Here, it can be shared to the second mobile terminal through the sharing widget configured on the first mobile terminal, or sent to the second mobile terminal through a social platform (such as WeChat). Here, the second mobile terminal can also be a smart phone, a tablet computer, etc.

[0092] After receiving the interaction content, the second mobile terminal can display it on its display screen. In this embodiment, the interaction content can be displayed in real time when the second mobile terminal receives it, or can be displayed when the user turns on the second mobile terminal (i.e., unlocks the second mobile terminal), so that the user will not miss the interaction content due to the screen lock or not being on the desktop. The embodiments of the present disclosure do not make specific limitations on this.

[0093] In the embodiments of the present disclosure, by sending the interaction content between the user and the dynamic effect image to the second mobile terminal for display, the target entity can actively interact with the user of the second mobile terminal, improving the interactivity.

[0094] As an optional embodiment of the present disclosure, the above first mobile terminal includes: a dot matrix display screen, which is composed of light beads arranged in an array; the above image processing method 200 further includes: performing array segmentation on the second picture, and each light bead maps at least one segmentation area of the second picture; determining the display parameters of the light beads mapped by each segmentation area to obtain a second picture in dot matrix form; and displaying the second picture in dot matrix form on the dot matrix display screen.

[0095] Exemplarily, the above dot matrix display screen can be located at any position on the first side (for example, the back) of the body of the first mobile terminal, or can be located at any position on the second side (for example, the front) of the body of the first mobile terminal. The embodiments of the present disclosure do not make specific limitations on this. The display technology adopted by the dot matrix display screen can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), or a mini LED.

[0096] In the embodiments of the present disclosure, taking the display technology adopted by the dot matrix display screen as a mini LED as an example, it can be composed of light beads arranged in an array. For example, a 100*100 square array, or for another example, a circular array including 5000 light beads. The embodiments of the present disclosure do not make specific limitations on the number of light beads and the shape of the array arrangement.

[0097] Figure 7 Shows a specific example diagram of the dot matrix display screen of some embodiments of the present disclosure, as Figure 7 shown, Figure 7 The dot matrix display screen in is composed of light beads in an 8*8 square array, where the black circles are the light beads. It should be noted that for the convenience of display, Figure 7 the 8*8 square array in is only an example and is not used to limit the present disclosure.

[0098] Exemplarily, in the embodiments of the present disclosure, there are many methods for performing array segmentation on the second picture. For example, the second picture can be segmented according to the number of lamp beads; for another example, the second picture can be segmented into a preset number (e.g., 3000) of segmentation regions. The embodiments of the present disclosure do not specifically limit the segmentation method of the second picture.

[0099] After the segmentation of the second picture is completed, the segmentation regions are mapped to each lamp bead. Each lamp bead maps at least one segmentation region of the second picture. For each segmentation region, the display parameters of the lamp bead it maps are determined. Here, the display parameters may include: the on / off state of the lamp bead, the brightness of the lamp bead, the color of the lamp bead, etc. The embodiments of the present disclosure do not specifically limit the display parameters.

[0100] As a specific example, assume that the number of lamp beads on the dot matrix display screen is 100, and the number of pixel points of the second picture is 10000. Then, the area surrounded by 100 pixel points (10*10) corresponds to one lamp bead.

[0101] Based on this, when determining the brightness in the display parameters of the lamp bead, the average brightness value of 100 pixel points, the median brightness value, or the average brightness value after removing the extreme brightness values can be used as the brightness of the lamp bead. In the embodiments of the present disclosure, taking the average brightness value of 100 pixel points as the brightness of the corresponding lamp bead as an example, if the brightness of all 100 pixel points is 100, the brightness of the corresponding lamp bead is also 100; if the brightness of all 100 pixel points is 1, the brightness of the corresponding lamp bead is also 1; if the brightness of 100 pixel points is 1-100 respectively, the brightness of the corresponding lamp bead is 50.

[0102] In the embodiments of the present disclosure, after determining the brightness of the lamp bead, it is analyzed whether the mapped specified content meets the expectations, such as whether there is distortion. If so, secondary adjustment can be performed. There are many specific adjustment methods, such as being adjusted by a trained AI model to enhance the display effect.

[0103] When determining the color in the display parameters of the lamp bead, the second picture can be grayscale processed, and all pixel points on the second picture can be characterized by different grayscales.

[0104] After determining the display parameters of the lamp beads mapped by each segmentation region, a dot matrix form of the second picture can be obtained, and then the dot matrix form of the second picture is displayed on the dot matrix display screen.

[0105] Correspondingly, the embodiments of the present disclosure also provide a hardware structure diagram, specifically as Figure 8 shown. The mobile terminal 800 can be the first mobile terminal for implementing the above image processing method 200. As Figure 8As shown, the mobile terminal 800 includes: a processor 810 and a memory 820. Among them, the memory 820 is configured to store program instructions; the processor 810 is configured to load and execute the program instructions stored in the memory 820 to implement the embodiments of the corresponding image processing method as shown above.

[0106] As an embodiment, the memory 820 can be any electronic, magnetic, optical or other physical storage device that can contain or store information such as program instructions, data, etc. For example, the memory 820 can be: a volatile memory, a non-volatile memory or a similar storage medium. Specifically, the memory 820 can be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), a solid state drive, any type of storage disk (such as an optical disk, a DVD, etc.), or a similar storage medium, or a combination thereof.

[0107] So far, the description of the Figure 8 shown mobile terminal is completed.

[0108] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many variations, changes and alternative ways may occur to those skilled in the art without departing from the spirit and scope of the present disclosure. It should be understood that various alternatives to the embodiments of the present disclosure described herein may be employed in practicing the present disclosure. The appended claims are intended to define the scope of protection of the present disclosure and thus cover equivalents or alternatives within the scope of these claims.

Claims

1. An image processing method, characterized in that, The method is applied to a first mobile terminal, and the method includes: Obtain a first picture including a target subject, and extract the target subject area from the first picture along the edge of the target subject to form a second picture; Determine the background information of the background area of the second picture according to the main color of the target subject on the second picture, and / or according to the scene to be shown in the second picture.

2. The method according to claim 1, wherein If the scene to be shown in the second picture is the album of the first mobile terminal, the background information includes the background area color and the background area ratio; the method further includes: Determine the background area color according to the main color of the target subject on the second picture, and determine the background area ratio according to the ratio of the target subject on the second picture.

3. The method according to claim 1, characterized in that, If the scene to be shown in the second picture is a widget on the desktop of the first mobile terminal, and there is a wallpaper set on the desktop, the background information includes the background area color, and the background area color is set to have a light-transmitting effect; so that the wallpaper is presented in the background area.

4. The method according to claim 2, characterized in that, The album also includes multiple third pictures, and the second picture is shown adjacent to the first picture in the album, or replaces the first picture and is displayed at the position where the first picture is located; The display frame of the second picture corresponds to the album zoom instruction, and the second picture is arranged and displayed according to a first preset correspondence; The display frame of the third picture corresponds to the album zoom instruction, and the third picture is arranged and displayed according to a second preset correspondence.

5. The method according to claim 2, wherein In the album, for each row where the second picture is located, the number of second pictures displayed side by side in the row is less than or equal to a first set number.

6. The method according to claim 2, wherein Form a background contour according to the background area ratio, so that the background contour is separated from the edge area of the picture unit configured on the album.

7. The method according to claim 1, wherein Before extracting the target subject area from the first picture along the edge of the target subject to form a second picture, the method further includes one or more of the following: Judge whether the edge of the target subject in the first picture is clear. If the edge of the target subject in the first picture is clear, extract the target subject area from the first picture along the edge of the target subject to form a second picture; Judge whether the target subject in the first picture is covered. If the target subject in the first picture is not covered, extract the target subject area from the first picture along the edge of the target subject to form a second picture; Judge whether the number of target subjects in the first picture is less than or equal to a second set number. If the number of target subjects in the first picture is less than or equal to the second set number, extract one or all of the target subject areas from the first picture along the edge of the target subject to form a second picture.

8. The method according to claim 1, characterized in that The background information includes: the background area color; determining the background area color according to the main color of the target subject on the second picture includes: Determine the candidate color of the background area based on the main color of the target subject; If the absolute difference between the candidate color and the color of the main body edge of the target main body in the second picture is greater than or equal to the set color difference threshold, then use the candidate color as the color of the background area; If the absolute difference between the candidate color and the color data of the main body edge of the target main body in the second picture is less than the set color difference threshold, then adjust the candidate color until the absolute difference between the adjusted candidate color and the color of the main body edge of the target main body in the second picture is greater than or equal to the set color difference threshold, and use the adjusted candidate color as the color of the background area.

9. The method according to claim 1, wherein If the target main body belongs to a preset category, then the method further includes: Input the second picture into a trained dynamic effect generation model to obtain a dynamic effect image of the target main body; Display the dynamic effect image of the target main body according to the scene to be shown in the second picture.

10. The method according to claim 9, characterized in that, After obtaining the dynamic effect image of the target main body, the method further includes: Generate interaction content in response to an interaction action of the user with the dynamic effect image of the target main body; Send the interaction content to a second mobile terminal for display.

11. The method according to claim 1, wherein The first mobile terminal includes: a dot matrix display screen, which is composed of lamp beads arranged in an array; the method further includes: Perform array segmentation on the second picture, and each lamp bead maps at least one segmentation area of the second picture; Determine the display parameters of the lamp beads mapped by each segmentation area to obtain a second picture in dot matrix form; Display the second picture in dot matrix form on the dot matrix display screen.

12. A mobile terminal, where the mobile terminal is a first mobile terminal, characterized in that, Includes: A processor configured to execute program instructions; And A memory configured to store the program instructions, and when the program instructions are loaded and executed by the processor, the processor is caused to execute the image processing method according to any one of claims 1-11.

13. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are loaded and executed by the processor, the processor is caused to execute the image processing method according to any one of claims 1-11.

Images