Image processing method and device, computer readable storage medium and electronic equipment

Abstract

The invention discloses an image processing method and device, a computer readable storage medium and electronic equipment. The method comprises the steps of selecting at least one original image in response to an image selection operation input by a user; generating a display image corresponding to each original image according to each original image in the at least one original image, wherein the transparency of a background part except a main body part in a part exceeding the virtual frame in the display image is fully transparent; and displaying the display image. In the technical scheme provided by the embodiment of the invention, the transparency of the background part except the main body part in the part exceeding the virtual frame in the displayed image is full transparent, so that the main body part in the displayed image can break through the virtual frame to be displayed, and the integrity and the stereoscopic impression of the displayed image are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, an apparatus, a computer-readable storage medium and an electronic device. Background Art

[0002] The gallery service card is another important presentation method and entrance for displaying images on the desktop in addition to icons. By presenting images on the desktop, it is convenient for users to decorate the desktop to improve the aesthetics of the desktop and the convenience of previewing pictures. However, after the user selects the size of the original image and the gallery service card, the existing gallery service card can only crop the original image selected by the user in the same proportion according to the size of the selected gallery service card, so that the displayed image can only show the center part of the original image, and cannot fully display the main part of the original image, and the completeness of the image display is low. In addition, since only a part of the original image can be displayed, the three-dimensional sense of the image display is weak. Summary of the invention

[0003] The object of the present invention is to provide an image processing method, an apparatus, a computer-readable storage medium and an electronic device, so as to improve the integrity and three-dimensional sense of displayed images.

[0004] The present application embodiment adopts the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an electronic device, and the method includes:

[0006] In response to an image selection operation input by a user, selecting at least one original image;

[0007] Generate a display image corresponding to each original image according to each original image in at least one original image, wherein the transparency of the background part of the display image beyond the virtual border except the main part is fully transparent;

[0008] Display the showcase image.

[0009] In the technical solution provided in the embodiment of the present application, the transparency of the background part except the main part in the part of the displayed image that exceeds the virtual border is fully transparent, so that the main part of the displayed image can be displayed beyond the virtual border, thereby improving the integrity and three-dimensional sense of the displayed image.

[0010] In an implementation of the first aspect, generating a display image corresponding to each original image according to each original image of at least one original image specifically includes:

[0011] Determine the picture information of each original image;

[0012] Determine the bounding box information of each original image;

[0013] A display image corresponding to each original image is generated according to the picture information and frame information of each original image.

[0014] In the technical solution provided in the embodiment of the present application, a display image corresponding to each original image is generated based on the picture information and border information of the original image, so as to generate different display images according to the picture information and border information of different original images, so as to enhance the richness of the display images.

[0015] In an implementation of the first aspect, the picture information includes a main part and a background part, and determining the picture information of each original image specifically includes:

[0016] Calculate each original image through a fine segmentation algorithm to generate mask information for each original image;

[0017] Calculate each original image through a depth estimation algorithm to generate depth information of each original image;

[0018] The main part of each original image is determined according to the mask information and the depth information through the Otsu binarization algorithm, and the part other than the main part of each original image is the background part.

[0019] In the technical solution provided in the embodiment of the present application, the main part of the original image can be further filtered out according to the mask information and the depth information through the Otsu binarization algorithm, thereby improving the accuracy of determining the main part.

[0020] In an implementation of the first aspect, the original image is a static image, the border information includes a virtual border, and determining the border information of each original image specifically includes:

[0021] According to the main part of each original image, determine the minimum bounding box of the main part of each original image;

[0022] The minimum bounding box is indented to generate a virtual border for each original image.

[0023] In the technical solution provided in the embodiment of the present application, the minimum external frame is retracted to generate a virtual border for each original image, so as to create an effect that the main part breaks through the virtual border and enhance the three-dimensional sense of the displayed image.

[0024] In an implementation of the first aspect, the original image is a dynamic image, the border information includes a virtual border, and determining the border information of each original image specifically includes:

[0025] According to the main part of each original image, determine the minimum bounding box of the motion range of the main part of each original image;

[0026] The minimum bounding box is indented to generate a virtual border for each original image.

[0027] In the technical solution provided in the embodiment of the present application, the minimum external frame of the movement range of the main part of each original image is determined according to the main part of each original image, so that each frame of the dynamic image can create an effect of the main part breaking through the virtual border, thereby enhancing the three-dimensional sense of the displayed image.

[0028] In an implementation of the first aspect, the border information further includes a cropping border, and determining the border information of each original image specifically includes:

[0029] Each minimum bounding box is expanded according to a set ratio to generate a cropping frame for each original image.

[0030] In the technical solution provided in the embodiment of the present application, the minimum external frame is expanded outward according to the set ratio, which can ensure that the main part can be scaled proportionally when it is scaled, thereby preventing the main part from being deformed.

[0031] In an implementation of the first aspect, if the pre-cropped border generated when the minimum bounding box is expanded according to a set ratio is larger than the original image, a cropped border of the original image is generated according to an overlapping portion of the pre-cropped border and the original image.

[0032] In the technical solution provided in the embodiment of the present application, a cropping frame can be generated according to the size of the original image to ensure the integrity of the generated display image.

[0033] In an implementation of the first aspect, generating a display image corresponding to each original image according to the picture information and frame information of each original image specifically includes:

[0034] Crop each original image according to the cropping border of each original image to generate multiple cropped images;

[0035] The transparency of the background part of each cropped image that exceeds the virtual border except the main part is set to be fully transparent, so as to generate a display image corresponding to each original image.

[0036] In an implementation manner of the first aspect, the shape of the minimum circumscribed frame includes a rectangle, a triangle, or a circle, the shape of the virtual border includes a rectangle, a triangle, or a circle, and the shape of the cropped border includes a rectangle, a triangle, or a circle.

[0037] In the technical solution provided in the embodiment of the present application, the edge of the displayed image can be of various shapes, thereby improving the richness of the displayed image.

[0038] In an implementation of the first aspect, displaying a presentation image specifically includes:

[0039] The display image corresponding to each original image is displayed cyclically according to the set time interval.

[0040] In an implementation of the first aspect, in response to an image selection operation input by a user, selecting at least one original image specifically includes:

[0041] In the resource selection interface of the gallery service card, in response to an image selection operation input by a user, at least one original image is selected;

[0042] Displaying the display image, specifically including: displaying the display image in a gallery service card on the desktop;

[0043] or,

[0044] In response to an image selection operation input by a user, selecting at least one original image specifically includes:

[0045] In the alarm clock wallpaper customization interface, in response to an image selection operation input by a user, at least one original image is selected;

[0046] Displaying a display image specifically includes: displaying a display image in an alarm ringing interface.

[0047] In the technical solution provided in the embodiment of the present application, the above-mentioned image processing method can be applied in different interfaces, and the above-mentioned image processing method has universality.

[0048] In a second aspect, an embodiment of the present application provides an image processing device, which is applied to an electronic device, including:

[0049] A selection module, configured to select at least one original image in response to an image selection operation input by a user;

[0050] An acquisition module, used for generating a display image corresponding to each original image according to each original image in at least one original image, wherein the transparency of the background part of the part exceeding the virtual border in the display image except the main part is fully transparent;

[0051] The display module is used to display the presentation image.

[0052] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the method in the first aspect.

[0053] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1a A schematic diagram of an original image provided by an embodiment of the present application;

[0055] Figure 1b A schematic diagram of displaying an image provided in the related art;

[0056] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application;

[0057] Figure 3 A flowchart of an image processing method provided in one embodiment of the present application;

[0058] Figure 4 A schematic diagram of determining a virtual border provided by an embodiment of the present application;

[0059] Figure 5a A schematic diagram of determining a cutting frame provided in an embodiment of the present application;

[0060] Figure 5b A schematic diagram of another method for determining a cutting frame provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of displaying an image provided by an embodiment of the present application;

[0062] Figure 7a A flowchart of an image processing method provided in one embodiment of the present application;

[0063] Figure 7b A flowchart of another image processing method provided by an embodiment of the present application;

[0064] Figure 8 A flowchart of another image processing method provided by an embodiment of the present application;

[0065] Fig. 9 A schematic diagram of the structure of an image processing device provided in one embodiment of the present application;

[0066] Fig.10 A schematic diagram of the structure of an acquisition module provided in one embodiment of the present application. DETAILED DESCRIPTION

[0067] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, but should not be construed as limiting the present invention.

[0068] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0069] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0070] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0071] Figure 1a A schematic diagram of an original image provided for an embodiment of the present application is shown in FIG1 , where the original image includes a main part (a motorcycle and a person riding the motorcycle) and a background part (land and trees, etc.), and the main part of the original image is located at the top of the entire image. In the related art, after the user selects the size of the original image and the gallery service card, the gallery service card can only crop the original image selected by the user in the same proportion according to the size of the selected gallery service card to generate a display image, so that the display image can only display the center part of the original image. Figure 1b A schematic diagram of displaying an image provided in the related art, such as Figure 1b As shown in the figure, the center part of the original image is the wheel part of the motorcycle and its corresponding background part, and the displayed image only includes the wheel part of the motorcycle and its corresponding background part, and cannot fully display the motorcycle and the person riding the motorcycle, making the image display less complete. In addition, since the displayed image can only display a part of the original image, the three-dimensional sense of the image display is weak.

[0072] In order to solve the technical problems in the related art, an embodiment of the present application provides an electronic device.

[0073] In some embodiments, electronic devices include but are not limited to Or devices with other operating systems.

[0074] In one embodiment of the present application, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a wearable device, a car computer, etc.

[0075] FIG1 is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. Exemplarily, as shown in FIG1, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor (GYRO for short) 180B, an air pressure sensor 180C, a magnetic sensor 180D, an accelerometer (ACC for short) 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0076] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0077] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0078] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0079] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0080] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0081] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0082] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0083] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0084] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0085] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0086] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0087] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0088] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0089] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0090] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0091] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0092] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0093] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0094] In order to solve the above-mentioned technical problems of large-scale image integrity and low three-dimensional sense, based on the above-mentioned electronic device, the embodiment of the present application can intelligently center the main part of the displayed image, and the main part can present the effect of breaking through the frame, thereby improving the visual integrity and three-dimensional sense of the image presentation.

[0095] Based on the electronic device in Figure 1, the following Figures 3 to 6 The image processing method provided in the embodiment of the present application is described in detail. Figure 3 A flowchart of an image processing method provided in one embodiment of the present application is shown in FIG. Figure 3 As shown, the method may include:

[0096] S301: The electronic device selects at least one original image in response to an image selection operation input by a user.

[0097] In one implementation of the present application, in the resource selection interface of the gallery service card, in response to the image selection operation input by the user, at least one original image is selected. In one embodiment of the present application, the size of the gallery service card may include, for example, 2*2, 4*2 or 4*4, etc. The user may long press the gallery service card to enter the resource selection interface of the gallery service card, and may select at least one original image in response to the image selection operation input by the user in the resource selection interface of the gallery service card. In one implementation of the present application, if the user does not input the image selection operation, all images in the gallery are selected by default. In another implementation of the present application, in the alarm wallpaper customization interface, in response to the image selection operation input by the user, at least one original image is selected. In one implementation of the present application, the original image may be an image in any form, and the original image may be a static image or a dynamic image, for example, the dynamic image includes a video. The multiple original images may be multiple static images and / or dynamic images. In one implementation of the present application, after selecting one or more original images in response to the image selection operation input by the user, the original image may be preprocessed to optimize the original image. For example, when the original image includes four subjects and the sky, the personal masking tool of intelligent High Dynamic Range Imaging (HDR) 4 can be used to optimize the contrast, lighting, and skin color of each subject, or the sky segmentation tool can be used to denoise the sky. For another example, when the original image includes one subject (the main part) and the background part, the portrait mode can be used to use the character mask and depth estimation to simulate various effects, such as blurring the background part. For another example, if the original image is an image of a person in stage lighting, the character mask can be used to highlight the main part (character) against a black background, or other masks can be used to correctly render the lighting. For another example, if the original image is in high-key monochrome, the character mask can be used to highlight the main part (character) against a white background. In addition, the original image can also be adjusted through semantic masks.

[0098] S302: The electronic device generates a display image corresponding to each original image according to each original image in at least one original image, and the transparency of the background part of the part exceeding the virtual border in the display image except the main part is fully transparent.

[0099] In one embodiment of the present application, the electronic device can directly obtain the display image corresponding to each original image generated previously to save the energy consumption of the electronic device when in use, or generate the display image corresponding to each original image in real time after the user selects at least one original image. Specifically, the electronic device determines the picture information of each original image; determines the frame information of each original image; generates the display image corresponding to each original image according to the picture information and frame information of each original image. In one embodiment of the present application, the picture information includes the main part and the background part. Specifically, the electronic device can calculate each original image through a fine segmentation algorithm to generate the mask information of each original image; calculate each original image through a depth estimation algorithm to generate the depth information of each original image; determine the main part of each original image according to the mask information and the depth information through the Otsu binarization algorithm (OTSU), and the part other than the main part of each original image is the background part. In one embodiment of the present application, the electronic device can directly obtain the depth information of the photo by taking a photo in a large aperture mode with its own depth information. In one implementation of the present application, the smaller the value of the depth information of the original image, the closer the picture corresponding to the depth information is to the front. For example, if the original image includes a person and the sky, the value of the depth information corresponding to the person is smaller, and the value of the depth information corresponding to the sky is larger. In one embodiment of the present application, the electronic device can filter the mask information according to the depth information through the Otsu binarization algorithm to determine the main part of each original image. For example, when the original image includes a person and the sky, the person is the main part, and the part other than the person is the background part. In one embodiment of the present application, the original image is a static image, and the border information includes a virtual border. Exemplarily, the generation process of the virtual border can be: the electronic device determines the minimum bounding box of the main part of each original image according to the main part of each original image; the minimum bounding box is indented to generate a virtual border for each original image. Figure 4 A schematic diagram of determining a virtual border is provided in an embodiment of the present application, such as Figure 4As shown, the electronic device shrinks the minimum bounding box to generate a virtual border for each original image. The shrinkage ratio of the minimum bounding box can be set according to actual conditions. For example, the shrinkage ratio can be any value between 5% and 10%. In one embodiment of the present application, the original image is a dynamic image, and the border information includes a virtual border. Exemplarily, the process of generating the virtual border can be: the electronic device determines the minimum bounding box of the motion range of the main part of each original image based on the main part of each original image; shrinks the minimum bounding box to generate a virtual border for each original image. In one embodiment of the present application, the border information also includes a cropping border. Specifically, the electronic device can expand each minimum bounding box according to a set ratio to generate a cropping border for each original image. In one implementation of the present application, the aspect ratio of the cropping border can be consistent with the aspect ratio of the gallery service card. Figure 5a A schematic diagram of determining a cropping frame is provided in an embodiment of the present application, such as Figure 5a As shown, if the size of the gallery service card is 2*2 or 4*4, the minimum bounding box is expanded according to the aspect ratio of 1:1, the aspect ratio of the cropping frame is 1:1, and the cropping frame is a square. Figure 5b A schematic diagram of another method for determining a cropping frame is provided in an embodiment of the present application, such as Figure 5b As shown, if the size of the gallery service card is 4*2, the minimum bounding box is expanded according to the aspect ratio of 2:1, the aspect ratio of the cropping border is 2:1, and the cropping border is a rectangle. In one embodiment of the present application, the minimum bounding box is expanded according to the aspect ratio of the gallery service card, which can ensure that the main part can be scaled proportionally when it is scaled, thereby preventing the main part from being deformed. In one embodiment of the present application, if the minimum bounding box is expanded according to the set ratio and the generated pre-cropped border is larger than the original image, the cropped border of the original image can be generated according to the overlapping part of the pre-cropped border and the original image, such as Figure 5b As shown in FIG. 1 , if the minimum bounding box is expanded according to the aspect ratio of 2:1, the generated pre-cropped border is larger than the original image, then the cropped border of the original image is generated according to the overlapping part of the pre-cropped border and the original image ( Figure 5bThe dotted part in the image). Specifically, the electronic device may crop each original image according to the cropping border of each original image to generate multiple cropped images; set the transparency of the background part other than the main part in the part of each cropped image that exceeds the virtual border to be fully transparent, and generate a display image corresponding to each original image. In one implementation of the present application, the electronic device may set the Alpha value of the background part other than the main part in the part of each cropped image that exceeds the virtual border to be fully transparent. In one implementation of the present application, the shape of the minimum bounding box may include a rectangle, a triangle, or a circle, the shape of the virtual border may include a rectangle, a triangle, or a circle, and the shape of the cropped border may include a rectangle, a triangle, or a circle. In one implementation of the present application, the shape of the minimum bounding box, the shape of the virtual border, and the shape of the cropped border may be the same shape or different shapes.

[0100] S303: The electronic device displays a presentation image.

[0101] In one implementation of the present application, the electronic device can cyclically display the display image corresponding to each original image according to a set time interval. In one embodiment of the present application, the set time interval can be set according to actual conditions, for example, the set time interval is 5 seconds. In one implementation of the present application, the electronic device can display the display image in the gallery service card on the desktop. Figure 6 A schematic diagram of displaying an image provided by an embodiment of the present application, such as Figure 6 As shown, when the size of the gallery service card is 4*2, the display image is displayed in the center of the gallery service card, and the part outside the cropped border is not displayed. When the size of the gallery service card is 4*4, a part of the main part (character) of Figure 6 protrudes from the virtual border, so that the main part can present the effect of breaking through the border, and the visual integrity and three-dimensional sense of the image presentation are improved. In one implementation of the present application, the electronic device can display the display image in the alarm ringing interface. Specifically, when the alarm set by the user rings, the electronic device displays the alarm ringing interface, and the alarm ringing interface can include greetings and display images.

[0102] Combine the following Figures 7a to 8 An image processing method provided by an embodiment of the present application is described in detail with a specific embodiment. Figure 7a A flowchart of an image processing method provided in one embodiment of the present application is provided. Figure 7b A flowchart of another image processing method provided in an embodiment of the present application is shown. Figure 8 A flowchart of another image processing method provided in an embodiment of the present application is shown in FIG. Figure 7a , Figure 7b or Figure 8 As shown, the method includes:

[0103] Step 802: The electronic device selects at least one original image in response to an image selection operation input by a user.

[0104] For example, when a user needs to add a gallery service card to the desktop, at least one original image can be selected in the resource selection interface of the gallery service card in response to an image selection operation input by the user. The size of the gallery service card can include 2*2, 4*2 or 4*4. The user can long press the gallery service card to enter the gallery service card interface. In the resource selection interface of the gallery service card, at least one original image can be selected in response to an image selection operation input by the user. Optionally, if the user does not input an image selection operation, all images in the gallery can be selected by default. For another example, when the user needs to customize the alarm wallpaper so that the main part of the customized image can break through the border in the alarm wallpaper interface, at least one original image can be selected in the alarm wallpaper customization interface in response to an image selection operation input by the user.

[0105] In an implementation of the present application, the original image may be an image in any form, and the original image may be a static image or a dynamic image. For example, the dynamic image may include a video.

[0106] Step 804: The electronic device calculates each original image using a fine segmentation algorithm to generate mask information for each original image.

[0107] Step 806: The electronic device calculates each original image using a depth estimation algorithm to generate depth information of each original image.

[0108] In one embodiment of the present application, a photo taken in a large aperture mode has its own depth information, and the depth information of the photo can be directly obtained.

[0109] In one implementation of the present application, the smaller the value of the depth information of the original image, the closer the picture corresponding to the depth information is to the front. For example, if the original image includes a person and a sky, the value of the depth information corresponding to the person is smaller, and the value of the depth information corresponding to the sky is larger.

[0110] Step 808: The electronic device determines the main part of each original image according to the mask information and the depth information by using the Otsu binarization algorithm, and the part of each original image other than the main part is the background part.

[0111] In one embodiment of the present application, the mask information is filtered according to the depth information by using the Otsu binarization algorithm to determine the main part of each original image. In one implementation of the present application, if the original image includes a person and the sky, the person is the main part, and the part other than the person is the background part.

[0112] Step 810: The electronic device determines a minimum bounding box of the main part of each original image according to the main part of each original image.

[0113] In one embodiment of the present application, Figure 7a As shown in FIG. 7b , the minimum bounding box of the main part of the original image (the motorcycle and the person riding the motorcycle) is a rectangle.

[0114] Step 812: The electronic device shrinks the minimum bounding box to generate a virtual border for each original image.

[0115] In one embodiment of the present application, the shrinkage ratio of the minimum external frame can be set according to actual conditions. For example, the shrinkage ratio can be any value between 5% and 10%. Figure 7a As shown in FIG. 7b , the virtual border of the original image is a rectangle.

[0116] Step 814: The electronic device expands each minimum bounding box according to a set ratio to generate a cropping frame for each original image.

[0117] For example, Figure 7a As shown, the ratio can be set according to the aspect ratio of the gallery service card. For example, if the aspect ratio of the gallery service card is 1:1, each external frame is expanded according to the ratio of 1:1 to generate a cropped frame of the original image.

[0118] For example, Figure 7b As shown, the ratio can be set according to the image aspect ratio of the alarm wallpaper customization interface. For example, if the image aspect ratio of the alarm wallpaper customization interface is 1:1, each external frame is expanded according to the ratio of 1:1 to generate a cropped frame of the original image.

[0119] Step 816: The electronic device crops each original image according to the cropping frame of each original image to generate a plurality of cropped images.

[0120] Step 818: The electronic device sets the transparency of the background portion of each cropped image that exceeds the virtual border except for the main body portion to be fully transparent, and generates a display image corresponding to each original image.

[0121] In one implementation of the present application, the Alpha value of the background portion other than the main portion in the portion beyond the virtual border in each cropped image is set to be fully transparent.

[0122] Step 820: The electronic device cyclically displays the display image corresponding to each original image according to a set time interval.

[0123] In one implementation of the present application, each display image is displayed cyclically according to a set time interval. In one embodiment of the present application, the set time interval can be set according to actual conditions, for example, the set time interval is 5 seconds.

[0124] For example, Figure 7a As shown, each display image can be cyclically displayed in the gallery service card on the desktop every 5 seconds. Figure 7b As shown, each display image can be cyclically displayed in the alarm clock ringing interface every 5 seconds. When the alarm clock set by the user rings, the electronic device displays the alarm clock ringing interface, which may include a greeting and at least one display image, for example, the greeting is "Good morning". In the embodiment of the present application, only the gallery service card and the display images displayed in the alarm clock ringing interface are displayed. The technical solution provided in the embodiment of the present application can also be applied to other interfaces, which is not limited here.

[0125] Figures 1a to 8 The schematic diagram shown is an exemplary display of an embodiment of the present application. The user interface of the electronic device may also be in other styles, which is not limited to the embodiment of the present application.

[0126] In the technical solution provided by the embodiment of the present application, in response to the image selection operation input by the user, at least one original image is selected; a display image corresponding to each original image is generated according to each original image in the at least one original image, and the transparency of the background part of the part of the display image that exceeds the virtual border except the main part is fully transparent; and the display image is displayed. In the technical solution provided by the embodiment of the present application, the transparency of the background part of the part of the display image that exceeds the virtual border except the main part is fully transparent, so that the main part of the display image can be displayed beyond the virtual border, thereby improving the integrity and three-dimensional sense of the display image.

[0127] In the technical solution provided in the embodiment of the present application, in order to solve the problem that the main part of the current gallery service card cannot be highlighted, a method is proposed to intelligently center the main part, and the main part can present the effect of breaking through the virtual border, thereby improving the comfort and three-dimensional sense of the gallery service card. The virtual border can be adaptively suspended and transformed according to the content of the image. The main part presented in the alarm clock ringing interface can also break through the virtual border, thereby improving the comfort and three-dimensional sense of the appearance presented in the alarm clock ringing interface.

[0128] In the technical solution provided in the embodiment of the present application, the three-dimensional sense and vividness of the displayed image are improved through a fine segmentation algorithm and a depth estimation algorithm, thereby enhancing the user experience.

[0129] In the technical solution provided in the embodiment of the present application, virtual borders of different shapes and sizes can be selected based on the shapes of different main parts, so that the gallery service card as a whole is more harmonious.

[0130] An embodiment of the present application provides an image processing device, Fig. 9 A schematic diagram of the structure of an image processing device provided in one embodiment of the present application is shown in FIG. Fig. 9 As shown, the device includes: a selection module 11, an acquisition module 12 and a display module 13.

[0131] The selection module 11 is used to select at least one original image in response to an image selection operation input by a user.

[0132] The acquisition module 12 is used to generate a display image corresponding to each original image according to each original image in at least one original image, and the transparency of the background part of the display image beyond the virtual border except the main part is fully transparent.

[0133] The display module 13 is used to display the presentation image.

[0134] In one embodiment of the present application, Fig.10 A schematic diagram of a structure of an acquisition module provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the acquisition module 12 includes: a first determination submodule 1201 , a second determination submodule 1202 and a generation submodule 1203 .

[0135] The first determining submodule 1201 is used to determine the picture information of each original image.

[0136] The second determining submodule 1202 is used to determine the border information of each original image.

[0137] The generation submodule 1203 is used to generate a display image corresponding to each original image according to the picture information and frame information of each original image.

[0138] In one embodiment of the present application, the picture information includes a main part and a background part. The first determination submodule 1201 is specifically used to calculate each original image through a fine segmentation algorithm to generate mask information of each original image; calculate each original image through a depth estimation algorithm to generate depth information of each original image; determine the main part of each original image according to the mask information and the depth information through the Otsu binarization algorithm, and the part of each original image except the main part is the background part.

[0139] In one embodiment of the present application, the original image is a static image, the border information includes a virtual border, and the second determination submodule 1202 is specifically used to determine the minimum bounding box of the main part of each original image according to the main part of each original image; the minimum bounding box is indented to generate a virtual border for each original image.

[0140] In one embodiment of the present application, the original image is a dynamic image, and the border information includes a virtual border. The second determination submodule 1202 is specifically used to determine the minimum bounding box of the motion range of the main part of each original image according to the main part of each original image; the minimum bounding box is indented to generate a virtual border for each original image.

[0141] In one embodiment of the present application, the frame information also includes a cropping frame, and the second determination submodule 1202 is specifically used to expand each minimum bounding box according to a set ratio to generate a cropping frame for each original image.

[0142] In one embodiment of the present application, if the pre-cropped frame generated when the minimum bounding box is expanded according to a set ratio is larger than the original image, a cropped frame of the original image is generated according to the overlapping portion of the pre-cropped frame and the original image.

[0143] In one embodiment of the present application, the generation submodule 1203 is specifically used to crop each original image according to the cropping border of each original image to generate multiple cropped images; set the transparency of the background part of each cropped image that exceeds the virtual border except the main part to be fully transparent, and generate a display image corresponding to each original image.

[0144] In one embodiment of the present application, the display module 13 is specifically configured to cyclically display the display image corresponding to each original image according to a set time interval.

[0145] In one embodiment of the present application, the selection module 11 is specifically used to select at least one or more original images in the resource selection interface of the gallery service card in response to an image selection operation input by a user; the display module 13 is specifically used to display the display image in the gallery service card on the desktop.

[0146] In one embodiment of the present application, the selection module 11 is specifically used to select at least one original image in the alarm wallpaper customization interface in response to an image selection operation input by a user; the display module 13 is specifically used to display the display image in the alarm ringing interface.

[0147] In one embodiment of the present application, the shape of the minimum external frame includes a rectangle, a triangle or a circle, the shape of the virtual frame includes a rectangle, a triangle or a circle, and the shape of the cropped frame includes a rectangle, a triangle or a circle.

[0148] In the technical solution provided by the embodiment of the present application, in response to the image selection operation input by the user, at least one original image is selected; a display image corresponding to each original image is generated according to each original image in the at least one original image, and the transparency of the background part of the part of the display image that exceeds the virtual border except the main part is fully transparent; and the display image is displayed. In the technical solution provided by the embodiment of the present application, the transparency of the background part of the part of the display image that exceeds the virtual border except the main part is fully transparent, so that the main part of the display image can be displayed beyond the virtual border, thereby improving the integrity and three-dimensional sense of the display image.

[0149] In the description of the embodiments of the present application, for the convenience of description, the device is described by dividing it into various modules according to its functions. The division of each module is merely a division of logical functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or one or more software and / or hardware.

[0150] Specifically, the device proposed in the embodiment of the present application can be fully or partially integrated into a physical entity during actual implementation, or it can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in a processor element or instructions in the form of software.

[0151] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0152] Specifically, in one embodiment of the present application, the above-mentioned one or more computer programs are stored in the above-mentioned memory, and the above-mentioned one or more computer programs include instructions. When the above-mentioned instructions are executed by the above-mentioned device, the above-mentioned device executes the method steps described in the embodiment of the present application.

[0153] Furthermore, the devices, apparatuses, and modules described in the embodiments of the present application may be implemented by computer chips or entities, or by products having certain functions.

[0154] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0155] In the several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0156] Specifically, in one embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is run on a computer, the computer executes the method provided in the embodiment of the present application.

[0157] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is run on a computer, the computer executes the method provided by the embodiment of the present application.

[0158] The embodiment description in the present application is described with reference to the flowchart and / or block diagram of the method, equipment (device) and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow chart and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart or multiple flows and / or one box or multiple boxes of the block diagram.

[0159] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0160] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0161] It should also be noted that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0162] In the embodiments of the present application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

[0163] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0164] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0165] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments of the present application can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0167] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: Used in electronic equipment, including: In response to an image selection operation input by a user, selecting at least one original image; Generate a display image corresponding to each original image according to each original image of the at least one original image, wherein the transparency of the background part of the part exceeding the virtual border in the display image except the main part is fully transparent; The presentation image is displayed.

2. The method according to claim 1, characterized in that The step of generating a display image corresponding to each original image according to each original image of the at least one original image specifically includes: Determining picture information of each of the original images; Determine the border information of each of the original images; A display image corresponding to each of the original images is generated according to the picture information and the frame information of each of the original images.

3. The method according to claim 2, characterized in that The picture information includes a main part and a background part, and the determining of the picture information of each original image specifically includes: Calculate each of the original images by a fine segmentation algorithm to generate mask information for each of the original images; Calculating each of the original images using a depth estimation algorithm to generate depth information of each of the original images; The main part of each of the original images is determined according to the mask information and the depth information by using the Otsu binarization algorithm, and the part of each of the original images other than the main part is the background part.

4. The method according to claim 3, characterized in that The original image is a static image, the frame information includes a virtual frame, and determining the frame information of each original image specifically includes: According to the main part of each original image, determining the minimum bounding box of the main part of each original image; The minimum bounding box is indented to generate a virtual frame for each of the original images.

5. The method according to claim 3, characterized in that: The original image is a dynamic image, the frame information includes a virtual frame, and the determining of the frame information of each original image specifically includes: Determine, according to the main part of each of the original images, a minimum bounding box of the motion range of the main part of each of the original images; The minimum bounding box is indented to generate a virtual frame for each of the original images.

6. The method according to claim 4 or 5, characterized in that: The frame information also includes a cropping frame. The step of determining the frame information of each original image specifically includes: Each of the minimum bounding boxes is expanded according to a set ratio to generate a cropping frame for each of the original images.

7. The method according to claim 6, characterized in that If the pre-cropped frame generated when the minimum bounding box is expanded according to the set ratio is larger than the original image, a cropped frame of the original image is generated according to the overlapping portion of the pre-cropped frame and the original image.

8. The method according to claim 6 or 7, characterized in that: Generating a display image corresponding to each original image according to the picture information and the frame information of each original image specifically includes: Cropping each of the original images according to the cropping border of each of the original images to generate a plurality of cropped images; The transparency of the background part of each of the cropped images that exceeds the virtual border, except for the main part, is set to be fully transparent, so as to generate a display image corresponding to each of the original images.

9. The method according to claim 8, characterized in that The shape of the minimum circumscribed frame includes a rectangle, a triangle or a circle, the shape of the virtual border includes a rectangle, a triangle or a circle, and the shape of the cropped border includes a rectangle, a triangle or a circle.

10. The method according to claim 1, characterized in that The displaying of the display image specifically includes: The display image corresponding to each original image is cyclically displayed according to a set time interval.

11. The method according to claim 1, characterized in that: The selecting at least one original image in response to the image selection operation input by the user specifically includes: selecting at least one original image in response to the image selection operation input by the user in the resource selection interface of the gallery service card; The displaying of the display image specifically includes: displaying the display image in a gallery service card on the desktop; or, The selecting at least one original image in response to the image selection operation input by the user specifically includes: selecting at least one original image in response to the image selection operation input by the user in the alarm wallpaper customization interface; The displaying of the display image specifically includes: displaying the display image in an alarm ringing interface.

12. An image processing device, characterized in that: Used in electronic equipment, including: A selection module, configured to select at least one original image in response to an image selection operation input by a user; an acquisition module, configured to generate a display image corresponding to each original image according to each original image in the at least one original image, wherein the transparency of the background part of the part exceeding the virtual border in the display image except the main part is fully transparent; The display module is used to display the display image.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 11.

14. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 11.

Images