Image processing method and apparatus, computer-readable storage medium and electronic device

Abstract

Disclosed in the present application are an image processing method and apparatus, a computer-readable storage medium and an electronic device. The method comprises: in response to an image selection operation input by a user, selecting at least one original image; on the basis of each original image among the at least one original image, generating a presentation image corresponding to each original image, wherein a background part, except a body part, in the part of the presentation image that exceeds a virtual frame is fully transparent; and displaying the presentation image. In the technical solution provided by the embodiments of the present application, a background part, except a body part, in the part of a presentation image that exceeds a virtual frame is fully transparent, and thus the body part of the presentation image can be displayed beyond the virtual frame, improving the integrity and stereoscopic impression of the presentation image.

Description

Image processing method, device, computer-readable storage medium, and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 1, 2023, with application number 202311135305.9 and application name “Image processing method, device, computer-readable storage medium and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the field of image processing technology, and in particular to an image processing method, an apparatus, a computer-readable storage medium, and an electronic device. Background Art

[0003] The gallery service card is another important way to display images on the desktop, in addition to icons. By presenting images on the desktop, users can decorate their desktop, enhance its aesthetics, and improve the convenience of previewing images. However, after the user selects the size of the original image and the gallery service card, the existing gallery service card can only crop the original image in the same proportion based on the selected gallery service card size. As a result, the displayed image only shows the center portion of the original image, not the entire main body of the original image, resulting in a low degree of image integrity. Furthermore, since only a portion of the original image can be displayed, the three-dimensional effect of the image display is weak.

[0004] Summary of the Invention

[0005] The object of the present invention is to provide an image processing method, an apparatus, a computer-readable storage medium and an electronic device, so as to improve the integrity and stereoscopic effect of displayed images.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides an image processing method, applied to an electronic device, the method comprising:

[0008] In response to an image selection operation input by a user, selecting at least one original image;

[0009] generating a display image corresponding to each original image according to each original image of the at least one original image, wherein the transparency of the background portion of the display image beyond the virtual border except the main portion is fully transparent;

[0010] Displays the showcase image.

[0011] In the technical solution provided in the embodiment of the present application, the transparency of the background part of the displayed image beyond the virtual border except the main part is fully transparent, so that the main part of the displayed image can be displayed beyond the virtual border, thereby improving the integrity and three-dimensional sense of the displayed image.

[0012] In an implementation of the first aspect, generating a display image corresponding to each original image according to each original image of at least one original image specifically includes:

[0013] Determine the picture information of each original image;

[0014] Determine the bounding box information of each original image;

[0015] A display image corresponding to each original image is generated according to the picture information and frame information of each original image.

[0016] In the technical solution provided in the embodiment of the present application, a display image corresponding to each original image is generated based on the picture information and border information of the original image, so as to generate different display images according to the picture information and border information of different original images, so as to enhance the richness of the display image.

[0017] In an implementation of the first aspect, the picture information includes a main portion and a background portion, and determining the picture information of each original image specifically includes:

[0018] Calculate each original image through a fine segmentation algorithm to generate mask information for each original image;

[0019] Calculate each original image through the depth estimation algorithm to generate the depth information of each original image;

[0020] The main part of each original image is determined according to the mask information and the depth information through the Otsu binarization algorithm, and the part other than the main part of each original image is the background part.

[0021] In the technical solution provided in the embodiment of the present application, the main part of the original image can be further filtered out according to the mask information and depth information through the Otsu binarization algorithm, thereby improving the accuracy of determining the main part.

[0022] In an implementation of the first aspect, the original image is a static image, the border information includes a virtual border, and determining the border information of each original image specifically includes:

[0023] Determine a minimum bounding box of the main part of each original image according to the main part of each original image;

[0024] The minimum bounding box is indented to generate a virtual border for each original image.

[0025] In the technical solution provided in the embodiment of the present application, the minimum external frame is retracted to generate a virtual border for each original image, so as to create an effect that the main part breaks through the virtual border and enhance the three-dimensional effect of the displayed image.

[0026] In an implementation of the first aspect, the original image is a dynamic image, the border information includes a virtual border, and determining the border information of each original image specifically includes:

[0027] Determine, based on the main part of each original image, a minimum bounding box of the motion range of the main part of each original image;

[0028] The minimum bounding box is indented to generate a virtual border for each original image.

[0029] In the technical solution provided in the embodiment of the present application, the minimum external frame of the motion range of the main part of each original image is determined based on the main part of each original image, so that each frame of the dynamic image can create the effect of the main part breaking through the virtual border, thereby enhancing the three-dimensional sense of the displayed image.

[0030] In an implementation of the first aspect, the border information further includes a cropping border, and determining the border information of each original image specifically includes:

[0031] Each minimum bounding box is expanded according to a set ratio to generate a cropping frame for each original image.

[0032] In the technical solution provided in the embodiment of the present application, the minimum external frame is expanded according to the set ratio, which can ensure that the main part can be scaled proportionally when it is scaled, thereby preventing the main part from being deformed.

[0033] In an implementation of the first aspect, if the pre-cropped frame generated when the minimum bounding box is expanded according to a set ratio is larger than the original image, a cropped frame of the original image is generated based on the overlapping portion of the pre-cropped frame and the original image.

[0034] In the technical solution provided in the embodiment of the present application, a cropping frame can be generated according to the size of the original image to ensure the integrity of the generated display image.

[0035] In an implementation of the first aspect, generating a display image corresponding to each original image according to the picture information and frame information of each original image specifically includes:

[0036] cropping each original image according to a cropping border of each original image to generate a plurality of cropped images;

[0037] The transparency of the background portion of each cropped image that exceeds the virtual border, excluding the main portion, is set to be fully transparent, to generate a display image corresponding to each original image.

[0038] In an implementation of the first aspect, the shape of the minimum circumscribed frame includes a rectangle, a triangle, or a circle, the shape of the virtual border includes a rectangle, a triangle, or a circle, and the shape of the cropped border includes a rectangle, a triangle, or a circle.

[0039] In the technical solution provided in the embodiment of the present application, the edges of the displayed image can be of various shapes, thereby improving the richness of the displayed image.

[0040] In an implementation of the first aspect, displaying a presentation image specifically includes:

[0041] The display image corresponding to each original image is displayed cyclically according to the set time interval.

[0042] In an implementation of the first aspect, selecting at least one original image in response to an image selection operation input by a user specifically includes:

[0043] In the resource selection interface of the gallery service card, in response to an image selection operation input by a user, at least one original image is selected;

[0044] Displaying the display image, specifically including: displaying the display image in a gallery service card on the desktop;

[0045] or,

[0046] In response to an image selection operation input by a user, selecting at least one original image specifically includes:

[0047] In the alarm clock wallpaper customization interface, in response to an image selection operation input by a user, at least one original image is selected;

[0048] Displaying a display image specifically includes: displaying a display image in an alarm ringing interface.

[0049] In the technical solution provided in the embodiment of the present application, the above-mentioned image processing method can be applied in different interfaces, and the above-mentioned image processing method has universality.

[0050] In a second aspect, an embodiment of the present application provides an image processing device, applied to an electronic device, comprising:

[0051] a selection module, configured to select at least one original image in response to an image selection operation input by a user;

[0052] an acquisition module, configured to generate a display image corresponding to each original image according to each original image of the at least one original image, wherein the transparency of the background portion of the display image beyond the virtual border, excluding the main portion, is fully transparent;

[0053] The display module is used to display the presentation image.

[0054] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the method in the first aspect.

[0055] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] FIG1a is a schematic diagram of an original image provided by an embodiment of the present application;

[0057] FIG1b is a schematic diagram of a display image provided in the related art;

[0058] FIG2 is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application;

[0059] FIG3 is a flowchart of an image processing method provided by an embodiment of the present application;

[0060] FIG4 is a schematic diagram of determining a virtual border according to an embodiment of the present application;

[0061] FIG5a is a schematic diagram of determining a cropping frame according to an embodiment of the present application;

[0062] FIG5 b is a schematic diagram of another method for determining a cropping frame according to an embodiment of the present application;

[0063] FIG6 is a schematic diagram of a display image provided by an embodiment of the present application;

[0064] FIG7a is a flowchart of an image processing method provided by an embodiment of the present application;

[0065] FIG7 b is a flowchart of another image processing method provided by an embodiment of the present application;

[0066] FIG8 is a flowchart of another image processing method provided by an embodiment of the present application;

[0067] FIG9 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

[0068] FIG10 is a schematic structural diagram of an acquisition module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0070] It should be clear that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0071] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0072] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0073] Figure 1a is a schematic diagram of an original image provided in an embodiment of the present application. As shown in Figure 1, the original image includes a main part (a motorcycle and a person riding a motorcycle) and a background part (land and trees, etc.). The main part of the original image is located at the top of the entire image. In the related art, after the user selects the size of the original image and the gallery service card, the gallery service card can only crop the original image selected by the user in the center according to the size of the selected gallery service card to generate a display image, so that the display image can only display the center part of the original image. Figure 1b is a schematic diagram of a display image provided in the related art. As shown in Figure 1b, the center part of the original image is the wheel part of the motorcycle and its corresponding background part. The display image only includes the wheel part of the motorcycle and its corresponding background part, and cannot fully display the motorcycle and the person riding the motorcycle, resulting in a low degree of completeness of the image display. In addition, since the display image can only display a part of the original image, the three-dimensional sense of the image display is weak.

[0074] In order to solve the technical problems in the related art, an embodiment of the present application provides an electronic device.

[0075] In some embodiments, electronic devices include but are not limited to Or devices with other operating systems.

[0076] In one embodiment of the present application, electronic devices include but are not limited to mobile phones, tablet computers, laptop computers, wearable devices, car computers, etc.

[0077] FIG1 is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. For example, as shown in FIG1 , the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor (GYRO for short) 180B, an air pressure sensor 180C, a magnetic sensor 180D, an accelerometer (ACC for short) 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0078] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0079] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0080] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0081] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0082] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0083] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0084] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0085] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0086] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0087] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0088] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0089] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0090] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0091] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0092] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0093] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0094] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0095] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0096] In order to solve the above-mentioned technical problems of large-scale image integrity and low three-dimensional sense, based on the above-mentioned electronic device, the embodiment of the present application can intelligently center the main part of the displayed image, and the main part can present the effect of breaking through the border, thereby improving the visual integrity and three-dimensional sense of the image presentation.

[0097] Based on the electronic device in Figure 1, the image processing method provided by an embodiment of the present application is described in detail below in conjunction with Figures 3 to 6. Figure 3 is a flow chart of an image processing method provided by an embodiment of the present application. As shown in Figure 3, the method may include:

[0098] S301: The electronic device selects at least one original image in response to an image selection operation input by a user.

[0099] In one implementation of the present application, in response to a user input of an image selection operation, at least one original image is selected in the resource selection interface of a gallery service card. In one embodiment of the present application, the gallery service card may have a size of, for example, 2*2, 4*2, or 4*4. The user can long-press the gallery service card to enter the resource selection interface of the gallery service card. In response to the user input of an image selection operation, at least one original image can be selected in response to the user input of an image selection operation. In one implementation of the present application, if the user does not input an image selection operation, all images in the gallery are selected by default. In another implementation of the present application, in response to a user input of an image selection operation, at least one original image is selected in the alarm wallpaper customization interface. In one implementation of the present application, the original image can be any image format, and can be a static image or a dynamic image, for example, a dynamic image includes a video. The multiple original images can be multiple static images and / or dynamic images. In one implementation of the present application, after one or more original images are selected in response to the user input of an image selection operation, the original images can be pre-processed to optimize the original images. For example, when the original image includes four subjects and the sky, you can use the personal mask tool of Intelligent High Dynamic Range Imaging (HDR) 4 to optimize the contrast, lighting, and skin color of each subject, or you can use the sky segmentation tool to denoise the sky. For another example, when the original image includes one subject (main part) and background part, you can use the character mask and depth estimation in portrait mode to simulate various effects, such as blurring the background part. For another example, if the original image is an image of a person under stage lighting, you can use the character mask to highlight the main part (person) against a black background, or you can use other masks to correctly render the lighting. For another example, if the original image is in high-key monochrome, you can use the character mask to highlight the main part (person) against a white background. In addition, the original image can also be adjusted through semantic masks.

[0100] S302: The electronic device generates a display image corresponding to each original image according to each original image of at least one original image, and the transparency of the background part of the part exceeding the virtual border in the display image except the main part is fully transparent.

[0101] In one embodiment of the present application, the electronic device can directly obtain the display image corresponding to each original image generated previously to save energy consumption of the electronic device when in use, or generate the corresponding display image in real time according to each original image after the user selects at least one original image. Specifically, the electronic device determines the picture information of each original image; determines the border information of each original image; generates the display image corresponding to each original image according to the picture information and border information of each original image. In one embodiment of the present application, the picture information includes the main part and the background part. Specifically, the electronic device can calculate each original image through a fine segmentation algorithm to generate the mask information of each original image; calculate each original image through a depth estimation algorithm to generate the depth information of each original image; determine the main part of each original image according to the mask information and depth information through the Otsu binarization algorithm (OTSU), and the part of each original image other than the main part is the background part. In one embodiment of the present application, the photo taken in the large aperture mode has its own depth information, and the electronic device can directly obtain the depth information of the photo. In one implementation of the present application, the smaller the depth information value of the original image, the closer the image to which the depth information corresponds is to the front. For example, if the original image includes a person and a sky, the depth information value corresponding to the person is smaller, while the depth information value corresponding to the sky is larger. In one embodiment of the present application, the electronic device can filter mask information based on the depth information using the Otsu binarization algorithm to determine the main portion of each original image. For example, if the original image includes a person and a sky, the person is the main portion, and the portion other than the person is the background portion. In one embodiment of the present application, the original image is a static image, and the border information includes a virtual border. Exemplarily, the virtual border generation process can include: the electronic device determines a minimum bounding box for the main portion of each original image based on the main portion of each original image; and indents the minimum bounding box to generate a virtual border for each original image. Figure 4 is a schematic diagram of determining a virtual border according to one embodiment of the present application. As shown in Figure 4, the electronic device indents the minimum bounding box to generate a virtual border for each original image. The indentation ratio of the minimum bounding box can be set according to actual conditions. For example, the indentation ratio can be any value between 5% and 10%. In one embodiment of the present application, the original image is a dynamic image, and the border information includes a virtual border. Exemplarily, the process of generating the virtual border may be: the electronic device determines the minimum bounding box of the motion range of the main portion of each original image based on the main portion of each original image; and shrinks the minimum bounding box to generate a virtual border for each original image. In one embodiment of the present application, the border information also includes a cropping border. Specifically, the electronic device may expand each minimum bounding box according to a set ratio to generate a cropping border for each original image.In one implementation of the present application, the aspect ratio of the cropping border can be consistent with the aspect ratio of the gallery service card. Figure 5a is a schematic diagram of determining the cropping border provided by an embodiment of the present application. As shown in Figure 5a, if the size of the gallery service card is 2*2 or 4*4, the minimum external frame is expanded according to the aspect ratio of 1:1, the aspect ratio of the cropping border is 1:1, and the cropping border is a square. Figure 5b is another schematic diagram of determining the cropping border provided by an embodiment of the present application. As shown in Figure 5b, if the size of the gallery service card is 4*2, the minimum external frame is expanded according to the aspect ratio of 2:1, the aspect ratio of the cropping border is 2:1, and the cropping border is a rectangle. In an embodiment of the present application, the minimum external frame is expanded according to the aspect ratio of the gallery service card, which can ensure that the main part can be scaled proportionally when it is scaled, thereby preventing the main part from being deformed. In one embodiment of the present application, if the pre-cropped border generated when the minimum bounding box is expanded according to a set ratio is larger than the original image, a cropped border of the original image can be generated based on the overlapping portion of the pre-cropped border and the original image, as shown in Figure 5b. If the pre-cropped border generated when the minimum bounding box is expanded according to a 2:1 aspect ratio is larger than the original image, a cropped border of the original image (the dotted portion in Figure 5b) is generated based on the overlapping portion of the pre-cropped border and the original image. Specifically, the electronic device can crop each original image according to the cropped border of each original image to generate multiple cropped images; set the transparency of the background portion of each cropped image that exceeds the virtual border, excluding the main body portion, to be fully transparent, to generate a display image corresponding to each original image. In one implementation of the present application, the electronic device can set the alpha value of the background portion of each cropped image that exceeds the virtual border, excluding the main body portion, to be fully transparent. In one implementation of the present application, the shape of the minimum circumscribed frame may include a rectangle, a triangle, or a circle, the shape of the virtual border may include a rectangle, a triangle, or a circle, and the shape of the cropped border may include a rectangle, a triangle, or a circle. In one implementation of the present application, the shape of the minimum circumscribed frame, the shape of the virtual border, and the shape of the cropped border may be the same shape or different shapes.

[0102] S303: The electronic device displays a presentation image.

[0103] In one implementation of the present application, the electronic device can cyclically display the display image corresponding to each original image according to a set time interval. In one embodiment of the present application, the set time interval can be set according to actual conditions, for example, the set time interval is 5 seconds. In one implementation of the present application, the electronic device can display the display image in the gallery service card on the desktop. Figure 6 is a schematic diagram of a display image provided in an embodiment of the present application. As shown in Figure 6, when the size of the gallery service card is 4*2, the display image is displayed in the center of the gallery service card, and the part outside the cropped border is not displayed. When the size of the gallery service card is 4*4, a part of the main body (character) in Figure 6 protrudes from the virtual border, so that the main body can appear to break through the border, thereby improving the visual integrity and three-dimensional sense of the image presentation. In one implementation of the present application, the electronic device can display the display image in the alarm ringing interface. Specifically, when the alarm set by the user rings, the electronic device displays the alarm ringing interface, which can include greetings and display images.

[0104] An image processing method provided by an embodiment of the present application is described in detail below with reference to FIG. 7a to FIG. 8 , using a specific embodiment as an example. FIG. 7a is a flow chart of an image processing method provided by an embodiment of the present application, FIG. 7b is a flow chart of another image processing method provided by an embodiment of the present application, and FIG. 8 is a flow chart of another image processing method provided by an embodiment of the present application. As shown in FIG. 7a , FIG. 7b , or FIG. 8 , the method includes:

[0105] Step 802: The electronic device selects at least one original image in response to an image selection operation input by the user.

[0106] For example, when a user needs to add a gallery service card to the desktop, at least one original image can be selected in the resource selection interface of the gallery service card in response to the image selection operation input by the user. The sizes of the gallery service card can include 2*2, 4*2 or 4*4. The user can long press the gallery service card to enter the gallery service card interface. In the resource selection interface of the gallery service card, at least one original image can be selected in response to the image selection operation input by the user. Optionally, if the user does not input an image selection operation, all images in the gallery can be selected by default. For another example, when the user needs to customize the alarm clock wallpaper so that the main part of the customized image can break through the border in the alarm clock wallpaper interface, at least one original image can be selected in response to the image selection operation input by the user in the alarm clock wallpaper customization interface.

[0107] In one implementation of the present application, the original image may be an image in any form, and the original image may be a static image or a dynamic image. For example, the dynamic image may include a video.

[0108] Step 804: The electronic device calculates each original image using a fine segmentation algorithm to generate mask information for each original image.

[0109] Step 806: The electronic device calculates each original image using a depth estimation algorithm to generate depth information of each original image.

[0110] In one embodiment of the present application, a photo taken in a large aperture mode has its own depth information, and the depth information of the photo can be directly obtained.

[0111] In one implementation of the present application, the smaller the value of the depth information of the original image, the closer the picture corresponding to the depth information is to the front. For example, if the original image includes a person and the sky, the value of the depth information corresponding to the person is smaller, and the value of the depth information corresponding to the sky is larger.

[0112] Step 808: The electronic device determines the main part of each original image according to the mask information and the depth information using the Otsu binarization algorithm, and the part of each original image other than the main part is the background part.

[0113] In one embodiment of the present application, the mask information is filtered based on the depth information using the Otsu binarization algorithm to determine the main portion of each original image. In one implementation of the present application, if the original image includes a person and a sky, the person is the main portion, and the portion other than the person is the background portion.

[0114] Step 810: The electronic device determines a minimum bounding box of the main part of each original image according to the main part of each original image.

[0115] In one embodiment of the present application, as shown in FIG. 7 a or 7 b , the minimum bounding box of the main part of the original image (the motorcycle and the person riding the motorcycle) is a rectangle.

[0116] Step 812: The electronic device shrinks the minimum bounding box to generate a virtual border for each original image.

[0117] In one embodiment of the present application, the shrinkage ratio of the minimum bounding box can be set according to actual conditions, for example, the shrinkage ratio can be any value between 5% and 10%. As shown in Figure 7a or 7b, the virtual frame of the original image is a rectangle.

[0118] Step 814: The electronic device expands each minimum bounding box according to a set ratio to generate a cropping frame for each original image.

[0119] For example, as shown in FIG7a, the ratio can be set according to the aspect ratio of the gallery service card. For example, if the aspect ratio of the gallery service card is 1:1, each external frame is expanded according to the ratio of 1:1 to generate a cropped frame of the original image.

[0120] For another example, as shown in FIG7b, the ratio can be set according to the image aspect ratio of the alarm clock wallpaper customization interface. For example, if the image aspect ratio of the alarm clock wallpaper customization interface is 1:1, each external frame is expanded according to the ratio of 1:1 to generate a cropped frame of the original image.

[0121] Step 816: The electronic device crops each original image according to the cropping frame of each original image to generate multiple cropped images.

[0122] Step 818: The electronic device sets the transparency of the background portion of each cropped image that exceeds the virtual border, excluding the main portion, to be fully transparent, and generates a display image corresponding to each original image.

[0123] In one implementation of the present application, the Alpha value of the background portion of each cropped image that exceeds the virtual border, excluding the main portion, is set to be fully transparent.

[0124] Step 820: The electronic device cyclically displays the display image corresponding to each original image according to a set time interval.

[0125] In one implementation of the present application, each display image is displayed cyclically according to a set time interval. In one embodiment of the present application, the set time interval can be set according to actual conditions, for example, the set time interval is 5 seconds.

[0126] For example, as shown in FIG7a, each display image can be displayed in a loop in the gallery service card on the desktop every 5 seconds. For another example, as shown in FIG7b, each display image can be displayed in a loop in the alarm clock ringing interface every 5 seconds. When the alarm clock set by the user rings, the electronic device displays the alarm clock ringing interface. The alarm clock ringing interface may include a greeting and at least one display image, for example, the greeting is "Good morning". In the embodiment of the present application, only the display images displayed in the gallery service card and the alarm clock ringing interface are displayed. The technical solution provided in the embodiment of the present application can also be applied to other interfaces and is not limited here.

[0127] The schematic diagrams shown in Figures 1a to 8 are exemplary displays of an embodiment of the present application. The user interface of the electronic device may also be in other styles, which is not limited in this embodiment of the present application.

[0128] In the technical solution provided by the embodiments of the present application, in response to an image selection operation input by a user, at least one original image is selected; a display image corresponding to each original image is generated based on each original image, and the transparency of the background portion of the display image, excluding the main portion, that exceeds the virtual border is fully transparent; and the display image is displayed. In the technical solution provided by the embodiments of the present application, the transparency of the background portion of the display image, excluding the main portion, that exceeds the virtual border is fully transparent, allowing the main portion of the display image to be displayed beyond the virtual border, thereby improving the integrity and three-dimensional effect of the display image.

[0129] The technical solution provided in the embodiments of this application addresses the problem of the main body of the current gallery service card not being highlighted. A method is proposed to intelligently center the main body, allowing it to appear to break through the virtual border, thereby improving the visual comfort and three-dimensionality of the gallery service card. The virtual border can be adaptively suspended and transformed based on the content of the image. The main body presented in the alarm clock ringing interface can also break through the virtual border, improving the visual comfort and three-dimensionality of the alarm clock ringing interface.

[0130] In the technical solution provided in the embodiment of the present application, the three-dimensional sense and vividness of the displayed image are improved through the fine segmentation algorithm and the depth estimation algorithm, thereby enhancing the user experience.

[0131] In the technical solution provided in the embodiment of the present application, virtual borders of different shapes and sizes can be selected based on the shapes of different main parts, making the gallery service card more harmonious as a whole.

[0132] An embodiment of the present application provides an image processing device. FIG9 is a structural diagram of an image processing device provided by an embodiment of the present application. As shown in FIG9 , the device includes: a selection module 11 , an acquisition module 12 and a display module 13 .

[0133] The selection module 11 is configured to select at least one original image in response to an image selection operation input by a user.

[0134] The acquisition module 12 is configured to generate a display image corresponding to each original image according to each original image in the at least one original image, wherein the transparency of the background portion of the display image beyond the virtual border except the main portion is fully transparent.

[0135] The display module 13 is used to display presentation images.

[0136] In one embodiment of the present application, Figure 10 is a structural diagram of an acquisition module provided in one embodiment of the present application. As shown in Figure 10, the acquisition module 12 includes: a first determination submodule 1201, a second determination submodule 1202 and a generation submodule 1203.

[0137] The first determining submodule 1201 is used to determine the picture information of each original image.

[0138] The second determining submodule 1202 is used to determine the border information of each original image.

[0139] The generating submodule 1203 is used to generate a display image corresponding to each original image according to the picture information and frame information of each original image.

[0140] In one embodiment of the present application, the picture information includes a main part and a background part. The first determination submodule 1201 is specifically used to calculate each original image through a fine segmentation algorithm to generate mask information of each original image; calculate each original image through a depth estimation algorithm to generate depth information of each original image; determine the main part of each original image according to the mask information and depth information through the Otsu binarization algorithm, and the part of each original image other than the main part is the background part.

[0141] In one embodiment of the present application, the original image is a static image, and the border information includes a virtual border. The second determination submodule 1202 is specifically used to determine the minimum external frame of the main part of each original image based on the main part of each original image; the minimum external frame is indented to generate a virtual border for each original image.

[0142] In one embodiment of the present application, the original image is a dynamic image, and the border information includes a virtual border. The second determination submodule 1202 is specifically used to determine the minimum bounding box of the motion range of the main part of each original image based on the main part of each original image; the minimum bounding box is indented to generate a virtual border for each original image.

[0143] In one embodiment of the present application, the frame information further includes a cropping frame, and the second determining submodule 1202 is specifically configured to expand each minimum bounding box according to a set ratio to generate a cropping frame for each original image.

[0144] In one embodiment of the present application, if the minimum bounding box is expanded according to a set ratio and the generated pre-cropped frame is larger than the original image, a cropped frame of the original image is generated based on the overlapping portion of the pre-cropped frame and the original image.

[0145] In one embodiment of the present application, the generation submodule 1203 is specifically used to crop each original image according to the cropping border of each original image to generate multiple cropped images; set the transparency of the background part of each cropped image that exceeds the virtual border except the main part to be fully transparent to generate a display image corresponding to each original image.

[0146] In one embodiment of the present application, the display module 13 is specifically configured to cyclically display the display image corresponding to each original image according to a set time interval.

[0147] In one embodiment of the present application, the selection module 11 is specifically used to select at least one or more original images in the resource selection interface of the gallery service card in response to the image selection operation input by the user; the display module 13 is specifically used to display the display image in the gallery service card on the desktop.

[0148] In one embodiment of the present application, the selection module 11 is specifically used to select at least one original image in the alarm wallpaper customization interface in response to an image selection operation input by the user; the display module 13 is specifically used to display the display image in the alarm ringing interface.

[0149] In one embodiment of the present application, the shape of the minimum circumscribed frame includes a rectangle, a triangle, or a circle, the shape of the virtual frame includes a rectangle, a triangle, or a circle, and the shape of the cropped frame includes a rectangle, a triangle, or a circle.

[0150] In the technical solution provided by the embodiments of the present application, in response to an image selection operation input by a user, at least one original image is selected; a display image corresponding to each original image is generated based on each original image, and the transparency of the background portion of the display image, excluding the main portion, that exceeds the virtual border is fully transparent; and the display image is displayed. In the technical solution provided by the embodiments of the present application, the transparency of the background portion of the display image, excluding the main portion, that exceeds the virtual border is fully transparent, allowing the main portion of the display image to be displayed beyond the virtual border, thereby improving the integrity and three-dimensional effect of the display image.

[0151] In the description of the embodiments of the present application, for the convenience of description, the device is described as being divided into various modules according to their functions. The division of each module is merely a division of logical functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0152] Specifically, the device proposed in the embodiment of the present application can be fully or partially integrated into a physical entity during actual implementation, or it can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; or they can all be implemented in the form of hardware; or some modules can be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. During the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0153] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0154] Specifically, in one embodiment of the present application, the above-mentioned one or more computer programs are stored in the above-mentioned memory, and the above-mentioned one or more computer programs include instructions. When the above-mentioned instructions are executed by the above-mentioned device, the above-mentioned device executes the method steps described in the embodiment of the present application.

[0155] Furthermore, the devices, apparatuses, and modules described in the embodiments of the present application may be implemented by computer chips or entities, or by products having certain functions.

[0156] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, apparatus, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0157] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this application.

[0158] Specifically, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes the method provided in the embodiment of the present application.

[0159] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program product is run on a computer, it enables the computer to execute the method provided in the embodiment of the present application.

[0160] The embodiment description in this application is described with reference to the flow chart and / or block diagram according to the method, equipment (device) and computer program product of embodiment of the present application.It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer, embedded processing machine or other programmable data processing equipment to produce a machine, so that the instruction executed by the processor of computer or other programmable data processing equipment produces the device for realizing the function specified in one flow chart flow chart or multiple flow charts and / or one block or multiple blocks of block diagram.

[0161] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0163] It should also be noted that, in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.

[0164] In the embodiments of the present application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, commodity, or apparatus comprising the element.

[0165] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0166] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

[0167] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments of the present application can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: Used in electronic equipment, including: In response to an image selection operation input by a user, selecting at least one original image; Generate a display image corresponding to each original image according to each original image of the at least one original image, wherein the transparency of the background part of the part exceeding the virtual border in the display image except the main part is fully transparent; The presentation image is displayed.

2. The method according to claim 1, characterized in that Generating a display image corresponding to each original image according to each original image of the at least one original image specifically includes: Determining picture information of each of the original images; Determine the border information of each of the original images; A display image corresponding to each of the original images is generated according to the picture information and the frame information of each of the original images.

3. The method according to claim 2, characterized in that The picture information includes a main part and a background part, and the determining of the picture information of each original image specifically includes: Calculate each of the original images by a fine segmentation algorithm to generate mask information for each of the original images; Calculating each of the original images using a depth estimation algorithm to generate depth information of each of the original images; The main part of each of the original images is determined according to the mask information and the depth information by using the Otsu binarization algorithm, and the part of each of the original images other than the main part is the background part.

4. The method according to claim 3, characterized in that The original image is a static image, the frame information includes a virtual frame, and the determining of the frame information of each original image specifically includes: Determine, according to the main part of each original image, the minimum bounding box of the main part of each original image; The minimum bounding box is indented to generate a virtual frame for each of the original images.

5. The method according to claim 3, characterized in that: The original image is a dynamic image, the frame information includes a virtual frame, and the determining of the frame information of each original image specifically includes: Determine, according to the main part of each of the original images, a minimum bounding box of the motion range of the main part of each of the original images; The minimum bounding box is indented to generate a virtual frame for each of the original images.

6. The method according to claim 4 or 5, characterized in that: The frame information also includes a cropping frame. The step of determining the frame information of each original image specifically includes: Each of the minimum bounding boxes is expanded according to a set ratio to generate a cropping frame for each of the original images.

7. The method according to claim 6, characterized in that If the pre-cropped frame generated when the minimum bounding box is expanded according to the set ratio is larger than the original image, a cropped frame of the original image is generated according to the overlapping portion of the pre-cropped frame and the original image.

8. The method according to claim 6 or 7, characterized in that: Generating a display image corresponding to each original image according to the picture information and the frame information of each original image specifically includes: Crop each of the original images according to the cropping border of each of the original images to generate a plurality of cropped images; The transparency of the background part of each of the cropped images that exceeds the virtual border, except for the main part, is set to be fully transparent, so as to generate a display image corresponding to each of the original images.

9. The method according to claim 8, characterized in that The shape of the minimum circumscribed frame includes a rectangle, a triangle or a circle, the shape of the virtual border includes a rectangle, a triangle or a circle, and the shape of the cropped border includes a rectangle, a triangle or a circle.

10. The method according to claim 1, characterized in that The displaying of the display image specifically includes: The display image corresponding to each original image is cyclically displayed according to a set time interval.

11. The method according to claim 1, characterized in that: The selecting at least one original image in response to the image selection operation input by the user specifically includes: selecting at least one original image in response to the image selection operation input by the user in the resource selection interface of the gallery service card; The displaying of the display image specifically includes: displaying the display image in a gallery service card on the desktop; or, The selecting at least one original image in response to the image selection operation input by the user specifically includes: selecting at least one original image in response to the image selection operation input by the user in the alarm wallpaper customization interface; The displaying of the display image specifically includes: displaying the display image in an alarm ringing interface.

12. An image processing device, characterized in that: Used in electronic equipment, including: A selection module, configured to select at least one original image in response to an image selection operation input by a user; an acquisition module, configured to generate a display image corresponding to each original image according to each original image in the at least one original image, wherein the transparency of the background part of the part exceeding the virtual border in the display image except the main part is fully transparent; The display module is used to display the display image.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 11.

14. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 11.

Images