Wallpaper image processing method, electronic equipment, storage medium and program product

By determining the overlap area between the control bounding box and the foreground subject mask bounding box, the problem of low efficiency in the foreground subject's coverage calculation of control in the depth of field wallpaper is solved, and efficient and accurate coverage judgment and depth of field wallpaper generation is achieved to ensure the control function and interface readability.

CN119027324APending Publication Date: 2024-11-26HUBEI QIGUANG TECHNOLOGY CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411053697.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the process of building depth of field wallpaper, the foreground body's over-coverage area of ​​the system control will affect the display and function of the control information. The existing technology determines whether each pixel is in the control area to calculate the coverage, which is inefficient and inaccurate. .

Method used

By obtaining wallpaper images and controls, determine the overlap area between the control bounding box and the foreground main mask bounding box, define the pixel alignment range, reduce the calculation amount and improve the efficiency and accuracy of coverage, and the ratio of the overlapping area to the control area is the area of ​​the control. Generate depth of field wallpaper when it is not greater than the threshold.

Benefits of technology

It significantly improves the computing efficiency and result accuracy of the foreground subject's coverage of controls, while ensuring the functional availability of system controls and the readability of interface information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027324A_ABST
    Figure CN119027324A_ABST
Patent Text Reader

Abstract

The invention provides a wallpaper image processing method, electronic equipment, a medium and a program product. The method comprises the following steps: acquiring a wallpaper image and a first control; processing the wallpaper image to obtain a foreground main body corresponding to the wallpaper image and a mask bounding box corresponding to the foreground main body in the wallpaper image; and in response to the fact that the first control is superposed on the wallpaper image, the first control and the mask bounding box have an overlapping region, and the ratio of the area of the overlapping region to the area of the first control is not greater than a proportion threshold value, combining the wallpaper image and the first control to generate the depth-of-field wallpaper. According to the embodiment of the invention, the range of pixel comparison is limited in the overlapping region of the control bounding box and the foreground main body mask bounding box, and the field depth wallpaper is generated when the ratio of the area of the overlapping region to the area of the control is not greater than the threshold value, so that the availability of system control functions and the readability of interface information can be ensured while the field depth wallpaper is generated; and the calculation efficiency and the result precision of the foreground main body coverage degree are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a wallpaper image processing method, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] Depth of field wallpaper can make the main body of the wallpaper stand out from the system controls or other interface elements, thus presenting a three-dimensional layered visual effect, providing the screen interface with rich visual layers and three-dimensional beauty. Regarding the image fusion of the system controls and the foreground main body of the wallpaper, if the foreground main body covers too much area of ​​the system controls, it will affect the information display of the system controls. Therefore, calculating the coverage of the foreground main body on the system controls is a key step to ensure the usability of the control functions and the readability of the interface information.

[0003] In the related art, calculating the coverage of the foreground subject to the system control usually requires determining whether each pixel in the overall image of the foreground subject is within the system control area, and then counting the number of foreground subject pixels within the system control area. However, in the process of constructing a depth of field wallpaper, if the foreground subject image is large in size and high in resolution, the calculation of the coverage will involve a large number of pixel checks. Summary of the invention

[0004] In a first aspect, an embodiment of the present application provides a method for processing a wallpaper image, comprising:

[0005] Acquire a wallpaper image and a first control; process the wallpaper image to obtain a foreground subject corresponding to the wallpaper image and a mask boundary box corresponding to the foreground subject in the wallpaper image;

[0006] In response to the first control being superimposed on the wallpaper image, there being an overlapping area between the first control and the mask bounding box, and a ratio of an area of ​​the overlapping area to an area of ​​the first control being no greater than a ratio threshold, the wallpaper image and the first control are combined to generate a depth of field wallpaper.

[0007] In a second aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and the processor implements any method of the embodiment of the present application when executing the computer program.

[0008] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of any one of the embodiments of the present application is implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any method of the embodiments of the present application when executed by a processor.

[0010] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0012] Figure 1 A block diagram of a mobile phone according to an embodiment of the present application is shown.

[0013] Figure 2 A flowchart of a wallpaper image processing method according to an embodiment of the present application is shown.

[0014] Figure 3 A schematic diagram showing the wallpaper foreground and subject segmentation effect according to an embodiment of the present application is shown.

[0015] Figure 4 A schematic diagram showing calculation of the ratio of the overlapping area to the first control bounding box according to an embodiment of the present application is shown.

[0016] Figure 5 A schematic diagram showing the proportion of foreground and main body in the overlapping area according to an embodiment of the present application is shown.

[0017] Figure 6 The figure shows an overall flow chart of determining the depth of field wallpaper generation condition according to an embodiment of the present application.

[0018] Figure 7 A structural block diagram of a wallpaper image processing device according to an embodiment of the present application is shown.

[0019] Figure 8 A block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0021] The following terms will be used below. It should be noted that the following is a brief description of the basic concepts involved in the embodiments of the present application, and it should be understood that the basic concepts introduced below do not limit the embodiments of the present application.

[0022] User Interface (UI): It is the medium for information exchange between computer systems and users, including the overall design of human-computer interaction, operating logic, and aesthetic interface.

[0023] Control: refers to the graphical interface elements that users can interact with. It allows users to execute specific functions or commands and can feedback important system information. It is a basic component of the user interface.

[0024] Mask: It is used to highlight or hide certain key parts of an image. A mask is usually a binary or grayscale image. In a binary mask image, pixel values ​​are usually divided into two types. One pixel value can represent that the corresponding image area is hidden, and the other pixel value can represent that the corresponding image area is visible.

[0025] Foreground Subject: Refers to the most prominent and visually striking element in an image or scene. The foreground subject is a key part of the composition and usually directly affects the visual effect and emotional expression of the image.

[0026] Depth of field wallpaper can visually highlight the foreground subject of the wallpaper above the system controls or other interface elements, thereby presenting a three-dimensional layered visual effect on the device screen, providing the screen interface with rich visual layers and three-dimensional beauty.

[0027] In order to form a visual layering effect, the foreground layer of the depth of field wallpaper often blocks or covers the system controls of the device interface. In the process of building a depth of field wallpaper, if the foreground subject covers too much area of ​​the system controls, it will affect the information display and function implementation of the system controls. For example, in the process of building a depth of field wallpaper, important notification controls blocked by the foreground layer may cause users to miss important information. For example, in the process of building a depth of field wallpaper, the use of date and time controls and the foreground combination of depth of field wallpapers needs to consider the degree of coverage of the foreground on the date and time controls. Too large a coverage area will affect the display of the time information of the controls. Therefore, calculating and judging the degree of coverage of the foreground subject on the system controls is a key step in ensuring the availability of control functions and the readability of interface information.

[0028] In the related art, it is possible to determine whether the coverage of the foreground subject on the system control meets the generation conditions of the depth of field wallpaper by judging whether each pixel of the foreground subject's overall image is within the system control area. Specifically, each pixel in the foreground subject can be checked to find the ratio of the number of pixels of the foreground subject falling within the system control area to the total number of pixels in the system control area, and obtain the coverage ratio of the foreground subject to the system control, and then determine whether the generated depth of field wallpaper interferes with the usability of the control function and the readability of the interface information. However, in the process of constructing the depth of field wallpaper, the foreground subject image may be large in size and high in resolution. At this time, the calculation of the coverage will involve all pixels of the foreground subject, thereby affecting the efficiency of the calculation process and the accuracy of the results.

[0029] The wallpaper image processing method provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), and can also be applied to databases, servers, and terminal artificial intelligence-based service response systems. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0030] For example, the electronic device may be a station (STATION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a computer, a laptop computer, a handheld communication device, a handheld computing device, and / or other devices for communicating on a wireless system and a next generation communication system, such as a mobile terminal in a 5G network, a mobile terminal in a future evolved Public Land Mobile Network (PLMN) or a mobile terminal in a future evolved Non-terrestrial Network (NTN), etc.

[0031] As an example but not limitation, when the electronic device is a wearable device, the wearable device can also be a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as gloves, watches, AR (Augmented Reality) head-mounted display devices, VR (Virtual Reality) head-mounted display devices, or MR (Mixed Reality) head-mounted display devices equipped with long-distance communication modules and / or short-distance communication modules. In some embodiments, the above-mentioned electronic device can be a device having the following features: Figure 1 The exemplary hardware structure of the exemplary mobile phone 100 shown in FIG. Figure 1 As shown, the mobile phone 100 may specifically include: a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a short-range wireless communication module 170, a processor 180, and a power supply 190. Those skilled in the art will appreciate that Figure 1 The structure of the mobile phone 100 shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Figure 1 The components of the mobile phone 100 are described in detail:

[0032] The RF circuit 110 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 180 for processing; in addition, the data related to the uplink is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 110 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, and the wireless communication can include Global System For Mobile Communications (Global System For Mobile Communications, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Time Division Code Division Multiple Access (Time-Division Code Division Multiple Access, TD-SCDMA), Long Term Evolution (Long Term Evolution, LTE), New Radio (New Radio, NR), GNSS, FM, low-orbit satellite connection and / or IR technology, etc. The GNSS may include a Global Positioning System (GPS), a Global Navigation Satellite System (GLONASS), a BeiDou Navigation Satellite System (BDS), a Quasi-Zenith Satellite System (QZSS) and / or a Satellite Based Augmentation System (SBAS), etc.

[0033] The memory 120 can be used to store software programs and modules. The processor 180 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as pictures, audio data, phone books, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Specifically, the memory 120 may store pictures taken by an electronic device or downloaded via a wireless network.

[0034] The input unit 130 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone 100. Specifically, the input unit 130 may include a touch panel 131 and other input devices 132. The touch panel 131, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 131 or near the touch panel 131 using any suitable object or accessory such as a finger (such as a single finger or multiple fingers), a stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 180, and can receive and execute the command sent by the processor 180. In addition, the touch panel 131 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 131, the input unit 130 may also include other input devices 132. Specifically, other input devices 132 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0035] The display unit 140 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 140 may include a display panel 141. Optionally, the display panel 141 may be configured in the form of a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), etc. Further, the touch panel 131 may cover the display panel 141. When the touch panel 131 detects a touch operation on or near it, it is transmitted to the processor 180 to determine the type of touch event. Subsequently, the processor 180 provides a corresponding visual output on the display panel 141 according to the type of touch event. Although in Figure 1 In the embodiment, the touch panel 131 and the display panel 141 are used as two independent components to implement the input and output functions of the mobile phone, but in some embodiments, the touch panel 131 and the display panel 141 can be integrated to implement the input and output functions of the mobile phone. For example, the display panel 141 is configured to display the wallpaper image and / or the depth of field wallpaper described in the embodiments of the present application.

[0036] The mobile phone 100 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 141 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0037] The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the mobile phone. The audio circuit 160 can transmit the received audio data to the speaker 161 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the processor 180 for processing, and then sent to another electronic device through the RF circuit 110, or the audio data is output to the memory 120 for further processing.

[0038] Communication technologies such as Wi-Fi, Bluetooth, Near Field Communication (NFC) and Ultra Wide Band (UWB) belong to short-range wireless transmission technologies. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the short-range wireless module 170, which provides users with wireless broadband Internet access. The above-mentioned short-range wireless module 170 may include a Wi-Fi chip, a Bluetooth chip, an NFC chip and a UWB chip. Through the Wi-Fi chip, the mobile phone 100 can achieve the function of Wi-Fi Direct connection with other electronic devices, and can also make the mobile phone 100 work in an AP mode (Access Point mode) that can provide wireless access services and allow other wireless devices to access, or work in a STA mode (Station mode) that can connect to an AP but does not accept wireless devices to access, thereby establishing point-to-point communication between the mobile phone 100 and other Wi-Fi devices.

[0039] The processor 180 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 120, and calling data stored in the memory 120, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 180 may include one or more processing units; optionally, the processor 180 may include, for example, an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), a controller, a video codec, a digital signal processor (Digital Signal Processor, DSP), a baseband processor, and / or a neural network processor (Neural-Network Processing Unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. In some examples, the processor 180 is configured to execute the wallpaper image processing method described in the embodiments of the present application.

[0040] The mobile phone 100 also includes a power source 190 (such as a battery) for supplying power to various components. For example, the power source can be logically connected to the processor 180 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system.

[0041] The mobile phone 100 may also include a camera. Optionally, the position of the camera on the mobile phone may be front-mounted or rear-mounted, which is not limited in the embodiments of the present application. In at least one embodiment, the mobile phone may collect a scene image of the current scene through the camera, and determine the scene information and the scene type by parsing the scene image.

[0042] The software system of the mobile phone 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software architecture of the electronic device.

[0043] In some embodiments of the software architecture of the mobile phone 100 of the present application, the Android system is divided into four layers, namely, the application layer, the application framework layer (Framework, FWK), the system layer and the hardware abstraction layer, and the layers communicate with each other through software interfaces.

[0044] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime (Android Runtime) and the system library, and the kernel layer. The application layer may include a series of application packages. The application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, instant messaging, payment, and banking. The application framework layer provides an application programming interface (Application Programming Interface, API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions. The application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0045] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures, such as managing the control enable / disable / hide / display of the preset application in the embodiment of the present application through the view system.

[0046] The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloads, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, the electronic device vibrates, the indicator light flashes, etc.

[0047] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. The system library can include multiple functional modules. For example: Surface Manager, Media Libraries, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.

[0048] The surface manager is used to manage the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, H.265, MP3, AAC, AMR, JPG, HEVC, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing. The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0049] The embodiment of the present application provides a method for processing wallpaper images, which can be applied to electronic devices such as the mobile phone 100 described above in the present application. By determining the overlap between the control bounding box and the foreground subject mask bounding box, the range of pixel comparison is limited to the overlapping area of ​​the control bounding box and the foreground subject mask bounding box, which significantly reduces the amount of calculation and improves the efficiency of calculating the coverage of the foreground subject to the control and the accuracy of the calculation results. In addition, the embodiment of the present application generates a depth of field wallpaper when the ratio of the area of ​​the overlapping area to the area of ​​the control is not greater than the ratio threshold, which can ensure the availability of the system control function and the readability of the interface information while generating a depth of field wallpaper with a three-dimensional aesthetic.

[0050] Figure 2 FIG. 1 is a flowchart showing a wallpaper image processing method according to an embodiment of the present application. Figure 2 , the method specifically comprises the following steps:

[0051] S201, obtaining a wallpaper image and a first control; processing the wallpaper image to obtain a foreground subject corresponding to the wallpaper image and a mask boundary box corresponding to the foreground subject in the wallpaper image.

[0052] S202, in response to the first control being superimposed on the wallpaper image, there being an overlapping area between the first control and the mask boundary box, and the ratio of the area of ​​the overlapping area to the area of ​​the first control being no greater than a ratio threshold, the wallpaper image and the first control are combined to generate a depth of field wallpaper.

[0053] The wallpaper image processing method provided in the embodiment of the present application can be applied to the depth of field wallpaper combination generation work of terminal devices such as personal computers, mobile terminals, smart TVs, wearable devices, etc. Exemplarily, the wallpaper image processing method provided in the embodiment of the present application can judge the depth of field wallpaper generation conditions for the wallpaper image and controls obtained by the user. It should be noted that the above application scenarios or application examples provided in the embodiment of the present application are for ease of understanding, and the embodiment of the present application does not specifically limit the application of the technical solution.

[0054] Exemplarily, the first control may be one or more graphical interface elements with specific interactive functions selected by the user. Functionally, the first control may include functional controls such as weather controls, time and date controls, and application notification controls; in form, the first control may be a button, a text box, a label, etc. The wallpaper image may be a wallpaper image with one or more foreground subjects. The embodiment of the present application does not limit the specific type and form of the first control in the technical solution and the specific content of the wallpaper image.

[0055] In some specific examples, after the first control is acquired, the first control bounding box can be determined first. Specifically, when the first control material image is loaded into the screen interface coordinate system, one or more first control material images selected by the user are organized in a unified area by horizontal or vertical arrangement as the first control bounding box. Optionally, the first control bounding box can be a rectangle, a circle, an ellipse, etc., and the embodiment of the present application does not limit the specific shape of the first control bounding box in the technical solution. Exemplarily, the first control bounding box is a rectangle. The regional attributes of the first control bounding box acquired and recorded include the coordinates of the upper left corner vertex of the first control bounding box in the screen interface coordinate system, the width of the rectangle, the height of the rectangle, etc. Among them, when the first control material image is loaded into the screen interface coordinate system, the default position coordinates of the upper left corner vertex of the first control rectangular bounding box are acquired. If the first control slides in the screen interface coordinate system, the coordinates of the upper left corner vertex of the first control bounding box in the current position in the screen interface coordinate system can be acquired in real time through the sliding process event according to the sliding offset. Further, the width and height of the first control area bounding box can be acquired through the first control material image.

[0056] Exemplarily, in step S201, the wallpaper image can be processed by traditional image algorithms such as image edge detection algorithms (such as Sobel operator, Roberts operator, Laplacian operator, etc.), watershed algorithm, etc., so as to obtain the foreground subject and the corresponding mask area of ​​the wallpaper image. It can also be obtained by image segmentation through pixel clustering using a machine learning clustering algorithm (such as k-Means). In addition, the foreground subject and the corresponding mask area can also be obtained through deep learning image segmentation models (such as U-net, real-time full scene (yoso) segmentation network, etc.). The embodiment of the present application does not limit the specific algorithm used in the technical solution to obtain the foreground subject and the corresponding mask area of ​​the wallpaper image. Exemplarily, the U-net network can be used to segment the wallpaper image, and the pixels can be further merged according to the output of the model to obtain the foreground subject mask area and the foreground subject rectangular area in the wallpaper image.

[0057] Figure 3 A schematic diagram of the wallpaper foreground and subject segmentation effect according to an embodiment of the present application is shown. Figure 4 A schematic diagram showing the calculation of the proportion of the overlapping area in the first control boundary frame according to an embodiment of the present application is shown. Figure 3 and Figure 4 As shown, in some specific examples, Figure 3 The sub-image (a) in is the original wallpaper image. Figure 3 The sub-image (b) in the figure is a mask image of the foreground subject of the original wallpaper image formed by processing the original wallpaper image using the U-net network and merging pixels. The mask image of the foreground subject shown in sub-image (b) can be represented by a single-channel bitmap, where the pixel value of the foreground subject mask area can be 255 (white), and the pixel value of the non-subject mask area can be 0 (black). Further, edge detection is performed on the mask image of the foreground subject, and the edge line is drawn in the original wallpaper image. At the same time, the rectangular bounding box of the subject area in the mask is calculated, that is, the region of interest (ROI) of the subject sub-image in the original image, and the boundary of the subject area is drawn in the original image as Figure 3 The embodiment of the present application does not limit the specific detection algorithm for edge detection of the mask image of the foreground subject in the technical solution. Figure 3 Sub-image (d) shows the rectangular bounding box of the foreground subject mask portion in the wallpaper image, and the foreground subject area image extracted from the original wallpaper image using the mask.

[0058] Exemplarily, when a rectangular foreground subject mask bounding box is loaded in the screen interface coordinate system, the foreground subject mask rectangular bounding box also has default coordinates, and the default position coordinates and area attributes of the upper left corner vertex of the foreground subject mask rectangular bounding box are obtained, including the coordinates of the upper left corner vertex of the foreground subject mask rectangular bounding box in the screen interface coordinate system, the width of the rectangle, the height of the rectangle, etc. Specifically, analogous to the first control bounding box, when the foreground subject mask bounding box slides, the current position coordinates of the upper left corner vertex of the foreground subject mask bounding box in the screen interface coordinate system can be calculated in real time according to the relative position of the sliding during the sliding process, and the width and height of the foreground subject mask bounding box can be further obtained through the known area attributes of the rectangular foreground subject mask bounding box. In this way, by determining the first control bounding box and processing the wallpaper image, the foreground subject corresponding to the wallpaper image and the mask bounding box corresponding to the foreground subject in the wallpaper image are obtained, and the range of pixel comparison can be limited to the overlapping area of ​​the control bounding box and the foreground subject mask bounding box, which significantly reduces the amount of calculation and improves the calculation efficiency of the foreground subject coverage of the control and the accuracy of the calculation results.

[0059] In step S202, in response to the first control being superimposed on the wallpaper image, there is an overlapping area between the first control and the mask bounding box, and the ratio of the area of ​​the overlapping area to the area of ​​the first control is not greater than the ratio threshold, the wallpaper image and the first control can be combined to generate a depth of field wallpaper. Specifically, when combining to generate the depth of field wallpaper, it is first determined whether there is an overlapping area between the foreground subject mask bounding box and the first control bounding box in the screen interface.

[0060] like Figure 4 As shown, in a specific embodiment, when the first control is placed in the screen interface coordinate system 401, the position and area attributes of the first control boundary box 402 are known. The vertex of the current first control area at the upper left corner can be recorded as p c , p c The coordinates are marked as (x c ,y c ), the width of the first control boundary box 402 is recorded as w c , the height of the first control boundary box 402 is recorded as h c When the foreground subject mask bounding box 406 is placed in the screen interface coordinate system 401, the position and region attributes of the foreground subject mask bounding box 406 are known. The top left corner vertex of the current foreground subject mask bounding box 406 can be recorded as p f , p f The coordinates are marked as (x f ,y f ), the width of the foreground subject mask bounding box 406 is denoted as w f , the height is denoted as h f .

[0061] In one implementation, determining an overlapping area between the first control and the mask bounding box includes:

[0062] Get the first coordinate (x c ,y c ), the second coordinate (x f ,y f ), the width w of the first control c , and height h c and the width w of the mask bounding box f and height h f ;

[0063] Determine the first reference coordinates (x1, y1) corresponding to the first reference point on the wallpaper image and the second reference coordinates (x2, y2) corresponding to the second reference point on the wallpaper image; where x1 is max(x c , x f ), y1 is max(y c ,y f ), x2 is min(x c +w c , x f +w f ), y2 is min(y c +h c ,y f +h f );

[0064] Determine a relative position relationship between the first reference point and the second reference point according to the first reference coordinate and the second reference coordinate;

[0065] According to the relative position relationship, an overlapping area between the first control and the mask boundary box is determined.

[0066] In one implementation, determining an overlapping area between the first control and the mask boundary box according to the relative position relationship includes:

[0067] In response to the relative position relationship that the second reference point is located to the lower right of the first reference point, determining that the overlapping region between the first control and the mask bounding box is not an empty set, and determining the area of ​​the overlapping region according to the first reference coordinates and the second reference coordinates; or,

[0068] In response to the relative position relationship that the second reference point is not located to the lower right of the first reference point, it is determined that the overlapping area between the first control and the mask bounding box is an empty set.

[0069] In a specific embodiment, the method for determining whether there is an overlapping area 403 between the foreground subject mask boundary box 406 and the first control boundary box 402 is as follows: define a first reference point p1 and a second reference point p2. Initialize the positions of p1 and p2 in the screen interface coordinate system 401, mark the coordinates of the first reference point as (x1, y1), and mark the coordinates of the second reference point as (x2, y2), where:

[0070] Assign the x1 coordinate of p1 to max(x c , x f ).

[0071] Assign the y1 coordinate of p1 to max(y c ,y f ).

[0072] Assign the x2 coordinate of p2 to min( x c+W c , x f +w f ).

[0073] Assign the y2 coordinate of p2 to min(y c +h c ,y f +h f ).

[0074] like Figure 4 As shown, the overlap between the first control bounding box 402 and the foreground main body mask bounding box 406 can be determined by judging whether p2 is at the lower right of p1, that is, whether the x2 coordinate value of p2 is greater than the x1 coordinate value of p1 and the y2 coordinate value of p2 is greater than the y1 coordinate value of p1. If the conditions are met, it is considered that there is an overlapping area 403 between the foreground main body mask bounding box 406 and the first control bounding box 402, that is, it is determined that the overlapping area between the first control and the mask bounding box is not an empty set, and the area of ​​the overlapping area 403 is determined according to the first reference coordinate and the second reference coordinate. On the contrary, if the conditions are not met, it is considered that there is no overlapping area 403 between the foreground main body mask bounding box 406 and the first control bounding box 402, that is, it is determined that the overlapping area between the first control and the mask bounding box is an empty set.

[0075] If there is no overlapping area between the foreground subject mask bounding box 406 and the first control bounding box 402, it is considered that the foreground subject of the wallpaper image and the first control do not meet the conditions for generating a depth of field wallpaper, and the depth of field wallpaper generation process is directly terminated. On the contrary, if there is an overlapping area between the foreground subject mask bounding box 406 and the first control bounding box 402, it is further determined whether the ratio of the area of ​​the overlapping area 403 to the area of ​​the first control is greater than the ratio threshold.

[0076] Exemplarily, it is determined whether the ratio of the area of ​​the overlapping region 403 to the area of ​​the first control boundary box 402 is greater than the ratio threshold, and the implementation method is described as follows:

[0077] The ratio of the area of ​​the overlapping region 403 to the area of ​​the first control boundary box 402 is recorded as λ1, and the area of ​​the overlapping region is calculated. The coordinates of p1 are recorded as (x1, y1), and the coordinates of p2 are recorded as (x2, y2). Then the area of ​​the overlapping region s r =|(x1-x2)(y1-y2)|.

[0078] Calculate the area of ​​the first control bounding box 402, the area s of the first control bounding box 402 c =w c h c .

[0079] Calculate the proportion λ1,

[0080] It is determined whether λ1 is less than or equal to the preset ratio threshold of the foreground subject to the first control, and the preset ratio threshold is recorded as λ. By setting the preset ratio threshold, it is possible to prevent the foreground subject from covering too large an area of ​​the system control, thereby affecting the information display and function realization of the system control, and ensuring the usability of the control function and the readability of the interface information. If λ1≤λ is satisfied, it is considered that the foreground subject of the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper, generate a depth of field wallpaper, and end the process. On the contrary, if λ1≤λ is not satisfied, based on the number of pixels of the foreground subject or the non-foreground subject in the overlapping area 403, it is continued to be determined whether the foreground subject of the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper. For example, the pixels in the overlapping area 403 are sampled, and the attribution relationship of the sampled pixels is determined by determining whether the pixels are located in the foreground subject mask area 405 or the non-foreground subject mask area 404. Based on the number of pixels of the foreground subject or the non-foreground subject in the overlapping area, it is continued to be determined whether the foreground subject of the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper. Exemplarily, the preset ratio threshold λ can be 25% or other suitable ratios. For example, it can also be determined based on the impact of the ratio of the overlapping area to the area of ​​the first control bounding box 402 on the visual effect of the depth of field wallpaper. The embodiment of the present application does not limit the specific value of the preset ratio threshold in the technical solution.

[0081] In this way, the embodiment of the present application determines the overlap between the control bounding box and the foreground subject mask bounding box, and limits the scope of pixel comparison to the overlapping area of ​​the control bounding box and the foreground subject mask bounding box, thereby significantly reducing the amount of calculation and improving the efficiency of calculating the coverage of the foreground subject over the control and the accuracy of the calculation results; in addition, the embodiment of the present application generates a depth of field wallpaper when the ratio of the area of ​​the overlapping area to the area of ​​the control is not greater than the ratio threshold, which can ensure the availability of the system control function and the readability of the interface information while generating a depth of field wallpaper with a three-dimensional aesthetic.

[0082] In one embodiment, the method for processing a wallpaper image further includes:

[0083] In response to the ratio of the area of ​​the overlapping area to the pixel area of ​​the first control being greater than a ratio threshold, and the number of pixels in the foreground subject located in the overlapping area is less than a first number threshold, or the number of pixels in the non-foreground subject of the wallpaper image located in the overlapping area is greater than or equal to a second number threshold, the wallpaper image and the first control are combined to generate a depth of field wallpaper.

[0084] As mentioned above, when the ratio λ1 of the overlapping area to the area of ​​the first control's bounding box is greater than the preset ratio threshold λ, it is necessary to check the degree of coverage or occlusion of the system control by the foreground subject to prevent the foreground subject from covering too large an area of ​​the system control and affecting the information display and function implementation of the system control. The embodiment of the present application continues to determine whether the foreground subject of the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper based on the number of pixels of the foreground subject or the non-foreground subject in the overlapping area.

[0085] Exemplarily, based on the area of ​​the first control bounding box and a preset ratio threshold, the upper limit of the number of pixels of the foreground subject mask in the overlapping area is calculated and recorded as the first quantity threshold. Based on the area of ​​the overlapping area and the solved first quantity threshold, the upper limit of the number of pixels of the non-foreground subject mask in the overlapping area is calculated and recorded as the second quantity threshold. When the number of pixels in the foreground subject located in the overlapping area is less than the first quantity threshold, or the number of pixels in the non-foreground subject of the wallpaper image located in the overlapping area is greater than or equal to the second quantity threshold, the wallpaper image and the first control can be combined to generate a depth of field wallpaper.

[0086] In one implementation, the number of pixels in the foreground subject located in the overlapping area is less than a first number threshold, and the wallpaper image and the first control are combined to generate a depth of field wallpaper, including:

[0087] The overlapping area is sampled in sequence according to the step length to obtain calibration pixels, wherein the step length is related to the total number of pixels in the overlapping area;

[0088] Determine the attribution relationship of the calibration pixel points according to the pixel values ​​corresponding to the calibration pixel points in the mask boundary box;

[0089] According to the attribution relationship, the number of pixels in the foreground subject located in the overlapping area is updated; and

[0090] In response to the number of pixels in the updated foreground subject located in the overlapping area being less than a first quantity threshold after sampling, the wallpaper image and the first control are combined to generate a depth of field wallpaper, wherein the first quantity threshold is inversely correlated with the step size.

[0091] In a specific embodiment, the pixels in the overlapping area are sampled, and by determining the ownership relationship of the sampled pixels, the number of mask pixels of the foreground subject in the overlapping area is counted to determine the degree of coverage or occlusion of the foreground subject on the system control. The process of determining the ownership relationship of the sampled pixels is referred to as pixel verification.

[0092] Specifically, first determine whether to perform stride sampling on the pixels in the overlapping area. Wallpaper images with higher resolution and larger size have more pixels. When using such wallpaper images to generate depth of field wallpapers, the number of checks will be proportional to the number of pixels. In order to ensure the calculation efficiency of the control coverage, the result is calculated within a given number of checks. The check threshold can be set to limit the number of pixels to be checked, thereby limiting the number of pixel checks. Exemplarily, the overlapping area can be sampled in sequence according to the step size to obtain calibrated pixels. Among them, the step size of the stride sampling can be set to be related to the total number of pixels in the overlapping area.

[0093] In one embodiment, the step size is related to the total number of pixels in the overlapping area, including:

[0094] The step size is determined according to the square root of the ratio of the total number of pixels in the overlapping area to the threshold value of the number of pixels to be compared.

[0095] In some embodiments, the verification threshold may be adaptively set according to the number of target pixel verifications. For example, if the number of pixels involved in the calculation is set to not exceed 1000, the verification threshold m=1000 is set.

[0096] If the number of pixels in the overlapping area is not greater than the verification threshold, stride sampling is not performed, that is, each pixel in the overlapping area participates in the verification. On the contrary, if the number of pixels in the overlapping area is greater than the verification threshold, stride sampling is performed. Exemplarily, the step length can be determined according to the square root of the ratio of the total number of pixels in the overlapping area to the threshold value of the number of pixels to be compared. After determining the step length, the first quantity threshold and the second quantity threshold are updated accordingly. Among them, the first quantity threshold and the second quantity threshold are inversely correlated with the step length.

[0097] In one embodiment, the mask boundary frame includes a first pixel point and a second pixel point, the first pixel point is a pixel point belonging to the foreground subject and the corresponding pixel value is a first preset value, and the second pixel point is a pixel point not belonging to the foreground subject and the corresponding pixel value is a second preset value;

[0098] Determining the attribution of the sampling pixel point according to the pixel value corresponding to the sampling pixel point in the mask boundary box includes: in response to the pixel value corresponding to the sampling pixel point being a first preset value, determining that the sampling pixel point belongs to the foreground subject; or, in response to the pixel value corresponding to the sampling pixel point being a second preset value, determining that the sampling pixel point does not belong to the foreground subject.

[0099] Exemplarily, the first pixel point is a pixel point belonging to the foreground subject and the corresponding pixel value is a first preset value, and the second pixel point is a pixel point not belonging to the foreground subject and the corresponding pixel value is a second preset value. The first preset value can be set to 255 and the second preset value can be set to 0. If the pixel value of the current sampling point is verified to be 255, it is considered that the current sampling point is in the foreground subject mask, and the number of pixels of the foreground subject located in the overlapping area is updated. On the contrary, if the pixel value of the current sampling point is verified to be 0, it is considered that the current sampling point is not in the foreground subject mask, and the number of pixels of the non-foreground subject located in the overlapping area is updated.

[0100] Furthermore, when the number of foreground subject mask pixels is greater than the upper limit of foreground subject mask pixels, it is determined that the foreground subject of the wallpaper image and the first control do not meet the conditions for generating a depth of field wallpaper, and the depth of field wallpaper generation process is terminated. If after sampling, the number of pixels in the updated foreground subject located in the overlapping area is less than the first number threshold, it is determined that the foreground subject of the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper, and the wallpaper image and the first control are combined to generate a depth of field wallpaper.

[0101] In this way, the embodiment of the present application uses a stride sampling method to perform statistical judgment on the number of pixels of the foreground subject mask in the overlapping area, and determines the pixel sampling step by setting an upper threshold value for the number of sampled pixels. This can limit the number of pixel checks when the image resolution is high and the size is large, thereby improving the calculation efficiency of the depth of field wallpaper control coverage while ensuring the accuracy of the results.

[0102] In one implementation, the number of pixels in the non-foreground subject of the wallpaper image that are located in the overlapping area is greater than or equal to a second number threshold, and the wallpaper image and the first control are combined to generate a depth of field wallpaper, including:

[0103] The overlapping area is sampled in sequence according to a step size to obtain calibrated pixels, wherein the step size is related to the total number of pixels in the overlapping area; the attribution relationship of the calibrated pixels is determined according to the pixel values ​​corresponding to the calibrated pixels in the mask boundary box; the number of pixels in the overlapping area in the non-foreground subject is updated according to the attribution relationship; and in response to the number of pixels in the overlapping area in the updated non-foreground subject is greater than or equal to a second quantity threshold after one sampling, the wallpaper image and the first control are combined to generate a depth of field wallpaper, wherein the second quantity threshold is inversely correlated with the step size.

[0104] In a specific embodiment, the pixels in the overlapping area are sampled, and by determining the attribution relationship of the sampled pixels, the number of non-foreground subject mask pixels in the overlapping area is counted to determine the degree of coverage or occlusion of the foreground subject on the system control.

[0105] If after one sampling, the number of pixels in the updated non-foreground subject located in the overlapping area is greater than or equal to the second quantity threshold, then it is judged that the foreground subject of the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper, and the wallpaper image and the first control are combined to generate a depth of field wallpaper.

[0106] In one implementation, the method for processing a wallpaper image further includes:

[0107] In response to the number of pixels in the updated non-foreground subject located in the overlapping area being less than a second threshold after a sampling process, and sampling of the overlapping area is not completed according to the step length, the overlapping area is continuously sampled according to the step length to obtain the next calibration pixel.

[0108] If after one sampling process, the number of pixels in the overlapping area of ​​the non-foreground subject is less than the second number threshold, and the overlapping area is not sampled, the next calibration pixel is obtained to determine whether the sampled pixel is in the foreground subject mask.

[0109] In a specific embodiment, based on the number of pixels of the foreground subject or the non-foreground subject in the overlapping area, it is determined whether the foreground subject of the wallpaper image and the first control meet the condition of generating the depth of field wallpaper, and the implementation method is as follows:

[0110] Based on the area s of the first control's bounding box c and the preset ratio threshold λ, calculate the upper limit of the pixels of the foreground subject mask in the overlapping area, which is recorded as the first quantity threshold n limit , for example, n limit =s c λ.

[0111] Calculate the upper limit of the number of pixels of the non-foreground subject mask in the overlapping area, recorded as the second number threshold ~n limit , illustratively, ~nlimit =s r -n limit .

[0112] As mentioned above, the upper limit n of the number of pixels of the foreground subject mask in the overlapping area is calculated. limit , and the upper limit of the number of pixels of the non-foreground subject mask in the overlap area is ~n limit After that, it is necessary to sample the pixels in the overlapping area, and determine the ownership of the sampled pixels by judging whether the pixels are in the foreground subject mask area or the non-foreground subject mask area. Count the number of foreground subject mask pixels or the number of non-foreground subject mask pixels in the overlapping area. When the number of foreground subject mask pixels is greater than the upper limit of foreground subject mask pixels or the number of non-foreground subject mask pixels is greater than the upper limit of non-foreground subject mask pixels, it is judged that the foreground subject of the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper, generate the depth of field wallpaper and end the process.

[0113] Specifically, first determine whether to perform stride sampling. For example, there are n pixels in the overlapping area, and the threshold of the number of pixels to be compared is set to m. The specific determination method is as follows:

[0114] If n≤m is satisfied, stride sampling is not performed, that is, each pixel in the overlapping area is involved in the verification, and the default sampling step is step=1. On the contrary, if n≤m is not satisfied, stride sampling is performed, and the implementation method is described as follows:

[0115] The step length is determined based on the square root of the ratio of the total number of pixels in the overlapping area to the threshold number of pixels to be compared. Specifically, the step length step is calculated, and let Available

[0116] Update n at the same time limit and ~n limit .in,

[0117] In this way, the embodiment of the present application uses a stride sampling method to perform statistical judgment on the number of pixels of the foreground subject mask in the overlapping area, and determines the pixel sampling step by setting an upper threshold value for the number of sampled pixels. This can limit the number of pixel checks when the image resolution is high and the size is large, thereby improving the calculation efficiency of the foreground subject coverage while ensuring the accuracy of the results.

[0118] Figure 5 A schematic diagram showing the proportion of foreground and subject in the overlapping area according to an embodiment of the present application is shown as follows: Figure 5 As shown, the method for determining whether a pixel is in the foreground subject mask area and determining the attribution relationship of the sampled pixel point is described as follows:

[0119] Step a: Initialize the foreground subject mask pixel count in the overlap area 403 to n s = 0, the non-foreground subject mask pixel count in the overlap area is ~n s =0.

[0120] Step b: In the overlapping area, sampling is performed in the horizontal and vertical directions with a step size of step.

[0121] Step c: determine whether the currently sampled pixel is in the foreground subject mask area 405. In response to the pixel value corresponding to the sampled pixel being the first preset value, determine that the sampled pixel belongs to the foreground subject; or, in response to the pixel value corresponding to the sampled pixel being the second preset value, determine that the sampled pixel does not belong to the foreground subject. Exemplarily, the first preset value can be set to 255 and the second preset value to 0. If the pixel value of the current sampled point is 255, it is considered that the current sampled point is in the foreground subject mask. If it exists in the foreground subject mask, then n s =n s +1. On the contrary, if the pixel value of the current sampling point is 0, it is considered that the current sampling point is not in the foreground subject mask area 405. If it does not exist in the foreground subject mask area 405, then ~n s =~n s +1.

[0122] Step d: Determine the number of mask pixels n of the current foreground subject s Is it more than n limit , if n s >n limit , then the ratio of the foreground subject pixel area in the overlapping area to the first control boundary box area exceeds λ, then it is determined that the wallpaper image and the first control do not meet the depth of field wallpaper generation conditions, and the depth of field wallpaper generation process ends. s >n limit If not, proceed to step e.

[0123] Step e: Determine the number of current non-foreground subject mask pixels ~n s Whether it exceeds ~n limit , if ~n s >~n limit , it is determined that the wallpaper image and the first control meet the depth of field wallpaper generation condition, and the process ends. This step is the reverse calculation of step d. When the number of pixels in the non-foreground subject mask area 404 in the overlapping area is greater than the threshold value ˜n limit When n is the number of foreground subject mask pixels remaining in the overlapping area 403 s Must be less than n limit If ~n s >~n limitIf not, continue to step f.

[0124] Step f, if ~n s >~n limit If it is not true, then determine whether the sampling is finished, that is, whether the sampling calculation in the overlapping area is completed. If the sampling is not finished, get the next sampling point and go to step c. On the contrary, if the sampling statistics are finished and n does not appear s >n limit , it is considered that the wallpaper image and the first control meet the depth of field wallpaper generation condition, and then the depth of field wallpaper is generated, and the process ends.

[0125] When the embodiment of the present application calculates the coverage of the foreground subject over the control in the overlapping area, the calculation is terminated when the number of foreground subject mask pixels is greater than the upper limit of the foreground subject mask pixels or the number of non-foreground subject mask pixels is greater than the upper limit of the non-foreground subject mask pixels. By using both forward and reverse statistical judgments in the sampling statistics process, the calculation efficiency of the control coverage and the accuracy of the calculation results can be improved in the depth of field wallpaper generation process.

[0126] Figure 6 FIG. 2 shows an overall flow chart of the depth of field wallpaper generation condition determination according to an embodiment of the present application. Figure 6 As shown, in a specific embodiment, the overall process of determining whether the wallpaper image and the first control meet the conditions for generating a depth of field wallpaper is as follows:

[0127] Acquire a wallpaper image and a first control; determine a first control boundary box, and process the wallpaper image to obtain a foreground subject corresponding to the wallpaper image and a mask boundary box corresponding to the foreground subject in the wallpaper image.

[0128] In response to the first control being superimposed on the wallpaper image, there is an overlapping area between the first control and the mask bounding box, and the ratio of the area of ​​the overlapping area to the area of ​​the first control is not greater than the ratio threshold, the wallpaper image and the first control can be combined to generate a depth of field wallpaper.

[0129] Specifically, when combining and generating a depth of field wallpaper, first determine whether there is an overlapping area between the foreground subject mask boundary box and the first control boundary box in the screen interface. If there is no overlapping area between the foreground subject mask boundary box and the first control boundary box, it is determined that the foreground subject of the wallpaper image and the first control do not meet the conditions for generating a depth of field wallpaper, and the depth of field wallpaper generation process is directly terminated. On the contrary, if there is an overlapping area between the foreground subject mask boundary box and the first control boundary box, it is continued to determine whether the ratio of the overlapping area area to the first control area area is greater than the ratio threshold.

[0130] Further, the preset ratio threshold is recorded as λ, and the ratio of the area of ​​the overlapping region to the area of ​​the first control boundary box is recorded as λ1. In response to the ratio λ1 of the area of ​​the overlapping region to the pixel area of ​​the first control being greater than the ratio threshold λ, the pixel points in the overlapping region are sampled, and the attribution relationship of the sampled pixel points is determined by judging whether the pixel is in the foreground subject mask area or the non-foreground subject mask area, and the number of foreground subject mask pixels and the number of non-foreground subject mask pixels in the overlapping region are counted.

[0131] Based on the area of ​​the first control bounding box and the preset ratio threshold, the upper limit of the pixels of the foreground subject mask in the overlapping area is calculated, which is recorded as the first quantity threshold n limit , calculate the upper limit of the pixels of the non-foreground subject mask in the overlapping area, recorded as the second quantity threshold ~n limit .

[0132] Determine whether to perform stride sampling. For example, there are n pixels in the overlapping area, and the threshold number of pixels to be compared is set to m. If n≤m, no stride sampling is performed, that is, every pixel in the overlapping area participates in the verification. Otherwise, stride sampling is performed, and the step length of stride sampling is determined according to the square root of the ratio of the total number of pixels in the overlapping area to the threshold number of pixels to be compared, and n is updated at the same time. limit and ~n limit .

[0133] In response to the pixel value corresponding to the sampling pixel being a first preset value, it is determined that the sampling pixel belongs to the foreground subject; or, in response to the pixel value corresponding to the sampling pixel being a second preset value, it is determined that the sampling pixel does not belong to the foreground subject. Determine the attribution relationship of the sampling pixel and count the number of foreground subject mask pixels or the number of non-foreground subject mask pixels in the overlapping area. The number of foreground subject pixels in the overlapping area is less than a first number threshold n limit In the case of, or the number of non-foreground subject pixels located in the overlapping area is greater than or equal to the second number threshold value ˜n limit In the case of the foreground subject of the wallpaper image and the first control satisfying the condition for generating a depth of field wallpaper, the depth of field wallpaper is generated and the process ends.

[0134] The embodiment of the present application generates a depth of field wallpaper when the ratio of the area of ​​the overlapping area to the area of ​​the control is not greater than the ratio threshold, which can ensure the availability of the system control functions and the readability of the interface information while generating a depth of field wallpaper with a three-dimensional aesthetic. In addition, the embodiment of the present application limits the scope of pixel comparison to the overlapping area of ​​the control boundary box and the foreground subject mask boundary box, and uses a stride sampling method to perform statistical judgment on the number of pixels of the foreground subject mask in the overlapping area, and uses forward and reverse statistical judgments simultaneously in the sampling and statistical process, which significantly reduces the amount of calculation and improves the calculation efficiency of the control coverage and the accuracy of the calculation results.

[0135] Corresponding to the method provided in the embodiment of the present application, the embodiment of the present application further provides a wallpaper image processing device, which is used to implement the wallpaper image processing method of any one of the above-mentioned implementation modes of the present application. Figure 7 A structural block diagram of a wallpaper image processing device according to an embodiment of the present application is shown as follows: Figure 7 As shown, the wallpaper image processing device includes:

[0136] The processing module 701 obtains a wallpaper image and a first control; processes the wallpaper image to obtain a foreground subject corresponding to the wallpaper image and a mask boundary box corresponding to the foreground subject in the wallpaper image.

[0137] The wallpaper generation module 702 combines the wallpaper image and the first control to generate a depth of field wallpaper in response to the first control being superimposed on the wallpaper image, the first control and the mask boundary box having an overlapping area, and the ratio of the area of ​​the overlapping area to the area of ​​the first control being no greater than a ratio threshold.

[0138] Figure 8 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 8 As shown, the electronic device includes: a memory 810 and a processor 820. The memory 810 stores a computer program that can be run on the processor 820. When the processor 820 executes the computer program, the method in the above embodiment is implemented. The number of the memory 810 and the processor 820 can be one or more.

[0139] The processor 820 may be a device on a chip (SOC), and the processor 820 may include a central processing unit (CPU), and may further include other types of processors.

[0140] The processor involved in the processor 820 may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units, NPU) and an image signal processor (Image Signal Processing, ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASICs, or one or more integrated circuits for controlling the execution of the program of the technical solution of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0141] The processor 820 may include one or more processing units. For example, the processor may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent components or integrated into one or more processors.

[0142] The memory 810 may include a code storage area and a data storage area. The code storage area may store an operating system. The data storage area may store data created during the use of the electronic device, etc. In addition, the memory 810 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, a flash memory component, a universal flash storage (Universal Flash Storage, UFS), etc.

[0143] The memory 810 can be a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or it can also be any computer-readable medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0144] The electronic device also includes:

[0145] The communication interface 830 is used to communicate with external devices and perform data exchange transmission.

[0146] If the memory 810, the processor 820 and the communication interface 830 are implemented independently, the memory 810, the processor 820 and the communication interface 830 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0147] In some embodiments, the processor 820 may include one or more interfaces. The interface may include an Inter-Integrated Circuit (I2C) interface, an Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a SIM card interface, and / or a USB interface, etc. Among them, the USB interface is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface can be used to connect a charger to charge an electronic device, and can also be used to transmit data between an electronic device and a peripheral device.

[0148] The electronic device may also include an external memory interface, which is used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 1001 through the external memory interface to implement a data storage function. For example, files such as music and videos are stored in the external memory card.

[0149] Optionally, in a specific implementation, if the memory 810, the processor 820 and the communication interface 830 are integrated on a chip, the memory 810, the processor 820 and the communication interface 830 can communicate with each other through an internal interface.

[0150] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0151] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0152] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0153] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0154] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM).

[0155] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0156] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0157] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0158] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0159] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0160] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0161] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0162] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing a wallpaper image, comprising: Get the wallpaper image and the first control; Processing the wallpaper image to obtain a foreground subject corresponding to the wallpaper image and a mask boundary box corresponding to the foreground subject in the wallpaper image; In response to the first control being superimposed on the wallpaper image, there being an overlapping area between the first control and the mask bounding box, and a ratio of an area of ​​the overlapping area to an area of ​​the first control being no greater than a ratio threshold, the wallpaper image and the first control are combined to generate a depth of field wallpaper.

2. The method for processing a wallpaper image according to claim 1, further comprising: In response to the ratio of the area of ​​the overlapping area to the pixel area of ​​the first control being greater than the ratio threshold, and the number of pixels in the foreground subject located in the overlapping area is less than a first number threshold, or the number of pixels in the non-foreground subject of the wallpaper image located in the overlapping area is greater than or equal to a second number threshold, the wallpaper image and the first control are combined to generate a depth of field wallpaper.

3. The method for processing wallpaper images according to claim 2, wherein: The number of pixels in the foreground subject located in the overlapping area is less than the first number threshold, and the wallpaper image and the first control are combined to generate a depth of field wallpaper, including: Sampling the overlapping area in sequence according to a step length to obtain calibration pixels, wherein the step length is related to the total number of pixels in the overlapping area; Determining the attribution relationship of the calibration pixel point according to the pixel value corresponding to the calibration pixel point in the mask boundary frame; According to the attribution relationship, updating the number of pixels in the foreground subject located in the overlapping area; and In response to the number of pixels in the updated foreground subject located in the overlapping area being less than the first quantity threshold after the sampling is completed, the wallpaper image and the first control are combined to generate a depth of field wallpaper, wherein the first quantity threshold is inversely correlated with the step size.

4. The method for processing a wallpaper image according to claim 2, wherein: The number of pixels in the non-foreground subject of the wallpaper image and located in the overlapping area is greater than or equal to a second number threshold, and the wallpaper image and the first control are combined to generate a depth of field wallpaper, including: Sampling the overlapping area in sequence according to a step length to obtain calibration pixels, wherein the step length is related to the total number of pixels in the overlapping area; Determining the attribution relationship of the calibration pixel point according to the pixel value corresponding to the calibration pixel point in the mask boundary frame; According to the attribution relationship, updating the number of pixels in the non-foreground subject located in the overlapping area; and In response to the number of pixels in the non-foreground subject located in the overlapping area being greater than or equal to a second quantity threshold after one sampling, the wallpaper image and the first control are combined to generate a depth of field wallpaper, wherein the second quantity threshold is inversely correlated with the step size.

5. The method for processing a wallpaper image according to claim 4, further comprising: In response to the fact that after a sampling process, the number of pixel points in the updated non-foreground subject located in the overlapping area is less than a second quantity threshold, and the sampling of the overlapping area is not completed according to the step length, the sampling of the overlapping area is continued according to the step length to obtain the next calibration pixel point.

6. The wallpaper image processing method according to any one of claims 3 or 4, wherein: The step size is related to the total number of pixels in the overlapping area, including: The step size is determined according to the square root of the ratio of the total number of pixels in the overlapping area to a threshold value for the number of pixels to be compared.

7. The method for processing wallpaper images according to claim 3, wherein: The mask boundary frame includes a first pixel point and a second pixel point, the first pixel point is a pixel point belonging to the foreground subject and the corresponding pixel value is a first preset value, and the second pixel point is a pixel point not belonging to the foreground subject and the corresponding pixel value is a second preset value; Determining the attribution relationship of the sampling pixel points according to the pixel values ​​corresponding to the sampling pixel points in the mask boundary box includes: In response to a pixel value corresponding to the sampling pixel being the first preset value, determining that the sampling pixel belongs to the foreground subject; or, In response to the pixel value corresponding to the sampling pixel point being the second preset value, it is determined that the sampling pixel point does not belong to the foreground subject.

8. The method for processing a wallpaper image according to claim 1, wherein: Determining an overlapping area between the first control and the mask boundary box includes: Get the first coordinate (x) corresponding to the first specified point of the first control c ,y c ), the second coordinate (x ) corresponding to the second specified point of the mask bounding box f ,y f ), the width w of the first control c , and height h c and the width w of the mask bounding box f and height h f ; Determine a first reference coordinate (x1, y1) corresponding to a first reference point on the wallpaper image and a second reference coordinate (x2, y2) corresponding to a second reference point on the wallpaper image; wherein x1 is max(x c , x f ), y1 is max(y c ,y f ), x2 is min(x c +w c ,x f +w f ), y2 is min(y c +h c ,y f +h f ); Determine a relative position relationship between the first reference point and the second reference point according to the first reference coordinates and the second reference coordinates; According to the relative position relationship, an overlapping area between the first control and the mask boundary box is determined.

9. The method for processing a wallpaper image according to claim 8, wherein: Determining an overlapping area between the first control and the mask boundary box according to the relative position relationship includes: In response to the relative position relationship that the second reference point is located to the lower right of the first reference point, determining that the overlapping region between the first control and the mask bounding box is not an empty set, and determining the area of ​​the overlapping region according to the first reference coordinates and the second reference coordinates; or, In response to the relative position relationship being that the second reference point is not located to the lower right of the first reference point, it is determined that an overlapping area between the first control and the mask bounding box is an empty set.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 9 when executing the computer program.

11. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Citation Information

Cited By

  • Processing method, intelligent terminal and storage medium

    CN121349354A

  • Processing method, intelligent terminal and storage medium

    CN121357281A