Display method, electronic equipment and computer readable medium

Abstract

The invention relates to the technical field of computers, in particular to a display method, electronic equipment and a computer readable medium, by adjusting display parameters influencing foreground elements in a set target interface, including display positions of the foreground elements in corresponding background image areas and other related parameters, the foreground elements in the target interface can be displayed. Therefore, the foreground element on the set target interface is arranged to the position where the interaction effect with the saliency main body can be generated. Thus, on the basis of the display method provided by the invention, when the background image is set for the target interface, foreground elements such as a time component and a notification component can be laid out to the position where interactivity is generated with characteristics such as actions of saliency subjects such as figures in the original background image, and the interestingness of the target interface can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and particularly to a display method, an electronic device, and a computer-readable medium. Background Art

[0002] As terminal electronic devices such as mobile phones, tablet computers, and smart watches play an increasingly important role in people's lives, the screen display content of electronic devices also carries more and more functions, and users' requirements for the aesthetics of screen content are also getting higher and higher. Among them, using images stored locally in the electronic device as the interface background, such as the lock screen wallpaper or the desktop wallpaper, etc., has become a common choice for users. The aforementioned images can be, for example, images taken by a camera and images downloaded by the user.

[0003] Due to the limitations of the screen size and card size of the terminal device, it is necessary to crop the image according to a certain aspect ratio. In addition, components such as a clock component, a lock screen notification, or a floating notification are also displayed on the lock screen interface or the desktop of the mobile phone, etc. These components are usually arranged at fixed positions on the lock screen interface desktop. This results in the image used as the interface background after cropping being often blocked by components such as the clock component, the lock screen notification, or the floating notification, and the aesthetic property of the composition formed by the cropped image and related components or floating windows is poor, affecting the user experience. Summary of the Invention

[0004] In view of this, the present application provides a display method, an electronic device, and a computer-readable medium, which can control an interactive effect to be generated between relevant components and the above-mentioned salient subject when the relevant components are laid out on the background image in some interface display scenarios.

[0005] In a first aspect, the present application provides a display method applied to an electronic device. The method includes: detecting an interface display instruction; in response to the interface display instruction, displaying a first interface, where the first interface includes at least one first foreground element and a first background image area in the first background image, and the display parameter of the first foreground element in the first background image area includes a position parameter of the first foreground element, and the position parameter is related to the semantic information of at least one display element in the first background image area.

[0006] For example, the above-mentioned first foreground element may be a time component, a notification component, etc. displayed on the desktop or the lock screen interface. The above-mentioned first background image may be an image of the wallpaper set by the user as the desktop or the lock screen interface, which may be described below as the original background image. The corresponding first background image area may be a partial area cropped from the first background image to fit the screen size of an electronic device such as a mobile phone. The display parameter including the position parameter of the first foreground element may be, for example, a parameter related to determining the display position of the time component, the notification component, etc. on the desktop or the lock screen interface. And at least one display element in the above-mentioned first background image area may be, for example, a prominent subject in the original background image, including prominent subjects such as people, animals, plants, delicious foods, buildings, or landscapes.

[0007] It can be understood that the above semantic information may be the result of image semantic understanding of the original background image. The semantic information may include descriptions of one or more prominent subjects included in the image, the actions or forms of the prominent subjects, etc., as well as related descriptions such as the corresponding action objects or form extension directions.

[0008] In a possible implementation of the above first aspect, the semantic information includes one or more of the following features: features related to actions; features related to forms.

[0009] In a possible implementation of the above first aspect, the features related to actions include one or more of the following features: features for describing actions; features for describing the objects of actions.

[0010] For example, among the above features related to actions, the features for describing actions may be features such as "kick", "run", "jump", "grasp", "support", etc.; the features for describing the objects of actions may be action object features such as "football", "flower", etc., which are not limited here.

[0011] In a possible implementation of the above first aspect, the features related to forms include one or more of the following features: features for describing forms; features for describing the extension directions of forms.

[0012] For example, among the above features related to forms, the features for describing forms may be features for describing the form of a "big tree"; the features for describing the extension directions of forms may be features for describing the extension directions of the branches and leaves of the big tree, etc., which are not limited here.

[0013] It can be understood that the above-mentioned action-related features and / or morphology-related features can be the features of any one of the above-mentioned at least one display element. If there are multiple salient objects in the original background image, the representative degree of each salient object can be calculated by means of a saliency detection algorithm or the like, and then the salient object with the highest representative degree or higher than the corresponding threshold can be determined as the display element to which the above-mentioned action-related features and / or morphology-related features belong.

[0014] The size of the above-mentioned first background image area is related to the screen size of the electronic device and the size parameter in the initial display parameters of the first foreground element. Among them, the screen size of the electronic device can include the width and height values of the screen. Based on the width and height values, the aspect ratio of the screen can be determined, and then the aspect ratio of the above-mentioned first background image area can be determined. Then, the appropriate cropping area in the original background image can be determined by using the aspect ratio as the above-mentioned first background image area. In addition, the initial display parameters of the above-mentioned first foreground element can be, for example, the size, color, font and other related parameters of a time component, a notification component, etc. The size of the time component, the notification component, etc. can be understood as the size parameter in the above-mentioned initial display parameters.

[0015] In a possible implementation of the above first aspect, before displaying the first interface, the method further includes: performing image semantic understanding on the first background image to obtain semantic information related to at least one display element and region position information corresponding to the features included in the semantic information; determining the first region position information corresponding to the first feature among the features included in the semantic information; and determining the position parameter of the first foreground element according to the first region position information.

[0016] For example, the above-mentioned first feature can be a feature describing an action, such as "kick". At this time, when determining the position parameter of the first foreground element according to the first region position information corresponding to the first feature, it can be the position information of the area where the virtual object corresponding to the action described by the first feature is located. For example, the position information of the area where the virtual object football or other objects of the "kick" action may be located is determined as the position parameter of the first foreground element.

[0017] In a possible implementation of the above first aspect, the first feature is a feature describing the object of an action, and determining the position parameter of the first foreground element according to the first region position information includes: determining the first region position information as the position parameter of the first foreground element.

[0018] For example, the above-mentioned first background image can be understood as the original background image. The semantic information related to at least one display element obtained through the above-mentioned image semantic understanding may include information such as the prominent subject located in the area of the first background image, the action of the prominent subject, and the object of the action. The object of the action is the object of the action of the prominent subject. For example, the object of the action of "kicking" can be "football", etc. The above-mentioned first feature can be, for example, the object of the action of the prominent subject, such as "football". The first regional position information corresponding to the first feature can be the position information of the football in the corresponding image.

[0019] The features included in the above-mentioned semantic information can be, for example, the fields in the semantic information extracted in the semantic segmentation process described in the following embodiments that can express different regional position information. Correspondingly, the regional position information corresponding to the above-mentioned features can correspond to the regional position information corresponding to each field in the semantic information. Therefore, the process of determining the position parameter of the first foreground element described in the above implementation can be understood as the process of replacing the initial position parameter of the first foreground element with the first regional position information corresponding to the first feature. For example, replacing the initial position parameters of time components, notification components, etc. with the position parameters determined corresponding to the position information of "football" in the corresponding image.

[0020] In a possible implementation of the above first aspect, the display parameter further includes the color parameter of the first foreground element, and the color parameter is related to one or more of the following colors: the initial color of the first foreground element; the background color of the area where the feature describing the object of the action is located; the background color of the area where the feature describing the stretching direction of the shape is located; the color of the first display element among at least one display element.

[0021] For example, in some embodiments, the color parameter included in the display parameters of the above-mentioned first foreground element may be the color parameter in the initial display parameters of the first foreground element, such as the preset color parameters corresponding to a time component, a notification component, etc., that is, the color parameter indicated by the above-mentioned first color information. In other embodiments, the color parameter included in the display parameters of the first foreground element may also be the color parameter corresponding to the background color of the area where the action object of the prominent subject in the original background image is located. For example, it may be the color parameter corresponding to the background color of the area where "football" or "flower" is located, etc. When the prominent subject in the original background image is a landscape or a building, such as "big tree", the color of the first foreground element may also be the background color of the area indicated by the stretching direction of the form of the prominent subject in the corresponding image. For example, the color of the time component may be the background color of the area in the stretching direction of the branches and leaves of the "big tree", etc. The color of the first foreground element may also be a color adapted to the color of the prominent subject. For example, the color of the time component may be a color similar to or contrasting with the color of the person's clothes, which is not limited herein.

[0022] In a possible implementation of the above first aspect, the method for determining the first background image area includes: based on the screen size of the electronic device, determining the size parameter of the first cropping frame used to crop the first background image area from the first background image; according to the first semantic information of the first display element, the position information of the first display element in the first background image, and the size parameter of the first display element, determining at least one cropping area that satisfies the size parameter of the first cropping frame and includes the first display element, where the at least one display element includes the first display element; and determining the first cropping area in the at least one cropping area as the first background image area.

[0023] In a possible implementation of the above first aspect, the size parameter of the first cropping frame is further related to the size parameter of the first foreground element, and the method for determining the first background image area includes: based on the screen size of the electronic device and the size parameter of the first foreground element, determining the size parameter of the first cropping frame.

[0024] That is, according to the screen size of the electronic device and the sizes of the first foreground elements such as the time component and the notification component, several cropping areas can be determined from the original background image. Further, according to the semantic information related to the display elements such as the prominent subject in the original background image, such as "an athlete kicking a football", the position information and size parameter of the prominent subject in the original background image, etc., one cropping area, such as the above-mentioned first cropping area, can be selected from several cropping areas that satisfy the size parameter of the first cropping frame as the above-mentioned first background image area.

[0025] In a possible implementation of the first aspect described above, at least one cropping region is a plurality of cropping regions, and determining a first cropping region among the at least one cropping region as the first background image region includes: performing an aesthetic score on the at least one cropping region, and determining the first cropping region whose aesthetic score meets the third condition as the first background image region, where the third condition includes that the aesthetic score is higher than a second threshold or the highest value among the aesthetic scores.

[0026] That is, the above-mentioned first background image region can be a cropping region among several cropping regions that meet the above-mentioned first cropping frame size parameters and whose corresponding aesthetic scores are higher than the corresponding thresholds, or it can also be the cropping region with the highest corresponding aesthetic score, which is not limited here.

[0027] In a possible implementation of the first aspect described above, the first interface includes a lock screen interface or a desktop, and the above-mentioned interface display instruction includes: a display instruction triggered by a first user operation indicating the display of the lock screen interface; or, a display instruction triggered by a second user operation indicating the display of the desktop.

[0028] The above-mentioned first interface can be a target interface for the user to perform operations such as setting the background image. This target interface can be, for example, a desktop or a lock screen interface displayed on an electronic device such as a mobile phone, which is not limited here.

[0029] It can be understood that when adjusting the display parameters of the foreground elements that can affect the layout of the target interface to be set, using the screen size and the size and position of the significant subject in the original background image as layout constraint conditions, some action-related features, morphology-related features, color features, and representative features of the foreground elements on the target interface to be set can be fused with the significant subject, and finally an interface display effect with strong interactivity and aesthetics can be produced.

[0030] In some embodiments, the process of determining the display parameters of the above-mentioned first foreground element or the process of the above-mentioned fusion can be implemented using a pre-trained machine learning model or neural network model, etc., for example, through a saliency detection and composition scoring model, etc., which is not limited here.

[0031] In a second aspect, the present application provides an electronic device, including: one or more processors; one or more memories; one or more programs are stored in the one or more memories, and when the one or more programs are executed by the one or more processors, the electronic device is caused to execute the display method provided in the first aspect and various possible implementations of the first aspect.

[0032] In a third aspect, the present application provides a computer-readable medium, on which instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the display method provided in the above first aspect and various possible implementations of the first aspect.

[0033] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, the display method provided in the above first aspect and various possible implementations of the first aspect is implemented.

[0034] For the beneficial effects of the above second aspect to the fourth aspect, reference may be made to the relevant descriptions in the above first aspect and various possible implementations of the first aspect, and details are not described herein again. Description of the Drawings

[0035] Figure 1 The figure shows a schematic diagram of an application scenario of a display method provided by an embodiment of the present application.

[0036] Figure 2a The figure shows a schematic diagram of a lock screen interface displayed by a mobile phone in response to an operation of a user setting an image including a human subject as a target interface background image to be set.

[0037] Figure 2b The figure shows another schematic diagram of a lock screen interface displayed by a mobile phone in response to an operation of a user setting an image including a human subject as a target interface background image to be set.

[0038] Figure 2c The figure shows a schematic diagram of a lock screen interface displayed by a mobile phone in response to an operation of a user setting a landscape image as a target interface background image to be set

[0039] Figure 2d The figure shows a schematic diagram of a desktop displayed by a mobile phone in response to an operation of a user setting an image including a human subject as a target interface background image to be set.

[0040] Figure 3 The figure shows a schematic diagram of a system framework and implementation principle capable of implementing the above display method provided by an embodiment of the present application.

[0041] Figure 4 The figure shows a schematic diagram of an implementation process of a display method provided by Embodiment 1 of the present application.

[0042] Figure 5a The figure shows a schematic diagram of an image browsing interface supporting user operations displayed on a mobile phone provided by Embodiment 1 of the present application.

[0043] Figure 5b The figure shows a schematic diagram of a settings interface that supports user operations for mobile phone display provided in Embodiment 1 of the present application.

[0044] Figure 5c The figure shows another schematic diagram of a settings interface that supports user operations for mobile phone display provided in Embodiment 1 of the present application.

[0045] Figure 5d The figure shows a schematic diagram of a lock screen interface for mobile phone display provided in Embodiment 1 of the present application.

[0046] Figure 6 The figure shows a schematic diagram of the process of a composition scoring model determining recommended layout parameters for the layout constraint conditions of an input model provided in Embodiment 1 of the present application.

[0047] Figure 7 The figure shows a schematic diagram of the technical implementation principle of image semantic understanding and natural language understanding adopted in a display method provided in Embodiment 1 of the present application.

[0048] Figure 8 The figure shows a schematic diagram of the implementation process of a display method provided in Embodiment 2 of the present application.

[0049] Figure 9 The figure shows a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0050] Figure 10 The figure shows a block diagram of the operating system software structure of an electronic device provided in an embodiment of the present application. Detailed implementation manners

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0052] It should also be stated that the numbering of the steps in the methods and processes in the embodiments of the present application is for the convenience of citation and does not limit the sequence. If there is a sequence between the steps, it shall be subject to the written description.

[0053] Figure 1 The figure shows a schematic diagram of the application scenario of a display method.

[0054] As Figure 1 shown, this scenario includes a mobile phone 100. In the locked screen state, the locked screen interface 101 displayed on the mobile phone 100 may include a locked screen wallpaper, a clock component 102, a locked screen notification 103, etc. This locked screen wallpaper can be set with a photo taken by the user using the mobile phone 100 or an image downloaded, for example Figure 1The wallpaper corresponding to the image of "an athlete playing football" shown. Currently, the lock screen wallpaper set by the user is usually a cropped image that adapts to the aspect ratio of the screen for the original background image. On this basis, the mobile phone 100 then renders and synthesizes the clock component, lock screen notifications, etc. to be laid out with the cropped image according to the preset layout rules and displays them as Figure 1 the lock screen interface 101 shown in the example.

[0055] However, since the cropping rules used by the mobile phone 100 to respond to the user's setting of the cropped image are usually fixed preset rules such as centered cropping, and the layout rules used by the mobile phone 100 when synthesizing the lock screen interface are usually also laid out according to the preset positions and preset sizes of the corresponding components, the components displayed on the lock screen desktop of the mobile phone 100 may block the features of the prominent subjects such as people in the lock screen wallpaper, such as the face, limbs, actions, etc. of the people. Also, when the mobile phone 100 receives notification information (such as WeChat TM , missed calls, etc.), the position where the lock screen notification 103 is displayed may also cover the features of the prominent subject in the lock screen wallpaper, which will result in a poor aesthetic appearance of the lock screen interface and the user's visual experience will also decline accordingly.

[0056] To solve the above problems, the present application provides a display method. Specifically, the method adjusts the display parameters affecting the foreground elements in the target interface to be set, including parameters such as the size of the foreground elements, the position relative to the prominent subject in the background image, and the color, etc., to achieve laying out the foreground elements on the target interface to be set at positions where they can interact with the prominent subject. Among them, the relevant features such as the size and position of the prominent subject in the background image can be determined according to the size and position of the relevant prominent subject in the corresponding original background image determined by image semantic understanding of the corresponding original background image. Based on this, the present application can use the size, position, etc. of the relevant prominent subject in the original background image, combined with the screen size of the electronic device, as the layout constraint conditions for adjusting the foreground elements on the target interface to be set, and adjust the preset size, position, color and other parameters of the foreground elements on the target interface to be set.

[0057] It can be understood that the screen size of the above-mentioned electronic device can constrain the interface size of the set target interface. For example, the aspect ratio of the set target interface can be the same as the aspect ratio of the screen. The above-mentioned foreground elements, such as the components displayed on the set target interface, include a clock component, a notification component, etc. Correspondingly, the display parameters of the foreground elements adjusted in this application can include the component size, component position, component color, etc. of the relevant components. The display parameters of the adjusted foreground elements, such as the component size, component position, component color, etc. of the relevant components, can control the interaction effect between the relevant components and the above-mentioned salient subject when the relevant components are laid out on the background image. In addition, in order to enhance this interaction effect, the cropping area corresponding to the above-mentioned background image can also be optimized through aesthetic scoring to improve the aesthetics of the composition of the set target interface formed by laying out the above-mentioned foreground elements on the background image.

[0058] In this way, the set target interface with the background image set based on the display method provided in this application can reflect the interaction and interest between the salient subject in the original background image and the relevant components. Moreover, in the process of adjusting the display parameters of foreground elements such as relevant components, optimizing the cropping area corresponding to the background image on the original background image through methods such as aesthetic scoring is also beneficial to improving the aesthetics of the composition of the set target interface.

[0059] Among them, the above-mentioned salient subject can include people, animals, plants, food, buildings or landscapes, etc. Correspondingly, the characteristics of the salient subject determined based on the image semantic information can include the action characteristics of people or animals, the morphological characteristics of plants, the color characteristics of food, and the characteristics of representative buildings or landscapes, etc., which are not limited here. The above-mentioned representative building can be the part with a representative degree higher than the corresponding threshold among multiple buildings in the original background image, or the part with a representative degree higher than the corresponding threshold among the representative landscapes or multiple landscape components. In some embodiments, when the number of salient subjects such as people or animals in the original background image is large, the representative degree of each subject can also be calculated to determine one of the people or animals as the salient subject for extracting relevant characteristics. The above-mentioned representative degree can be obtained by processing the original background image through a saliency detection algorithm or other methods, which will not be elaborated here.

[0060] In the embodiments of the present application, image semantic information can be used for a digital computer or a machine controlled by a digital computer to understand the content included in an image. For example, when the prominent subject in the original background image is a person, the user can understand that the image is of a player kicking a football based on the person's actions and the object of the action. Similarly, the machine can also extract information about the person as "player", the person's action as "kick", the object of the action as "football", and information such as the color of the object of the action, etc., based on the image semantic information of the original background image. Therefore, the image semantic information can include, but is not limited to, color features, texture features, shape features, etc., and can express the feature information corresponding to the content that is close to the user's understanding.

[0061] It can be understood that when the present application adjusts the display parameters of the foreground elements that can affect the layout of the target interface to be set, using the screen size and the size, position, etc. of the prominent subject in the original background image as layout constraint conditions, the foreground elements on the target interface to be set can be fused with some action-related features, morphology (mophe)-related features, color features, and representative features, etc. of the prominent subject, so as to be able to produce an interface display effect with strong interactivity and aesthetics. In some embodiments, the above fusion process can be implemented using a pre-trained machine learning model or neural network model, etc., which is not limited herein.

[0062] The above morphology-related features can include, for example, the shape of the prominent subject and the stretching direction of the morphology, such as the shape of the branches and leaves of a big tree and the stretching direction of the branches and leaves, etc., which is not limited herein.

[0063] It can be understood that the target interface to be set applicable to the display method provided by the present application can include a lock screen interface or a desktop, etc. displayed on an electronic device such as a mobile phone. In other embodiments, the above target interface to be set can also include other interfaces suitable for adjusting the display parameters by using the above display method, which is not limited herein.

[0064] As an example, Figures 2a to 2d According to the embodiments of the present application, a schematic diagram of an interface corresponding to a mobile phone to which a display method is applied is shown.

[0065] Refer to Figure 2aAs shown, on the lock screen interface 210 displayed on the mobile phone 100, the cropped image set as the lock screen wallpaper is the above-mentioned "athlete playing football" image. At this time, based on the display method provided in this application, after cropping the image of the "athlete playing football" related part in the original background image to obtain the cropped image, the mobile phone 100 can adapt the clock component 211 and the lock screen notification 212 that need to be laid out and displayed on the lock screen interface 210 above the action "kick" of the "athlete" 213. In this way, it can not only avoid the clock component, lock screen notification, etc. from blocking the prominent subjects such as the "athlete" on the wallpaper, but also enable the lock screen interface 210 displayed on the mobile phone 100 to show the interactive effect that the "athlete" 213 "kicks" the football and the lock screen notification 212 and the clock component 211, improving the interactivity and interest, and thus being beneficial to the user's visual experience.

[0066] In some other embodiments, the cropped image set as the lock screen wallpaper can also be an image with other people as the prominent subject, and the subject shows other forms of body movements. Refer to Figure 2b As shown, on the lock screen interface 220 displayed on the mobile phone 100, the cropped image set as the lock screen wallpaper can also be the image of "a person crossing a cliff". At this time, based on the display method provided in this application, the mobile phone 100 can adapt the clock component and the lock screen notification that need to be laid out and displayed on the lock screen interface 220 above the action of "waving" the arm when the "person" crosses. In this way, it can not only avoid the clock component, lock screen notification, etc. from blocking the prominent subjects such as the "person" on the lock screen wallpaper, but also enable the lock screen interface 220 displayed on the mobile phone 100 to show the interactive effect that the "person"'s waving arm holds up the lock screen notification and the clock component, improving the interactivity and interest, and thus being beneficial to the user's visual experience.

[0067] In some embodiments, the prominent subject in the above-mentioned cropped image set as the lock screen wallpaper can also be a plant or a landscape, etc. Refer to Figure 2c As shown, on the lock screen interface 230 displayed on the mobile phone 100, the cropped image set as the lock screen wallpaper can be a landscape picture with a "big tree" and "flocks of birds". At this time, based on the display method provided in this application, the mobile phone 100 can adapt the clock component and the lock screen notification that need to be laid out and displayed on the lock screen interface 230 above the prominent subject of the "big tree". In this way, it can not only avoid the clock component, lock screen notification, etc. from blocking the prominent subjects such as the "big tree" on the lock screen wallpaper, but also enable the lock screen interface 230 displayed on the mobile phone 100 to show the interactive effect that the representative feature of the "big tree", the "tree top", holds up the lock screen notification and the clock component above the "tree top", improving the interactivity and interest, and thus being beneficial to the user's visual experience.

[0068] In some embodiments, the mobile phone 100 may also be configured to display the display method provided by the present application on the desktop or other application interfaces. Correspondingly, the components to be displayed on the desktop or other application interfaces may also include card components, application icons, etc., which are not limited here.

[0069] refer to Figure 2d As shown, on the desktop 240 displayed by the mobile phone 100, the cropped image set as the desktop wallpaper continues to use the above-mentioned "athlete playing football" image as an example. Based on the display method provided by the present application, the mobile phone 100 can display the WeChat image that needs to be displayed on the desktop 240. TM The icons of application 241 and theme application 242 are adapted to the positions above the ball kicked by the "athlete" when kicking the football, below the "swinging" arm, etc. In this way, it is possible to avoid the application icons or some card components from blocking the "athlete" and other prominent subjects on the desktop wallpaper, and also to make the desktop 240 displayed by the mobile phone 100 show the "athlete" kicking the football and the "WeChat" TM The interactive effect of "application 241" and "theme application 242" is improved to improve interactivity and fun, thereby facilitating the user's visual experience.

[0070] As an example, Figure 3 According to an embodiment of the present application, a system framework and a schematic diagram of the implementation principle capable of implementing the above-mentioned display method are shown.

[0071] It is understood that for the lock screen interface, the components 401 to be displayed may include the above-mentioned clock components and lock screen notifications, etc. For the desktop, the components 401 to be displayed on the mobile phone 100 may include clock cards, suggestion cards, application cards and other card components providing different services, as well as some application icons, etc., which are not limited here.

[0072] Continue to refer Figure 3 The wallpaper service 320 may have an image semantic understanding function, and may perform image semantic understanding on the original background image 300. Figure 3 The tasks of "① semantic understanding" and "② semantic segmentation" are shown. The "① semantic understanding" can, for example, determine that the semantic information corresponding to the original background image 300 is "athletes playing football". Correspondingly, the "② semantic segmentation" process can segment the original background image 300, and then obtain the position information of the corresponding areas of the fields such as "athletes" and "football". In this way, the fields such as "athletes" and "football" in the semantic information can be associated with the position information of the corresponding areas.

[0073] It can be understood that the above semantic understanding function can be based on some image semantic understanding systems or models. For example, an image processing model that generates a harmonious text layout on natural images through deep aesthetic learning. This model can be trained based on a convolutional neural networks (CNN) model or a recurrent neural networks (RNN) model, without limitation here.

[0074] For the semantic information corresponding to the above image semantic understanding function and the segmented fields, the wallpaper service 320 can be based on a pre-configured natural language understanding (NLU) model, such as Figure 3 the "③ Aesthetic Composition NLU" shown, for intent recognition and slot filling. That is, determine the intent and slots expressed by the semantic information corresponding to the original background image 300, referring to Figure 3 the "Intent {}" and "Slot {}" shown.

[0075] Continue to refer to Figure 3 , and use the "Intent {}" and "Slot {}" recognized by the above NLU model as layout constraint conditions. Combine with the relevant layout rules of the above component 401 provided by the desktop application 310, and replace the parameters such as the component position, component size, and component color in the relevant layout rules of the component 401, then the recommended layout parameters that can layout the component 401 to interact with the salient subject of the original background image 300 can be determined. Finally, based on the determined recommended layout parameters, the mobile phone 100 can display the corresponding interface layout effect.

[0076] Specifically, Figure 3 For the specific implementation process of the display method shown, reference can be made to the detailed description in the following text in combination with relevant embodiments and drawings, which will not be elaborated here.

[0077] It can be understood that the electronic devices applicable to the display method provided in the embodiments of the present application may include, but are not limited to, mobile phones, tablet computers, desktop computers, laptops, handheld computers, netbooks, and augmented reality (AR) or virtual reality (VR) devices, smart TVs, wearable devices such as smart watches, in-vehicle devices, portable game consoles, portable music players, reader devices, and electronic devices with one or more displays and one or more processors.

[0078] Based on the above Figure 1 shown application scenarios, the following combines specific embodiments to introduce the specific implementation process of the display method provided in the present application.

[0079] Example 1

[0080] This embodiment of the present application introduces the specific implementation process of a display method in the above Figure 1 shown application scenario. This implementation process can reasonably layout the positions and sizes of each component to be displayed on the set target interface in combination with the relevant features of the significant subject in the original background image, forming an interactive and aesthetically pleasing layout of the set target interface. Moreover, the specific implementation process of the display method provided in this embodiment of the present application can also perform an aesthetic score on the original background image set as the background image of the set target interface to screen for a suitable cropping area, which is beneficial to improving the aesthetics of the composition of the set target interface.

[0081] Specifically, Figure 4 According to this embodiment of the present application, a schematic diagram of the implementation process of a display method is shown. It can be understood that Figure 4 the execution subject of each step in the shown implementation process can be the above-mentioned electronic devices such as mobile phones and tablet computers, which is not limited here.

[0082] Continuing with the mobile phone 100 as an example of the execution subject below, in combination with Figure 4 the specific implementation process of the display method provided by the present application will be introduced in detail.

[0083] As Figure 4 shown, this implementation process may include the following steps:

[0084] 401: Detect a user operation of setting the original background image as the background image of the set target interface.

[0085] Exemplarily, the above-mentioned original background image may be an image stored in the mobile phone 100, including photos taken by the user using the mobile phone 100, downloaded images or images in videos, etc., which is not limited here. Correspondingly, the above-mentioned set target interface may include an interface with a background image and one or more components displayed above the background image, such as the lock screen interface or desktop displayed on the mobile phone 100. In some other embodiments, the above-mentioned set target interface may also be a chat interface of an instant messaging application, etc., which is not limited here. Correspondingly, the above-mentioned user operation of setting the background image may include setting the lock screen wallpaper, desktop wallpaper or setting the chat background of the chat interface, etc.

[0086] As an example, Figures 5a to 5d shows a schematic diagram of the set target interface during the process of the user setting the lock screen interface on the mobile phone.

[0087] Refer to Figure 5aWhen the user browses the images in the gallery of the mobile phone 100, the user can click on the control 511 on the image browsing interface 510 and select the "Set as" option 512, so that the mobile phone 100 displays Figure 5b the setting interface 520 shown.

[0088] Continue to refer to Figure 5b When the user clicks on the "Wallpaper" option 521 on the setting interface 520, the user can set the current image as the wallpaper. At this time, in response to the user's setting operation, the mobile phone 100 can display Figure 5c the setting interface 530 shown.

[0089] Continue to refer to Figure 5c As shown, the user can click on the "Apply" control 531 on the setting interface 530 to set the current image as the "lock screen wallpaper" and / or set it as the "desktop wallpaper". For example, the user can click on the "Set as Lock Screen" option 532 on the setting interface 530 to set the current image as the lock screen wallpaper. For another example, the user can also click on the "Set as Desktop" option 533 on the setting interface 530 to set the current image as the desktop wallpaper. For yet another example, the user can further click on the "Set Both" option 534 on the setting interface 530 to set the current image as both the lock screen wallpaper and the desktop wallpaper. There is no limitation here.

[0090] Correspondingly, when the mobile phone 100 detects the above setting operation of the user, it can execute the following steps 402 to 410 to complete the process of rendering and synthesizing the lock screen interface corresponding to the above setting operation, and finally it can display Figure 5d the lock screen interface 540 shown in the example. For the specific display effect of the lock screen interface after setting, reference can be made to the relevant description in step 410 below, which will not be elaborated here.

[0091] 402: Perform image semantic understanding on the original background image to obtain image semantic information.

[0092] Exemplarily, when the mobile phone 100 detects the above setting operation performed by the user, it can perform image semantic understanding on the original background image that is about to be set as the background image, that is, the original background image, and then obtain the image semantic information corresponding to the original background image.

[0093] It can be understood that the above image semantic understanding is based on natural language processing (NLP) of the image content. In the embodiments of the present application, the implementation process of the above image semantic understanding can be implemented by using some image semantic understanding systems or models. A complete image understanding system usually can include the following four layers: data layer, description layer, cognitive layer, and application layer.

[0094] Among them, the data layer is used to obtain image data, including binary images, grayscale images of color photos, color images, depth images, etc., mainly involving the processes of image compression and transmission. The main operation object of this layer is pixels.

[0095] The description layer is used to extract features, measure the similarity between features, which is reflected as the distance of feature vectors. This measurement process can adopt data dimensionality reduction methods such as subspace, including independent component analysis (ICA), principal component analysis (PCA), etc. After dimensionality reduction, the image pixel feature data can be further symbolized to facilitate subsequent image understanding calculations. The main task of this layer is to realize the symbolization of pixel representation.

[0096] The cognitive layer is used for image understanding, that is, learning and inference on the symbolized pixels. It can be understood that the cognitive layer is the "engine" of the image understanding system, and the tasks of this layer are very complex, also including the establishment of relevant databases, etc. The main processing object of the cognitive layer is symbols, such as the above symbolized pixel representations, etc., without limitation here.

[0097] The application layer can design corresponding classifiers, learning algorithms, etc. according to task requirements, such as classification tasks, recognition tasks, detection tasks, etc. The classifiers, learning algorithms, etc. designed by the application layer can finally output the classification results and image semantic information, etc. of the image to be understood. Details are not elaborated here.

[0098] In the embodiment of this application, there is no specific limitation on the system or model, etc. used to implement the image semantic understanding corresponding to this step 402. Electronic devices such as the mobile phone 100 can use a pre-trained image semantic understanding system or model, etc. to perform image semantic understanding on the original background image set as the target interface background image by the user, and obtain the image semantic information corresponding to this image.

[0099] 403: Based on the image semantic information, extract the relevant features of the salient subject.

[0100] Exemplarily, the mobile phone 100 performs semantic segmentation based on the image semantic information obtained corresponding to the above image semantic understanding process, and extracts the fields in the semantic information that can express the position information of different regions. Among them, the fields determined corresponding to this semantic segmentation process may include fields describing the prominent subjects in the original background image, such as people, animals, plants, food, buildings, or landscapes, etc. Moreover, the fields determined corresponding to this semantic segmentation process may also include the relevant features of the prominent subjects, such as action-related features "kick", "run", "jump", "hold", "support", etc., and the prominent subjects.

[0101] Optionally, when there are multiple prominent subjects in the original background image, the mobile phone 100 may determine the representative prominent subject among them according to a preset prominent subject screening rule as a constraint condition for subsequent interface composition. In some embodiments, the representative prominent subject may include relatively common prominent subjects, such as the person corresponding to the face that is often recognized in the historical record of face recognition by the mobile phone 100, etc. In other embodiments, the representative prominent subject may also include the subject with a higher degree of prominence, such as the subject in the center position or the subject with a more forward standing position among multiple prominent subjects, or for example, the subject with the largest action amplitude among multiple prominent subjects, which is not limited herein.

[0102] Optionally, when there are many relevant features of the prominent subject in the original background image, for example, when the person subject in the original background image has multiple action-related features including raising the arm, lifting the leg, and kicking the ball, etc., the mobile phone 100 may determine the representative feature among them according to a preset relevant feature screening rule as a constraint condition for subsequent interface composition. In some embodiments, the representative feature may be, for example, the feature with a larger action amplitude among multiple action-related features. In other embodiments, the representative feature may also include, for example, the feature that the object of the action is larger or the position in the picture is more forward, which is not limited herein.

[0103] 404: Obtain the preset layout rules of one or more components to be displayed on the set target interface.

[0104] Exemplarily, the set target interface affected by the above user operation may include one or more components to be displayed, including the clock component, notification component, etc. on the lock screen interface, the card component on the desktop, and the message bubble component on the chat interface, etc., which is not limited herein. Among them, due to the different contents to be displayed, the display positions (i.e., component positions), component sizes (such as aspect ratios, etc.), and component colors, etc. of each component on the set target interface may all be different.

[0105] For example, in the operating system of an electronic device such as mobile phone 100, corresponding layout rules can be preset for clock components, notification components, etc. displayed on the lock screen interface. For example, the above layout rules include centering the clock component in the upper area of the set target interface, displaying the clock information in a preset font size, and displaying relevant parameters such as the component color, aspect ratio, etc. of the clock component. The above layout rules can also include displaying the lock screen notification below the clock component, displaying the notification information in a preset font size, and relevant parameters such as the window color, font color, and window aspect ratio of the lock screen notification, which will not be elaborated here.

[0106] 405: Determine the composition layout parameters of the set target interface according to the relevant features of the significant subject and the preset layout rules of one or more components.

[0107] Exemplarily, after mobile phone 100 determines the relevant features of the significant subject in the original background image set as the background image and the preset layout rules of one or more components to be displayed, it can comprehensively consider the relevant features of the significant subject, component color, component position, component size, and screen size, etc., to determine the composition layout parameters of the set target interface. This determination process can be implemented, for example, through means such as intent recognition and slot replacement.

[0108] It can be understood that a subfield of NLP can include natural language understanding (NLU). Among them, the two most critical tasks of NLU include intent recognition and slot filling. Among them, intent recognition is the process of recognizing the purpose of the significant subject expressed in the image semantic information corresponding to the above original background image. For example, when the significant subject is a person or an animal, etc., the intents recognized by this intent recognition process can include some common behavioral actions of people or animals, etc., such as kicking, striding, supporting, holding, looking, eating, etc. Under each intent, one or more slots can be correspondingly configured.

[0109] A slot refers to the key information that the NLU model needs to extract from the above image semantic information, such as the action subject, action object, etc. corresponding to the action-related features identified as "intent". Slots can include attributes such as slot values. Among them, a slot value refers to the specific parameter of a slot and can also be called the entity of the slot.

[0110] For the above Figure 5a or Figure 5b image semantic information corresponding to the exemplified image, the intent recognition results achieved by mobile phone 100 running the NLU model can include:

[0111]

[0112] In some other embodiments, for the action-related features of different salient subjects or other features with interactive intentions, the mobile phone 100 can also identify other combinations of intentions and operations based on the NLU model. For example, combinations formed by the intention of "holding" and the action subject "hand", the action object "rape flower", etc. are not limited herein.

[0113] For the above intention recognition results, after adaptively expanding the attribute fields of "category, position, color", etc. for the content of the above original background image as the foreground and background of the target interface to be set, the slot values corresponding to the corresponding fields can be added to obtain the following intention recognition results including slot values:

[0114]

[0115] Among them, [920, 810, 100, 100] in the above slot can represent the relative coordinates of a specified position on the original background image. This position coordinate can represent, for example, the relative coordinate corresponding to the position that can interact with the action of "kicking" of the above action subject "foot". For example, the relative coordinate corresponding to the position 513 above the football "kicked" by the athlete shown above. It can be understood that the above relative coordinate can be the coordinate parameter relative to a certain corner point of the original background image, or the coordinate parameter relative to the image preview origin preset in the mobile phone 100, etc., which is not limited herein. Figure 5a As shown above, the relative coordinates corresponding to the position 513 above the football "kicked" by the athlete, etc. It can be understood that the above relative coordinate can be the coordinate parameter relative to a certain corner point of the original background image, or the coordinate parameter relative to the image preview origin preset in the mobile phone 100, etc., which is not limited herein.

[0116] Correspondingly, [600, 200, 300, 400] in the above slot represents the relative coordinates of the position where the above action object "football" is located; [0, 400, 920, 1120] represents the relative coordinates of the position where the foreground object "athlete" to which the above action subject "foot" belongs is located; [0, 0, 1080, 330] represents the relative coordinates of the position where the background "football field" is located, etc., which will not be elaborated herein.

[0117] It can be understood that in specific applications, the intention categories and the slots configured under each intention category can be preset. For example, for images containing salient subjects such as people and animals, intention categories corresponding to action-related features such as kicking, striding, holding, picking up, hugging, looking, eating, etc. can be preset. For landscape images containing salient subjects such as buildings, delicious foods, representative scenic spots or obvious feature objects, the upper, lower, left, right, etc. orientations of the subject or other interactive features can be preset as intention categories, which are not limited herein.

[0118] Based on the above-mentioned intent recognition results including slot values, and then performing slot replacement, the composition layout parameters corresponding to the significant subject and related components can be obtained. For example, based on the intent recognition results of the above example, the mobile phone 100 can replace the component position coordinate parameters in the preset layout rules of the related components with [920, 810, 100, 100] in the "action subject" slot above, so as to modify the preset component position corresponding to the related components to the position indicated by [920, 810, 100, 100] on the original background image. As mentioned before, at this position, the component can interact with the "foot" of the action subject, that is, the "athlete" in the original background image. At the same time, the preset component color in the preset layout rules of the related components can also be replaced with the color channel parameters corresponding to "black" in the above-mentioned "action subject" slot.

[0119] In this way, an interaction relationship can be established between the related components and the significant subject of the related image, improving the interactivity and interest between the components on the set target interface and the background image.

[0120] Based on the composition layout parameters determined in the above step 405, the process of performing aesthetic scoring on the composition determined based on the composition layout parameters and screening out the recommended layout parameters corresponding to the recommended composition in steps 406 to 408 will be continued below.

[0121] 406: Based on the crop box size parameters in the composition layout parameters, multiple crop regions are determined in the original background image.

[0122] Exemplarily, the above-mentioned crop box size parameters may include parameters such as the length and width of the crop box. By implementing the recommended layout parameters determined corresponding to the above step 405 by the mobile phone 100, the size and position of the significant subject, the size and position of the related components, and the aspect ratio of the display screen, etc. can be determined. Based on these parameters, the mobile phone 100 can determine the size parameters of the crop box for cropping the above-mentioned original background image. Among them, the length of the above-mentioned crop box can be less than or equal to the original length of the original background image, and the width of the above-mentioned crop box can be less than or equal to the width of the original background image. And, the aspect ratio of the above-mentioned crop box can be the same as the aspect ratio of the screen of the mobile phone 100.

[0123] It can be understood that when the above-mentioned cropping frame is located at different positions on the original background image, the corresponding cropped areas selected by the frame will also change. Therefore, the mobile phone 100 can determine one or more cropped areas in the original background image based on the size parameters of the above-mentioned cropping frame. Among them, the images in each cropped area can be used as the background image of the target interface to be set, and combined with relevant components to form a composition result. However, the aesthetic degrees of the compositions formed by combining different cropped areas with relevant components are different. For specific details, reference can be made to the relevant description in step 408 below, which will not be elaborated here.

[0124] 407: Based on the component-related parameters in the composition layout parameters, determine the composition formed by arranging each component into each cropped area.

[0125] Exemplarily, the component-related parameters in the above-mentioned recommended layout parameters may include relevant parameters such as component size, component position, and component color, which are not limited here. Based on the above component-related parameters, the mobile phone 100 can determine the positions, sizes, etc. of each component arranged on the images of each cropped area, that is, it can determine the composition formed by the background image and the components in each cropped area.

[0126] It can be understood that in the process of arranging the components into the above-mentioned cropped areas to determine the composition of the target interface to be set, the size of the components displayed on the target interface to be set can be fixed as the size of the component in the preset layout rule, that is, the size parameters of the relevant components can remain unchanged. In some other embodiments, in the process of the mobile phone 100 implementing step 405 to determine the recommended layout parameters correspondingly, the size of the components displayed on the target interface to be set can also be scaled and adjusted, such as adaptively adjusting the component size according to the size of the significant subject, etc., which are not limited here.

[0127] 408: Perform an aesthetic score on the compositions formed by arranging the relevant components in each cropped area.

[0128] Exemplarily, the above-mentioned composition is formed by combining the images of each cropped area related to the significant subject of the above-mentioned original background image with relevant components. System services of the mobile phone 100, such as the wallpaper service, etc., can provide the above-mentioned aesthetic scoring function. For example, the wallpaper service of the mobile phone 100 can respond to the invocation of applications such as the desktop or wallpaper, and perform an aesthetic score on the multiple compositions formed by arranging the above-mentioned relevant components into each cropped area. Furthermore, the mobile phone 100 can continue to execute step 409 below according to the aesthetic score result, and screen out the composition layout parameters with the highest score or other composition layout parameters that meet the preset score screening conditions.

[0129] 409: Based on the aesthetic score result, select the composition layout parameters corresponding to the composition that meets the score screening conditions as the recommended layout parameters.

[0130] Exemplarily, the above scoring and screening conditions may include, for example, the condition of screening out the highest score in the aesthetic score ranking, or the condition of screening out the aesthetic scores higher than a preset score threshold, etc. Among them, the preset score threshold may include a single score value, or multiple score values and multi-level threshold conditions, etc., which are not limited herein. Based on the preset scoring and screening conditions, the mobile phone 100 can screen out the compositions that meet the scoring and screening conditions according to the aesthetic score results, which are the recommended compositions. The composition layout parameters corresponding to the recommended compositions can be used as the recommended layout parameters and provided to the desktop application or wallpaper application, etc., to continue to execute the following step 410 for drawing, rendering, and composite display.

[0131] It can be understood that the process of the mobile phone 100 executing the above steps 406 to 409 can be implemented by integrating constraint-based saliency detection and a composition scoring model. Among them, the above composition scoring model can screen out the compositions with higher aesthetics according to the aesthetic score results of each composition formed by respectively arranging relevant components in each cropping area, and recommend them to the mobile phone 100 for continued drawing, rendering, and composite display.

[0132] Specifically, the process of the mobile phone 100 implementing the above steps 406 to 409 to perform aesthetic scoring on relevant compositions and determine the recommended layout parameters corresponding to the recommended compositions based on the above composition scoring model will be described in detail below in combination with relevant drawings, and will not be elaborated herein.

[0133] 410: In response to the user operation of locking the screen or switching back to the desktop, display the target interface rendered and composed according to the selected recommended layout parameters.

[0134] Exemplarily, after the mobile phone 100 determines the recommended layout parameters that can ensure both interactivity and aesthetics, it can respond to the user's lock screen operation and display the lock screen interface drawn, rendered, and composed based on the recommended layout parameters. Alternatively, the mobile phone 100 can also respond to the user's operation of switching back to the desktop and display the desktop drawn, rendered, and composed based on the recommended layout parameters.

[0135] As an example, based on the implementation results of the above steps 401 to 410, the mobile phone 100 can display Figure 5d the lock screen interface shown. Referring to Figure 5d , a lock screen notification 542 can be displayed at the position 541 above the football "kicked" by the athlete on the lock screen interface 540, and a clock component 543, etc., can be displayed near the notification 542. In this way, the above lock screen notification 542, clock component 543, etc., can establish an interactive connection with the "kicking" action of the athlete in the background image of the lock screen interface 540, making the lock screen interface 540 have interactivity and aesthetics.

[0136] For the processes of determining the composition layout parameters corresponding to steps 405 to 409 above and determining the recommended layout parameters corresponding to the recommended composition by performing aesthetic scoring on the relevant composition, reference can be made to Figure 6 the implementation process shown

[0137] Specifically Figure 6 According to the embodiments of the present application, a schematic diagram of the process of a composition scoring model for determining recommended layout parameters according to the layout constraint conditions of the input model is shown. It can be understood that Figure 6 the process shown can be implemented by a composition scoring model or related algorithms pre-set in the mobile phone 100. As an example, the above composition scoring model can be a neural network model, for example, it can be trained by CNN or RNN

[0138] As Figure 6 shown, the input end of the composition scoring model 600 may include an embedding layer 610, also known as a mapping layer. In the embodiments of the present application, this layer can convert (or map) the intent and slot fields (referred to as fields for short) determined based on processes such as understanding the semantic information of the image, intent recognition, and slot filling into vectors with a fixed size that can be recognized by a computer. In addition, the embedding layer can also change a onehot vector into a dense vector, or convert a high-dimensional sparse feature vector into a low-dimensional dense feature vector, etc., which will not be limited here

[0139] Continuing to refer to Figure 6 , the fields input to the embedding layer 610 of the composition scoring model 600 may at least include the component size and the screen size, and may also include the component position, component color, action-related features of the salient subject (i.e., the intent field), and the action object, etc., which will not be limited here. Among them, the screen size may include the aspect ratio (ratio) of the screen, such as 16:9 or other ratios, and the component size may also include the length and width of the component, such as [200, 800], etc. It can be understood that the aspect ratio of the component and the aspect ratio of the screen can be the same or similar to improve the aesthetics of the composition. The process of performing embedding mapping for each constraint condition, for example, includes mapping the above component position, screen aspect ratio, component color, action-related features of the salient subject, etc. into an N-dimensional vector, such as a 1024-dimensional vector, etc

[0140] After each field input to the embedding layer is mapped into a vector of the corresponding dimension, it can be further output as a layout constraint condition to the adaptive weight module 620 of the composition scoring model 600 to calculate the corresponding weights for each layout constraint condition. Among them, the adaptive weight module 620 can obtain the ability to calculate the corresponding weights for each layout constraint condition through algorithms or neural network training

[0141] It can be understood that the process of calculating the corresponding weights for each layout constraint condition described above may include adaptively configuring higher weight values for relatively important layout constraint conditions, such as component size, component position, screen size, and action-related features, etc.; it may also include adaptively configuring lower weight values for relatively less important layout constraint conditions, such as component color, action object, etc., which is not limited here. After the weight values corresponding to each layout constraint condition are determined, the adaptive weight module 620 can output each layout constraint condition and the corresponding weight values to the scoring network 630 of the composition scoring model 600. Among them, the weight values corresponding to each layout constraint condition can affect the aesthetic scoring results of the composition scoring model 600 for relevant compositions. For specific details, please continue to refer to the relevant descriptions below.

[0142] Continue to refer to Figure 6 , after the mobile phone 100 encodes the original background image 300 through an encoder, the encoded information obtained by the corresponding encoding can be provided to the wallpaper service preset with the above composition scoring model 600. Then, when the mobile phone 100 runs the wallpaper service, the received encoded information can be decoded through a decoder and can also be used as the input of the scoring network 630 of the composition scoring model 600.

[0143] Furthermore, the above scoring network 630 can comprehensively consider the above layout constraint conditions and the corresponding weight values, analyze the salience of the salient subject in the decoded original background image, determine the cropping area related to the salient subject and matching the size of the cropping frame, calculate the aesthetic score of the relevant composition, and finally screen out the cropping area that meets the score screening conditions and the corresponding recommended layout parameters. The recommended layout parameters are the parameters determined by the mobile phone 100 when performing the above step 409.

[0144] In some other embodiments, the content corresponding to the above steps 402 to 403 can be described as an image semantic understanding (NLP) process and a natural language understanding (NLU) process for image semantic information. The above processes can determine the intention information and slot information related to the salient subject in the original background image.

[0145] Refer to Figure 7 , for the original background image 300, the image semantic information determined by the above NLP process can be, for example, Figure 7 "Athletes playing football" as shown, and the intention information determined by the above NLU process can include, for example, Figure 7 "Subject: {name: Athletes, feet}" as shown, and the slot information can include, for example, Figure 7 "Action: {name: Kick}, Slot: {name: Football}" as shown.

[0146] The content corresponding to the execution of the above steps 404 to 407 can be described as a slot replacement process. This process can utilize the slot information determined and extended by the above NLU process to replace the layout parameters related to the components to be layout-displayed on the target interface to be set, so as to determine the interaction relationship between each component and the salient subject. For the slot replacement process, the determined composition layout parameters can define the component positions, component sizes, component colors, etc. of each component in the target interface to be set, and can also define the size of the cropping frame for cropping out a part of the cropping area image containing the salient subject from the original background image. Refer to Figure 7 , for example, the above slot replacement process can be based on Figure 7 the relative coordinates of the reference positions in the replacement area shown, adjust the relative coordinates of the component positions in the preset layout rules of the relevant components, and adjust the component sizes of the relevant components based on the recommended size of the replacement area, so as to form Figure 7 the component area shown.

[0147] The content corresponding to the execution of the above steps 408 to 409 can be described as a composition recommendation process. This process can perform an aesthetic score on the composition result determined corresponding to the composition layout parameters determined after the above slot replacement. This aesthetic scoring process can be performed synchronously during the process of mapping the cropping area corresponding to the size of the cropping frame in the above composition layout parameters onto the original background image. This aesthetic scoring process can output a recommended composition that meets the score screening conditions in a timely manner according to the preset score threshold, or can determine a recommended composition that meets the score screening conditions after sorting the aesthetic scores corresponding to each cropping area. There is no limitation here.

[0148] Refer to Figure 7 , for example, the above composition recommendation process can crop out a cropping frame of the background image on the original background image 300. The cropping area corresponding to the cropping frame and serving as the background image can be the cropping area corresponding to the composition with the highest score in the above aesthetic scoring.

[0149] Based on the above introduction in combination with the relevant drawings, it can be understood that the display method provided by the embodiments of the present application can optimize the layout parameters of at least one component to be displayed on the target interface to be set on the background image according to the relevant features of the salient subject in the original background image set as the background image of the target interface to be set, so that an interaction connection is generated between the components on the target interface to be set and the salient subject, making the target interface to be set interactive and interesting. Moreover, in the process of determining the recommended layout parameters, the display method provided by the embodiments of the present application can screen out a cropping area with higher aesthetics based on the result of the aesthetic score for the composition of the target interface to be set, which is also beneficial to improving the aesthetics of the display of the target interface to be set.

[0150] It can be understood that in some other embodiments, the display method provided in this application may also include other implementation manners. The following will be introduced in combination with another Embodiment 2.

[0151] Embodiment 2

[0152] This embodiment of the application introduces the specific implementation process of a display method in the above Figure 1 shown application scenario. While reasonably arranging the positions and sizes of each component to be displayed on the target interface to be set to form a target interface layout to be set with interactivity and aesthetics, since it reduces the calculation and processing processes such as using a scoring network to perform saliency analysis on the original background image and performing aesthetic scoring on relevant compositions, it is beneficial to improve the efficiency of an electronic device such as a mobile phone in responding to user operations.

[0153] Specifically, Figure 8 According to the embodiment of the application, a schematic flowchart of an implementation process of a display method is shown. It can be understood that Figure 8 the execution subject of each step in the shown implementation process can be the above-mentioned electronic devices such as mobile phones and tablet computers, which will not be limited here.

[0154] Continuing to take the mobile phone 100 as an example of the execution subject below, in combination with Figure 8 the specific implementation process of the display method provided in this application will be introduced in detail.

[0155] As Figure 8 shown, the implementation process may include the following steps:

[0156] 801: Detect a user operation of setting the original background image as the background image of the target interface to be set.

[0157] 802: Perform image semantic understanding on the original background image to obtain image semantic information.

[0158] 803: Based on the image semantic information, extract relevant features of the salient subject.

[0159] 804: Obtain the preset layout rules of one or more components to be displayed on the target interface to be set.

[0160] The execution processes of the above steps 801 to 804 are the same as those of steps 401 to 404 in the above Embodiment 1. Specifically, reference can be made to the relevant descriptions of the above steps 401 to 404, which will not be elaborated here.

[0161] 805: Determine the recommended layout parameters of the target interface to be set according to the relevant features of the salient subject and the preset layout rules of one or more components.

[0162] Exemplarily, after determining the relevant features of the prominent subject in the original background image set as the background image, and the preset layout rules of one or more components that need to be displayed, the mobile phone 100 can comprehensively consider the relevant features of the prominent subject, component color, component position, component size, and screen size, etc., and then determine the recommended layout parameters of the set target interface.

[0163] It can be understood that in the recommended layout parameters, the cropped area used as the background image can be an area obtained by centering a prominent subject on the original background image, or an area cropped based on other preset cropping rules, which is not limited here.

[0164] 806 : In response to a user operation of locking the screen or switching back to the desktop, displaying a target interface synthesized by rendering according to the recommended layout parameters.

[0165] For example, after determining the recommended layout parameters that can ensure both interactivity and aesthetics, the mobile phone 100 can respond to the lock screen operation implemented by the user and display the lock screen interface drawn, rendered and synthesized based on the recommended layout parameters. Alternatively, the mobile phone 100 can also respond to the operation of switching back to the desktop implemented by the user and display the desktop drawn, rendered and synthesized based on the recommended layout parameters.

[0166] It can be understood that in the embodiment of the present application, the above-mentioned recommended layout parameters do not need to go through the calculation process such as embedding mapping of the composition scoring model, calculating the weights corresponding to each layout constraint condition, and screening out the recommended layout parameters corresponding to the recommended composition after aesthetic scoring of the relevant composition. Therefore, based on the implementation of the above-mentioned steps 801 to 806, the embodiment of the present application can achieve a rapid response to the user's operation of setting the background image of the set target interface, and improve the processing efficiency of cutting the relevant original background image and forming the set target interface composition. Moreover, based on the above-mentioned steps 801 to 806, the embodiment of the present application can also improve the interactivity and fun between the components in the set target interface and the salient subject of the background image.

[0167] Figure 9 According to an embodiment of the present application, a schematic diagram of the hardware structure of an electronic device is shown.

[0168] In some embodiments of the present application, the electronic device may be the mobile phone 100. In other embodiments, the electronic device may also be other electronic devices such as a tablet computer, which is not limited here.

[0169] like Figure 9As shown in the figure, the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identity module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0170] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0171] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0172] The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0173] In the embodiments of the present application, the processor 110 of the mobile phone 100 may read the instructions related to the execution content of steps 401 to 410 in Embodiment 1 above, or the instructions related to the execution content of steps 801 to 806 in Embodiment 2 above through the above-mentioned controller, and control the execution of the relevant instructions to implement the display method provided by the present application.

[0174] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the above-mentioned memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0175] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a universal serial bus (USB) interface, etc.

[0176] The USB interface 130 is an interface that conforms to the USB standard specification. Specifically, it may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the mobile phone 100, and can also be used for data transmission between the mobile phone 100 and peripheral devices. It can also be used to connect a headset to play audio through the headset. This interface can also be used to connect other electronic devices, such as AR devices, etc.

[0177] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only for illustrative purposes and does not constitute a limitation on the structure of the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may also adopt different interface connection methods in the above-mentioned embodiments, or a combination of multiple interface connection methods.

[0178] The charging management module 140 is used to receive charging input from a charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0179] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, the wireless communication module 160, etc.

[0180] The wireless communication function of the mobile phone 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0181] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas.

[0182] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G / 5.5G, etc. applied to the mobile phone 100.

[0183] The wireless communication module 160 can provide solutions for wireless communications applied to the mobile phone 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0184] The mobile phone 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0185] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.

[0186] The mobile phone 100 can implement the shooting function through the ISP, camera 193, video codec, GPU, display screen 194, application processor, etc. Among them, the ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to being able to process digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos.

[0187] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0188] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.). The data storage area can store the data created during the use of the mobile phone 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121 and / or the instructions stored in the memory provided in the processor.

[0189] The mobile phone 100 can implement the audio function through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, etc. For example, music playback, recording, etc.

[0190] The keys 190 include a power-on key, volume keys, etc. The keys 190 can be mechanical keys or touch keys. The mobile phone 100 can receive key inputs and generate key signal inputs related to the user settings and function controls of the mobile phone 100.

[0191] The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. For touch operations applied to different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminder, receiving messages, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0192] The indicator 192 can be an indicator light and can be used to indicate the charging state, power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0193] The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the mobile phone 100.

[0194] Figure 10 According to an embodiment of the present application, a schematic diagram of the operating system software structure of an electronic device is shown.

[0195] It can be understood that the operating system of the mobile phone 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In an embodiment of the present invention, taking the Android system with a layered architecture as an example, the system software structure of the mobile phone 100 is exemplarily described.

[0196] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android TM system is divided into four layers, from top to bottom are the application layer, the application framework layer, Android TM runtime (Android TM runtime), and the kernel layer.

[0197] As Figure 10 shown, the application layer can include a series of application packages. The application packages can include applications such as the desktop, gallery, wallpaper, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0198] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0199] The application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a wallpaper service, etc.

[0200] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0201] The content provider is used to store and obtain data, and make this data accessible to application programs. This data may include videos, images, audio, incoming and outgoing calls, browsing history and bookmarks, phone books, etc.

[0202] The view system includes visible controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build application programs. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

[0203] The phone manager is used to provide the communication functions of the mobile phone 100. For example, the management of call states (including answering, hanging up, etc.).

[0204] The resource manager provides various resources for application programs, such as localized strings, icons, pictures, layout files, video files, etc.

[0205] The notification manager enables application programs to display notification information in the status bar. It can be used to convey notification-type messages, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform that the download is completed, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or a scroll bar text, such as the notification of a background-running application program, and can also be a notification that appears on the screen in the form of a dialogue window. For example, prompting text information in the status bar, emitting a prompt tone, vibrating the electronic device, flashing the indicator light, etc.

[0206] In the embodiment of the present application, the notification manager can also make the notification messages of application programs be displayed in the form of lock screen notifications on the lock screen interface. The display position of this lock screen notification on the lock screen interface can be determined based on the display scheme provided by the present application.

[0207] The wallpaper service can provide functions for the desktop applications, wallpaper applications, etc. in the above application program layer to process the original background image into a wallpaper and determine the composition of the target interface to be set. The wallpaper service can respond to the calls of desktop applications or wallpaper applications, etc., and implement the relevant content of steps 402 to 409 in Embodiment 1 of the present application, or implement the relevant content of steps 802 to 805 in Embodiment 2 of the present application. The specific implementation process can refer to the relevant descriptions of the above relevant steps and will not be elaborated here.

[0208] Android TM The Runtime includes core libraries and a virtual machine. Android TM runtime is responsible for Android TM system scheduling and management.

[0209] The core libraries consist of two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core libraries of Android.

[0210] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0211] The system libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (such as OpenGL ES), 2D graphics engines (such as SGL), etc.

[0212] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media libraries support the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media libraries can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is the drawing engine for 2D drawing.

[0213] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, and sensor drivers.

[0214] The embodiments of the present application also provide a computer program product for implementing the display methods provided in the above embodiments.

[0215] The embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as computer program modules or module codes executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memories and / or storage elements), at least one input device, and at least one output device.

[0216] A computer program module or module code can be applied to input instructions to perform the various functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0217] The module code can be implemented in a high-level modular language or an object-oriented programming language to communicate with the processing system. When needed, the module code can also be implemented in assembly language or machine language. In fact, the mechanisms described in this application are not limited to the scope of any particular programming language. In any case, the language can be a compiled language or an interpreted language.

[0218] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions can be distributed via a network or via other computer-readable media. Thus, a machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to, a floppy disk, a compact disc, a CD-ROM, a magneto-optical disk, a read only memory (ROM), a random access memory (RAM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic or optical card, a flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) in electrical, optical, acoustic, or other forms using the Internet. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0219] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technique disclosed in embodiments of the present application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0220] The disclosure of embodiments of the present application also relates to an apparatus for performing operations in the text. The apparatus may be specifically constructed for the required purpose or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer-readable medium, such as, but not limited to, any type of disk, including a floppy disk, optical disk, CD-ROM, magneto-optical disk, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical card, application specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, and each may be coupled to a computer system bus. In addition, the computers mentioned in the specification may include a single processor or may be architectures involving multiple processors for increased computing power.

[0221] In addition, the language used in this specification has been principally selected for readability and guidance purposes and may not have been selected to delineate or circumscribe the disclosed subject matter. Accordingly, the disclosure of embodiments of the present application is intended to illustrate rather than limit the scope of the concepts discussed herein.

Claims

1. A display method, applied to an electronic device, characterized in that, it includes: detecting an interface display instruction; in response to the interface display instruction, displaying a first interface, wherein the first interface includes at least one first foreground element and a first background image area in a first background image, where the display parameter of the first foreground element in the first background image area includes the position parameter of the first foreground element, and the position parameter is related to the semantic information of at least one display element in the first background image area.

2. The method according to claim 1, characterized in that, the semantic information includes one or more of the following features: features related to actions; features related to morphology.

3. The method according to claim 2, characterized in that, the features related to actions include one or more of the following features: features for describing actions; features for describing the object of an action.

4. The method according to claim 2, characterized in that, the features related to morphology include one or more of the following features: features for describing morphology; features for describing the stretching direction of morphology.

5. The method according to any one of claims 1 to 4, characterized in that, before displaying the first interface, the method further includes: performing image semantic understanding on the first background image to obtain semantic information related to the at least one display element and regional position information corresponding to the features included in the semantic information; determining first regional position information corresponding to a first feature among the features included in the semantic information; determining the position parameter of the first foreground element according to the first regional position information.

6. The method according to claim 5, characterized in that, the first feature is a feature for describing the object of an action, and the determining the position parameter of the first foreground element according to the first regional position information includes: determining the first regional position information as the position parameter of the first foreground element.

7. The method according to any one of claims 1 to 4, characterized in that, the display parameter further includes the color parameter of the first foreground element, and the color parameter is related to one or more of the following colors: the initial color of the first foreground element; the background color of the area where the feature for describing the object of an action is located; the background color of the area where the feature for describing the stretching direction of morphology is located; the color of the first display element among the at least one display element.

8. The method according to claim 5, characterized in that, the method for determining the first background image area includes: based on the screen size of the electronic device, determining the size parameter of a first cropping frame for cropping out the first background image area on the first background image; according to the first semantic information of the first display element, the position information of the first display element in the first background image, and the size parameter of the first display element, determining at least one cropping area that meets the size parameter of the first cropping frame and includes the first display element, where the at least one display element includes the first display element; Determine the first cropping area in the at least one cropping area as the first background image area.

9. The method according to claim 8, wherein, the size parameter of the first cropping frame is further related to the size parameter of the first foreground element, and the manner of determining the first background image area includes: Based on the screen size of the electronic device and the size parameter of the first foreground element, determine the size parameter of the first cropping frame.

10. The method according to claim 8 or 9, wherein, the at least one cropping area is a plurality of cropping areas, and the determining the first cropping area in the at least one cropping area as the first background image area includes: Performing an aesthetic score on the at least one cropping area, and determining the first cropping area whose aesthetic score meets the third condition as the first background image area, where the third condition includes that the aesthetic score is higher than the second threshold or the highest value in the aesthetic scores.

11. The method according to any one of claims 1 to 10, wherein, the first interface includes a lock screen interface or a desktop, and the interface display instruction includes: A display instruction triggered by a first user operation indicating the display of the lock screen interface; or, A display instruction triggered by a second user operation indicating the display of the desktop.

12. An electronic device, wherein, comprising: One or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device executes the display method according to any one of claims 1 to 11.

13. A computer-readable medium, wherein, Instructions are stored on the readable medium, and when the instructions are executed on a computer, the computer executes the display method according to any one of claims 1 to 11.

Images