Window Display Method, Device, Electronic Device and Storage Medium

Abstract

This application relates to the field of computer technology, and specifically discloses a window display method, device, electronic device, and storage medium. The method includes: obtaining the position information of an image acquisition component in a terminal relative to a display screen in the terminal, and obtaining the display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; obtaining the layout information of an image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; and displaying the image window in the reference display area according to the layout information. This solution can achieve flexible layout of the image window according to the position of the image acquisition component and the reference display area.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly relates to a window display method, apparatus, electronic device, and storage medium. Background Art

[0002] In scenarios such as face payment and face recognition verification, an image window is displayed on a display screen for a user to preview a face image captured by a camera in the image window. In related technologies, the image window is displayed in a fixed size, and the layout form of the image window is single. Summary of the Invention

[0003] Embodiments of this application provide a window display method, apparatus, electronic device, and storage medium to solve the problem in related technologies that the image window has a fixed size and a single layout form.

[0004] According to one aspect of the embodiments of this application, a window display method is provided, including: obtaining position information of an image acquisition component in a terminal relative to a display screen in the terminal, and obtaining display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; obtaining layout information of an image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; and displaying the image window in the reference display area according to the layout information.

[0005] According to one aspect of the embodiments of this application, a window display method is provided, including: receiving reported data, where the reported data includes position information of an image acquisition component in a terminal relative to a display screen in the terminal and display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; obtaining layout information of an image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; and returning the layout information to the terminal so that the terminal displays the image window in the reference display area according to the layout information.

[0006] According to one aspect of the embodiments of this application, a window display apparatus is provided, including: an obtaining module, configured to obtain position information of an image acquisition component in a terminal relative to a display screen in the terminal, and obtain display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; a first layout information obtaining module, configured to obtain layout information of an image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; and a display module, configured to display the image window in the reference display area according to the layout information.

[0007] In some embodiments of the present application, based on the foregoing solution, the layout information includes window position parameters; the first layout information obtaining module includes: a relative position information determining unit, configured to determine the relative position information between the image acquisition component and the reference display area according to the display area information and the position information of the image acquisition component relative to the display screen in the terminal; a window position parameter determining unit, configured to obtain the window position parameters corresponding to the relative position information.

[0008] In some embodiments of the present application, based on the foregoing solution, the layout information includes window size parameters; the first layout information obtaining module includes: a size parameter determining unit, configured to determine the size parameters of the reference display area according to the display area information; an adjustment coefficient determining unit, configured to determine an adjustment coefficient according to the size parameters of the reference display area; a scaling ratio determining unit, configured to adjust the reference ratio according to the adjustment coefficient to obtain a scaling ratio; a target window size parameter determining unit, configured to scale the size parameters of the reference display area according to the scaling ratio to obtain the window size parameters.

[0009] In some embodiments of the present application, based on the foregoing solution, the layout information further includes layout direction parameters; the first layout information obtaining module further includes: a layout direction parameter obtaining module, configured to obtain the layout direction parameters from the display area information.

[0010] In some other embodiments of the present application, based on the foregoing solution, the first layout information obtaining module includes: an input unit, configured to input the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area into a neural network model; an output unit, configured to output the layout information of the image window by the neural network model according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; wherein, the neural network model is determined by training the model according to a plurality of training samples, and the training samples include sample position information, sample display area information, and sample layout information; the sample position information indicates the position of the sample image acquisition component relative to the sample display screen in the sample terminal; the sample display area information indicates the sample reference display area in the sample display screen; the sample layout information indicates the sample target display area occupied by the sample image window in the sample reference display area, and the sample target display area is adapted to the position of the sample image acquisition component and the sample reference display area.

[0011] In some embodiments of the present application, based on the foregoing solution, the window display device further includes: a training sample acquisition module for acquiring a plurality of training samples; a predicted layout information obtaining module for outputting predicted layout information by the neural network model according to the sample position information and sample area information in the training samples; an adjustment module for adjusting the weight parameters of the neural network model according to the predicted layout information and the sample layout information in the training samples until the neural network model converges.

[0012] In some embodiments of the present application, based on the foregoing solution, the window display device further includes: a height difference information acquisition module for acquiring the height difference information between the user to be captured and the image acquisition component; a reference position range determination module for determining a reference position range according to the height difference information; a window position parameter adjustment module for adjusting the window position parameters in the layout information according to the reference position range if the position of the image window indicated by the layout information is not within the reference position range.

[0013] In some embodiments of the present application, based on the foregoing solution, the acquisition module includes: a split screen information acquisition unit for acquiring the split screen information of the display screen if the target application needs to call an image window; a first determination unit for, if the split screen information indicates that the display screen is split, taking the split screen display area corresponding to the target application as the reference display area and acquiring the display area information of the split screen display area corresponding to the target application; in this embodiment, the display module is configured to: control the layout of the image window in the split screen display area corresponding to the target application according to the layout information.

[0014] In some other embodiments of the present application, based on the foregoing solution, the acquisition module includes: a split screen information acquisition unit for acquiring the split screen information of the display screen if the target application needs to call an image window; a first display area information acquisition unit for acquiring the display area information of the split screen display area corresponding to the target application if the split screen information indicates that the display screen is split; a second determination unit for, if it is determined according to the display area information of the split screen display area corresponding to the target application that the size of the split screen display area corresponding to the target application is lower than a set size threshold, taking the full screen display area as the reference display area and acquiring the display area information of the full screen display area; in this embodiment, the display module is configured to: display the image window in the full screen display area according to the layout information.

[0015] In some embodiments of the present application, based on the foregoing solution, the acquisition module further includes: a third determination unit, configured to use the full-screen display area as the reference display area and obtain the display area information of the full-screen display area if the split-screen information indicates that the display screen is not split-screen; in this embodiment, the display module is configured to: display the image window in the full-screen display area according to the layout information.

[0016] According to one aspect of the embodiments of the present application, there is provided a window display device, including: a receiving module, configured to receive reported data, where the reported data includes the position information of an image acquisition component in a terminal relative to a display screen in the terminal and the display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; a second layout information obtaining module, configured to obtain layout information of an image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; and a return module, configured to return the layout information to the terminal, so that the terminal displays the image window in the reference display area according to the layout information.

[0017] According to one aspect of the embodiments of the present application, there is provided an electronic device, including: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the above-mentioned window display method is implemented.

[0018] According to one aspect of the embodiments of the present application, there is provided a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the above-mentioned window display method is implemented.

[0019] According to one aspect of the embodiments of the present application, there is provided a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device obtains the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method.

[0020] In the solution of the present application, according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area in the display screen, the layout information of the image window is determined, and the image window is laid out in the reference display area according to the layout information, so as to realize the flexible layout of the image window according to the position information of the image acquisition component relative to the display screen and the display area information of the reference display area, enrich the layout modes of the image window in different display screens, and effectively solve the problem of single layout mode of the image window in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the present application can be applied.

[0023] Figure 2 A flowchart of a window display method shown according to an embodiment of the present application.

[0024] Figure 3 A schematic structural diagram of a neural network shown according to an embodiment.

[0025] Figure 4 A schematic diagram of the calculation process in a single neuron shown according to an embodiment.

[0026] Figure 5 A schematic diagram showing model training according to an embodiment of the present application.

[0027] Figure 6 A schematic diagram of the display of an image window shown according to a specific embodiment of the present application.

[0028] Figure 7 A schematic diagram of the display of an image window shown according to another specific embodiment of the present application.

[0029] Figure 8 A schematic diagram of the display of an image window shown according to another specific embodiment of the present application.

[0030] Figure 9 A flowchart of the step of obtaining the display area information of the reference display area in the display screen in step 210 shown according to an embodiment of the present application.

[0031] Figure 10 A flowchart of the step of obtaining the display area information of the reference display area in the display screen in step 210 shown according to another embodiment of the present application.

[0032] Figure 11 A schematic diagram of the cross-split screen display of an image window shown according to a specific embodiment of the present application.

[0033] Figure 12 A flowchart of a window display method shown according to another embodiment of the present application.

[0034] Figure 13It is a flowchart of a window display method shown according to a specific embodiment of the present application.

[0035] Figure 14 It is a block diagram of a window display device shown according to an embodiment.

[0036] Figure 15 It shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. Detailed Embodiments

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0038] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0039] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the drawings are only illustrative and do not necessarily include all the content and operations / steps, nor do they necessarily have to be executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0041] It should be noted that: "a plurality of" mentioned herein refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0042] Figure 1 It shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.

[0043] As Figure 1 shown, the system architecture may include a terminal 110 and a server 120. The terminal 110 and the server 120 establish a communication connection through a wired or wireless network. The terminal 110 may be a smart phone, a laptop computer, a desktop computer, a payment terminal device, an access control terminal, a display device equipped with an image acquisition component, a bank self-service terminal, a mall self-service terminal, a television, etc., which are not specifically limited herein.

[0044] The terminal 110 is provided with a display screen and an image acquisition component. In Figure 1 , the image acquisition component in the terminal 110 is a camera 111. The camera is provided with an image sensor. The image sensor may be a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or a combination of an image sensor and a structured light sensor. The camera 111 is a front camera in the terminal 110. Thus, when the user faces the display screen, the image window can display the image collected by the camera for the user to preview the image.

[0045] The full-screen display area 112 in the display screen of the terminal 110 can be used to display the user interfaces of various applications. When an application in the terminal 110 needs to call the image window, the image window 113 can also be superimposed and displayed on the user interface of the application. The image window is used to preview the user image collected by the image acquisition component when the user faces the display screen. The collected user image may be, for example, the user's face image, iris image, etc., which are not specifically limited herein. Figure 1 shows a face image 114 displayed in the image window 113.

[0046] The server 120 may be the server corresponding to the application that can call the image window in the terminal 110, or the server corresponding to the software development kit (SDK) that calls the image window in the terminal.

[0047] In some embodiments of the present application, the terminal 110 may send reporting data to the server 120. The reporting data includes the position information of the image acquisition component in the terminal relative to the display screen in the terminal and the display area information of the reference display area in the display screen. The reference display area may be a full-screen display area or a split-screen display area. After receiving the reporting data, the server 120 predicts the layout information of the image window according to the position information in the reporting data and the display area information of the reference display area. Among them, the layout information indicates the display position and size of the image window. In some embodiments, the layout information further indicates the layout direction of the image window. After that, the server 120 returns the layout information of the image window to the terminal 110, so that the terminal 110 displays the image window in the set display area according to the layout information, ensuring that the target display area where the image window is located in the reference display area is adapted to the position of the image acquisition component and the reference display area.

[0048] In some embodiments of the present application, when the user to be recognized looks at the image window (such as the image in the preview image window), the image acquisition component facilitates the image acquisition component to acquire a front image (such as a frontal face image) of the user to be recognized.

[0049] In some embodiments of the present application, the terminal 110 may send reporting data to the server 120 when the target application needs to call the image window. The server 120 predicts the layout information of the image window according to the real-time reporting data of the terminal 110. Correspondingly, after receiving the layout information returned by the server 120, the terminal 110 layouts the image window according to the layout information and displays the image window.

[0050] In some embodiments of the application, the terminal 110 can also independently determine the time to send the reporting data to the server 120. For example, when the network condition of the terminal 110 is good and the number of programs running in the terminal 110 is small. After receiving the layout information returned by the server 120, the terminal 110 can store the layout information locally in the terminal 110. When the terminal 110 needs to call the image window, it obtains the locally stored layout information to layout the image window in the reference display area and display the image window. In this embodiment, the terminal 110 independently selects the time to send the reporting data to the server, rather than sending the reporting data when the application in the terminal needs to call the image window, which can avoid the problem that the terminal 110 cannot obtain the layout information in time due to the network communication quality problem when the image window needs to be called, and thus cannot display the image window in time.

[0051] In Figure 1In the system architecture shown, the server 120 predicts the layout information of the image window based on the position information of the image acquisition component in the terminal relative to the display screen and the area display information of the reference display area in the display screen. In other embodiments, if the computing and processing capabilities of the terminal 110 meet the requirements, the terminal 110 itself can also predict the layout information of the image window based on the position information of the image acquisition component relative to the display screen and the area display information of the reference display area in the display screen, without the participation of the server 120.

[0052] It should be understood that Figure 1 the numbers of the terminal 110 and the server 120 in are merely illustrative. According to the implementation requirements, any number of terminals and servers can be provided. For example, the server 120 can be a server cluster composed of multiple servers, etc.

[0053] The following elaborates in detail on the implementation details of the technical solution of the embodiments of the present application:

[0054] Figure 2 shows a flowchart of a window display method according to an embodiment of the present application. This method can be executed by a computer device with processing capabilities, such as Figure 1 the terminal shown, implemented through the interaction between the terminal device and the server, etc., which is not specifically limited herein. Referring to Figure 2 shown, this method at least includes steps 210 to 230, which are introduced in detail as follows:

[0055] Step 210, obtain the position information of the image acquisition component in the terminal relative to the display screen in the terminal, and obtain the display area information of the reference display area in the display screen; the reference display area includes the full-screen display area or the split-screen display area.

[0056] The position information of the image acquisition component in the terminal relative to the display screen in the terminal is used to indicate the position of the image acquisition relative to the display screen when using the display screen or the plane where the display screen is located as the position reference. This position information can be the coordinate information of the image acquisition component in the coordinate system of the plane where the display screen is located. The position information of the image acquisition component relative to the display screen in the terminal can be determined according to the position parameters of the image acquisition component stored in the terminal. This position parameter can be the distance indicating the image acquisition component from the edge of the display screen. Thus, based on the position parameter of the image acquisition component and the size parameter of the display screen, the position information of the image acquisition component relative to the display screen is determined.

[0057] The reference display area refers to the area for determining the display area occupied by the image window in the display screen. It can be the full-screen display area of the display screen or the split-screen display area of any split screen in the display screen.

[0058] The split-screen display area refers to the display area occupied by the split screen in a display screen that supports split-screen when the display screen is in the split-screen state. When the display screen is in the split-screen state, the display screen includes at least two split screens. In this case, the split-screen display area used as the reference display area is the split-screen display area corresponding to the split screen of the application that needs to call the image window.

[0059] The full-screen display area is the largest display area provided by the display screen for display. When the display screen is not split, the user interface displayed on the display screen fills the full-screen display area. When the display screen is in the split-screen state, the display area obtained by combining the split-screen display areas corresponding to all split screens is the full-screen display area. It can be understood that if the display screen does not support split-screen display, the reference display area in step 210 is the full-screen display area.

[0060] The display area information of the reference display area is used to indicate the position and size of the reference display area in the display screen. Therefore, the display area information of the reference display area includes a position parameter for indicating the position of the reference display area in the display screen and a size parameter for indicating the size of the reference display area. In some embodiments, the display area information further includes a layout direction parameter for indicating the layout direction of the reference display area, and the layout direction parameter is used to indicate whether the display screen is in landscape or portrait orientation. In some embodiments, the Orientation parameter of the display screen can be used as the layout direction parameter of the display screen, and the value of the Orientation parameter is used to identify whether the display screen is in landscape or portrait orientation.

[0061] In a specific embodiment, whether to use the full-screen display area or the split-screen display area as the reference display area can be set according to actual needs. For example, regardless of whether the display screen supports split-screen, the full-screen display area is used as the reference display area. Another example is that if the display screen is in the split-screen state when an image window needs to be called, the split-screen display area corresponding to the split screen of the application that needs to call the image window is used as the reference display area; conversely, if the display screen is not in the split-screen state when an image window needs to be called, the full-screen display area is used as the reference display area; another example is that if the size of the split-screen display area is lower than a set threshold, the full-screen display area is used as the reference display area, and only when the size of the split-screen display area is not lower than the set threshold, the split-screen display area is used as the reference display area.

[0062] In the case where the reference display area is the full-screen display area of the display screen, the display area information of the reference display area is the display area information of the full-screen display area, where the display area information of the full-screen display area can be obtained by reading the device parameters of the display screen, that is, the device parameters of the display screen include parameters indicating the position and size of the largest display area in the display screen.

[0063] When the reference display area is a split-screen display area, the display area information of the reference display area is the display area information of the split-screen display area. When the display screen is in a split-screen state, the display screen records the split-screen parameters of the display screen, such as the number of split-screens, the position of the split-screen boundary, etc. In this case, the display area information of the split-screen display area corresponding to the split-screen can be determined according to the split-screen parameters of the display screen and the display area information of the full-screen display area of the display screen.

[0064] Step 220: Obtain the layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area.

[0065] The image window is used to preview the image captured by the image acquisition component. When the image acquisition component is the front image acquisition component in the terminal, such as the front camera of a smart phone, the user can preview the user image of himself captured by the front image acquisition component through the image window.

[0066] The layout information of the image window is used to indicate the layout of the image window in the reference display area, that is, it indicates the position and size of the image window in the reference display area. Therefore, the layout information of the image window defines the display area occupied by the image window in the reference display area. For the sake of distinction, the display area occupied by the image window in the reference display area is called the target display area. The parameter used to indicate the position of the image window in the reference display area in the layout information is called the window position parameter, and the parameter used to indicate the size of the image window in the reference display area in the information is called the window size parameter. Therefore, the layout information may include a window position parameter and a window size parameter.

[0067] In some embodiments of the present application, the layout information of the image window may further include a layout direction parameter indicating the layout direction of the image window. When the image window is a centrally symmetric figure, such as a circle, the effect of laying out the image window in all directions in the plane is the same. Therefore, in this scenario, the layout information of the image window may not include the target layout direction. When the image window is a non-centrally symmetric figure, such as a rectangle or an arc shape adapted to the human body contour, the number of target window position parameters can be reduced, and the position of the image window can be determined by combining the target layout direction and some window position parameters.

[0068] In the solution of the present application, the layout information of the image window is determined according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area, and the image window is displayed in the reference display area according to the layout information, so that the displayed image window can be adapted to the position of the image acquisition component and the reference display area, that is, the adaptability between the position and size of the image window and the position of the image acquisition component and the reference display area. Therefore, in step 220, the layout information of the image window can be determined with the target that the target display area occupied by the image window in the reference display area is adapted to the position of the image acquisition component and the reference display area.

[0069] In some embodiments of the present application, the image window is used to preview the user image collected by the image acquisition component when the user faces the display screen. Since it is the captured object (user) who previews the user image displayed in the image window, the image window has the function of guiding the line-of-sight direction of the captured user. In this case, if the display position of the image window is inappropriate, it may lead to incorrect guidance of the user's line of sight. For example, guiding in the opposite direction of the image acquisition component will cause the image acquisition component to be unable to capture the front face image of the user, but capture a non-front face image such as the side face of the user. Compared with the front face image, it is more difficult to perform face recognition based on the non-front face image, and the accuracy of face recognition is also reduced.

[0070] Therefore, the adaptability of the target display area where the image window is located to the position of the image acquisition component and the reference display area is reflected in: (1) The size of the target display area is adapted to the size of the reference display area, ensuring that the user's face area in the captured image is displayed as much as possible in the image window, and making the size of the displayed image window convenient for the user to operate on the display screen, avoiding the situation where the displayed image window is too large or too small and inconvenient for the user to operate; (2) The display position of the image window is adapted to the position of the image acquisition component, so that the displayed image window correctly guides the user's line-of-sight direction, and when the user previews the image displayed in the image window, the image acquisition component captures the front face image of the user; (3) Further, since the displayed image window has a guiding effect on the user's line-of-sight direction, if the size of the displayed image window is large, the displayed image window gives the user a large range of line-of-sight direction choices. In this case, it may also cause the deflection direction of the user image captured by the image acquisition component to be relatively large compared to the front face image when the user faces the display screen to preview the image window. Therefore, considering the guiding effect of the image window on the user's line-of-sight direction, it is necessary to limit the size of the image window to avoid the situation where it is inconvenient to capture the front face image of the user due to the too large image window.

[0071] In the solution of this application, based on the principle that the target display area indicated by the layout information is adapted to the position of the image acquisition component and to the reference display area, the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area are used as the basis for determining the layout information, and the position and size of the image window are defined by comprehensively considering the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area.

[0072] In the solution of this application, the shape of the image window can be set according to actual needs, such as rectangular, circular, oval, a shape adapted to the human contour, etc. In a specific embodiment, the shape of the image window can be specified in advance, and the layout information can be determined according to the parameters required to determine the position and size of the image window in accordance with this shape.

[0073] In some embodiments of this application, the layout information of the image window can be output by a trained neural network model. During the process of training the neural network model, the neural network model learns, through training samples, the law between the position information of the image acquisition component relative to the display screen and the display area information of the reference display area in the display screen, and the layout of the image window adapted to the position of the image acquisition component and the reference display area. Thus, after the training is completed, the neural network model can accurately predict the layout information of the image window adapted to the position of the image acquisition component and the reference display area based on the position information of the image acquisition component relative to the display screen and the display area information of the reference display area in the display screen.

[0074] In some embodiments of this application, still based on the principle that the target display area indicated by the layout information is adapted to the position of the image acquisition component and to the reference display area, the window position parameter indicating the position of the image window and the window size parameter indicating the size of the image window are respectively calculated based on the position information of the image acquisition component relative to the display screen and the display area information of the reference display area in the display screen. In the case where a layout direction is required, the layout direction parameter indicating the layout direction of the image window can be further determined. The specific calculation process is described below and will not be elaborated here.

[0075] Step 230, display the image window in the reference display area according to the layout information.

[0076] After the terminal obtains the layout information of the image window, according to the position and size indicated by the layout information, the image window is correspondingly displayed in the reference display area, so that the position of the target display area occupied by the image window in the reference display area is the position indicated by the layout information, and the size of the target display area is the size indicated by the layout information. In some embodiments of this application, the image window is displayed in the form of a floating window on the user interface of the terminal.

[0077] In the solution of the present application, according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area in the display screen, the layout information of the image window is determined, and the image window is laid out in the reference display area according to the layout information, so as to realize the flexible layout of the image window according to the position information of the image acquisition component relative to the display screen and the display area information of the reference display area. It can ensure the adaptability of the target display area occupied by the displayed image window in the reference display area to the position of the image acquisition component and the reference display area, enrich the layout methods of the image window in different display screens, and effectively solve the problem of the single layout method of the image window in the related art.

[0078] Moreover, since the image window is used to preview the user image collected by the image acquisition component when the user faces the display screen, the displayed image window has the function of guiding the user's line of sight. In this solution, the image window displayed according to the layout information is adapted to the position of the image acquisition component and the reference display area, and the user's line of sight can be correctly guided through the image window, so that when the user looks at the image window, it is convenient for the image acquisition component to collect the front image of the user.

[0079] In some embodiments of the present application, step 220 includes: inputting the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area into a neural network model; the neural network model outputs the layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; wherein, the neural network model is determined by training the model according to a number of training samples, and the training samples include sample position information, sample display area information and sample layout information; the sample position information indicates the position of the sample image acquisition component in the sample terminal relative to the sample display screen in the sample terminal; the sample display area information indicates the sample reference display area in the sample display screen; the sample layout information indicates the sample target display area occupied by the sample image window in the sample reference display area, and the sample target display area is adapted to the position of the sample image acquisition component and the sample reference display area.

[0080] Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable machines to have the functions of perception, reasoning, and decision-making.

[0081] Machine learning algorithms are a type of algorithms that enable computers to automatically analyze and obtain rules from data and use the rules to predict unknown data. Neural networks are one of the machine learning algorithms. A neural network is a mathematical model or computational model that mimics the structure and function of a biological neural network and is used to estimate or approximate a function. A neural network is composed of a large number of nodes (or neurons) interconnected with each other. Each node represents a specific output function, called an activation function. The connection between each two nodes represents a weight value for the signal passing through this connection, which is called a weight, and this is equivalent to the memory of the neural network. The output of the network depends on the connection method, weight values, and activation functions of the network.

[0082] Figure 3 It is a schematic diagram of the structure of a neural network shown according to an embodiment, as Figure 3 shown. This neural network includes a plurality of interconnected neurons 310. After the neurons in the input layer receive the input, they perform operations and judgments based on this input and transmit the information to the neurons in the next layer, and so on until the output is obtained. Figure 4 It shows the calculation process in a single neuron. In Figure 4 , a1~an are the components of the input vector, w1~wn are the weights of the respective synapses of the neuron, b is the bias, f is the transfer function, usually a non-linear function; SUM represents summation. Figure 4 The calculation process in the neuron can be summarized as follows: after weighting the components of the input vector according to the weights, a scalar result is obtained through a non-linear transfer function.

[0083] In this embodiment, the neural network model for outputting layout information can be constructed based on convolutional neural networks, recurrent neural networks, fully connected networks, etc., and specific limitations are not provided here. Before putting this neural network model into online application, it is necessary to train this neural network model first. The training process can be as Figure 5As shown, the pre-constructed neural network model is trained with a number of training samples (training sample 1, training sample 2... training sample n). After the training is completed, a neural network model that can be applied is obtained.

[0084] Among them, the training samples include sample position information, sample display area information, and sample layout information. For the convenience of distinction, the terminal from which the data for training the neural network model is sourced is called the sample terminal, the image acquisition component in the sample terminal is called the sample image acquisition component, and the display screen in the sample terminal is called the sample display screen. The reference display area in the sample display screen is called the sample reference display area, the image window adapted to the position of the sample image acquisition component and the sample reference display area is called the sample image window, and correspondingly, the target display area occupied by the sample image window in the sample reference display area is called the sample target display area. The layout information of the sample image window is called the sample layout information.

[0085] In some embodiments of the present application, multiple sample terminals with different position information of the image acquisition component relative to the display screen and / or different reference display area information in the display screen can be selected. Thus, multiple sets of sample information can be determined, where each set of sample information includes sample position information and sample display area information. The different position information of the image acquisition component relative to the display screen can be that the image component is arranged at different positions in the display screen, and the different reference display area information in the display screen can be that all the display areas in the display screen are different (for example, the shape and / or size of the full-screen display area are different), or the split-screen display areas in the display screen are different.

[0086] In some embodiments of the present application, for each set of sample information (sample position information and sample display area information), image windows are respectively displayed in the sample display screen according to a preset set of layout information. After the image acquisition component in the sample terminal is turned on, during the process where the user faces the image windows displayed according to each set of layout information, the frontal face image of the user is collected, and the image quality score of the collected frontal face image is calculated. The layout information corresponding to the highest image quality score among the set of layout information is used as the sample layout information corresponding to this sample information (sample position information and sample display area information). Thus, a sample information (sample position information and sample display area information) and the sample layout information corresponding to this sample information are used as a training sample. By repeating the above process, multiple training samples can be constructed.

[0087] In some embodiments of the present application, in order to further ensure the reliability of the sample layout information determined for each group of sample information, for each layout information under each group of sample information, frontal face images can be collected for multiple users respectively, and the image quality scores of the frontal face images of each user under each layout information can be calculated. Then, the sample layout information corresponding to the sample information can be determined by integrating the image quality scores corresponding to each layout information of the multiple users. For example, calculate the average value of the image quality scores corresponding to the multiple users under each layout information, and use the layout information corresponding to the highest average image quality score as the sample layout information corresponding to the sample information. In some embodiments of the present application, the training process of the neural network model may include the following process: obtain a number of training samples; the neural network model outputs predicted layout information according to the sample position information and sample area information in the training samples; adjust the weight parameters of the neural network model according to the predicted layout information and the sample layout information in the training samples until the neural network model converges.

[0088] The predicted layout information refers to the layout information output by the neural network model according to the sample position information and sample area information in the training samples.

[0089] In some embodiments of the present application, the function value of the loss function of the neural network model is calculated based on the sample local information and the predicted layout information. The function value of the loss function is calculated based on the predicted layout information and the sample layout information in the training samples, and then the weight parameters of the neural network model are adjusted according to the function value of the loss function until the neural network model converges. The loss function can be an absolute value loss function, a logarithmic loss function, a squared loss function, an exponential loss function, etc., and is not specifically limited herein.

[0090] Through the above training process, the neural network model learns the ability to predict the layout information of the image window based on the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area. Thus, it can be ensured that during the online application process of the neural network model, based on the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area, the layout information of the image window that is adapted to the position of the image acquisition component and the size of the reference display area can be accurately output, realizing the automatic determination of an image window that is adapted in position and size according to the position information of the image acquisition component relative to the display screen and the reference display area, and improving the efficiency of determining the layout information.

[0091] In some embodiments of the present application, the training samples include, in addition to sample position information, sample display area information, and sample layout information, the age information of the sample user. In this embodiment, the sample target display area occupied by the sample image window in the sample reference display area determined according to the sample layout information is related to the age of the sample user. In other words, the sample layout information is adapted not only to the position of the sample image acquisition component and the sample reference display area, but also to the age of the sample user.

[0092] To construct the training samples, for each set of sample information (including sample position information and sample display area information), an image window is displayed in the sample display screen according to each set of layout information in a preset multiple sets of layout information. After the image acquisition component in the sample terminal is turned on, on the one hand, the user scores the experience of the image in the image window previewed by the user to obtain an experience score; on the other hand, for each set of layout information, an image quality score is obtained by scoring the frontal face image of the sample user acquired by the image acquisition component. Then, according to the experience score and the corresponding image quality score corresponding to each set of layout information, a comprehensive score corresponding to each set of layout information is calculated, and the layout information with the highest comprehensive score is used as the sample layout information corresponding to the sample position information and the sample display area information.

[0093] According to the above process, the sample layout information for each sample user corresponding to each set of sample information (sample position information, sample display area information) can be determined accordingly. On this basis, the age information of a sample user, a set of sample information (sample position information and sample display area information), and the sample layout information of the sample user corresponding to this sample information are used as a training sample, and multiple training samples are thus obtained.

[0094] In the process of training the neural network model, the sample position information, the sample display area information, and the age information of the sample user are input into the neural network model. The neural network model performs layout parameter prediction and outputs predicted layout information, and then adjusts the parameters of the neural network model according to the predicted layout information and the corresponding sample layout information.

[0095] Since the sample layout information is based on the experience scores given by sample users for the displayed image window, and users in different age groups may have different experience scores for image windows with different layouts. For example, under the same other conditions, compared with young users, the size of the image window with a higher experience score for elderly users is larger. Therefore, the trained neural network model can learn the relationship between the layout information of the sample window, the age information of the user, the position information of the image acquisition component in the terminal relative to the display screen in the terminal, and the display area information of the reference display area in the display screen. Thus, the layout information can be predicted based on the age information of the user, the position information of the image acquisition component in the terminal relative to the display screen in the terminal, and the display area information of the reference display area in the display screen, and the layout information adapted to the input information of the model can be output.

[0096] In the solution of this embodiment, the neural network model can further predict the layout information in a targeted manner by combining the age of the user on the basis of referring to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area in the display screen, ensuring that the image window displayed according to the predicted layout information is adapted to the age of the user, and further improving the user experience.

[0097] In some embodiments of the present application, an age recognition model for age recognition may also be set in the terminal. The age recognition model may be a model constructed by a convolutional neural network, a fully connected neural network, etc. After the image acquisition component in the terminal captures the face image of the user, the age of the user is predicted based on the face image, and the predicted age is input into the neural network model. Thus, there is no need for the user to input the age, and the terminal automatically recognizes the age of the user, and the neural network model automatically predicts the layout information. In some other embodiments of the present application, the layout information includes window position parameters; step 220 further includes: determining the relative position information between the image acquisition component and the reference display area according to the display area information and the position information of the image acquisition component relative to the display screen in the terminal; obtaining the window position parameter corresponding to the relative position information. The relative position information between the image acquisition component and the reference display area is used to indicate the relative position between the image acquisition component and the reference display area, such as relative orientation, relative distance, etc.

[0098] In some embodiments of the present application, a mapping relationship is established in advance between the relative position information between the image acquisition component and the reference display area and the window position parameters of the adapted image window. Therefore, after determining a relative position information, based on this mapping relationship, the window position parameter corresponding to this relative position information is determined.

[0099] By establishing the mapping relationship between the relative position information between the image acquisition component and the reference display area and the window position parameters of the adapted image window, the adaptability between the position of the image window and the position of the image acquisition component can be defined. For example, in a large display screen, if the camera is located lower, it can be defined that the image window is displayed in the lower area of the display screen, facilitating the user's operation on the display screen; similarly, if the camera is installed on the left side of the display screen, the position of the image window can be close to the left side of the display screen, attracting the user's attention to the left, so that the face image captured by the camera is the user's frontal face image.

[0100] In some embodiments of the present application, the data of the sample terminal can be collected, such as the data in each training sample in the above embodiments, and the mapping relationship between the relative position information between the image acquisition component and the reference display area and the window position parameters of the image window can be determined based on the data in the training sample.

[0101] Figure 6 It is a display schematic diagram of the image window shown according to a specific embodiment of the present application. As Figure 6 shown, the full-screen display area 112 in the display screen is evenly divided into 9 sub-screens, including the labeled sub-screens 1-7 and the two sub-screens surrounded by sub-screens 1-4. In this embodiment, the determined target window position parameters indicate that the center point of the image window is located on the symmetry axis between sub-screen 1 and sub-screen 2 (or the symmetry axis between sub-screen 3 and sub-screen 4) and on the symmetry axis between sub-screen 1 and sub-screen 3 (or the symmetry axis between sub-screen 2 and sub-screen 4). Therefore, the layout and display of the image window are carried out according to the position indicated by the target window position parameters as Figure 6 shown.

[0102] In some embodiments of the present application, the layout information includes window size parameters; step 220 includes: determining the size parameters of the reference display area according to the display area information; determining the adjustment coefficient according to the size parameters of the reference display area; adjusting the reference ratio according to the adjustment coefficient to obtain the scaling ratio; scaling the size parameters of the reference display area according to the scaling ratio to obtain the window size parameters.

[0103] The window size parameter refers to the parameter in the layout information used to indicate the size of the image window. For example, for a rectangular image window, its window size parameter can be the parameter used to indicate the length and width of the image window. For another example, for a circular image window, its window size parameter can be the parameter used to indicate the diameter or radius of the image window.

[0104] The reference ratio refers to the ratio coefficient between the size of the initially set image window and the size of the reference display area. If the reference ratio is not adjusted, for display screens of different sizes, the size of the reference display area is scaled according to this reference ratio to obtain the size of the image window. This will make the ratio between the size of the displayed image window and the size of the display screen constant for each display screen. As a result, for a display screen with a smaller size, the size of the image window will be reduced based on the size of the display screen, resulting in a smaller size of the image window, and it may be impossible to display the full-face image of the user in the image window; for a display screen with a larger size, the window size determined according to this reference ratio may still be relatively large, resulting in an overly large image window displayed on the display screen. When the user looks at the image acquisition component, it is still not convenient for the image acquisition component to accurately capture the frontal face image of the user.

[0105] Therefore, to avoid this situation, in the solution of this embodiment, the adjustment coefficient is determined according to the size parameter of the reference display area, and then the reference ratio is adjusted through the adjustment coefficient to obtain the scaling ratio, which can ensure that for reference display areas of different sizes, the appropriate scaling ratio applicable to the reference display area is used to scale on the basis of the size of the reference display area, thereby enriching the layout flexibility and diversity of the image window. In some embodiments of the present application, to avoid the size of the image window being too large or too small, the adjustment coefficient can be set in segments. If the reference ratio is set as the ratio of the size of the image window to the size of the reference display area, the adjustment coefficient can be determined according to the following principle:

[0106] If the size parameter of the reference display area is lower than the first size threshold, the adjustment coefficient is determined according to the first mapping relationship, so that the scaling ratio obtained after adjustment according to the adjustment coefficient is greater than the reference ratio;

[0107] If the size parameter of the reference display area is not lower than the first size threshold and lower than the second size threshold, the adjustment coefficient is determined according to the second mapping relationship, so that the scaling ratio obtained after adjustment according to the adjustment coefficient is equal to the reference ratio, or ensure that the scaling ratio obtained after adjustment according to the adjustment coefficient is within the set scaling range;

[0108] If the size parameter of the reference display area is not lower than the second size threshold, the adjustment coefficient is determined according to the third mapping relationship, so that the scaling ratio obtained after adjustment according to the adjustment coefficient is less than the reference ratio; wherein, the second size threshold is greater than the first size threshold. The first mapping relationship, the second mapping relationship, and the third mapping relationship are all used to indicate the mapping relationship between the size parameter of the reference display area and the adjustment coefficient.

[0109] In a specific embodiment, taking the full-screen display area as the reference display area, it can be set that: if the size parameter of the reference display area is lower than the first size threshold, the adjustment coefficient is greater than 1, and the scaling ratio obtained after adjusting according to the adjustment coefficient is 1 or close to 1; if the size parameter of the reference display area is not lower than the first size threshold and lower than the second size threshold, the size parameter of the full-screen display area is scaled according to the scaling ratio obtained after adjusting according to the adjustment coefficient, and the obtained target window size parameter is the specified window size parameter; if the size parameter of the reference display area is not lower than the second size threshold, the adjustment coefficient is less than 1. The specified window size parameter can be set according to actual needs and is not specifically limited herein.

[0110] Figure 7 It is a display schematic diagram of an image window shown according to a specific embodiment; in this embodiment, Figure 7 A shows a display schematic diagram of the image window 113 in the full-screen display area when the size parameter of the full-screen display area 112 is lower than the first size threshold, as Figure 7 shown in A. At this time, the image window 113 basically fills the full-screen display area, so as to ensure that the image window 113 can sufficiently display the full-face image. Figure 7 B shows a display schematic diagram of the image window 113 in the full-screen display area 112 when the size parameter of the full-screen display area 112 is not lower than the first size threshold and less than the second size threshold, where the size of the image window is the size indicated by the specified window size parameter. Figure 7 C shows a display schematic diagram of the image window 113 in the full-screen display area when the size parameter of the full-screen display area 112 is not lower than the second size threshold, where the size of the image window 113 is scaled according to a scaling ratio greater than the reference ratio based on the size of the full-screen display area, so as to avoid the size of the image window being too large.

[0111] In some embodiments of the present application, the layout information further includes a layout direction parameter; step 220 further includes: obtaining the layout direction parameter from the display area information.

[0112] The layout direction parameter refers to the parameter in the layout information used to indicate the layout direction of the image window. In this embodiment, taking the layout direction parameter in the display area information as the layout parameter of the image window can ensure that the layout direction of the displayed image window is consistent with the layout direction of the reference display area.

[0113] Figure 8 It is a display schematic diagram of an image window shown according to a specific embodiment. In this embodiment, taking the full-screen display area as the reference display area, Figure 8 In A, the layout direction of the full-screen display area 112 is the landscape orientation. Correspondingly, the image window 113 is also arranged in the landscape orientation. InFigure 8 In B, the layout direction of the full-screen display area 112 is the portrait orientation. Correspondingly, the image window 113 is also arranged in the portrait orientation.

[0114] In some embodiments of the present application, as Figure 9 shown, the step of obtaining the display area information of the reference display area in the display screen in step 210 further includes:

[0115] Step 910, if the target application needs to call the image window, obtain the split-screen information of the display screen. Herein, the target application does not specifically refer to a certain application in the terminal, but generally refers to the application in the terminal that currently needs to call the image window, such as an application that needs to perform face verification, an application that needs to perform face payment, etc. The split-screen information of the display screen is used to indicate whether the display screen is split.

[0116] Step 920, determine whether the display screen is split according to the split-screen information of the display screen.

[0117] If it is determined that the display screen is split, then execute step 930, use the split-screen display area corresponding to the target application as the reference display area, and obtain the display area information of the split-screen display area corresponding to the target application. Correspondingly, in this embodiment, in step 230, display the image window in the split-screen display area corresponding to the target application according to the layout information.

[0118] In this embodiment, if the display screen is split, then use the split-screen display area of the split corresponding to the target application as the reference display area. Thus, the obtained layout information of the image window is used to indicate the layout of the image window in the split-screen display area corresponding to the target application.

[0119] In some embodiments, the display area information of the split corresponding to the target application can be determined according to the split parameters in the split-screen information and the area display information of the full-screen display area of the display screen, that is, according to the size of the full-screen display area indicated by the area display information of the full-screen display area, perform split-screen division according to the split parameters, so as to obtain the display area information of the split-screen display area used to indicate the size and position of the split-screen display area where the target application is located. In other embodiments, after the display screen is split, the size and position of each split-screen display area are correspondingly determined according to the split parameters. Therefore, if the display area information of the split-screen display area is stored in the terminal, then in step 930, the corresponding storage location in the terminal can be directly accessed to obtain the display area information of the split-screen display area corresponding to the target application.

[0120] Please continue to refer to Figure 9, if it is determined that the display screen is not split-screen, step 940 is executed to use the full-screen display area as the reference display area and obtain the display area information of the full-screen display area. Correspondingly, in step 230, the image window is displayed in the full-screen display area according to the layout information.

[0121] In this embodiment, when the display screen is in a split-screen display state, if the target application needs to call an image window, the split-screen display area corresponding to the split screen where the target application is located is used as the reference display area, so that the image window is displayed in the split-screen display area corresponding to the split screen where the target application is located. Therefore, the displayed image window does not affect the display of the content in other split screens.

[0122] Figure 10 It is a flowchart of the step of obtaining the display area information of the reference display area in the display screen in step 210 shown in another embodiment of the present application.

[0123] Compared with Figure 9 in the flowchart, after step 920, Figure 10 in the embodiment shown, if it is determined that the display screen is split-screen according to the split-screen information, step 1010 is executed to obtain the display area information of the split-screen display area corresponding to the target application. Step 1020, determine whether the size of the split-screen display area corresponding to the target application is lower than a set size threshold according to the display area information of the split-screen display area corresponding to the target application. Among them, the set threshold size can be set according to actual needs and is not specifically limited here. This set size threshold is used to indicate the minimum size of the reference display area.

[0124] If it is lower than the set threshold, step 940 is executed to use the full-screen display area as the reference display area and obtain the display area information of the full-screen display area; correspondingly, in step 230, the image window is displayed in the full-screen display area according to the layout information. If the size of the split-screen display area corresponding to the target application is lower than the set size threshold, it indicates that the split-screen display area where the target application is located is small. If the image window is displayed in this split-screen display area, the image window may be small, which is not convenient for the user to preview their own user image. Therefore, in order to ensure the image preview effect and facilitate the user's operation, in this case, the full-screen display area is used as the reference display area. In this case, there may be an image window displayed across the split screens in the full-screen display area according to the layout information.

[0125] Figure 11 It is a schematic diagram of an image window displayed across split screens according to a specific embodiment of the present application. As Figure 11 shown, the display screen includes two split screens, namely the first split screen 1110 and the second split screen 1120, and the target display area occupied by the image window 113 in the display screen spans these two split screens.

[0126] Please continue to refer to Figure 10 . If the size of the split-screen display area corresponding to the target application is not less than the set threshold, step 1030 is executed to use the split-screen display area of the split screen corresponding to the target application as the reference display area. Correspondingly, in step 230, the image window is displayed in the full-screen display area according to the layout information. In this application scenario, when the size of the split-screen display area corresponding to the target application is not less than the set threshold, it indicates that the split-screen display area corresponding to the target application meets the minimum size requirement for image window display.

[0127] Please continue to refer to Figure 10 . If it is determined according to the split-screen information that the display screen is not split, step 940 is executed to use the full-screen display area as the reference display area and obtain the display area information of the full-screen display area. Correspondingly, in step 230, the image window is displayed in the full-screen display area according to the layout information.

[0128] In the case where the size of the split-screen display area of the split screen where the target application is located is small, for example, lower than the set threshold, if this split-screen display area is used as the reference display area, it may cause the image window in this split-screen display area to be small, which is not convenient for the user to preview the user image and results in a poor user experience. In this embodiment, the reference display area is flexibly selected according to the split-screen state of the display screen and the size of the split-screen display area. When the size of the split-screen display area is lower than the set threshold, using the full-screen display area as the reference display area can effectively ensure that the size of the displayed image window is convenient for the user to preview the image.

[0129] In some examples of this application, for some terminals that do not support the split-screen display function, it can also be directly using the full-screen display area of the display screen as the reference display area. Or for terminals with the size parameter of the full-screen display area of the display screen lower than the second set threshold, regardless of whether the display screen is split or not, it can also be directly using the full-screen display area of the display screen as the reference display area. Among them, the second set threshold can be the same as the set threshold or greater than the set threshold, and no specific limitation is made here.

[0130] In some embodiments of this application, before step 230, the method further includes: obtaining the height difference information between the user to be captured and the image capture component; determining the reference position range according to the height difference information; if the position of the image window indicated by the layout information is not within the reference position range, adjusting the window position parameter in the layout information according to the reference position range.

[0131] The user to be captured refers to the user whose user image is to be captured by the image capture component, and can also be understood as the user facing the display screen currently. The height difference information between the user to be captured and the image capture component is used to indicate the height difference in the vertical direction between the user to be captured and the image capture component.

[0132] In some embodiments of the present application, the height difference between the user to be captured and the image capture component can be determined by using the feature region on the user's face image of the user to be captured as a reference. Specifically, when the user to be captured faces the display screen, the image capture component pre-captures the face image of the user to be captured, and selects a feature region from the pre-captured face image. The feature region can be the region where the eyes are located, the region where the nose is located, the top region of the head, etc. in the pre-captured face image, and no specific limitation is made here. Then, the height difference between the user to be recognized and the image capture component is estimated according to the position of the feature region in the pre-captured face image. For example, a mapping relationship between the height difference between the user and the image capture component and the position of the feature region in the pre-captured face image is pre-constructed. The mapping relationship can be constructed by referring to the height differences between a number of known users and the image capture component and the positions of the corresponding feature regions of the known users in the face images of the known users. On the basis of constructing this mapping relationship, in the application process, the height difference between the user to be recognized and the image capture component can be determined according to this mapping relationship and the position of the feature region of the user to be recognized in the pre-captured face image.

[0133] In some other embodiments of the present application, the height difference between the user to be captured and the image capture component can also be determined according to the proximity sensor and the infrared sensor in the terminal. Specifically, when the proximity sensor and the like are used to measure the first vertical height of the projection area of the pre-captured user on the display screen when the user to be captured projects along the direction perpendicular to the display screen, and then in combination with the second vertical height position of the image capture component in the display screen, the height difference between the user to be recognized and the image capture component is calculated. Of course, the above is only an exemplary example, and other methods that can calculate the height difference between the user to be captured and the image capture component can also be applied to this solution.

[0134] The reference position range is used to indicate the position range in the vertical direction of the image window that can completely display the face area. The reference position range is related to the height difference between the user and the image capture component. Therefore, the reference position range can be determined according to the mapping relationship between the height difference between the user and the image capture component and the position range of the image window that completely displays the face area. In some embodiments of the present application, the reference position ranges corresponding to different height differences can be preset in advance. Then, after determining the height difference between the user to be recognized and the image capture component, the reference position range corresponding to this height difference can be obtained.

[0135] In this embodiment, since the reference position range is used to indicate the position range of the image window that can completely display the face region in the vertical direction, therefore, adjusting the window position parameter in the layout information according to the reference position range is actually adjusting the window position parameter indicating the vertical height of the image window in the layout information according to the vertical height range indicated by the reference position range. The window position parameter indicating the vertical height of the image window in the layout information may be the coordinate of the feature point in the image window on the vertical coordinate axis, and the feature point in the image window may be the center point, edge point, vertex, point on the axis of symmetry, etc. of the image window. Among them, the feature point of the image window may be one or more, and the positions of the one or more feature points are used to determine the position of the image window.

[0136] After adjusting the window position parameter in the layout information according to the reference position range, the position of the image window indicated by the window position parameter in the layout information in the vertical direction is within the reference position range. Since the reference position range defines the position range of the image window that can completely display the face region in the vertical direction, therefore, after displaying the image window according to the layout information, the face region of the user to be recognized can be completely displayed in the displayed image window, and there will be no situation where the entire face region cannot be displayed in the image window due to the too high or too low display position of the image window.

[0137] In some embodiments of the present application, in the scenario of face payment, after the image window is displayed in the terminal according to the layout information, the user can preview their own image in the image window. During the process of looking at the image window for image preview, the image acquisition component can capture the frontal face image of the user, perform face recognition through the captured frontal face image of the user to determine the account information corresponding to the face image, and then perform payment according to the account information and the order to be paid.

[0138] In some embodiments of the present application, in the scenario of face verification login or face verification access, after the image window is displayed in the terminal according to the layout information, during the process of the user previewing the image in the image window (that is, the user looking at the image window), the frontal face image of the user can be captured, and then face recognition is performed based on the frontal face image of the user to determine whether the user is an authorized user. If so, the user is logged in through face verification or an access action is performed for the user.

[0139] Figure 12 is a flowchart of a window display method shown according to an embodiment of the present application. This method can be applied to a server, such as Figure 12 shown, and this method includes:

[0140] Step 1210: Receive the reported data, where the reported data includes the position information of the image acquisition component in the terminal relative to the display screen in the terminal and the display area information of the reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area.

[0141] Step 1220: Obtain the layout information of the image window according to the position information of the image acquisition component in the terminal relative to the display screen in the terminal and the display area information of the reference display area.

[0142] Step 1230: Return the layout information to the terminal so that the terminal displays the image window in the reference display area according to the layout information. In this embodiment, the server predicts the layout information based on the reported data sent by the terminal and sends the layout information to the terminal so that the terminal displays the image window in the reference display area of the display screen according to the layout information. In this embodiment, the server predicts the layout information instead of the terminal. Therefore, the processing pressure on the terminal and the requirements for the computing and processing performance of the terminal are reduced.

[0143] Specifically in Step 1220, the server can predict the layout information in the manner listed in the above embodiments, such as predicting the layout information through the trained neural network model, etc., which will not be elaborated here.

[0144] Figure 13 is a flowchart of a window display method shown according to a specific embodiment of the present application. As Figure 13 shown, the terminal device executes Step 1310 to send the reported data to the background service, the background service executes Step 1320 to input the device parameters into the neural network model, and then the neural network model outputs the layout information to the background service through Step 1330. The background service issues the layout information to the terminal device through Step 1340, and the terminal executes Step 1350 to perform the layout according to the layout information and display the image window in the display screen of the terminal device. The reported data includes the position information of the image acquisition component in the terminal relative to the display screen in the terminal and the display area information of the reference display area in the display screen.

[0145] The following introduces the device embodiments of the present application, which can be used to execute the methods in the above embodiments of the present application. For the details not disclosed in the device embodiments of the present application, please refer to the above method embodiments of the present application.

[0146] Figure 14 is a block diagram of a window display device shown according to an embodiment. As Figure 14As shown in the figure, the window display device includes: an acquisition module 1410, configured to acquire the position information of an image acquisition component in the terminal relative to the display screen in the terminal, and acquire the display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; a first layout information obtaining module 1420, configured to obtain the layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; and a display module 1430, configured to display the image window in the reference display area according to the layout information.

[0147] In some other embodiments of the present application, the layout information includes window position parameters; the first layout information obtaining module 1420 includes: a relative position information determining unit, configured to determine the relative position information between the image acquisition component and the reference display area according to the display area information and the position information of the image acquisition component relative to the display screen in the terminal; and a window position parameter determining unit, configured to obtain the window position parameters corresponding to the relative position information.

[0148] In some other embodiments of the present application, the layout information includes window size parameters; the first layout information obtaining module 1420 includes: a size parameter determining unit, configured to determine the size parameters of the reference display area according to the display area information; an adjustment coefficient determining unit, configured to determine an adjustment coefficient according to the size parameters of the reference display area; a scaling ratio determining unit, configured to adjust the reference ratio according to the adjustment coefficient to obtain a scaling ratio; and a target window size parameter determining unit, configured to scale the size parameters of the reference display area according to the scaling ratio to obtain the window size parameters.

[0149] In some embodiments of the present application, the layout information further includes layout direction parameters; the first layout information obtaining module 1420 further includes: a layout direction parameter obtaining module, configured to obtain the layout direction parameters from the display area information.

[0150] In some embodiments of the present application, the first layout information obtaining module 1420 includes: an input unit for inputting the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area into the neural network model; an output unit for outputting the layout information of the image window by the neural network model according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; wherein, the neural network model is determined by training the model according to a number of training samples, and the training samples include sample position information, sample display area information, and sample layout information; the sample position information indicates the position of the sample image acquisition component in the sample terminal relative to the sample display screen in the sample terminal; the sample display area information indicates the sample reference display area in the sample display screen; the sample layout information indicates the sample target display area occupied by the sample image window in the sample reference display area, and the sample target display area is adapted to the position of the sample image acquisition component and the sample reference display area.

[0151] In some embodiments of the present application, the window display device further includes: a training sample acquisition module for acquiring a number of training samples; a predicted layout information obtaining module for outputting predicted layout information by the neural network model according to the sample position information and sample area information in the training samples; an adjustment module for adjusting the weight parameters of the neural network model according to the predicted layout information and the sample layout information in the training samples until the neural network model converges.

[0152] In some embodiments of the present application, the window display device further includes: a height difference information acquisition module for acquiring the height difference information between the user to be captured and the image acquisition component; a reference position range determination module for determining the reference position range according to the height difference information; a window position parameter adjustment module for adjusting the window position parameters in the layout information according to the reference position range if the position of the image window indicated by the layout information is not within the reference position range.

[0153] In some embodiments of the present application, the acquisition module 1410 includes: a split screen information acquisition unit for acquiring the split screen information of the display screen if the target application needs to call an image window; a first determination unit for using the split screen display area corresponding to the target application as the reference display area and acquiring the display area information of the split screen display area corresponding to the target application if the split screen information indicates that the display screen is split; in this embodiment, the display module 1430 is configured to control the layout of the image window in the split screen display area corresponding to the target application according to the layout information.

[0154] In some other embodiments of the present application, the acquisition module 1410 includes: a split-screen information acquisition unit, configured to acquire split-screen information of the display screen if a target application needs to call an image window; a first display area information acquisition unit, configured to acquire display area information of the split-screen display area corresponding to the target application if the split-screen information indicates that the display screen is split; a second determination unit, configured to use the full-screen display area as a reference display area and acquire display area information of the full-screen display area if it is determined that the size of the split-screen display area corresponding to the target application is lower than a set size threshold according to the display area information of the split-screen display area corresponding to the target application; in this embodiment, the display module 1430 is configured to: display the image window in the full-screen display area according to the layout information.

[0155] In some embodiments of the present application, the acquisition module 1410 further includes: a third determination unit, configured to use the full-screen display area as a reference display area and acquire display area information of the full-screen display area if the split-screen information indicates that the display screen is not split; in this embodiment, the display module 1430 is configured to: display the image window in the full-screen display area according to the layout information.

[0156] The present application also discloses a window display device, including: a receiving module, configured to receive reported data, where the reported data includes position information of an image acquisition component in a terminal relative to a display screen in the terminal and display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; a second layout information obtaining module, configured to obtain layout information of an image window according to the position information of the image acquisition component in the terminal relative to the display screen and the display area information of the reference display area; a return module, configured to return the layout information to the terminal, so that the terminal displays the image window in the reference display area according to the layout information.

[0157] Figure 15 FIG. shows a schematic structural diagram of a computer system of an electronic device suitable for implementing embodiments of the present application.

[0158] It should be noted that Figure 15 The shown computer system 1500 of the electronic device is only an example, and should not impose any limitation on the functions and usage scopes of the embodiments of the present application.

[0159] Such as Figure 15As shown, the computer system 1500 includes a Central Processing Unit (CPU) 1501, which can perform various appropriate actions and processes according to the program stored in the Read-Only Memory (ROM) 1502 or the program loaded from the storage section 1508 into the Random Access Memory (RAM) 1503, such as executing the methods in the above embodiments. In the RAM 1503, various programs and data required for system operation are also stored. The CPU 1501, ROM 1502, and RAM 1503 are connected to each other via a bus 1504. An Input / Output (I / O) interface 1505 is also connected to the bus 1504.

[0160] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, etc.; an output section 1507 including, for example, a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc. and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed so that a computer program read from it can be installed into the storage section 1508 as needed.

[0161] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication section 1509, and / or installed from the removable medium 1511. When the computer program is executed by the Central Processing Unit (CPU) 1501, various functions defined in the system of the present application are executed.

[0162] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0164] The units involved in the embodiments of the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. In some cases, the names of these units do not constitute a limitation on the units themselves.

[0165] On the other hand, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions, and when the computer-readable storage instructions are executed by a processor, the methods in any of the above embodiments are implemented.

[0166] According to one aspect of the present application, there is also provided an electronic device, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the methods in any of the above embodiments are implemented.

[0167] According to one aspect of the embodiments of the present application, there is provided a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods in any of the above embodiments.

[0168] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0169] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented in software, or in a manner combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the methods according to the embodiments of the present application.

[0170] Other embodiments of the present application will be readily contemplated by those skilled in the art after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common general knowledge or conventional technical means in the technical field not disclosed in the present application.

[0171] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A window display method, characterized in that, comprising: obtaining the position information of an image acquisition component in a terminal relative to a display screen in the terminal, and obtaining the display area information of a reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; obtaining the layout information of an image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; displaying the image window in the reference display area according to the layout information; wherein, the obtaining the layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area includes: inputting the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area into a neural network model; outputting the layout information of the image window by the neural network model according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; wherein, the neural network model is determined by training the model according to a number of training samples, and the training samples include sample position information, sample display area information, and sample layout information; the sample position information indicates the position of a sample image acquisition component in a sample terminal relative to a sample display screen in the sample terminal; the sample display area information indicates a sample reference display area in the sample display screen; the sample layout information indicates a sample target display area occupied by a sample image window in the sample reference display area, and the sample target display area is adapted to the position of the sample image acquisition component and the sample reference display area.

2. The method according to claim 1, characterized in that, the layout information includes window position parameters.

3. The method according to claim 1, characterized in that, the layout information includes window size parameters.

4. The method according to claim 2 or 3, characterized in that, the layout information further includes layout direction parameters.

5. The method according to claim 1, characterized in that, the method further includes: obtaining a number of training samples; outputting predicted layout information by the neural network model according to the sample position information and sample area information in the training samples; adjusting the weight parameters of the neural network model according to the predicted layout information and the sample layout information in the training samples until the neural network model converges.

6. The method according to any one of claims 1 to 3 and 5, characterized in that, before displaying the image window in the reference display area according to the layout information, the method further includes: obtaining the height difference information between a user to be collected and the image acquisition component; determining a reference position range according to the height difference information; if the position of the image window indicated by the layout information is not within the reference position range, adjusting the window position parameters in the layout information according to the reference position range.

7. The method according to claim 1, characterized in that, Obtaining the display area information of the reference display area in the display screen includes: If the target application needs to call an image window, obtaining the split-screen information of the display screen; If the split-screen information indicates that the display screen is split, using the split-screen display area corresponding to the target application as the reference display area, and obtaining the display area information of the split-screen display area corresponding to the target application; Displaying the image window in the reference display area according to the layout information includes: Displaying the image window in the split-screen display area corresponding to the target application according to the layout information.

8. The method according to claim 1, wherein, Obtaining the display area information of the reference display area in the display screen includes: If the target application needs to call an image window, obtaining the split-screen information of the display screen; If the split-screen information indicates that the display screen is split, obtaining the display area information of the split-screen display area corresponding to the target application; If it is determined according to the display area information of the split-screen display area corresponding to the target application that the size of the split-screen display area corresponding to the target application is lower than the set size threshold, using the full-screen display area as the reference display area, and obtaining the display area information of the full-screen display area; Displaying the image window in the reference display area according to the layout information includes: Displaying the image window in the full-screen display area according to the layout information.

9. The method according to claim 7 or 8, wherein, After obtaining the split-screen information of the display screen, the method further includes: If the split-screen information indicates that the display screen is not split, using the full-screen display area as the reference display area, and obtaining the display area information of the full-screen display area; Displaying the image window in the reference display area according to the layout information includes: Displaying the image window in the full-screen display area according to the layout information.

10. A window display method, wherein, includes: Receiving reported data, the reported data including the position information of the image acquisition component in the terminal relative to the display screen in the terminal and the display area information of the reference display area in the display screen; The reference display area includes a full-screen display area or a split-screen display area; Obtaining the layout information of the image window according to the position information of the image acquisition component in the terminal relative to the display screen and the display area information of the reference display area; Returning the layout information to the terminal so that the terminal displays the image window in the reference display area according to the layout information; wherein, the obtaining the layout information of the image window according to the position information of the image acquisition component in the terminal relative to the display screen and the display area information of the reference display area includes: Inputting the position information of the image acquisition component in the terminal relative to the display screen and the display area information of the reference display area into a neural network model; The neural network model outputs the layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; wherein, the neural network model is determined by training the model according to a number of training samples, and the training samples include sample position information, sample display area information, and sample layout information; the sample position information indicates the position of the sample image acquisition component in the sample terminal relative to the sample display screen in the sample terminal; the sample display area information indicates the sample reference display area in the sample display screen; the sample layout information indicates the sample target display area occupied by the sample image window in the sample reference display area, and the sample target display area is adapted to the position of the sample image acquisition component and the sample reference display area.

11. A window display device, characterized in that, comprising: an acquisition module, configured to acquire the position information of the image acquisition component in the terminal relative to the display screen in the terminal, and acquire the display area information of the reference display area in the display screen; the reference display area includes a full-screen display area or a split-screen display area; a first layout information obtaining module, configured to obtain the layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; a display module, configured to display the image window in the reference display area according to the layout information; the first layout information obtaining module includes: an input unit, configured to input the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area into the neural network model; an output unit, configured to output the layout information of the image window by the neural network model according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; wherein, the neural network model is determined by training the model according to a number of training samples, and the training samples include sample position information, sample display area information, and sample layout information; the sample position information indicates the position of the sample image acquisition component in the sample terminal relative to the sample display screen in the sample terminal; the sample display area information indicates the sample reference display area in the sample display screen; the sample layout information indicates the sample target display area occupied by the sample image window in the sample reference display area, and the sample target display area is adapted to the position of the sample image acquisition component and the sample reference display area.

12. The device according to claim 11, characterized in that, the layout information includes window position parameters.

13. The device according to claim 11, characterized in that, the layout information includes window size parameters.

14. The device according to claim 12 or 13, characterized in that, the layout information further includes layout direction parameters.

15. The device according to claim 11, characterized in that, the window display device further includes: A training sample acquisition module for acquiring a number of training samples; A predicted layout information obtaining module for outputting predicted layout information by the neural network model according to the sample position information and sample area information in the training samples; An adjustment module for adjusting the weight parameters of the neural network model according to the predicted layout information and the sample layout information in the training samples until the neural network model converges.

16. The apparatus according to any one of claims 11-13 and 15, wherein, the window display device further includes: a height difference information acquisition module for acquiring height difference information between the user to be captured and the image acquisition component; a reference position range determination module for determining a reference position range according to the height difference information; a window position parameter adjustment module for adjusting the window position parameters in the layout information according to the reference position range if the position of the image window indicated by the layout information is not within the reference position range.

17. The apparatus according to claim 11, wherein, the acquisition module includes: a split screen information acquisition unit for acquiring split screen information of the display screen if a target application needs to call an image window; a first determination unit for taking the split screen display area corresponding to the target application as the reference display area and acquiring display area information of the split screen display area corresponding to the target application if the split screen information indicates that the display screen is split; Correspondingly, the display module is configured to control the layout of the image window in the split screen display area corresponding to the target application according to the layout information.

18. The apparatus according to claim 11, wherein, the acquisition module includes: a split screen information acquisition unit for acquiring split screen information of the display screen if a target application needs to call an image window; a first display area information acquisition unit for acquiring display area information of the split screen display area corresponding to the target application if the split screen information indicates that the display screen is split; a second determination unit for taking the full screen display area as the reference display area and acquiring display area information of the full screen display area if it is determined according to the display area information of the split screen display area corresponding to the target application that the size of the split screen display area corresponding to the target application is lower than a set size threshold; Correspondingly, the display module is configured to display the image window in the full screen display area according to the layout information.

19. The apparatus according to claim 17 or 18, wherein, the acquisition module further includes: a third determination unit for taking the full screen display area as the reference display area and acquiring display area information of the full screen display area if the split screen information indicates that the display screen is not split; Correspondingly, the display module is configured to display the image window in the full screen display area according to the layout information.

20. A window display device, wherein, it includes: A receiving module, configured to receive reported data, where the reported data includes position information of an image acquisition component in a terminal relative to a display screen in the terminal and display area information of a reference display area in the display screen; The reference display area includes a full-screen display area or a split-screen display area; A second layout information obtaining module, configured to obtain layout information of an image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; A returning module, configured to return the layout information to the terminal, so that the terminal displays the image window in the reference display area according to the layout information; The obtaining layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area includes: Inputting the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area into a neural network model; Outputting, by the neural network model, the layout information of the image window according to the position information of the image acquisition component relative to the display screen in the terminal and the display area information of the reference display area; Wherein, the neural network model is determined by training the model according to a plurality of training samples, and the training samples include sample position information, sample display area information, and sample layout information; the sample position information indicates the position of a sample image acquisition component in a sample terminal relative to a sample display screen in the sample terminal; the sample display area information indicates a sample reference display area in the sample display screen; the sample layout information indicates a sample target display area occupied by a sample image window in the sample reference display area, and the sample target display area is adapted to the position of the sample image acquisition component and the sample reference display area.

21. An electronic device, characterized in that, it includes: A processor; A memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1-10 is implemented.

22. A computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the method according to any one of claims 1-10 is implemented.

23. A computer program product, characterized in that, the computer program product includes computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1-10 is implemented.

Images