Display method and electronic equipment

Abstract

A display method and an electronic device (100, 1100), applied to a folding screen electronic device (100, 1100) comprising a first display portion (401) and a second display portion (402), the method comprising: when it is determined that the folding screen electronic device (100, 1100) is in a semi-folded state, at a first time, determining a three-dimensional model (300) of a preset object based on an observation point of a user; a first projection image (403) of a preset object is displayed through a first display part (401), a second projection image (404) of the preset object is displayed through a second display part (402), and when it is detected that an observation point of a user moves from a first position to a second position, a third projection image (405, 501) of the preset object is displayed through the first display part (401) based on the second position. A fourth projection image (406, 502) of the preset object is displayed through a second display portion (402). According to the invention, the three-dimensional visual effect of the preset object can be presented based on the folding attribute of the folding screen electronic device (100, 1100), and the utilization rate of the folding attribute of the folding screen electronic device (100, 1100) is improved.

Description

Display method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on June 27, 2023, with application number 202310770373.6 and application name “Display Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of folding screen technology, and in particular to a display method and electronic device. Background Art

[0003] With the rapid development of electronic technology, electronic devices are being updated more frequently, and their forms are gradually diversified. Folding screen electronic devices have emerged. When the folding screen electronic device is in a folded state, its size is similar to that of a conventional electronic device with the same appearance size. When the folding screen is unfolded, the size of the display screen can be twice that of the conventional folding screen electronic device, or even more.

[0004] At present, compared with conventional electronic devices, foldable screen electronic devices, in addition to having larger display screens, urgently need to develop special functions with folding properties to reflect the unique value of foldable screen electronic devices and enhance users' unique experience of foldable screen electronic devices.

[0005] Summary of the Invention

[0006] The present application provides a display method and electronic device. When a foldable screen electronic device is in a semi-folded state, a projected image is displayed on a display screen based on the user's observation point and a three-dimensional model of a preset object. The folding properties of the foldable screen electronic device are utilized to make the projected image present a three-dimensional effect, thereby improving the utilization rate of the folding properties of the foldable screen electronic device, increasing the playability of the foldable screen electronic device, and enhancing the user's gaming experience.

[0007] In the first aspect, the present application provides a display method, which is applied to a folding screen electronic device including a first display part and a second display part, including: determining that the folding screen electronic device is in a semi-folded state, and the semi-folded state indicates that the angle between the first display part and the second display part is between a first threshold value and a second threshold value; at a first moment, determining that the user's observation point is at a first position, the first display part displays a first projection image of a preset object, and the second display part displays a second projection image of the preset object, wherein the first projection image and the second projection image are determined based on a three-dimensional model of the preset object and the first position; at a second moment after the first moment, determining that the user's observation point is at a second position different from the first position, the first display part displays a third projection image of the preset object, and the second display part displays a fourth projection image of the preset object, wherein the third projection image and the fourth projection image are determined based on the three-dimensional model of the preset object and the second position.

[0008] In an embodiment of the present application, a foldable electronic device with a screen in a semi-folded state displays a first projected image and a second projected image of a preset object through a first display portion and a second display portion, respectively, based on the position of a user's observation point and a three-dimensional model of the preset object. Because the foldable electronic device has a certain folding angle, the first and second projected images viewed by the user from the first position present a three-dimensional effect. Furthermore, when the user's observation point moves from the first position to the second position, the foldable electronic device re-displays the projected image of the preset object based on the user's second position based on the detected change in observation point, so that the third and fourth projected images viewed by the user at the second position are different from the first and second projected images viewed at the first position. The projected images can change as the user's observation point moves, presenting different viewing effects according to the angle of the user's observation point. The method provided in the present application presents a three-dimensional visual effect of a preset object based on the folding properties of the foldable electronic device. At the same time, the image viewed by the user can change as the observation point moves, thereby improving the utilization rate of the folding properties of the foldable electronic device and enhancing the user's gaming experience.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: establishing a preset coordinate system based on the folding screen electronic device, representing the first position and the three-dimensional model of the preset object in the preset coordinate system; and determining the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object.

[0010] It should be understood that the method for determining the first projection image and the second projection image has the same principle as the method for determining the third projection image and the fourth projection image. To avoid repetition, the method is described in the embodiment of the present application by taking the determination of the first projection image and the second projection image as an example.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the first projection image is represented by the coordinates of multiple first intersections, and the second projection image is represented by the coordinates of multiple second intersections; the determining of the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object includes: determining, based on the preset coordinate system, a first plane expression of the plane where the first display part is located in the preset coordinate system, a second plane expression of the plane where the second display part is located in the preset coordinate system, and the first coordinates of the first position in the preset coordinate system; determining multiple straight line expressions consisting of the first coordinate and multiple second coordinates, the multiple second coordinates being the coordinates of all or part of the data points of the three-dimensional model of the preset object in the preset coordinate system; determining the coordinates of the multiple first intersections based on the multiple straight line expressions and the first plane expression; determining the coordinates of the multiple second intersections based on the multiple straight line expressions and the second plane expression.

[0012] It should be understood that the three-dimensional model of a preset object stored in a foldable screen electronic device refers to a data model used to describe the three-dimensional form of the preset object. Optionally, the model can be a three-dimensional model of a virtual object constructed using three-dimensional modeling software, or a three-dimensional model of a physical object obtained by capturing the outline of the physical object using a three-dimensional camera. The data of the three-dimensional model stored in the foldable screen electronic device can be point cloud data. This application does not specifically limit the method for obtaining the three-dimensional model or the data type.

[0013] It should also be understood that the preset coordinate system can be any three-dimensional coordinate system. Regardless of whether the coordinates of all or part of the data points of the three-dimensional model are based on the preset coordinate system as the reference system, they should be converted into coordinate expressions based on the preset coordinate system as the reference system during specific implementation. However, this application does not specifically limit the preset coordinate system.

[0014] In an embodiment of the present application, the first projection image is represented by the coordinates of multiple first intersection points, and the second projection image is represented by the coordinates of multiple second intersection points. When the folding screen electronic device is in a semi-folded state, the preset object seen by the user with the first coordinate position as the observation point can present a three-dimensional stereoscopic effect, which is beneficial to improving the user's gaming experience.

[0015] In combination with the first aspect, in certain implementations of the first aspect, the first projection image is represented by the coordinates and colors of multiple first intersections, and the second projection image is represented by the coordinates and colors of multiple second intersections; the method also includes: determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple first intersections; determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple second intersections.

[0016] It should be understood that the three-dimensional model of the preset object stored in the folding screen electronic device includes the data point coordinates on the three-dimensional model and the colors of each data point, and the colors of the multiple first intersections and the multiple second intersections should respectively correspond to the multiple data points on the three-dimensional model of the preset object.

[0017] Optionally, the colors of the data points included in the three-dimensional model of the preset object may be represented by three primary colors: red, green, and blue (RGB).

[0018] Optionally, the colors of multiple first intersections and multiple second intersections should correspond one-to-one to the colors of multiple data points on the three-dimensional model of the preset object, or they can be assigned preset coefficients to make the corresponding colors lighter or darker. This application does not make any specific restrictions on this.

[0019] In combination with the first aspect, in certain implementations of the first aspect, when the folding screen electronic device is in a semi-folded state, the connecting part between the first display part and the second display part forms a curved transition part, and the method further includes: at the first moment, displaying a fifth projection image through the curved transition part, and the fifth projection image is determined based on the first position and the three-dimensional model of the preset object; at the second moment, displaying a sixth projection image through the curved transition part, and the sixth projection image is determined based on the second position and the three-dimensional model of the preset object.

[0020] It should be understood that when there is a curved transition portion in the connecting portion between the first display portion and the second display portion, the above-mentioned first projection image and the third projection image can only be displayed in the first display portion, and the second projection image and the fourth projection image can only be displayed in the second display portion. This causes the curved surface of the connecting portion of the two display portions to produce a bright line because no image is displayed, affecting the user's visual experience.

[0021] The method provided in an embodiment of the present application displays a fifth projection image / sixth projection image on the curved transition portion when a curved transition portion exists at the connection between the first display portion and the second display portion of a foldable screen electronic device. At the first moment, the user sees the fifth projection image in addition to the first and second projection images based on their observation point; at the second moment, by displaying the sixth projection image on the curved transition portion, the user sees the sixth projection image in addition to the third and fourth projection images based on their observation point. The display of the fifth projection image / sixth projection image on the curved surface can reduce the bright lines generated by the curved surface at the bend of the screen and optimize the user's visual experience.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the fifth projection image is represented by the coordinates of multiple third intersections. Before displaying the fifth projection image through the curved transition part, the method further includes: determining the surface expression of the curved transition part in the preset coordinate system based on the surface radius of the curved transition part and the preset coordinate system; and determining the coordinates of the multiple third intersections based on the multiple straight line expressions and the surface expression.

[0023] It should be understood that the method for determining the coordinates of the plurality of third intersection points is similar to the method for determining the coordinates of the plurality of first intersection points described above, and will not be described in detail here.

[0024] In combination with the first aspect, in certain implementations of the first aspect, before determining the multiple straight-line expressions consisting of the first coordinate and multiple second coordinates, the method also includes: judging whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part based on the size of the first display part and the size of the second display part; determining the multiple straight-line expressions consisting of the first coordinate and multiple second coordinates includes: determining the multiple straight-line expressions consisting of the first coordinate and multiple second coordinates when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part.

[0025] It should be understood that the display range of the first display part and the display range of the second display part are fixed, but the user's observation point can be moved. As the user's observation point changes, the projected image of the three-dimensional model of the preset object may exceed the display range of the first display part and / or the display range of the second display part.

[0026] If the first projection image exceeds the display range of the first display part, and / or the second projection image exceeds the display range of the second display part, before determining the multiple straight line expressions composed of the first coordinate and multiple second coordinates, the three-dimensional model of the preset object should be reduced based on the preset coordinate system until the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part.

[0027] In combination with the first aspect, in certain implementations of the first aspect, the display range of the first display part is represented by multiple first boundary coordinates, and the display range of the second display part is represented by multiple second boundary coordinates, and the determining whether the projected image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part based on the size of the first display part and the size of the second display part includes: determining multiple first boundary coordinates of the first display part in the preset coordinate system and multiple second boundary coordinates of the second display part in the preset coordinate system based on the size of the first display part and the size of the second display part; determining a first straight line expression formed by the first coordinate and the coordinates of preset data points on the three-dimensional model of the preset object, the preset data point being at least one data point among the front, back, left, right, top and bottom vertices of the three-dimensional model in the preset coordinate system; determining the coordinates of a fourth intersection based on the first straight line expression and the first plane expression, and determining the coordinates of a fifth intersection based on the first straight line expression and the second plane expression; and determining whether the coordinates of the fourth intersection are within the display range of the first display part represented by the multiple first boundary coordinates, and whether the coordinates of the fifth intersection are within the display range of the second display part represented by the multiple second boundary coordinates.

[0028] It should be understood that when calculating the intersection of a straight line expression and a first plane expression or a second plane expression, if too many data points on the three-dimensional model of the preset object are used, it may cause a computing burden on the foldable screen electronic device. In order to reduce the computing power consumption of the foldable screen electronic device, one possible implementation method is to only calculate the fourth intersection and the fifth intersection of the first straight line expression composed of the preset data points and the first coordinates with the first plane expression and the second plane expression. Since the preset data points are at least one data point among the front, back, left, right, top and bottom vertices of the three-dimensional model, that is, the edge points of the three-dimensional model, if the fourth intersection is within the display range of the first display part and the fifth intersection is within the display range of the second display part, then the projected image of the three-dimensional model of the preset object will not exceed the display range of the first display part and will not exceed the display range of the second display part.

[0029] Optionally, the front-back, left-right, top-bottom orientations of the three-dimensional model of the preset object may be based on the user's observation point, which is not limited in this application.

[0030] The method provided in the present application pre-judges whether the projection image of the three-dimensional model of the preset object will exceed the display range of the first display part and / or the display range of the second display part based on preset data points before the first display part displays the first projection image of the preset object and the second display part displays the second projection image of the preset object. If it is judged that the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, subsequent display processing is performed based on all or part of the data points of the three-dimensional model of the preset object in the preset coordinate system, which is beneficial to saving computing power of folding screen electronic devices, reducing the calculation time of the display process, and bringing a smoother experience to users.

[0031] In combination with the first aspect, in some implementations of the first aspect, the method further includes: based on the size of the first display part and the size of the second display part, judging whether the first projection image and the second projection image exceed the display range of the first display part and / or the second display part; displaying the first projection image of the preset object through the first display part and displaying the second projection image of the preset object through the second display part includes: when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, displaying the first projection image of the preset object through the first display part and displaying the second projection image of the preset object through the second display part.

[0032] However, if the first projected image exceeds the display range of the first display part, and / or the second projected image exceeds the display range of the second display part, it is necessary to proportionally reduce the displayed first projected image and the second projected image based on the folding line of the folding screen electronic device to obtain an updated first projected image and an updated second projected image, until the updated first projected image does not exceed the display range of the first display part and the updated second projected image exceeds the display range of the second display part, then the updated first projected image is displayed through the first display part, and the updated second projected image is displayed through the second display part.

[0033] The method provided in the present application can make timely adjustments when the first projected image exceeds the display range of the first display part and / or the second projected image exceeds the display range of the second display part, so that the user ultimately sees the complete projected image, avoiding affecting the user experience due to the projected image exceeding the display range.

[0034] In combination with the first aspect, in certain implementations of the first aspect, the preset coordinate system is established with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis, and the side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180 degrees.

[0035] Compared with an arbitrarily established coordinate system, the method provided in the present application is easier to locate the boundary between the first display part and the second display part when calculating the projection image of a preset object, which can save the computing power of the folding screen electronic device.

[0036] In combination with the first aspect, in some implementations of the first aspect, when the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system.

[0037] In combination with the first aspect, in some implementations of the first aspect, when the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

[0038] In combination with the first aspect, in certain implementations of the first aspect, the user's observation point is the midpoint of the line between the user's two eyes; determining the first position includes: determining the user's left eye position and the user's right eye position through a camera; and determining the midpoint position of the left eye position and the right eye position.

[0039] In a second aspect, the present application provides a display device, comprising: a processing module and a display module, the processing module being used to: determine that the folding screen electronic device is in a semi-folded state, and at a first moment, detect a first position of a user's observation point; the display module being used to: based on the first position, display a first projection image of a preset object through the first display part, and display a second projection image of the preset object through the second display part, wherein the first projection image and the second projection image are determined based on a three-dimensional model of the preset object and the first position, and the semi-folded state indicates that the angle between the first display part and the second display part is between a first threshold value and a second threshold value; the processing module is also used to: at a second moment after the first moment, detect a second position of the user's observation point; the display module being used to: based on the second position, display a third projection image of the preset object through the first display part, and display a fourth projection image of the preset object through the second display part, wherein the third projection image and the fourth projection image are determined based on the three-dimensional model of the preset object and the second position.

[0040] Optionally, the processing module is also used to: establish a preset coordinate system based on the folding screen electronic device, and represent the first position and the three-dimensional model of the preset object in the preset coordinate system; and determine the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object.

[0041] Optionally, the processing module is also used to: determine, based on the preset coordinate system, a first plane expression of the plane where the first display part is located in the preset coordinate system, a second plane expression of the plane where the second display part is located in the preset coordinate system, and the first coordinate of the first position in the preset coordinate system; determine multiple straight line expressions consisting of the first coordinate and multiple second coordinates, where the multiple second coordinates are the coordinates of all or part of the data points of the three-dimensional model of the preset object in the preset coordinate system; determine the coordinates of the multiple first intersections based on the multiple straight line expressions and the first plane expression; and determine the coordinates of the multiple second intersections based on the multiple straight line expressions and the second plane expression.

[0042] Optionally, the processing module is also used to: determine the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple first intersections; and determine the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple second intersections.

[0043] Optionally, the display module is also used to: at the first moment, display a fifth projection image through the curved transition part, and the fifth projection image is determined based on the first position and the three-dimensional model of the preset object; at the second moment, display a sixth projection image through the curved transition part, and the sixth projection image is determined based on the second position and the three-dimensional model of the preset object.

[0044] Optionally, the processing module is also used to: determine the surface expression of the surface transition part in the preset coordinate system based on the surface radius of the surface transition part and the preset coordinate system; and determine the coordinates of the multiple third intersection points based on the multiple straight line expressions and the surface expressions.

[0045] Optionally, the processing module is also used to: determine whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part based on the size of the first display part and the size of the second display part; determine the multiple straight line expressions consisting of the first coordinate and multiple second coordinates, including: and, when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, determine the multiple straight line expressions consisting of the first coordinate and multiple second coordinates.

[0046] Optionally, the processing module is also used to: determine multiple first boundary coordinates of the first display part in the preset coordinate system, and multiple second boundary coordinates of the second display part in the preset coordinate system based on the size of the first display part and the size of the second display part; determine a first straight line expression composed of the first coordinate and the coordinates of preset data points on the three-dimensional model of the preset object, the preset data point being at least one data point among the front, back, left, right, top and bottom vertices of the three-dimensional model in the preset coordinate system; determine the coordinates of the fourth intersection based on the first straight line expression and the first plane expression, and determine the coordinates of the fifth intersection based on the first straight line expression and the second plane expression; and, determine whether the coordinates of the fourth intersection are within the display range of the first display part represented by the multiple first boundary coordinates, and whether the coordinates of the fifth intersection are within the display range of the second display part represented by the multiple second boundary coordinates.

[0047] Optionally, the processing module is also used to: determine whether the first projection image and the second projection image exceed the display range of the first display part and / or the second display part based on the size of the first display part and the size of the second display part; and, when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, display the first projection image of the preset object through the first display part and display the second projection image of the preset object through the second display part.

[0048] Optionally, the processing module is also used to establish the preset coordinate system with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis, and the side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180.

[0049] Optionally, when the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system.

[0050] Optionally, when the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

[0051] Optionally, the processing module is further configured to: determine the position of the user's left eye and the position of the user's right eye through a camera; and determine a midpoint between the left eye position and the right eye position.

[0052] In a third aspect, the present application provides another foldable screen electronic device, comprising a processor coupled to a memory and configured to execute instructions in the memory to implement the method of any possible implementation of the first aspect described above. Optionally, the foldable screen electronic device further comprises a memory. Optionally, the foldable screen electronic device further comprises a communication interface, the processor being coupled to the communication interface.

[0053] In a fourth aspect, a computer program product is provided, comprising: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute the method in any possible implementation of the first aspect.

[0054] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program (also referred to as code, or instructions) which, when run on a computer, enables the computer to execute the method in any possible implementation of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] FIG1 is a schematic structural diagram of a foldable screen electronic device provided in an embodiment of the present application;

[0056] FIG2 is a software structure block diagram of a foldable screen electronic device provided in an embodiment of the present application;

[0057] FIG3 is an example diagram of a three-dimensional model of a preset object provided in this application;

[0058] FIG4 is a diagram illustrating an example of a display interface of a display method provided by the present application;

[0059] FIG5 is a diagram illustrating an example of a display interface of another display method provided by the present application;

[0060] FIG6 is a schematic diagram of a first position provided in an embodiment of the present application;

[0061] FIG7 is a schematic diagram of the foldable screen electronic device provided in this application;

[0062] FIG8 is a schematic flowchart of a method for determining a first projection image and a second projection image provided by the present application;

[0063] FIG9 is a schematic flow chart of a display range determination method provided in an embodiment of the present application;

[0064] FIG10 is an example diagram of a preset coordinate system provided in an embodiment of the present application;

[0065] Figure 11 is a schematic block diagram of a folding screen electronic device provided in this application. DETAILED DESCRIPTION

[0066] The technical solution in this application will be described below with reference to the accompanying drawings.

[0067] To facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or execution order, and the words "first" and "second" do not necessarily mean different.

[0068] It should be noted that in the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0069] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0070] With the rapid development of electronic technology, electronic devices are being updated more frequently, and their forms are becoming increasingly diverse, leading to the emergence of foldable screen electronic devices. When folded, these devices are roughly the same size as conventional electronic devices of the same external dimensions. However, when unfolded, the display size can be twice that of conventional devices, or even greater.

[0071] At present, compared with conventional electronic devices, foldable screen electronic devices have a larger display screen, but their user experience is no different from that of conventional electronic devices. They do not fully utilize the folding properties of the foldable screen to bring more new and substantial functions to foldable screen electronic devices.

[0072] It can be seen that the utilization rate of the folding properties of foldable screen electronic devices is low, and there is an urgent need to develop special functions with folding properties to reflect the unique value of foldable screen electronic devices and enhance users' unique experience of foldable screen electronic devices.

[0073] In view of this, the present application provides a display method and electronic device, which, when the folding screen electronic device is in a semi-folded state, displays its projected images on two display parts based on the user's observation point and the three-dimensional model of the preset object, and utilizes the folding properties of the folding screen electronic device to make the projected image present a three-dimensional effect, thereby improving the utilization rate of the folding properties of the folding screen electronic device, increasing the playability of the folding screen electronic device, and enhancing the user's gaming experience.

[0074] It should be understood that the display method provided in the embodiment of the present application is applied to a folding screen electronic device including a first display part and a second display part. The above-mentioned semi-folded state refers to the angle between the first display part and the second display part being between the first threshold value and the second threshold value. The present application does not specifically limit the values ​​of the first threshold value and the second threshold value. It should be noted that in the folding screen electronic device described herein, the first display part and the second display part are two display parts divided from a complete display screen, and the folding screen electronic device can be folded and unfolded according to the intersection line of the first display part and the second display part. The first display part and the second display part can display images and videos as a whole, and can also display images and videos separately.

[0075] It should be understood that when the first threshold is 0 degrees and the second threshold is 180 degrees, the foldable screen electronic device is in a semi-folded state facing inward, and when the first threshold is 180 degrees and the second threshold is 360 degrees, the foldable screen electronic device is in a semi-folded state facing outward. Both the semi-folded state facing inward and the semi-folded state facing outward are applicable to this application. The folding form of the foldable screen electronic device presented in the drawings of the embodiments of this application does not constitute a specific limitation on the folding form of the foldable screen electronic device involved in this application.

[0076] In order to better understand the embodiments of the present application, the following describes the foldable screen electronic device involved in the embodiments of the present application. The foldable screen electronic device involved in the present application can be any electronic device with a foldable screen, such as a mobile phone or a tablet computer. The present application does not specifically limit the type of foldable screen electronic device involved.

[0077] Figure 1 is a structural schematic diagram of the folding screen electronic device 100 provided in an embodiment of the present application, but the structure shown in Figure 1 does not constitute a limitation on the folding screen electronic device involved in this application.

[0078] The foldable screen electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0079] It should be understood that the foldable screen electronic device 100 can implement display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. In the embodiment of the present application, the GPU is used to perform mathematical and geometric calculations and generate or change the displayed projected image according to the instructions of the processor 110, and display it through the display screen 194.

[0080] In an embodiment of the present application, a foldable screen electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. In one implementation of the present application, the foldable display screen of the electronic device may include a first display portion and a second display portion. When the foldable screen electronic device 100 is in a semi-folded state, the two display portions respectively display projected images of a preset object, presenting a three-dimensional visual effect to the user.

[0081] In an embodiment of the present application, the folding screen electronic device 100 can determine the user's observation point through a camera 193, a distance sensor 180F, and an application processor.

[0082] The gyroscope sensor 180B can be used to determine the motion posture of the folding screen electronic device 100. In an embodiment of the present application, the folding angle of the angle between the two display parts can be determined by the gyroscope sensor 180B.

[0083] It should be understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the foldable screen electronic device 100. In the embodiments of this application, the foldable screen electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0084] The software system of the foldable screen electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the foldable screen electronic device 100.

[0085] Figure 2 is a software structure block diagram of the folding screen electronic device 100 provided in an embodiment of the present application.

[0086] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: from top to bottom: the application layer, the application framework layer, the Android runtime layer, the hardware abstraction layer, and the Linux kernel layer.

[0087] The application layer can include a series of application packages.

[0088] The application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. As shown in Figure 2, in the embodiment of the present application, the application layer includes at least camera, gallery, etc.

[0089] The application framework layer provides an application programming interface (API) and programming framework for the application layer's applications. The application framework layer includes some predefined functions.

[0090] As shown in FIG. 2 , the application framework layer may include a window manager, a resource manager, a notification manager, and the like.

[0091] A window manager is used to manage window programs. It can obtain the display screen size, determine whether a status bar is present, lock the screen, take screenshots, and more. In an embodiment of the present application, the window manager can be used to obtain the sizes of the first and second display parts. These sizes can be used to determine whether the projected image of a 3D model of a preset object exceeds the display range of the first and / or second display parts, and then display the projected image.

[0092] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0093] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads, message reminders, and more. The Notification Manager can also display notifications in the form of icons or scrolling text in the top status bar of the system, such as notifications from applications running in the background, or in the form of dialog windows on the screen. Examples include displaying text messages in the status bar, sounding notifications, vibrating foldable electronic devices, and flashing indicator lights.

[0094] The Android runtime layer includes the C / C++ program library and the Android runtime library.

[0095] The Android runtime library includes the core library and virtual machine, which is responsible for the scheduling and management of the Android system.

[0096] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0097] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files in the application layer and application framework layer as binary files. The virtual machine is responsible for performing functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0098] The C / C++ library may include multiple functional modules, such as media libraries, a 3D graphics processing engine (e.g., OpenGL ES), a 2D graphics processing engine (e.g., SGL), and a local service module.

[0099] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. In an embodiment of the present application, the media library stores three-dimensional model data of multiple preset objects, each model corresponding to its implementation scenario. For example, the three-dimensional model of the preset object in a game scenario can be a cat, the three-dimensional model of the preset object in an off-screen display scenario can be a flying phoenix, and the three-dimensional model of the preset object in a screen wallpaper scenario can be at least one object in the wallpaper. This application does not limit this.

[0100] The 3D graphics processing engine is used to implement 3D graphics drawing, image rendering, compositing, and layer processing. In an embodiment of the present application, the 3D graphics processing library stores the program code for implementing the present application. In one possible implementation, the program code also includes a face recognition algorithm library, an eye recognition algorithm library, and the like.

[0101] The two-dimensional graphics processing engine is a drawing engine for two-dimensional drawing. In the embodiment of the present application, the two-dimensional graphics processing engine converts the projection images of the first display part and the second display part into a two-dimensional format picture.

[0102] The local service module includes system program codes and is used to provide support for system services.

[0103] The Hardware Abstraction Layer (HAL) is an interface layer between the operating system kernel and the hardware circuitry. Its purpose is to abstract the hardware. To protect the intellectual property of hardware manufacturers, it hides the platform-specific hardware interface details, providing the operating system with a virtual hardware platform, making it hardware-independent and portable across multiple platforms. From the perspective of software and hardware testing, both hardware and software testing can be performed independently within the HAL, enabling parallel testing of both software and hardware. In simple terms, hardware control actions are placed within the HAL.

[0104] In the embodiment of the present application, the hardware abstraction layer at least includes the camera driver code.

[0105] The Linux kernel layer is the layer between hardware and software. The Linux kernel layer includes at least camera drivers, memory drivers, inter-process communication drivers, display drivers, and sensor drivers.

[0106] It is worth noting that the present application can be applied to any of the scenarios described below, and the following enumeration of possible scenarios does not serve as a specific limitation on the applicable scenarios of the present application.

[0107] As one possible scenario, if a foldable electronic device provides an always on display (AOD) function and the user has enabled this function, when the foldable electronic device detects that the screen is off and in a semi-folded state, it can display a projected image of a preset object corresponding to the screen-off display based on the user's observation point. The following describes the software and hardware workflow of the foldable electronic device 100 in conjunction with the screen-off display scenario when the foldable electronic device is in a semi-folded state.

[0108] When the screen of a foldable electronic device is in a semi-folded state, if it is detected that the screen of the foldable electronic device is in an off-screen state, the application layer program corresponding to the off-screen display calls the camera driver of the Linux kernel layer to turn on the front camera of the foldable electronic device, triggering the front camera to automatically capture the current image and pass it to the human eye recognition algorithm library of the system runtime layer. The human eye recognition algorithm library determines the user's observation point based on the recognized human eye position information. At the same time, the application layer program corresponding to the off-screen display also calls the sensor driver to obtain the current screen folding angle formed by the first display portion and the second display portion of the foldable electronic device. The program code in the three-dimensional graphics processing engine constructs a three-dimensional coordinate system based on the user's observation point and the screen folding angle of the foldable electronic device, and calculates the projected image coordinates of the three-dimensional model of the preset object in this coordinate system. After the two-dimensional image processing engine converts these coordinates into a displayable image, it passes it to the application layer for display on the first display portion and the second display portion respectively. Optionally, the preset object displayed on the off-screen can be a flying phoenix, so that the phoenix appears three-dimensional from the user's perspective.

[0109] As another possible scenario, a foldable screen electronic device is installed with an application (APP), which has a startup screen when it is started. If the APP detects that the screen is in a semi-folded state when it is started, it can display a projected image corresponding to the startup screen based on the user's position, so that the user can view the startup screen with a three-dimensional effect.

[0110] In addition, a display method provided by the present application can also be applied to the game APP interface to display objects in the game, so that they present a three-dimensional effect in the user's vision.

[0111] It should be understood that in the implementation scenario of the APP startup screen, the implementation scenario of the game APP, or any other feasible scenario, the software and hardware workflow of the folding screen electronic device 100 is similar to the above-mentioned screen-off display scenario, and will not be repeated here.

[0112] The following specific embodiments are used to describe the technical solutions of the present application in detail. The following specific embodiments can be implemented independently or in combination with each other. The same or similar concepts or processes may not be described in detail in some embodiments.

[0113] For example, the three-dimensional model 300 shown in FIG3 is used as the three-dimensional model of the preset object, and a display method provided in this application is described in combination with FIG4. The hardware structure of the folding screen electronic device shown in FIG4 can be as shown in FIG1, and the software structure can be as shown in FIG2, but this application does not make any specific limitations on this.

[0114] Figure 4, a, shows an example of the display interface of a foldable electronic device in a semi-folded state at a first moment. At this first moment, the foldable electronic device detects that the user's observation point is at a first position. Based on this first position, the foldable electronic device displays a first projected image 403 of a predetermined object on the first display portion 401 and a second projected image 404 of the predetermined object on the second display portion 402. The method for determining the user's observation point will be described in detail below.

[0115] Figure 4(b) shows an example of the display interface of a foldable electronic device in a semi-folded state at a second moment. At this second moment, the foldable electronic device detects that the user's observation point is at a second position. Based on this second position, the foldable electronic device displays a third projected image 405 of a preset object on the first display portion 401 and a fourth projected image 406 of the preset object on the second display portion 402.

[0116] First projected image 404, second projected image 403, third projected image 405, and fourth projected image 406 are all determined based on the three-dimensional model 300 of the preset object, as shown in FIG3 . Therefore, the projected images viewed by the user at different observation points represent the stereoscopic effects of the three-dimensional model 300 at the corresponding viewing angles. Alternatively, the three-dimensional model of the preset object can be a three-dimensional model of any other object, which is not specifically limited in this application.

[0117] For ease of understanding, the first projection image 403, the second projection image 404, the third projection image 405, and the fourth projection image 406 in Figure 4 only show the outlines of the projection images. In specific implementation, each projection image can be displayed as an image with color filling or shadow, and this application does not limit this.

[0118] In Figures 4a and 4b, dashed line 407 represents the folding line between the first display portion 401 and the second display portion 402. The screen of the foldable electronic device can be folded and unfolded along this folding line. Similarly, this folding line is the dividing line between the first projected image 403 and the second projected image 404, as well as the dividing line between the third projected image 405 and the fourth projected image 406.

[0119] From a (the first moment) in Figure 4 to b (the second moment) in Figure 4, the semi-folded state of the folding screen electronic device has not changed, but the user's observation point moves from the first position to the second position, and the image displayed on the first display part 401 is updated from the first projection image 403 to the third projection image 405, and the image displayed on the second display part 402 is updated from the second projection image 404 to the fourth projection image 406.

[0120] It should be understood that, in the process of the user's observation point moving relative to the folding screen electronic device, the projected images on the first display part and the second display part of the folding screen electronic device will continue to change with the change of the user's observation point. The first moment and the second moment mentioned above are merely examples of taking the observation points of two users at different positions. The change of the user's observation point can be a continuous process. The projection image can be determined at multiple moments during the change of the user's observation point. In other words, the folding screen electronic device can collect the user's observation point at a certain frequency, and display the projection image corresponding to the position based on the collected user's observation point, so that even if the user is moving relative to the folding screen electronic device, he can approximately see the continuously changing projection image with a three-dimensional effect, as if observing a three-dimensional object.

[0121] The first and second positions shown in FIG4 , and the movement route from the first position to the second position are all exemplary. The user's observation point can be any position where the user can see the display screen of the foldable screen electronic device, for example, it can also be the third position shown in FIG5 . Based on the third position, the foldable screen electronic device displays a third projection image 501 of the preset object through the first display part 401 and displays a fourth projection image 502 of the preset object through the second display part 402. The visual effect of the projection image seen by the user based on the third position is different from that at the first and second positions. The first position and the second position shown in FIG4 and the third position shown in FIG5 do not constitute a specific position limitation on the user's observation point in this application.

[0122] In an embodiment of the present application, the screen of a foldable electronic device is in a semi-folded state. Based on the position of a user's observation point and a three-dimensional model of a preset object, the first and second display portions of the foldable electronic device respectively display a first projected image and a second projected image of the preset object. Because the foldable electronic device has a certain folding angle, the first and second projected images viewed by the user from the first position present a three-dimensional effect. Furthermore, when the user's observation point moves from the first position to the second position, the foldable electronic device re-displays the projected image of the preset object based on the user's second position, based on the detected change in observation point. This causes the third and fourth projected images viewed by the user at the second position to differ from the first and second projected images viewed at the first position. The projected images can change as the user's observation point moves, presenting different viewing effects depending on the user's observation angle, as if the user were viewing a three-dimensional object. The method provided in this application presents a three-dimensional visual effect of a preset object based on the folding property of the foldable electronic device. Furthermore, the image viewed by the user can change as the observation point moves, thereby improving the utilization of the folding property of the foldable electronic device and enhancing the user's gaming experience.

[0123] As an optional embodiment, the user's observation point can be the midpoint of the line connecting the user's eyes. The following describes the method for determining the user's observation point, using a foldable screen electronic device's method for determining a first position at a first moment as an example. The method includes: determining the positions of the user's left eye and the user's right eye using the foldable screen electronic device's camera; and determining the midpoint between the left and right eye positions as the user's observation point, i.e., the first position described above. This method is described below with reference to Figure 6.

[0124] As shown in Figure 6, plane 601 is used to represent the plane where the first display part or the second display part of the folding screen electronic device is located. N is any point in plane 601, and the midpoint M between the user's left eye L and right eye R is the user's observation point. At the first moment, determining the first position is to determine the position information of point M.

[0125] In one possible implementation, the foldable screen electronic device obtains the user's portrait through a camera, identifies the user's left eye and right eye according to a preconfigured face recognition model and / or eye recognition model, and calculates the distances RL and RN between the left eye L and the right eye R and any point N in the plane 601 through the sensor, and further calculates the distance to the midline MN of the RL side in △NLR; the sensor can also be used to calculate the angle α between the plane 602 formed by the user's left eye L and right eye R and point N and the plane 601, which is also the angle between the straight line where M and N are located and the plane 601, and the position information of M includes the distance to MN and the angle α.

[0126] In a possible implementation, the first coordinate of M in the preset coordinate system can be determined based on the preset coordinate system, the distance between MN and the angle α. The preset coordinate system can be any three-dimensional coordinate system.

[0127] In one possible implementation, the folding screen electronic device may also be equipped with at least two cameras, and the first position of the user's observation point may be determined based on the distance between the at least two cameras and the user's left eye and right eye. This application does not specifically limit this determination method.

[0128] In one possible implementation, the foldable screen electronic device can determine the position of the user's observation point at a preset time interval. Optionally, the preset time interval can be as short as possible so that when the position of the user's observation point continues to change, the position can be determined in a timely manner and the projected image can be displayed based on the position in a timely manner, ensuring that the user has a continuous stereoscopic visual experience. This application does not specifically limit the value of the preset time interval.

[0129] As an optional embodiment, when the foldable screen electronic device is in a semi-folded state, the connection between the first display portion and the second display portion forms a curved transition portion. At a first moment, in addition to the display on the first and second display portions, the curved transition portion can also display a fifth projected image based on the first position and a three-dimensional model of a preset object. At a second moment, the curved transition portion can also display a sixth projected image based on the second position and a three-dimensional model of a preset object.

[0130] For example, as shown in FIG7 , KHGF represents the curved transition portion formed by the connection between the first and second display portions. The area enclosed by KH and fold line 407 is adjacent to the first display portion 401, and the area enclosed by FG ​​and fold line 407 is adjacent to the second display portion 402. At a first moment, the fifth projected image is displayed on the curved transition portion KHGF; at a second moment, the sixth projected image is displayed on the curved transition portion KHGF.

[0131] It should be understood that when there is a curved surface at the connection between the first display part and the second display part, the first and third projection images can only be displayed on the first display part, and the second and fourth projection images can only be displayed on the second display part. This causes the curved transition part connecting the two display parts to produce a bright line because no image is displayed, affecting the user's visual experience.

[0132] The method provided in an embodiment of the present application, in the case where a curved transition portion exists at the connection between the first display portion and the second display portion of a foldable screen electronic device, at a first moment, by displaying a fifth projected image at the curved transition portion, a user, based on their observation point, sees not only the first and second projected images but also the fifth projected image. At a second moment, by displaying a sixth projected image at the curved transition portion, a user, based on their observation point, sees not only the third and fourth projected images but also the sixth projected image. The display of the fifth and sixth projected images on the curved surface can reduce the bright lines generated by the curved surface at the bend of the screen, thereby optimizing the user's visual experience.

[0133] As an optional embodiment, before displaying the first projection image of the preset object through the first display part and displaying the second projection image of the preset object through the second display part, the method provided in the present application also includes: establishing a preset coordinate system based on the folding screen electronic device, representing the first position and the three-dimensional model of the preset object in the preset coordinate system; and determining the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object.

[0134] It should be understood that the preset coordinate system can be any three-dimensional coordinate system, and the coordinate representations involved in the embodiments of the present application are all calculated with the preset coordinate system as the reference system.

[0135] The method for determining the first projection image and the second projection image has the same principle as the method for determining the third projection image and the fourth projection image. To avoid redundancy, the embodiment of the present application will subsequently describe the method using the determination of the first projection image and the second projection image as an example.

[0136] Figure 8 is a schematic flowchart of a method 800 for determining a first projection image and a second projection image provided in the present application. The first projection image is represented by the coordinates of multiple first intersections, and the multiple first intersections can form the first projection image. The second projection image is represented by the coordinates of multiple second intersections, and the multiple second intersections can form the second projection image.

[0137] The method 800 includes the following steps:

[0138] S801. Based on a preset coordinate system, determine a first plane expression of a plane where the first display part is located in the preset coordinate system, a second plane expression of a plane where the second display part is located in the preset coordinate system, and a first coordinate of a first position in the preset coordinate system.

[0139] S802: Determine a plurality of straight line expressions consisting of a first coordinate and a plurality of second coordinates, where the plurality of second coordinates are coordinates of all or part of data points of a three-dimensional model of a preset object in a preset coordinate system.

[0140] S803. Determine the coordinates of multiple first intersection points based on multiple straight line expressions and the first plane expression.

[0141] S804. Determine the coordinates of multiple second intersection points based on the multiple straight line expressions and the second plane expressions.

[0142] The above S803 and S804 can be executed simultaneously, or S803 can be executed first and then S804, or S804 can be executed first and then S803. This application does not limit the execution order of S803 and S804.

[0143] It should be understood that the three-dimensional model of a preset object stored in a foldable screen electronic device refers to a data model used to describe the three-dimensional form of the preset object. Optionally, the model can be a three-dimensional model of a virtual object constructed using three-dimensional modeling software, or a three-dimensional model of a physical object obtained by capturing the outline of the physical object using a three-dimensional camera. The data of the three-dimensional model stored in the foldable screen electronic device can be point cloud data. This application does not specifically limit the method for obtaining the three-dimensional model or the data type.

[0144] It should also be understood that the preset coordinate system can be an arbitrarily established three-dimensional coordinate system. Regardless of whether the coordinates of all or part of the data points of the three-dimensional model are based on the preset coordinate system as the reference system, when implementing method 800, they should be converted into coordinate expressions based on the preset coordinate system as the reference system. However, this application does not make specific limitations on the preset coordinate system.

[0145] In an embodiment of the present application, the first projection image is represented by the coordinates of multiple first intersection points, and the second projection image is represented by the coordinates of multiple second intersection points. When the folding screen electronic device is in a semi-folded state, the preset object seen by the user with the first coordinate position as the observation point can present a three-dimensional stereoscopic effect, which is beneficial to improving the user's gaming experience.

[0146] As an optional embodiment, the above-mentioned fifth projection image can also be represented by the coordinates of multiple third intersection points. Before displaying the fifth projection image through the curved transition part, the surface expression of the curved transition part in the preset coordinate system can be determined based on the surface radius of the curved transition part and the preset coordinate system, and the coordinates of multiple third intersection points can be determined based on multiple straight line expressions and surface expressions.

[0147] It should be understood that the sixth projection image can also be represented by the coordinates of multiple intersection points. The method for determining these coordinates is similar to the method for determining the coordinates of the multiple third intersection points and will not be further described. The method for determining the coordinates of the multiple third intersection points differs from the method for determining the coordinates of the multiple first intersection points described above in the determination of the surface expression. The determination of this surface expression will be described in detail later in conjunction with FIG. 7 and will not be further explained here.

[0148] It should also be understood that the display range of the first display portion and the display range of the second display portion are fixed, but the user's observation point is movable. In one possible implementation, as the user's observation point changes, the projected image of the three-dimensional model of the preset object may appear to exceed the display range of the first display portion and / or the display range of the second display portion. Therefore, before or after displaying the projected image of the three-dimensional model of the preset object, it can be determined whether the display range of the projected image exceeds the display range of the first display portion and / or the second display portion. If so, the projected image is then displayed after appropriate adjustments are made to ensure that the preset object viewed by the user is complete; if not, the projected image is displayed directly.

[0149] Specifically, the step of performing a judgment before displaying the projection image of the three-dimensional model of the preset object can be performed before S802. After performing the judgment, the above S802 is updated to: when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, determine multiple straight line expressions consisting of the first coordinate and multiple second coordinates.

[0150] As an optional embodiment, FIG9 illustrates a method 900 for determining whether the projected image of a three-dimensional model of a preset object exceeds the display range of the first display portion and / or the display range of the second display portion based on the size of the first display portion and the size of the second display portion. The method 900 includes the following steps:

[0151] S901: Determine a plurality of first boundary coordinates of the first display part in a preset coordinate system and a plurality of second boundary coordinates of the second display part in the preset coordinate system based on the size of the first display part and the size of the second display part.

[0152] It should be understood that the size of the first display portion and the size of the second display portion can be stored in the foldable screen electronic device as screen parameters when the foldable screen electronic device is shipped. Optionally, the size of the first display portion and the size of the second display portion can be the length and width of the first display portion and the second display portion.

[0153] S902. Determine a first straight line expression formed by the first coordinate and the coordinates of a preset data point on the three-dimensional model of the preset object, where the preset data point is at least one data point among the front, back, left, right, top and bottom vertices of the three-dimensional model in the preset coordinate system.

[0154] S903. Determine the coordinates of the fourth intersection point based on the first straight line expression and the first plane expression, and determine the coordinates of the fifth intersection point based on the first straight line expression and the second plane expression.

[0155] S904: Determine whether the coordinates of the fourth intersection point are within the display range of the first display portion represented by the plurality of first boundary coordinates, and whether the coordinates of the fifth intersection point are within the display range of the second display portion represented by the plurality of second boundary coordinates.

[0156] If the judgment result of the above S904 is yes, it means that the projection image of the three-dimensional model of the preset object does not exceed the display range of the first display part and / or the display range of the second display part, and the steps S802 to S804 in the above method 800 can be continued.

[0157] If the result of the above determination in S904 is negative, it indicates that the projected image of the three-dimensional model of the preset object exceeds the display range of the first display portion or exceeds the display range of the second display portion. Therefore, before the above step S802, the three-dimensional model of the preset object should be scaled down based on the preset coordinate system until the projected image of the three-dimensional model of the preset object does not exceed the display range of the first display portion or the display range of the second display portion. Then, steps S802 to S804 of the above method 800 should be executed based on the scaled down three-dimensional model of the preset object.

[0158] It should be understood that when calculating the intersection of a straight line expression and a first plane expression or a second plane expression, if too many data points on the three-dimensional model of the preset object are used, it may cause a computing burden on the foldable screen electronic device. In order to reduce the computing power consumption of the foldable screen electronic device, one possible implementation method is to only calculate the fourth intersection and the fifth intersection of the first straight line expression composed of the preset data points and the first coordinates with the first plane expression and the second plane expression. Since the preset data points are at least one data point among the front, back, left, right, top and bottom vertices of the three-dimensional model, that is, the edge points of the three-dimensional model, if the fourth intersection is within the display range of the first display part and the fifth intersection is within the display range of the second display part, then the projected image of the three-dimensional model of the preset object will not exceed the display range of the first display part and will not exceed the display range of the second display part.

[0159] The method provided in the embodiment of the present application pre-judges whether the projection image of the three-dimensional model of the preset object will exceed the display range of the first display part and / or the display range of the second display part based on preset data points before the first display part displays the first projection image of the preset object and the second display part displays the second projection image of the preset object. If it is judged that the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, subsequent display processing is performed based on all or part of the data points of the three-dimensional model of the preset object in the preset coordinate system, which is beneficial to saving computing power of folding screen electronic devices, reducing the calculation time of the display process, and bringing a smoother experience to users.

[0160] If no judgment is made as to whether the projected image will exceed the display range before the above S802, or if no valid result is obtained after the judgment is made, then after the first projected image of the preset object is displayed through the first display part and the second projected image of the preset object is displayed through the second display part, it is also possible to judge whether the first projected image and the second projected image exceed the display range of the first display part and / or the second display part based on the size of the first display part and the size of the second display part. If the first projected image does not exceed the display range of the first display part and the second projected image does not exceed the display range of the second display part, the first projected image of the preset object is displayed through the first display part, and the second projected image of the preset object is displayed through the second display part.

[0161] It should be understood that the judgment of whether the projected image will exceed the display range after the projected image of the three-dimensional model of the preset object is displayed is different from the above-mentioned method 900 in that: after S901, there is no need to execute S902 and the steps after S902, only whether the coordinates of the multiple first intersection points obtained in method 800 are all within the display range of the first display part represented by the multiple first boundary coordinates, and whether the coordinates of the multiple second intersection points are all within the display range of the second display part represented by the multiple second boundary coordinates. If so, it means that the first projected image does not exceed the display range of the first display part, and the second projected image does not exceed the display range of the second display part, and the currently displayed projected image does not need to be adjusted; if not, it means that the first projected image exceeds the display range of the first display part, and / or the second projected image exceeds the display range of the second display part.

[0162] If it is determined that the first projection image exceeds the display range of the first display part, and / or the second projection image exceeds the display range of the second display part, it is necessary to proportionally reduce the displayed first projection image and the second projection image based on the folding line of the folding screen electronic device to obtain an updated first projection image and an updated second projection image, until the updated first projection image does not exceed the display range of the first display part and the updated second projection image exceeds the display range of the second display part, then the updated first projection image is displayed through the first display part and the updated second projection image is displayed through the second display part.

[0163] The method provided in the present application can make timely adjustments when the first projected image exceeds the display range of the first display part and / or the second projected image exceeds the display range of the second display part, so that the user ultimately sees the complete projected image, avoiding affecting the user experience due to the projected image exceeding the display range.

[0164] As an optional embodiment, the preset coordinate system is established with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis. The side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180 degrees.

[0165] For example, the preset coordinate system may be as shown in FIG10. In the preset coordinate system shown in FIG10, point O is the origin of the preset coordinate system, one side of the first display portion 401 is the Z axis of the preset coordinate system, one side of the second display portion 402 is the Y axis of the preset coordinate system, and the folding line between the first display portion 401 and the second display portion 402 is the X axis of the preset coordinate system.

[0166] At the first moment, if the user's observation point is P, then point P is the first position, the coordinates of point P in the preset coordinate system are the first coordinates, A, B, C, and D are all points on the three-dimensional model of the preset object, the intersection of PA and the plane where the second display part 402 is located is A', the intersection of PB and the plane where the second display part 402 is located is B', the intersection of PC and the plane where the first display part 401 is located is C', and the intersection of PD and the plane where the first display part 401 is located is D'. A' and B' are included in multiple first intersections, C' and D' are included in multiple second intersections, multiple first intersections form a first projection image, and multiple second intersections form a second projection image.

[0167] As an optional embodiment, when a preset coordinate system is established in the manner shown in Figure 10, when the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system; when the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

[0168] In one possible implementation, the three-dimensional model of the preset object includes the coordinates of all data points of the preset object. The first projection image and the second projection image obtained based on the coordinates of all data points can reflect all features of the preset object, giving the user a more realistic visual experience.

[0169] In one possible implementation, the three-dimensional model of the preset object includes the coordinates of some data points of the preset object. By establishing a grid and dividing the three-dimensional model including the coordinates of all data points of the preset object, and taking each unit grid as a data point, the coordinates of some data points of the preset object can be obtained. This can save computing power of the folding screen electronic device and update the projection image of the preset object more quickly when the user's observation point moves.

[0170] As an optional embodiment, the first projection image is represented by the coordinates and colors of multiple first intersections, and the second projection image is represented by the coordinates and colors of multiple second intersections. The method 800 also includes: determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple first intersections; determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple second intersections.

[0171] It should be understood that the three-dimensional model of the preset object stored in the folding screen electronic device includes the coordinates of the data points on the three-dimensional model and the colors of each data point. The data form of any data point thereon can be expressed as V(x, y, z, (w)), wherein (x, y, z) represents the three-dimensional coordinates of the point in the preset coordinate system, and w represents the color information of the point. The colors of the multiple first intersection points and the multiple second intersection points should respectively correspond to the multiple data points on the three-dimensional model of the preset object. Referring to Figure 10, the colors of points C' and D' on the first projection image should respectively correspond to the colors of points C and D in the three-dimensional model of the preset object, and the colors of points A' and B' on the second projection image should respectively correspond to the colors of points A and B in the three-dimensional model of the preset object. The colors of points A', B', C' and D' can be obtained from the data of A, B, C and D in the three-dimensional model of the preset object.

[0172] Optionally, the colors of the data points included in the three-dimensional model of the preset object may be represented by three primary colors: red, green, and blue (RGB).

[0173] Optionally, the colors of multiple first intersections and multiple second intersections should correspond one-to-one to the colors of multiple data points on the three-dimensional model of the preset object, or they can be assigned preset coefficients to make the corresponding colors lighter or darker. This application does not make any specific restrictions on this.

[0174] Below, taking the user's observation point P, the screen folding angle of the foldable screen electronic device, the preset coordinate system, and the three-dimensional model of the preset object shown in Figure 10 as an example, the coordinate calculation process of the projection point of the three-dimensional model of the preset object on the screen of the foldable screen electronic device is described in detail. In Figure 10, A, B, C, and D are data points on the three-dimensional model of the preset object, A' and B' are data points of the image projected on the second display portion 402, and C' and D' are data points on the image projected on the first display portion 401. The coordinate calculation process of the data points A', B', C', and D' is the same. The following only uses the coordinate calculation process of C' as an example for explanation. C' is obtained by projection of C.

[0175] It should be understood that the steps of solving the coordinates of C' in the preset coordinate system are divided into three steps: ① Solve the expression of the straight line PC based on the coordinates of points P and C; ② Determine the plane expression of the plane XOZ where the first display part 401 is located; ③ Solve the coordinates of the intersection C' of the straight line PC and the plane XOZ.

[0176] As shown in Figure 10, point P is the user's observation point, and point C is the data point on the three-dimensional model of the preset object. The coordinates of these two points are known, and the expression of the straight line PC can be obtained: l = C + t (PC). This straight line expression is a point-to-point straight line equation expression, where l is any point on the straight line and t is the parameter of the straight line.

[0177] In the preset coordinate system shown in Figure 10, the plane expression of the plane XOZ where the first display part 401 is located can be expressed as: ax+by+cz=d, where a, b, and c are components of the normal vector of the plane XOZ, and d is the constant term of the equation.

[0178] Substituting the linear expression l=C+t(PC) into the plane expression ax+by+cz=d, we can obtain: a(lx)+b(ly)+c(lz)=d,

[0179] Where l is a point in vector form.

[0180] Substituting l=C+t(PC) we get: a[Cx+t(Px-Cx)]+b[Cy+t(Py-Cy)]+c[Cz+t(Pz-Cz)]=d,

[0181] Solve the equation to get the value of parameter t, and substitute the value of parameter t into the straight line expression to get:

[0182] Solving this system of equations can obtain the coordinates of the intersection point C’ of the straight-line expression and the plane expression, that is: C’(x

[0186] , y C’ , z C’ ).

[0183] For the KHFG surface shown in FIG. 7, the surface expression of this surface in the preset coordinate system can be expressed as: (x - e) 2 +(y - f) 2 +(z - g) 2 = r 2 ,

[0184] At this time, e, f, and g represent the center of the cylinder where the KHFG surface is located, r represents the radius of the circle of this cylinder, and the part of this surface in the first display part 401 should satisfy 0 < x ≤ the length of the first display part, 0 < z < half of the width of the first display part, and 0 < |y| < half of the width of the first display part.

[0185] Similarly, the straight-line expression can be substituted into the surface expression to solve the coordinates of the projection point on the surface. When solving, an iterative method such as Newton's method or the bisection method can be used to gradually approximate the root of the equation, which will not be elaborated here.

[0186] A display method has been described above in conjunction with FIGS. 1 to 10. Next, a foldable screen electronic device will be described in conjunction with FIG. 11.

[0187] FIG. 11 shows a foldable screen electronic device 1100 provided by an embodiment of the present application. The foldable screen electronic device 1100 includes a processor 1101, a memory 1102, and a display 1103. Among them, the processor 1101 and the memory 1102 communicate with each other through an internal connection path. The memory 1102 is used to store instructions, the processor 1101 is used to execute the instructions stored in the memory 1102, and the display 1103 is used to display the processing result of the processor 1102.

[0188] It should be understood that the foldable screen electronic device 1100 can be used to execute the respective steps and / or processes corresponding to the foldable screen electronic device in the above method embodiments. Optionally, the memory 1102 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A part of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 1101 may be used to execute the instructions stored in the memory, and when the processor 1101 executes the instructions stored in the memory, the processor 1101 is used to execute the respective steps and / or processes of the above method embodiments corresponding to the foldable screen electronic device.

[0189] It should be understood that in the embodiments of the present application, the processor of the foldable screen electronic device may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0190] During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor executes the instructions in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0191] The present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to implement the method shown in the above method embodiment.

[0192] The present application also provides a computer program product, which includes a computer program (also referred to as code or instructions). When the computer program runs on a computer, the computer can execute the method shown in the above method embodiment.

[0193] The above description is merely a specific embodiment of the present application, but the scope of protection of the embodiments of the present application is not limited thereto. Any person skilled in the art can easily conceive of changes or substitutions within the technical scope disclosed in the embodiments of the present application, and such changes or substitutions should be included in the scope of protection of the embodiments of the present application. Therefore, the scope of protection of the embodiments of the present application should be based on the scope of protection of the claims.

Claims

1. A display method, characterized in that: Applied to a folding screen electronic device including a first display portion and a second display portion, the method includes: Determining that the folding screen electronic device is in a semi-folded state, where the semi-folded state means that an angle between the first display part and the second display part is between a first threshold and a second threshold; At a first moment, it is determined that the user's observation point is at a first position, the first display part displays a first projection image of a preset object, and the second display part displays a second projection image of the preset object, wherein the first projection image and the second projection image are determined based on the three-dimensional model of the preset object and the first position; At a second moment after the first moment, it is determined that the user's observation point is at a second position different from the first position, the first display part displays a third projection image of the preset object, and the second display part displays a fourth projection image of the preset object, wherein the third projection image and the fourth projection image are determined based on the three-dimensional model of the preset object and the second position.

2. The method according to claim 1, characterized in that The method further comprises: Establishing a preset coordinate system based on the folding screen electronic device, and representing a three-dimensional model of the first position and the preset object in the preset coordinate system; The first projection image and the second projection image are determined based on the preset coordinate system, the first position, and the three-dimensional model of the preset object.

3. The method according to claim 2, characterized in that The first projection image is represented by the coordinates of a plurality of first intersection points, and the second projection image is represented by the coordinates of a plurality of second intersection points; The determining the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object includes: Based on the preset coordinate system, determine a first plane expression of a plane where the first display part is located in the preset coordinate system, a second plane expression of a plane where the second display part is located in the preset coordinate system, and a first coordinate of the first position in the preset coordinate system; Determine a plurality of straight line expressions formed by the first coordinate and a plurality of second coordinates, wherein the plurality of second coordinates are coordinates of all or part of the data points of the three-dimensional model of the preset object on the preset coordinate system; Determine the coordinates of the plurality of first intersection points based on the plurality of straight line expressions and the first plane expression; Based on the plurality of straight line expressions and the second plane expression, the coordinates of the plurality of second intersection points are determined.

4. The method according to claim 3, characterized in that The first projection image is represented by the coordinates and colors of a plurality of first intersection points, and the second projection image is represented by the coordinates and colors of a plurality of second intersection points; The method further comprises: Determining color information of a plurality of data points on the three-dimensional model of the preset object corresponding to the plurality of first intersection points; Color information of a plurality of data points on the three-dimensional model of the preset object corresponding to the plurality of second intersection points is determined.

5. The method according to claim 4, characterized in that When the folding screen electronic device is in a semi-folded state, a connecting portion between the first display portion and the second display portion forms a curved transition portion, and the method further includes: At the first moment, a fifth projection image is displayed through the curved transition portion, wherein the fifth projection image is determined based on the first position and the three-dimensional model of the preset object; At the second moment, a sixth projection image is displayed through the curved transition portion, where the sixth projection image is determined based on the second position and the three-dimensional model of the preset object.

6. The method according to claim 5, characterized in that The fifth projection image is represented by coordinates of a plurality of third intersection points, and before displaying the fifth projection image through the curved transition portion, the method further includes: Based on the surface radius of the curved surface transition portion and a preset coordinate system, determining a surface expression of the curved surface transition portion in the preset coordinate system; Based on the plurality of straight line expressions and the curved surface expressions, coordinates of the plurality of third intersection points are determined.

7. The method according to any one of claims 3 to 6, characterized in that Before determining a plurality of straight line expressions formed by the first coordinate and a plurality of second coordinates, the method further comprises: Based on the size of the first display part and the size of the second display part, determining whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part; The step of determining a plurality of straight line expressions formed by the first coordinate and a plurality of second coordinates comprises: When the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, a plurality of straight line expressions consisting of the first coordinate and a plurality of second coordinates are determined.

8. The method according to claim 7, characterized in that The display range of the first display part is represented by a plurality of first boundary coordinates, and the display range of the second display part is represented by a plurality of second boundary coordinates, and judging whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part based on the size of the first display part and the size of the second display part includes: Determine a plurality of first boundary coordinates of the first display portion in the preset coordinate system and a plurality of second boundary coordinates of the second display portion in the preset coordinate system based on the size of the first display portion and the size of the second display portion; Determine a first straight line expression formed by the first coordinate and the coordinates of a preset data point on the three-dimensional model of the preset object, wherein the preset data point is at least one data point among the front and back, left and right, and top and bottom vertices of the three-dimensional model in the preset coordinate system; Determine the coordinates of a fourth intersection point based on the first straight line expression and the first plane expression, and determine the coordinates of a fifth intersection point based on the first straight line expression and the second plane expression; It is determined whether the coordinates of the fourth intersection point are within the display range of the first display portion represented by the plurality of first boundary coordinates, and whether the coordinates of the fifth intersection point are within the display range of the second display portion represented by the plurality of second boundary coordinates.

9. The method according to any one of claims 3 to 6, characterized in that The method further comprises: Based on the size of the first display part and the size of the second display part, determining whether the first projection image and the second projection image exceed the display range of the first display part and / or the second display part; The method of displaying a first projection image of a preset object through the first display part and displaying a second projection image of the preset object through the second display part comprises: When the first projection image does not exceed the display range of the first display portion, and the second projection image does not exceed the display range of the first display portion, When the projection image of the preset object exceeds the display range of the second display part, the first display part displays the first projection image of the preset object, and the second display part displays the second projection image of the preset object.

10. The method according to any one of claims 2 to 9, characterized in that The preset coordinate system is established with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis. The side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180 degrees.

11. The method according to claim 10, characterized in that When the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system.

12. The method according to claim 10, characterized in that When the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

13. The method according to any one of claims 1 to 12, characterized in that The user's observation point is the midpoint of the line between the user's eyes; Determining the first position includes: Determine the left eye position and the right eye position of the user by using a camera; A midpoint position between the left eye position and the right eye position is determined.

14. A folding screen electronic device, characterized in that: include: A processor, wherein the processor is coupled to a memory, wherein the memory is used to store a computer program, and when the processor calls the computer program, the folding screen electronic device executes the method as described in any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program comprising instructions for implementing the method according to any one of claims 1 to 13.

16. A computer program product, characterized in that The computer program product includes a computer program code, and when the computer program code is executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 13.

Images