Display method and electronic equipment

Abstract

The invention provides a display method and electronic equipment, relates to the technical field of folding screens, and is applied to folding screen electronic equipment comprising a first display part and a second display part, and the method comprises the following steps: when it is determined that the folding screen electronic equipment is in a semi-folding state, determining the folding screen electronic equipment to be in a semi-folding state based on an observation point of a user and a three-dimensional model of a preset object at a first moment; a first projection image of a preset object is displayed through a first display part, a second projection image of the preset object is displayed through a second display part, and when it is detected that an observation point of a user moves from a first position to a second position, a third projection image of the preset object is displayed through the first display part based on the second position. And displaying a fourth projection image of the preset object through a second display part. According to the method provided by the invention, the three-dimensional visual effect of the preset object can be presented based on the folding attribute of the folding screen electronic equipment, and the utilization rate of the folding attribute of the folding screen electronic equipment is improved.

Description

Technical Field

[0001] The present application relates to the field of folding screen technology, and in particular to a display method and electronic device. Background Art

[0002] With the rapid development of electronic technology, electronic devices are being updated more frequently and their forms are gradually diversified. Folding screen electronic devices have emerged. When the folding screen electronic device is in a folded state, its size is similar to that of a conventional electronic device with the same appearance size. When the folding screen is unfolded, the size of the display screen can be twice that of the conventional folding screen electronic device, or even more.

[0003] At present, compared with conventional electronic devices, in addition to having a larger display screen, foldable screen electronic devices urgently need to develop special functions with folding properties to reflect the unique value of foldable screen electronic devices and enhance users' unique experience of foldable screen electronic devices. Summary of the invention

[0004] The present application provides a display method and an electronic device. When a foldable electronic device is in a semi-folded state, a projected image is displayed on a display screen based on a user's observation point and a three-dimensional model of a preset object. The folding property of the foldable electronic device is utilized to make the projected image present a three-dimensional effect, thereby improving the utilization rate of the folding property of the foldable electronic device, increasing the playability of the foldable electronic device, and enhancing the user's gaming experience.

[0005] In a first aspect, the present application provides a display method, which is applied to a folding screen electronic device including a first display part and a second display part, including: determining that the folding screen electronic device is in a semi-folded state, and the semi-folded state indicates that the angle between the first display part and the second display part is between a first threshold value and a second threshold value; at a first moment, determining that the user's observation point is in a first position, the first display part displays a first projection image of a preset object, and the second display part displays a second projection image of the preset object, wherein the first projection image and the second projection image are determined based on a three-dimensional model of the preset object and the first position; at a second moment after the first moment, determining that the user's observation point is in a second position different from the first position, the first display part displays a third projection image of the preset object, and the second display part displays a fourth projection image of the preset object, wherein the third projection image and the fourth projection image are determined based on the three-dimensional model of the preset object and the second position.

[0006] In an embodiment of the present application, a folding screen electronic device with a screen in a semi-folded state displays a first projection image and a second projection image of a preset object through a first display part and a second display part respectively based on the position of a user's observation point and a three-dimensional model of a preset object. Since the folding screen electronic device has a certain folding angle, the first projection image and the second projection image viewed by the user based on the first position present a three-dimensional effect. Moreover, when the user's observation point moves from the first position to the second position, the folding screen electronic device re-displays the projection image of the preset object based on the second position of the user based on the detected change of the observation point, so that the third projection image and the fourth projection image viewed by the user at the second position are different from the first projection image and the second projection image viewed at the first position. The projection image can change with the movement of the user's observation point, and presents different viewing effects according to the angles of different user observation points. The method provided in the present application presents a three-dimensional visual effect of a preset object based on the folding property of the folding screen electronic device. At the same time, the image viewed by the user can change with the movement of the observation point, thereby improving the utilization rate of the folding property of the folding screen electronic device and enhancing the user's gaming experience.

[0007] In combination with the first aspect, in some implementations of the first aspect, the method also includes: establishing a preset coordinate system based on the folding screen electronic device, representing the three-dimensional model of the first position and the preset object in the preset coordinate system; and determining the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object.

[0008] It should be understood that the method for determining the first projection image and the second projection image is based on the same principle as the method for determining the third projection image and the fourth projection image. To avoid repetition, the method is described by taking determining the first projection image and the second projection image as an example in the embodiment of the present application.

[0009] In combination with the first aspect, in certain implementations of the first aspect, the first projection image is represented by the coordinates of multiple first intersections, and the second projection image is represented by the coordinates of multiple second intersections; the determining of the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object includes: determining, based on the preset coordinate system, a first plane expression of the plane where the first display part is located in the preset coordinate system, a second plane expression of the plane where the second display part is located in the preset coordinate system, and a first coordinate of the first position in the preset coordinate system; determining multiple straight line expressions consisting of the first coordinates and multiple second coordinates, the multiple second coordinates being the coordinates of all or part of the data points of the three-dimensional model of the preset object on the preset coordinate system; determining the coordinates of the multiple first intersections based on the multiple straight line expressions and the first plane expressions; determining the coordinates of the multiple second intersections based on the multiple straight line expressions and the second plane expressions.

[0010] It should be understood that the three-dimensional model of the preset object stored in the foldable screen electronic device refers to a data model used to describe the three-dimensional form of the preset object. Optionally, the model can be a three-dimensional model of a virtual object constructed by three-dimensional model software, or a three-dimensional model of a physical object obtained by collecting the outline of the physical object with a three-dimensional camera. The data of the three-dimensional model stored in the foldable screen electronic device can be point cloud data. This application does not specifically limit the method for obtaining the three-dimensional model and its data type.

[0011] It should also be understood that the preset coordinate system can be any three-dimensional coordinate system. Regardless of whether the coordinates of all or part of the data points of the three-dimensional model are based on the preset coordinate system, they should be converted into coordinate expressions based on the preset coordinate system during specific implementation. However, this application does not specifically limit the preset coordinate system.

[0012] In an embodiment of the present application, the first projection image is represented by the coordinates of multiple first intersection points, and the second projection image is represented by the coordinates of multiple second intersection points. When the folding screen electronic device is in a semi-folded state, the preset object seen by the user with the first coordinate position as the observation point can present a three-dimensional stereoscopic effect, which is beneficial to improving the user's gaming experience.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the first projection image is represented by the coordinates and colors of multiple first intersections, and the second projection image is represented by the coordinates and colors of multiple second intersections; the method also includes: determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple first intersections; determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple second intersections.

[0014] It should be understood that the three-dimensional model of the preset object stored in the folding screen electronic device includes the data point coordinates on the three-dimensional model and the colors of each data point, and the colors of the multiple first intersections and the multiple second intersections should respectively correspond to the multiple data points on the three-dimensional model of the preset object.

[0015] Optionally, the colors of the data points included in the three-dimensional model of the preset object may be represented by three primary colors: red, green, and blue (RGB).

[0016] Optionally, the colors of the multiple first intersections and the multiple second intersections can correspond one-to-one to the colors of multiple data points on the three-dimensional model of the preset object, or can be assigned preset coefficients to make the corresponding colors lighter or darker. This application does not make any specific limitations on this.

[0017] In combination with the first aspect, in certain implementations of the first aspect, when the folding screen electronic device is in a semi-folded state, the connecting part between the first display part and the second display part forms a curved transition part, and the method further includes: at the first moment, displaying a fifth projection image through the curved transition part, and the fifth projection image is determined based on the first position and the three-dimensional model of the preset object; at the second moment, displaying a sixth projection image through the curved transition part, and the sixth projection image is determined based on the second position and the three-dimensional model of the preset object.

[0018] It should be understood that when there is a curved transition portion in the connection portion between the first display portion and the second display portion, the first projection image and the third projection image can only be displayed in the first display portion, and the second projection image and the fourth projection image can only be displayed in the second display portion. This causes the curved surface of the connection portion between the two display portions to produce a bright line because no image is displayed, thereby affecting the user's visual experience.

[0019] The method provided in the embodiment of the present application displays the fifth projection image / sixth projection image on the curved transition portion when there is a curved transition portion at the connection portion between the first display portion and the second display portion of the folding screen electronic device. At the first moment, the user sees the fifth projection image in addition to the first and second projection images based on his observation point; at the second moment, by displaying the sixth projection image on the curved transition portion, the user sees the sixth projection image in addition to the third and fourth projection images based on his observation point. The display of the fifth projection image / sixth projection image on the curved surface can reduce the bright lines generated by the curved surface at the bend of the screen and optimize the user's visual experience.

[0020] In combination with the first aspect, in certain implementations of the first aspect, the fifth projection image is represented by the coordinates of multiple third intersection points. Before displaying the fifth projection image through the curved transition portion, the method further includes: determining the surface expression of the curved transition portion in the preset coordinate system based on the surface radius of the curved transition portion and the preset coordinate system; and determining the coordinates of the multiple third intersection points based on the multiple straight line expressions and the surface expression.

[0021] It should be understood that the method for determining the coordinates of the plurality of third intersections is similar to the method for determining the coordinates of the plurality of first intersections described above, and will not be described in detail herein.

[0022] In combination with the first aspect, in certain implementations of the first aspect, before determining the multiple straight-line expressions consisting of the first coordinate and multiple second coordinates, the method also includes: judging whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part based on the size of the first display part and the size of the second display part; determining the multiple straight-line expressions consisting of the first coordinate and multiple second coordinates includes: determining the multiple straight-line expressions consisting of the first coordinate and multiple second coordinates when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part.

[0023] It should be understood that the display range of the first display part and the display range of the second display part are fixed, but the user's observation point is movable. As the user's observation point changes, the projection image of the three-dimensional model of the preset object may exceed the display range of the first display part and / or the display range of the second display part.

[0024] If the first projection image exceeds the display range of the first display part, and / or the second projection image exceeds the display range of the second display part, then before determining the multiple straight line expressions composed of the first coordinate and multiple second coordinates, the three-dimensional model of the preset object should be reduced based on the preset coordinate system until the first projection image does not exceed the display range of the first display part, and the second projection image does not exceed the display range of the second display part.

[0025] In combination with the first aspect, in some implementations of the first aspect, the display range of the first display part is represented by a plurality of first boundary coordinates, and the display range of the second display part is represented by a plurality of second boundary coordinates, and the determining, based on the size of the first display part and the size of the second display part, whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part comprises: determining, based on the size of the first display part and the size of the second display part, a plurality of first boundary coordinates of the first display part in the preset coordinate system, and a plurality of second boundary coordinates of the second display part in the preset coordinate system; determining a first straight line expression formed by the first coordinate and the coordinates of a preset data point on the three-dimensional model of the preset object, the preset data point being at least one data point among the front, back, left, right, top and bottom vertices of the three-dimensional model in the preset coordinate system; determining the coordinates of a fourth intersection based on the first straight line expression and the first plane expression, and determining the coordinates of a fifth intersection based on the first straight line expression and the second plane expression; determining whether the coordinates of the fourth intersection are within the display range of the first display part represented by the plurality of first boundary coordinates, and whether the coordinates of the fifth intersection are within the display range of the second display part represented by the plurality of second boundary coordinates.

[0026] It should be understood that when calculating the intersection of the straight line expression and the first plane expression or the second plane expression, if too many data points on the three-dimensional model of the preset object are used, it may cause a computing power burden on the folding screen electronic device. In order to reduce the computing power consumption of the folding screen electronic device, a possible implementation method is: only calculate the fourth intersection and the fifth intersection of the first straight line expression composed of the preset data point and the first coordinate and the first plane expression and the second plane expression. Since the preset data point is at least one data point in the front, back, left, right, top and bottom vertices of the three-dimensional model, that is, the edge point of the three-dimensional model, if the fourth intersection is within the display range of the first display part and the fifth intersection is within the display range of the second display part, the projected image of the three-dimensional model of the preset object will not exceed the display range of the first display part, and will not exceed the display range of the second display part.

[0027] Optionally, the front-back, left-right, top-bottom orientations of the three-dimensional model of the preset object may be based on the user's observation point, which is not limited in the present application.

[0028] The method provided in the present application pre-judges whether the projection image of the three-dimensional model of the preset object will exceed the display range of the first display part and / or the display range of the second display part based on preset data points before the first display part displays the first projection image of the preset object and the second display part displays the second projection image of the preset object, and if it is judged that the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, subsequent display processing is performed based on all or part of the data points of the three-dimensional model of the preset object in the preset coordinate system, which is beneficial to saving computing power of folding screen electronic devices, reducing the calculation time of the display process, and bringing a smoother experience to users.

[0029] In combination with the first aspect, in some implementations of the first aspect, the method also includes: based on the size of the first display part and the size of the second display part, judging whether the first projection image and the second projection image exceed the display range of the first display part and / or the second display part; displaying the first projection image of the preset object through the first display part and displaying the second projection image of the preset object through the second display part includes: when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, displaying the first projection image of the preset object through the first display part and displaying the second projection image of the preset object through the second display part.

[0030] However, if the first projection image exceeds the display range of the first display part, and / or the second projection image exceeds the display range of the second display part, it is necessary to proportionally reduce the displayed first projection image and the second projection image based on the folding line of the folding screen electronic device to obtain an updated first projection image and an updated second projection image, until the updated first projection image does not exceed the display range of the first display part and the updated second projection image exceeds the display range of the second display part, then the updated first projection image is displayed through the first display part, and the updated second projection image is displayed through the second display part.

[0031] The method provided in the present application can make timely adjustments when the first projection image exceeds the display range of the first display part and / or the second projection image exceeds the display range of the second display part, so that the user finally sees the complete projection image, avoiding affecting the user experience due to the projection image exceeding the display range.

[0032] In combination with the first aspect, in certain implementations of the first aspect, the preset coordinate system is established with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis, and the side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180 degrees.

[0033] Compared with an arbitrarily established coordinate system, the method provided in the present application is easier to locate the boundary between the first display part and the second display part when calculating the projection image of a preset object, and can save the computing power of the folding screen electronic device.

[0034] In combination with the first aspect, in some implementations of the first aspect, when the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system.

[0035] In combination with the first aspect, in some implementations of the first aspect, when the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

[0036] In combination with the first aspect, in certain implementations of the first aspect, the observation point of the user is the midpoint of the line between the user's eyes; determining the first position includes: determining the user's left eye position and the user's right eye position through a camera; determining the midpoint position of the left eye position and the right eye position.

[0037] In a second aspect, the present application provides a display device, comprising: a processing module and a display module, the processing module being used to: determine that the folding screen electronic device is in a semi-folded state, and at a first moment, detect a first position at which a user's observation point is located; the display module being used to: based on the first position, display a first projection image of a preset object through the first display part, and display a second projection image of the preset object through the second display part, wherein the first projection image and the second projection image are determined based on a three-dimensional model of the preset object and the first position, and the semi-folded state indicates that the angle between the first display part and the second display part is between a first threshold value and a second threshold value; the processing module is also used to: at a second moment after the first moment, detect a second position at which the user's observation point is located; the display module being used to: based on the second position, display a third projection image of the preset object through the first display part, and display a fourth projection image of the preset object through the second display part, wherein the third projection image and the fourth projection image are determined based on the three-dimensional model of the preset object and the second position.

[0038] Optionally, the processing module is also used to: establish a preset coordinate system based on the folding screen electronic device, represent the first position and the three-dimensional model of the preset object in the preset coordinate system; and determine the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object.

[0039] Optionally, the processing module is also used to: determine, based on the preset coordinate system, a first plane expression of the plane where the first display part is located in the preset coordinate system, a second plane expression of the plane where the second display part is located in the preset coordinate system, and a first coordinate of the first position in the preset coordinate system; determine a plurality of straight line expressions consisting of the first coordinate and a plurality of second coordinates, wherein the plurality of second coordinates are the coordinates of all or part of the data points of the three-dimensional model of the preset object in the preset coordinate system; determine the coordinates of the plurality of first intersections based on the plurality of straight line expressions and the first plane expression; and determine the coordinates of the plurality of second intersections based on the plurality of straight line expressions and the second plane expression.

[0040] Optionally, the processing module is also used to: determine the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple first intersections; and determine the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple second intersections.

[0041] Optionally, the display module is also used to: at the first moment, display a fifth projection image through the curved transition portion, and the fifth projection image is determined based on the first position and the three-dimensional model of the preset object; at the second moment, display a sixth projection image through the curved transition portion, and the sixth projection image is determined based on the second position and the three-dimensional model of the preset object.

[0042] Optionally, the processing module is also used to: determine the surface expression of the surface transition part in the preset coordinate system based on the surface radius of the surface transition part and the preset coordinate system; determine the coordinates of the multiple third intersection points based on the multiple straight line expressions and the surface expression.

[0043] Optionally, the processing module is also used to: determine whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part based on the size of the first display part and the size of the second display part; determine the multiple straight line expressions consisting of the first coordinate and multiple second coordinates, including: and, when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, determine the multiple straight line expressions consisting of the first coordinate and multiple second coordinates.

[0044] Optionally, the processing module is also used to: determine multiple first boundary coordinates of the first display part in the preset coordinate system, and multiple second boundary coordinates of the second display part in the preset coordinate system based on the size of the first display part and the size of the second display part; determine a first straight line expression composed of the first coordinate and the coordinates of preset data points on the three-dimensional model of the preset object, the preset data point being at least one data point among the front, back, left, right, top and bottom vertices of the three-dimensional model in the preset coordinate system; determine the coordinates of the fourth intersection based on the first straight line expression and the first plane expression, and determine the coordinates of the fifth intersection based on the first straight line expression and the second plane expression; and, determine whether the coordinates of the fourth intersection are within the display range of the first display part represented by the multiple first boundary coordinates, and whether the coordinates of the fifth intersection are within the display range of the second display part represented by the multiple second boundary coordinates.

[0045] Optionally, the processing module is also used to: determine whether the first projection image and the second projection image exceed the display range of the first display part and / or the second display part based on the size of the first display part and the size of the second display part; and, when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, display the first projection image of the preset object through the first display part, and display the second projection image of the preset object through the second display part.

[0046] Optionally, the processing module is also used to establish the preset coordinate system with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis, and the side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180.

[0047] Optionally, when the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system.

[0048] Optionally, when the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

[0049] Optionally, the processing module is further used to: determine the left eye position of the user and the right eye position of the user through a camera; and determine the midpoint position of the left eye position and the right eye position.

[0050] In a third aspect, the present application provides another folding screen electronic device, including a processor, the processor is coupled to a memory, and can be used to execute instructions in the memory to implement the method in any possible implementation of the first aspect above. Optionally, the folding screen electronic device also includes a memory. Optionally, the folding screen electronic device also includes a communication interface, and the processor is coupled to the communication interface.

[0051] In a fourth aspect, a computer program product is provided, comprising: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute a method in any possible implementation of the first aspect.

[0052] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program (also referred to as code, or instruction) which, when executed on a computer, enables the computer to execute a method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the structure of a foldable screen electronic device provided in an embodiment of the present application;

[0054] Figure 2 A software structure block diagram of a foldable screen electronic device provided in an embodiment of the present application;

[0055] Figure 3 An example diagram of a three-dimensional model of a preset object provided in this application;

[0056] Figure 4 This is an example diagram of a display interface of a display method provided in this application;

[0057] Figure 5 Another display method provided by this application is a display interface example diagram

[0058] Figure 6 A schematic diagram of a first position provided in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of the foldable screen electronic device provided in this application;

[0060] Figure 8 A schematic flowchart of a method for determining a first projection image and a second projection image provided by the present application;

[0061] Fig. 9 A schematic flow chart of a display range determination method provided in an embodiment of the present application;

[0062] Fig.10 An example diagram of a preset coordinate system provided in an embodiment of the present application;

[0063] Fig.11 A schematic block diagram of a foldable screen electronic device provided in this application. DETAILED DESCRIPTION

[0064] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0065] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0066] It should be noted that in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0067] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0068] With the rapid development of electronic technology, electronic devices are being updated more frequently, and their forms are becoming more diverse, with the emergence of foldable screen electronic devices. When a foldable screen electronic device is folded, its size is similar to that of a conventional electronic device with the same appearance size. When the foldable screen is unfolded, the size of the display screen can be twice that of the conventional electronic device, or even more.

[0069] At present, compared with conventional electronic devices, foldable screen electronic devices have a larger display screen, but their user experience is no different from that of conventional electronic devices. The folding properties of the foldable screen are not fully utilized to bring more new and substantial functions to foldable screen electronic devices.

[0070] It can be seen that the utilization rate of the folding properties of foldable screen electronic devices is low, and there is an urgent need to develop special functions with folding properties to reflect the unique value of foldable screen electronic devices and enhance users' unique experience of foldable screen electronic devices.

[0071] In view of this, the present application provides a display method and an electronic device. When the folding screen electronic device is in a semi-folded state, the projected images are displayed on two display parts based on the user's observation point and the three-dimensional model of the preset object. The folding properties of the folding screen electronic device are utilized to make the projected image present a three-dimensional effect, thereby improving the utilization rate of the folding properties of the folding screen electronic device, increasing the playability of the folding screen electronic device, and enhancing the user's gaming experience.

[0072] It should be understood that the display method provided in the embodiment of the present application is applied to a folding screen electronic device including a first display part and a second display part. The above-mentioned semi-folded state refers to the angle between the first display part and the second display part being between the first threshold value and the second threshold value. The present application does not specifically limit the values ​​of the first threshold value and the second threshold value. It should be noted that in the folding screen electronic device described herein, the first display part and the second display part are two display parts divided from a complete display screen, and the folding screen electronic device can be folded and unfolded according to the intersection line of the first display part and the second display part. The first display part and the second display part can display images and videos as a whole, and can also display images and videos separately.

[0073] It should be understood that when the first threshold is 0 degrees and the second threshold is 180 degrees, the folding screen electronic device is in a semi-folded state facing inwards, and when the first threshold is 180 degrees and the second threshold is 360 degrees, the folding screen electronic device is in a semi-folded state facing outwards. Both the semi-folded state facing inwards and the semi-folded state facing outwards of the folding screen electronic device are applicable to this application. The folding form of the folding screen electronic device presented in the drawings of the embodiments of this application does not constitute a specific limitation on the folding form of the folding screen electronic device involved in this application.

[0074] In order to better understand the embodiments of the present application, the folding screen electronic device involved in the embodiments of the present application is introduced below. The folding screen electronic device involved in the present application can be any electronic device with a foldable screen, such as a mobile phone, a tablet computer, etc., and the present application does not specifically limit the type of the folding screen electronic device involved.

[0075] Figure 1 This is a schematic diagram of the structure of the folding screen electronic device 100 provided in an embodiment of the present application, but Figure 1 The structure shown does not constitute a limitation of the folding screen electronic device involved in this application.

[0076] The folding screen electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0077] It should be understood that the folding screen electronic device 100 can realize the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. In the embodiment of the present application, the GPU is used to perform mathematical and geometric calculations, and generate or change the displayed projection image according to the instructions of the processor 110, and display it through the display screen 194.

[0078] In an embodiment of the present application, the foldable screen electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In one implementation of the present application, the foldable display screen of the electronic device may include a first display part and a second display part. When the foldable screen electronic device 100 is in a semi-folded state, the projection images of the preset objects are displayed respectively through the two display parts, which can present a three-dimensional visual effect to the user.

[0079] In an embodiment of the present application, the folding screen electronic device 100 can determine the user's observation point through a camera 193, a distance sensor 180F, an application processor, etc.

[0080] The gyro sensor 180B can be used to determine the motion posture of the folding screen electronic device 100. In the embodiment of the present application, the folding angle of the angle between the two display parts can be determined by the gyro sensor 180B.

[0081] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the folding screen electronic device 100. In the embodiment of the present application, the folding screen electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0082] The software system of the folding screen electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the folding screen electronic device 100.

[0083] Figure 2 This is a software structure block diagram of the foldable screen electronic device 100 provided in an embodiment of the present application.

[0084] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: application layer, application framework layer, Android runtime layer, hardware abstraction layer and Linux kernel layer.

[0085] The application layer may include a series of application packages.

[0086] The application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. Figure 2 As shown, in the embodiment of the present application, the application layer includes at least a camera, a gallery, etc.

[0087] The application framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. The application framework layer includes some predefined functions.

[0088] like Figure 2 As shown, the application framework layer may include a window manager, a resource manager, a notification manager, and the like.

[0089] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. In the embodiment of the present application, the window manager can be used to obtain the size of the first display part and the second display part, and the size can be used to determine whether the projected image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the second display part, and display the projected image.

[0090] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0091] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, a folding screen electronic device vibrates, an indicator light flashes, etc.

[0092] The Android runtime layer includes the C / C++ program library and the Android runtime library.

[0093] The Android runtime library includes the core library and the virtual machine, which is responsible for the scheduling and management of the Android system.

[0094] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0095] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0096] The C / C++ library may include multiple functional modules, such as media libraries, 3D graphics processing engine (such as OpenGL ES), 2D graphics processing engine (such as SGL), and local service module.

[0097] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. In an embodiment of the present application, the media library stores three-dimensional model data of multiple preset objects, each model corresponding to its implementation scene. For example, the three-dimensional model of the preset object in the game scene can be a cat, the three-dimensional model of the preset object in the off-screen display scene can be a flying phoenix, and the three-dimensional model of the preset object in the screen wallpaper scene is at least one object in the wallpaper, which is not limited by the present application.

[0098] The 3D graphics processing engine is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. In an embodiment of the present application, the 3D graphics processing library stores program codes for implementing the present application. In a possible implementation, the program code also includes a face recognition algorithm library, a human eye recognition algorithm library, etc.

[0099] The two-dimensional graphics processing engine is a drawing engine for two-dimensional drawing. In the embodiment of the present application, the two-dimensional graphics processing engine converts the projection images of the first display part and the second display part into a picture in a two-dimensional format.

[0100] The local service module includes the code of the system program and is used to provide support for the system service.

[0101] The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. In order to protect the intellectual property rights of hardware manufacturers, it hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. From the perspective of software and hardware testing, the testing of software and hardware can be completed based on the hardware abstraction layer, making it possible to conduct software and hardware testing in parallel. In layman's terms, the actions of controlling the hardware are placed in the hardware abstraction layer.

[0102] In the embodiment of the present application, the hardware abstraction layer at least includes the driver code of the camera.

[0103] The Linux kernel layer is the layer between hardware and software. The Linux kernel layer contains at least camera driver, storage driver, inter-process communication driver, display driver, and sensor driver.

[0104] It is worth noting that the present application can be applied to any scenario described below, and the enumeration of possible scenarios below does not serve as a specific limitation on the applicable scenarios of the present application.

[0105] As a possible scenario, the foldable screen electronic device provides an always on display (AOD) function and the user has enabled the function. When the foldable screen electronic device detects that the screen is off and in a semi-folded state, the projection image of the preset object corresponding to the screen off display can be displayed based on the user's observation point. The following is an example of the software and hardware workflow of the foldable screen electronic device 100 in conjunction with the screen off display scenario when the foldable screen electronic device is in a semi-folded state.

[0106] When the screen of the folding screen electronic device is in a semi-folded state, if it is detected that the screen of the folding screen electronic device is in an off-screen state, the application layer program corresponding to the off-screen display calls the camera driver of the Linux kernel layer to turn on the front camera of the folding screen electronic device, triggering the front camera to automatically capture the current picture and pass it to the human eye recognition algorithm library of the system runtime layer. The human eye recognition algorithm library determines the user's observation point based on the identified human eye position information. At the same time, the application layer program corresponding to the off-screen display also calls the sensor driver to obtain the current screen folding angle formed by the first display part and the second display part of the folding screen electronic device. The program code in the three-dimensional graphics processing engine constructs a three-dimensional coordinate system based on the user's observation point and the screen folding angle of the folding screen electronic device, and calculates the projection image coordinates of the three-dimensional model of the preset object in the coordinate system. After the coordinates are converted into a displayable picture by the two-dimensional image processing engine, they are passed to the application layer and displayed in the first display part and the second display part respectively. Optionally, the preset object of the off-screen display can be a flying phoenix, and the phoenix presents a three-dimensional effect from the user's perspective.

[0107] As another possible scenario, a foldable screen electronic device is installed with an application (APP), which has a startup screen when it is started. If the APP detects that the screen is in a semi-folded state when it is started, it can display a projection image corresponding to the startup screen based on the user's position, so that the user can view the startup screen with a three-dimensional effect.

[0108] In addition, a display method provided by the present application can also be applied to a game APP interface to display objects in the game, so that they present a three-dimensional effect in the user's vision.

[0109] It should be understood that in the implementation scenario of the APP startup screen, the implementation scenario of the game APP, or any other feasible scenario, the software and hardware workflow of the folding screen electronic device 100 is similar to the above-mentioned screen off display scenario, and will not be repeated here.

[0110] The following specific embodiments are used to describe the technical solutions of the present application in detail. The following specific embodiments can be implemented independently or in combination with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0111] For example, Figure 3 The three-dimensional model 300 shown is used as a three-dimensional model of a preset object, combined with Figure 4 A display method provided by the present application is described. Figure 4 The hardware structure of the folding screen electronic device shown in can be as follows Figure 1 As shown, the software structure can be as follows Figure 2 However, this application does not make any specific limitation to this.

[0112] Figure 4 a in the figure is an example of the display interface of the folding screen electronic device in a half-folded state at the first moment. At the first moment, when the folding screen electronic device detects that the user's observation point is at the first position, based on the first position, the first projection image 403 of the preset object is displayed through the first display part 401, and the second projection image 404 of the preset object is displayed through the second display part 402; the method for determining the user's observation point will be specifically described below.

[0113] Figure 4 b in the figure is an example diagram of the display interface of the folding screen electronic device in a semi-folded state at the second moment. At the second moment, the folding screen electronic device detects that the user's observation point is at the second position, and based on the second position, displays the third projection image 405 of the preset object through the first display part 401, and displays the fourth projection image 406 of the preset object through the second display part 402.

[0114] The first projection image 404, the second projection image 403, the third projection image 405, and the fourth projection image 406 are all based on Figure 3 The three-dimensional model 300 of the preset object shown is determined. Therefore, the content presented by the projection image seen by the user at different observation points is the stereoscopic effect of the three-dimensional model 300 at the perspective corresponding to the different observation points. Optionally, the three-dimensional model of the preset object can be a three-dimensional model of an object of any other form, which is not specifically limited in this application.

[0115] For ease of understanding, Figure 4 The first projection image 403, the second projection image 404, the third projection image 405, and the fourth projection image 406 only show the outline of the projection image. In a specific implementation, each projection image may be displayed as an image filled with color or with a shadow, which is not limited in this application.

[0116] exist Figure 4 In a and b of FIG. 4 , the dotted line 407 represents the screen folding line of the first display part 401 and the second display part 402, and the screen of the folding screen electronic device can be folded and unfolded based on the folding line. Similarly, the folding line is the dividing line between the first projection image 403 and the second projection image 404, and is also the dividing line between the third projection image 405 and the fourth projection image 406.

[0117] from Figure 4 a(first moment) to Figure 4 At b (the second moment) in the figure, the semi-folded state of the folding screen electronic device has not changed, but the user's observation point has moved from the first position to the second position, and the image displayed on the first display part 401 is updated from the first projection image 403 to the third projection image 405, and the image displayed on the second display part 402 is updated from the second projection image 404 to the fourth projection image 406.

[0118] It should be understood that in the process of the user's observation point moving relative to the folding screen electronic device, the projected images on the first display part and the second display part of the folding screen electronic device will continue to change with the change of the user's observation point. The first moment and the second moment in the above text are just examples of taking the observation points of two users at different positions. The change of the user's observation point can be a continuous process. Multiple moments can be taken during the change of the user's observation point to determine the projection image. In other words, the folding screen electronic device can collect the user's observation point at a certain frequency, and display the projection image corresponding to the position based on the collected user's observation point, so that even if the user is moving relative to the folding screen electronic device, he can approximately see the continuously changing projection image with a three-dimensional effect, as if observing a three-dimensional object.

[0119] Figure 4 The first position and the second position shown in the figure, and the moving route from the first position to the second position are all exemplary. The user's observation point can be any position where the user can see the display screen of the folding screen electronic device, for example, Figure 5 In the third position shown, the folding screen electronic device displays a third projection image 501 of the preset object through the first display part 401 based on the third position, and displays a fourth projection image 502 of the preset object through the second display part 402. The projection image seen by the user based on the third position has a visual effect different from that in the first position and the second position. Figure 4 The first position, the second position and Figure 5 The third positions shown do not constitute a specific position limitation for the user's observation point in this application.

[0120] In an embodiment of the present application, the screen of the foldable screen electronic device is in a semi-folded state. Based on the position of the user's observation point and the three-dimensional model of the preset object, the first display part and the second display part of the foldable screen electronic device respectively display the first projection image and the second projection image of the preset object. Since the foldable screen electronic device has a certain folding angle, the first projection image and the second projection image viewed by the user based on the first position present a three-dimensional effect, and when the user's observation point moves from the first position to the second position, the foldable screen electronic device re-displays the projection image of the preset object based on the second position of the user based on the detected change of the observation point, so that the third projection image and the fourth projection image viewed by the user at the second position are different from the first projection image and the second projection image viewed at the first position. The projection image can change with the movement of the user's observation point, and present different viewing effects according to the angles of different user observation points, as if viewing a three-dimensional object. The method provided in the present application presents a three-dimensional visual effect of a preset object based on the folding property of the foldable screen electronic device. At the same time, the image viewed by the user can change with the movement of the observation point, thereby improving the utilization rate of the folding property of the foldable screen electronic device and enhancing the user's gaming experience.

[0121] As an optional embodiment, the user's observation point can be the midpoint of the line between the user's eyes. The following describes the method for determining the user's observation point by taking the method for the folding screen electronic device to determine the first position at the first moment as an example. The method includes: determining the user's left eye position and the user's right eye position through the camera of the folding screen electronic device; determining the midpoint of the left eye position and the right eye position as the user's observation point, that is, the first position described above. Figure 6 To describe the method.

[0122] like Figure 6 As shown, plane 601 is used to represent the plane where the first display part or the second display part of the folding screen electronic device is located, N is any point in plane 601, the midpoint M of the user's left eye L and right eye R is the user's observation point, and at the first moment, determining the first position is to determine the position information of point M.

[0123] In one possible implementation, the foldable screen electronic device obtains the user's portrait through a camera, identifies the user's left eye and right eye according to a preconfigured face recognition model and / or eye recognition model, and calculates the distances RL and RN between the left eye L and the right eye R and any point N in the plane 601 through the sensor, and further calculates the distance to the midline MN of the RL side in △NLR; the sensor can also be used to calculate the angle α between the plane 602 formed by the user's left eye L and right eye R and point N and the plane 601, which is also the angle between the straight line where M and N are located and the plane 601, and the position information of M includes the distance of MN and the angle α.

[0124] In a possible implementation, according to the preset coordinate system, the distance between MN and the angle α, the first coordinate of M in the preset coordinate system can be determined. The preset coordinate system can be any three-dimensional coordinate system.

[0125] In one possible implementation, the folding screen electronic device may also be installed with at least two cameras, and the first position of the user's observation point may be determined based on the distance between the at least two cameras and the user's left eye and right eye. This application does not specifically limit this determination method.

[0126] In a possible implementation, the folding screen electronic device can determine the position of the user's observation point once every preset time interval. Optionally, the preset time interval can be as small as possible, so that when the position of the user's observation point continues to change, its position can be determined in time, and the projected image can be displayed based on the position in time, so as to ensure that the user has a continuous stereoscopic visual experience. This application does not specifically limit the value of the preset time interval.

[0127] As an optional embodiment, when the folding screen electronic device is in a semi-folded state, the connection portion between the first display portion and the second display portion forms a curved transition portion. At the first moment, in addition to displaying on the first display portion and the second display portion, a fifth projection image can also be displayed through the curved transition portion, and the fifth projection image is determined based on the first position and the three-dimensional model of the preset object; at the second moment, a sixth projection image can also be displayed through the curved transition portion, and the sixth projection image is determined based on the second position and the three-dimensional model of the preset object.

[0128] For example, Figure 7 As shown, KHGF represents the curved transition portion formed by the connection portion of the first display portion and the second display portion. The area enclosed by KH and the folding line 407 is adjacent to the first display portion 401, and the area enclosed by FG ​​and the folding line 407 is adjacent to the second display portion 402. At the first moment, the fifth projection image is displayed on the curved transition portion KHGF; at the second moment, the sixth projection image is displayed on the curved transition portion KHGF.

[0129] It should be understood that when there is a curved surface at the connection part between the first display part and the second display part, the first projection image and the third projection image can only be displayed on the first display part, and the second projection image and the fourth projection image can only be displayed on the second display part. This causes the curved transition part connecting the two display parts to produce a bright line because no image is displayed, thereby affecting the user's visual experience.

[0130] The method provided by the embodiment of the present application, when there is a curved transition part at the connection part between the first display part and the second display part of the folding screen electronic device, at the first moment, by displaying the fifth projection image on the curved transition part, the user sees not only the first projection image and the second projection image but also the fifth projection image based on his observation point, and at the second moment, by displaying the sixth projection image on the curved transition part, the user sees not only the third projection image and the fourth projection image but also the sixth projection image based on his observation point. The display of the fifth projection image / sixth projection image on the curved surface can reduce the bright lines generated by the curved surface at the bend of the screen and optimize the user's visual experience.

[0131] As an optional embodiment, before displaying a first projection image of a preset object through a first display part and displaying a second projection image of the preset object through a second display part, the method provided in the present application also includes: establishing a preset coordinate system based on the folding screen electronic device, representing a three-dimensional model of the first position and the preset object in the preset coordinate system; and determining the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object.

[0132] It should be understood that the preset coordinate system can be any three-dimensional coordinate system, and the coordinate representations involved in the embodiments of the present application are all calculated using the preset coordinate system as a reference system.

[0133] The method for determining the first projection image and the second projection image has the same principle as the method for determining the third projection image and the fourth projection image. To avoid repetition, the embodiment of the present application will describe the method by taking determining the first projection image and the second projection image as an example.

[0134] Figure 8 A schematic flowchart of a method 800 for determining a first projection image and a second projection image provided in the present application, wherein the first projection image is represented by the coordinates of a plurality of first intersection points, and the plurality of first intersection points can form the first projection image, and the second projection image is represented by the coordinates of a plurality of second intersection points, and the plurality of second intersection points can form the second projection image.

[0135] The method 800 comprises the following steps:

[0136] S801. Based on a preset coordinate system, determine a first plane expression of a plane where a first display part is located in the preset coordinate system, a second plane expression of a plane where a second display part is located in the preset coordinate system, and a first coordinate of a first position in the preset coordinate system.

[0137] S802. Determine a plurality of straight line expressions formed by a first coordinate and a plurality of second coordinates, where the plurality of second coordinates are coordinates of all or part of data points of a three-dimensional model of a preset object in a preset coordinate system.

[0138] S803. Determine the coordinates of multiple first intersection points based on multiple straight line expressions and the first plane expression.

[0139] S804. Determine the coordinates of multiple second intersection points based on multiple straight line expressions and second plane expressions.

[0140] The above S803 and S804 can be executed simultaneously, or S803 can be executed first and then S804, or S804 can be executed first and then S803. This application does not limit the execution order of S803 and S804.

[0141] It should be understood that the three-dimensional model of the preset object stored in the foldable screen electronic device refers to a data model used to describe the three-dimensional form of the preset object. Optionally, the model can be a three-dimensional model of a virtual object constructed by three-dimensional model software, or a three-dimensional model of a physical object obtained by collecting the outline of the physical object with a three-dimensional camera. The data of the three-dimensional model stored in the foldable screen electronic device can be point cloud data. This application does not specifically limit the method for obtaining the three-dimensional model and its data type.

[0142] It should also be understood that the preset coordinate system can be an arbitrarily established three-dimensional coordinate system. Regardless of whether the coordinates of all or part of the data points of the three-dimensional model are based on the preset coordinate system, when implementing method 800, they should be converted into coordinate expressions based on the preset coordinate system as the reference system. However, this application does not make any specific limitations on the preset coordinate system.

[0143] In an embodiment of the present application, the first projection image is represented by the coordinates of multiple first intersection points, and the second projection image is represented by the coordinates of multiple second intersection points. When the folding screen electronic device is in a semi-folded state, the preset object seen by the user with the first coordinate position as the observation point can present a three-dimensional stereoscopic effect, which is beneficial to improving the user's gaming experience.

[0144] As an optional embodiment, the above-mentioned fifth projection image can also be represented by the coordinates of multiple third intersection points. Before the fifth projection image is displayed through the curved transition part, the surface expression of the curved transition part in the preset coordinate system can be determined based on the surface radius of the curved transition part and the preset coordinate system, and the coordinates of the multiple third intersection points can be determined based on multiple straight line expressions and surface expressions.

[0145] It should be understood that the sixth projection image can also be represented by multiple intersection coordinates, and the method for determining the coordinates is similar to the method for determining the coordinates of the multiple third intersections, and will not be repeated. The difference between the method for determining the coordinates of the multiple third intersections and the method for determining the coordinates of the multiple first intersections is in determining the surface expression, which will be combined later in Figure 7 The determination of the surface expression is described in detail and will not be elaborated on here.

[0146] It should also be understood that the display range of the first display part and the display range of the second display part are fixed, but the user's observation point can be moved. In a possible implementation, as the user's observation point changes, the projection image of the three-dimensional model of the preset object may exceed the display range of the first display part and / or the display range of the second display part. Therefore, before or after displaying the projection image of the three-dimensional model of the preset object, it can be judged whether the display range of the projection image exceeds the display range of the first display part and / or the second display part. If it exceeds, the projection image is displayed after corresponding adjustments to ensure that the preset object seen by the user is complete; if it does not exceed, the projection image is directly displayed.

[0147] Specifically, the step of performing a judgment before displaying the projection image of the three-dimensional model of the preset object can be performed before S802. After performing the judgment, the above S802 is updated to: when the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, determine multiple straight line expressions consisting of the first coordinate and multiple second coordinates.

[0148] As an optional embodiment, Fig. 9 A method 900 for performing a judgment before displaying a projection image of a three-dimensional model of a preset object. The display range of a first display portion is represented by a plurality of first boundary coordinates, and the display range of a second display portion is represented by a plurality of second boundary coordinates. The method 900 is a possible method for judging whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display portion and / or the display range of the second display portion based on the size of the first display portion and the size of the second display portion. The method 900 includes the following steps:

[0149] S901. Determine a plurality of first boundary coordinates of the first display part in a preset coordinate system and a plurality of second boundary coordinates of the second display part in the preset coordinate system based on the size of the first display part and the size of the second display part.

[0150] It should be understood that the size of the first display portion and the size of the second display portion can be stored in the folding screen electronic device as screen parameters when the folding screen electronic device leaves the factory. Optionally, the size of the first display portion and the size of the second display portion can be the length and width of the first display portion and the second display portion.

[0151] S902, determining a first straight line expression formed by the first coordinate and the coordinates of a preset data point on the three-dimensional model of the preset object, where the preset data point is at least one data point among the front and back, left and right, and top and bottom vertices of the three-dimensional model in the preset coordinate system.

[0152] S903. Determine the coordinates of the fourth intersection based on the first straight line expression and the first plane expression, and determine the coordinates of the fifth intersection based on the first straight line expression and the second plane expression.

[0153] S904: Determine whether the coordinates of the fourth intersection point are within the display range of the first display portion represented by the plurality of first boundary coordinates, and whether the coordinates of the fifth intersection point are within the display range of the second display portion represented by the plurality of second boundary coordinates.

[0154] If the judgment result of the above S904 is yes, it means that the projection image of the three-dimensional model of the preset object does not exceed the display range of the first display part and / or the display range of the second display part, and the steps S802 to S804 in the above method 800 can continue to be executed.

[0155] If the result of the above S904 judgment is no, it means that the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part, or exceeds the display range of the second display part. Before the above S802, the three-dimensional model of the preset object should be reduced based on the preset coordinate system until the projection image of the three-dimensional model of the preset object does not exceed the display range of the first display part and does not exceed the display range of the second display part, and then based on the three-dimensional model of the preset object after the reduction processing, the steps S802 to S804 in the above method 800 are executed.

[0156] It should be understood that when calculating the intersection of the straight line expression and the first plane expression or the second plane expression, if too many data points on the three-dimensional model of the preset object are used, it may cause a computing power burden on the folding screen electronic device. In order to reduce the computing power consumption of the folding screen electronic device, a possible implementation method is: only calculate the fourth intersection and the fifth intersection of the first straight line expression composed of the preset data point and the first coordinate and the first plane expression and the second plane expression. Since the preset data point is at least one data point in the front, back, left, right, top and bottom vertices of the three-dimensional model, that is, the edge point of the three-dimensional model, if the fourth intersection is within the display range of the first display part and the fifth intersection is within the display range of the second display part, the projected image of the three-dimensional model of the preset object will not exceed the display range of the first display part, and will not exceed the display range of the second display part.

[0157] The method provided in the embodiment of the present application pre-judges whether the projection image of the three-dimensional model of the preset object will exceed the display range of the first display part and / or the display range of the second display part based on preset data points before the first display part displays the first projection image of the preset object and the second display part displays the second projection image of the preset object, and when it is judged that the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, subsequent display processing is performed based on all or part of the data points of the three-dimensional model of the preset object in the preset coordinate system, which is beneficial to saving computing power of folding screen electronic devices, reducing the calculation time of the display process, and bringing a smoother experience to users.

[0158] If a judgment on whether the projected image will exceed the display range is not made before the above S802, or if no valid result is obtained after the judgment is made, then after the first projected image of the preset object is displayed through the first display part and the second projected image of the preset object is displayed through the second display part, it is also possible to judge whether the first projected image and the second projected image exceed the display range of the first display part and / or the second display part based on the size of the first display part and the size of the second display part. If the first projected image does not exceed the display range of the first display part and the second projected image does not exceed the display range of the second display part, the first projected image of the preset object is displayed through the first display part, and the second projected image of the preset object is displayed through the second display part.

[0159] It should be understood that the judgment of whether the projected image will exceed the display range after the projected image of the three-dimensional model of the preset object is displayed is different from the above-mentioned method 900 in that: after S901, there is no need to execute S902 and the steps after S902, and it is only determined whether the coordinates of the multiple first intersection points obtained in method 800 are all within the display range of the first display part represented by the multiple first boundary coordinates, and whether the coordinates of the multiple second intersection points are all within the display range of the second display part represented by the multiple second boundary coordinates. If so, it means that the first projected image does not exceed the display range of the first display part, and the second projected image does not exceed the display range of the second display part, and the currently displayed projection image does not need to be adjusted; if not, it means that the first projected image exceeds the display range of the first display part, and / or the second projected image exceeds the display range of the second display part.

[0160] If it is determined that the first projection image exceeds the display range of the first display part, and / or the second projection image exceeds the display range of the second display part, it is necessary to proportionally reduce the displayed first projection image and the second projection image based on the folding line of the folding screen electronic device, and obtain an updated first projection image and an updated second projection image, until the updated first projection image does not exceed the display range of the first display part and the updated second projection image exceeds the display range of the second display part, then the updated first projection image is displayed through the first display part, and the updated second projection image is displayed through the second display part.

[0161] The method provided in the present application can make timely adjustments when the first projection image exceeds the display range of the first display part and / or the second projection image exceeds the display range of the second display part, so that the user finally sees the complete projection image, avoiding affecting the user experience due to the projection image exceeding the display range.

[0162] As an optional embodiment, a preset coordinate system is established with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis, and the side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180 degrees.

[0163] For example, the preset coordinate system can be as follows Fig.10 As shown. Fig.10 In the preset coordinate system shown, point O is the origin of the preset coordinate system, a side of the first display portion 401 is the Z axis of the preset coordinate system, a side of the second display portion 402 is the Y axis of the preset coordinate system, and the folding line between the first display portion 401 and the second display portion 402 is the X axis of the preset coordinate system.

[0164] At the first moment, if the user's observation point is P, then point P is the first position, the coordinates of point P in the preset coordinate system are the first coordinates, A, B, C, and D are all points on the three-dimensional model of the preset object, the intersection of PA and the plane where the second display part 402 is located is A', the intersection of PB and the plane where the second display part 402 is located is B', the intersection of PC and the plane where the first display part 401 is located is C', and the intersection of PD and the plane where the first display part 401 is located is D'. A' and B' are included in multiple first intersections, C' and D' are included in multiple second intersections, multiple first intersections form a first projection image, and multiple second intersections form a second projection image.

[0165] As an optional embodiment, Fig.10When the preset coordinate system is established in the manner shown, when the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system; when the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

[0166] In one possible implementation, the three-dimensional model of the preset object includes the coordinates of all data points of the preset object. The first projection image and the second projection image obtained based on the coordinates of all data points can reflect all features of the preset object, giving the user a more realistic visual experience.

[0167] In one possible implementation, the three-dimensional model of the preset object includes the coordinates of some data points of the preset object. The coordinates of some data points of the preset object can be obtained by establishing a grid and dividing the three-dimensional model including the coordinates of all data points of the preset object, with each unit grid being a data point. This can save computing power of the folding screen electronic device and update the projection image of the preset object more quickly when the user's observation point moves.

[0168] As an optional embodiment, the first projection image is represented by the coordinates and colors of multiple first intersections, and the second projection image is represented by the coordinates and colors of multiple second intersections. The method 800 also includes: determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple first intersections; determining the color information of multiple data points on the three-dimensional model of the preset object corresponding to the multiple second intersections.

[0169] It should be understood that the three-dimensional model of the preset object stored in the foldable screen electronic device includes the coordinates of the data points on the three-dimensional model and the colors of each data point. The data format of any data point on it can be expressed as V(x, y, z, (w)), where (x, y, z) represents the three-dimensional coordinates of the point in the preset coordinate system, and w represents the color information of the point. The colors of the multiple first intersections and the multiple second intersections should correspond to the multiple data points on the three-dimensional model of the preset object respectively. Fig.10 For example, the colors of points C' and D' on the first projection image should correspond to the colors of points C and D in the three-dimensional model of the preset object, respectively; the colors of points A' and B' on the second projection image should correspond to the colors of points A and B in the three-dimensional model of the preset object, respectively; and the colors of points A', B', C' and D' can be obtained from the data of A, B, C and D in the three-dimensional model of the preset object.

[0170] Optionally, the colors of the data points included in the three-dimensional model of the preset object may be represented by three primary colors: red, green, and blue (RGB).

[0171] Optionally, the colors of the multiple first intersections and the multiple second intersections can correspond one-to-one to the colors of multiple data points on the three-dimensional model of the preset object, or can be assigned preset coefficients to make the corresponding colors lighter or darker. This application does not make any specific limitations on this.

[0172] Below, Fig.10 Taking the user's observation point P, the screen folding angle of the foldable screen electronic device, the preset coordinate system, and the three-dimensional model of the preset object shown in the figure as an example, the coordinate calculation process of the projection point of the three-dimensional model of the preset object on the screen of the foldable screen electronic device is described in detail. Fig.10 A, B, C, and D are data points on the three-dimensional model of the preset object, A' and B' are data points on the projected image on the second display part 402, and C' and D' are data points on the projected image on the first display part 401. The coordinate calculation process of the data points A', B', C', and D' is the same. The following only takes the coordinate calculation process of C' as an example for explanation. C' is obtained by projection of C.

[0173] It should be understood that the steps of solving the coordinates of C' in the preset coordinate system are divided into three steps in total: ① Solve the expression of the straight line PC based on the coordinates of points P and C; ② Determine the plane expression of the plane XOZ where the first display part 401 is located; ③ Solve the coordinates of the intersection C' of the straight line PC and the plane XOZ.

[0174] like Fig.10 As shown, point P is the user's observation point, and point C is the data point on the three-dimensional model of the preset object. The coordinates of these two points are known, and the expression of the straight line PC can be obtained: l=C+t(PC). The straight line expression is a point-to-point straight line equation expression, where l is any point on the straight line and t is the parameter of the straight line.

[0175] In such Fig.10 In the preset coordinate system shown, the plane expression of the plane XOZ where the first display part 401 is located can be expressed as: ax+by+cz=d, wherein a, b, c are components of the normal vector of the plane XOZ, and d is the constant term of the equation.

[0176] Substituting the linear expression l=C+t(PC) into the plane expression ax+by+cz=d yields:

[0177] a(lx)+b(ly)+c(lz)=d,

[0178] Where l is a point in vector form.

[0179] Substituting l=C+t(PC) into the equation, we get:

[0180] a[Cx + t(Px - Cx)] + b[Cy + t(Py - Cy)] + c[Cz + t(Pz - Cz)] = d,

[0181] Solving this equation to obtain the value of parameter t, and substituting the value of parameter t into the linear expression, we get:

[0182]

[0183] Solving this system of equations can obtain the coordinates of the intersection point C’ of the linear expression and the plane expression, that is:

[0184] C’(x C’ , y C’ , z C’ ).

[0185] For the KHFG surface as shown in Figure 7 , the surface expression in the preset coordinate system can be expressed as:

[0186] (x - e) 2 + (y - f) 2 + (z - g) 2 = r 2 ,

[0187] At this time, e, f, and g represent the center of the cylinder where the KHFG surface is located, r represents the radius of the circle of the cylinder, and the part of the surface in the first display part 401 should satisfy 0 < x ≤ the length of the first display part, 0 < z < half of the width of the first display part, and 0 < |y| < half of the width of the first display part.

[0188] Similarly, the linear expression can be substituted into the surface expression to solve for the coordinates of the projection point on the surface. When solving, iterative methods such as Newton's method or the bisection method can be used to gradually approximate the root of the equation, which will not be elaborated here.

[0189] The above describes a display method in combination with Figures 1 to 10 . Next, a folding screen electronic device will be described in combination with Fig.11 .

[0190] Fig.11 This is a folding screen electronic device 1100 provided by an embodiment of the present application. The folding screen electronic device 1100 includes a processor 1101, a memory 1102, and a display 1103. Among them, the processor 1101 and the memory 1102 communicate with each other through an internal connection path. The memory 1102 is used to store instructions, the processor 1101 is used to execute the instructions stored in the memory 1102, and the display 1103 is used to display the processing result of the processor 1102.

[0191] It should be understood that the folding screen electronic device 1100 can be used to execute the various steps and / or processes corresponding to the folding screen electronic device in the above-mentioned method embodiment. Optionally, the memory 1102 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 1101 can be used to execute instructions stored in the memory, and when the processor 1101 executes instructions stored in the memory, the processor 1101 is used to execute the various steps and / or processes of the above-mentioned method embodiment corresponding to the folding screen electronic device.

[0192] It should be understood that in the embodiments of the present application, the processor of the folding screen electronic device may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0193] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor executes the instructions in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0194] The present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to implement the method shown in the above method embodiment.

[0195] The present application also provides a computer program product, which includes a computer program (also referred to as code or instruction). When the computer program runs on a computer, the computer can execute the method shown in the above method embodiment.

[0196] The above is only a specific implementation of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application, which should be included in the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be based on the protection scope of the claims.

Claims

1. A display method, characterized in that: Applied to a folding screen electronic device including a first display portion and a second display portion, the method includes: Determining that the folding screen electronic device is in a semi-folded state, where the semi-folded state means that an angle between the first display part and the second display part is between a first threshold and a second threshold; At a first moment, it is determined that the user's observation point is at a first position, the first display part displays a first projection image of a preset object, and the second display part displays a second projection image of the preset object, wherein the first projection image and the second projection image are determined based on the three-dimensional model of the preset object and the first position; At a second moment after the first moment, it is determined that the user's observation point is at a second position different from the first position, the first display part displays a third projection image of the preset object, and the second display part displays a fourth projection image of the preset object, wherein the third projection image and the fourth projection image are determined based on the three-dimensional model of the preset object and the second position.

2. The method according to claim 1, characterized in that The method further comprises: Establishing a preset coordinate system based on the folding screen electronic device, and representing a three-dimensional model of the first position and the preset object in the preset coordinate system; The first projection image and the second projection image are determined based on the preset coordinate system, the first position, and the three-dimensional model of the preset object.

3. The method according to claim 2, characterized in that The first projection image is represented by the coordinates of a plurality of first intersection points, and the second projection image is represented by the coordinates of a plurality of second intersection points; The determining the first projection image and the second projection image based on the preset coordinate system, the first position and the three-dimensional model of the preset object includes: Based on the preset coordinate system, determine a first plane expression of a plane where the first display part is located in the preset coordinate system, a second plane expression of a plane where the second display part is located in the preset coordinate system, and a first coordinate of the first position in the preset coordinate system; Determine a plurality of straight line expressions formed by the first coordinate and a plurality of second coordinates, wherein the plurality of second coordinates are coordinates of all or part of the data points of the three-dimensional model of the preset object on the preset coordinate system; Determine the coordinates of the plurality of first intersection points based on the plurality of straight line expressions and the first plane expression; Based on the plurality of straight line expressions and the second plane expression, the coordinates of the plurality of second intersection points are determined.

4. The method according to claim 3, characterized in that The first projection image is represented by the coordinates and colors of a plurality of first intersection points, and the second projection image is represented by the coordinates and colors of a plurality of second intersection points; The method further comprises: Determining color information of a plurality of data points on the three-dimensional model of the preset object corresponding to the plurality of first intersection points; Color information of a plurality of data points on the three-dimensional model of the preset object corresponding to the plurality of second intersection points is determined.

5. The method according to claim 4, characterized in that When the folding screen electronic device is in a semi-folded state, a connecting portion between the first display portion and the second display portion forms a curved transition portion, and the method further includes: At the first moment, a fifth projection image is displayed through the curved transition portion, wherein the fifth projection image is determined based on the first position and the three-dimensional model of the preset object; At the second moment, a sixth projection image is displayed through the curved transition portion, where the sixth projection image is determined based on the second position and the three-dimensional model of the preset object.

6. The method according to claim 5, characterized in that The fifth projection image is represented by coordinates of a plurality of third intersection points, and before displaying the fifth projection image through the curved transition portion, the method further includes: Based on the surface radius of the curved surface transition portion and a preset coordinate system, determining a surface expression of the curved surface transition portion in the preset coordinate system; Based on the plurality of straight line expressions and the curved surface expressions, coordinates of the plurality of third intersection points are determined.

7. The method according to any one of claims 3 to 6, characterized in that Before determining a plurality of straight line expressions formed by the first coordinate and a plurality of second coordinates, the method further includes: Based on the size of the first display part and the size of the second display part, determining whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part; The step of determining a plurality of straight line expressions formed by the first coordinate and a plurality of second coordinates comprises: When the first projection image does not exceed the display range of the first display part and the second projection image does not exceed the display range of the second display part, a plurality of straight line expressions consisting of the first coordinate and a plurality of second coordinates are determined.

8. The method according to claim 7, characterized in that The display range of the first display part is represented by a plurality of first boundary coordinates, and the display range of the second display part is represented by a plurality of second boundary coordinates, and judging whether the projection image of the three-dimensional model of the preset object exceeds the display range of the first display part and / or the display range of the second display part based on the size of the first display part and the size of the second display part includes: Determine a plurality of first boundary coordinates of the first display portion in the preset coordinate system and a plurality of second boundary coordinates of the second display portion in the preset coordinate system based on the size of the first display portion and the size of the second display portion; Determine a first straight line expression formed by the first coordinate and the coordinates of a preset data point on the three-dimensional model of the preset object, wherein the preset data point is at least one data point among the front and back, left and right, and top and bottom vertices of the three-dimensional model in the preset coordinate system; Determine the coordinates of a fourth intersection point based on the first straight line expression and the first plane expression, and determine the coordinates of a fifth intersection point based on the first straight line expression and the second plane expression; It is determined whether the coordinates of the fourth intersection point are within the display range of the first display portion represented by the plurality of first boundary coordinates, and whether the coordinates of the fifth intersection point are within the display range of the second display portion represented by the plurality of second boundary coordinates.

9. The method according to any one of claims 3 to 6, characterized in that The method further comprises: Based on the size of the first display part and the size of the second display part, determining whether the first projection image and the second projection image exceed the display range of the first display part and / or the second display part; The method of displaying a first projection image of a preset object through the first display part and displaying a second projection image of the preset object through the second display part comprises: When the first projection image does not exceed the display range of the first display part, and the second projection image does not exceed the display range of the second display part, the first projection image of the preset object is displayed through the first display part, and the second projection image of the preset object is displayed through the second display part.

10. The method according to any one of claims 2 to 9, characterized in that The preset coordinate system is established with the folding line between the first display part and the second display part as the X-axis, the side of the second display part as the Y-axis, and the side of the first display part as the Z-axis. The side of the first display part and the side of the second display part are in the same straight line when the angle between the first display part and the second display part is 180 degrees.

11. The method according to claim 10, characterized in that When the angle between the first display part and the second display part is 90 degrees, the preset coordinate system is an orthogonal coordinate system.

12. The method according to claim 10, characterized in that When the angle between the first display part and the second display part is not 90 degrees, the preset coordinate system is a non-orthogonal coordinate system.

13. The method according to any one of claims 1 to 12, characterized in that The user's observation point is the midpoint of the line between the user's eyes; Determining the first position includes: Determine the left eye position and the right eye position of the user by using a camera; A midpoint position between the left eye position and the right eye position is determined.

14. A folding screen electronic device, characterized in that: include: A processor, wherein the processor is coupled to a memory, wherein the memory is used to store a computer program, and when the processor calls the computer program, the folding screen electronic device executes the method as described in any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program comprising instructions for implementing the method according to any one of claims 1 to 13.

16. A computer program product, characterized in that The computer program product includes a computer program code, and when the computer program code is executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 13.

Images