Split-screen display method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the field of terminal display, in particular to a split-screen display method, electronic equipment and a computer readable storage medium. The method is applied to the electronic equipment, and the electronic equipment comprises a display screen. According to the method provided by the embodiment of the invention, the electronic equipment can automatically rotate the display direction of the preview flow of the camera under the condition that the camera is turned on through the split-screen application in the landscape state, or the screen is rotated after the camera is turned on through the split-screen application, so that the display content conforms to the viewing direction of a user, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal display, and in particular to a split-screen display method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of terminal technology, users' functional requirements for electronic devices are becoming more and more diverse. For example, in order to meet users' needs to use multiple applications in the same time period, more electronic devices can support split-screen mode. In this split-screen mode, the screen of the electronic device can display two or more windows so that users can view the content displayed in multiple windows. However, at present, when starting the camera through a split-screen application in horizontal mode, the display direction of the camera preview stream cannot adaptively match the user's viewing direction, affecting the user experience. Summary of the invention

[0003] The present application provides a split-screen display method, an electronic device, and a computer-readable storage medium. When a camera is started through a split-screen application in landscape mode, the display direction of the camera preview stream can be automatically adjusted so that the displayed content conforms to the user's viewing direction, thereby improving the user experience.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, a split-screen display method is provided, which is applied to an electronic device, wherein the electronic device includes a display screen, and the method includes:

[0006] The display screen displays a first interface in horizontal mode, wherein the first interface includes a first split-screen window of a first application, and the first split-screen window includes a first icon of a camera;

[0007] The electronic device receives a first operation of the user on the first icon, where the first operation is used to start a camera;

[0008] In response to the first operation, the electronic device adjusts a second direction according to the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction, wherein the second direction is a direction of the image in the camera viewfinder relative to the display screen;

[0009] The electronic device displays a second interface according to the adjusted second direction, where the second interface includes the second split-screen window, and the second split-screen window includes an image in a camera viewfinder.

[0010] Among them, the first application is not a camera application.

[0011] In an embodiment of the present application, when the electronic device does not rotate, the electronic device can be said to be in a portrait screen state (or vertical screen state); when the electronic device is rotated 90° counterclockwise or clockwise, the electronic device can be said to be in a landscape screen state; when the electronic device is rotated 180° counterclockwise or clockwise, the electronic device can be said to be in an inverted screen state.

[0012] Taking the Android system as an example, in the Android system, the content displayed to the user by the application is the user interface. When the application is opened, an Activity is started accordingly. Activity is a component of the application, and each Activity has an application window associated with it for displaying the user interface of the application. That is, in the embodiment of the present application, the application window refers to the window associated with the Activity for displaying the user interface of the application. The split-screen window refers to the application window in the split-screen mode.

[0013] In the embodiment of the present application, the electronic device opens the camera through application A (first application) in the landscape and split-screen modes, and the image in the camera viewfinder is "straightened" to match the user's viewing direction. Through the method in the embodiment of the present application, when the electronic device opens the camera through the split-screen application in the landscape state, the electronic device can automatically correct the direction of the camera preview stream, effectively reducing the occurrence of the "skewed" display direction of the preview stream caused by the rotation of the split-screen window driving the preview stream rotation, so that the camera display content matches the user's viewing direction, thereby improving the user experience.

[0014] In an implementation of the first aspect, the electronic device adjusts the second direction of the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction, including:

[0015] The electronic device determines whether the second direction needs to be adjusted;

[0016] If the second direction needs to be adjusted, the electronic device adjusts the second direction according to the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction.

[0017] In the embodiment of the present application, judgment is made first and then adjustment is made. In this way, unnecessary adjustment steps can be effectively reduced.

[0018] In an implementation of the first aspect, the electronic device determines whether the second direction needs to be adjusted, including:

[0019] The electronic device detects a display mode of an application window of a current camera;

[0020] If the display mode of the current camera application window is a split-screen mode, the electronic device determines that the second direction needs to be adjusted.

[0021] It should be noted that when the application window of the camera is displayed in a split-screen mode, the application window of the camera may be referred to as a split-screen window.

[0022] In one implementation, the activity task management service extension module (HnActivityTaskManagerServiceEX) may obtain the display mode of the camera application window through the activity record module (ActivityRecord).

[0023] In the embodiment of the present application, it is equivalent to executing the second direction adjustment solution in the split-screen mode. In this way, the impact on the camera preview stream in the full-screen mode can be reduced.

[0024] In an implementation of the first aspect, after the electronic device detects a display mode of an application window of a current camera, the method further includes:

[0025] If the display mode of the application window of the current camera is a split-screen mode, the electronic device determines whether the first application is in a first preset list, wherein the first preset list is configured in the system component;

[0026] If the first application is in the first preset list, the electronic device determines that the second direction needs to be adjusted.

[0027] In one implementation, the first preset list includes application package names. If the application package name of the first application is in the first preset list, it means that the first application is in the first preset list.

[0028] In an implementation of the first aspect, after the electronic device determines whether the first application is in a first preset list, the method further includes:

[0029] If the application is not in the first preset list, the electronic device determines whether the first application is in a second preset list, wherein the second preset list is configured in the application component;

[0030] If the first application is in the second preset list, the electronic device determines that the second direction needs to be adjusted.

[0031] In some implementations, the second preset list may exist in the configuration file of the CCM. Accordingly, the package name of the first application is searched in the second preset list of the configuration file; if it exists, it is determined that the first application is in the second preset list of the configuration file of the CCM, and true is returned; if it does not exist, it is determined that the first application is not in the second preset list of the configuration file of the CCM, and false is returned.

[0032] In the embodiment of the present application, the first preset list corresponds to the system component, and the second preset list corresponds to the product component (also called application component). The system component is effective for the entire system, while the product component is effective for a certain product. By adding a system-side list, if a problem common to all products occurs, it can be solved uniformly through the system-side list, without the need for single configuration of each product.

[0033] In some other embodiments, when there is no first preset list, after detecting the display mode of the application window of the current camera, it is determined whether the first application is in the second preset list, that is, there is no need to determine whether the Activity is in the first preset list.

[0034] In an implementation of the first aspect, the electronic device adjusts the second direction of the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction, including:

[0035] The electronic device calculates a first direction correction value according to a first direction of a second split-screen window of the current camera relative to the display screen;

[0036] The electronic device adjusts the second direction according to the direction correction value to obtain the adjusted second direction.

[0037] In an implementation of the first aspect, the electronic device calculates the first direction correction value according to a first direction of a second split-screen window of the current camera relative to a display screen, including:

[0038] If the first screen direction is greater than the first window direction, the first direction correction value is the difference between the first screen direction and the first window direction, wherein the first screen direction is the direction of the current display screen, and the first window direction is the direction of the current second split-screen window;

[0039] If the first screen direction is equal to the first window direction, the first direction correction value is a first preset value, and the first preset value indicates that there is no need to adjust the display direction of the camera;

[0040] If the first screen direction is smaller than the first window direction, the first direction correction value is the sum of the difference between the first screen direction and the first window direction and the second preset value.

[0041] In the embodiment of the present application, the screen direction refers to the direction of the display screen of the electronic device. In order to conform to the user's viewing habits, the screen direction can be defined as a direction that conforms to the user's viewing habits. Taking a mobile phone as an example, the position of the charging port can be defined as the bottom of the mobile phone, and the position of the camera on the display screen can be defined as the top of the mobile phone. Usually, the direction from the bottom to the top of the mobile phone conforms to the user's viewing direction, so the direction from the bottom to the top of the mobile phone can be defined as the screen direction of the mobile phone.

[0042] In some embodiments, the algorithm for adjusting the display direction of the camera may include the following steps:

[0043] If the current screen direction is greater than the direction of the camera application window, the output result includes the package name of the application that calls the camera and a first correction value, wherein the first correction value is the difference between the screen direction and the direction of the camera application window.

[0044] If the current screen direction is equal to the direction of the camera application window, the output result includes a second correction value, and the second correction value indicates that there is no need to adjust the display direction of the camera. For example, the second correction value is -1.

[0045] If the current screen direction is smaller than the direction of the camera application window, the output result includes the registration number of the application that called the camera and a third correction value, wherein the third correction value is the sum of the difference between the screen direction and the direction of the camera application window and a second preset value. For example, the second preset value is 4.

[0046] It should be noted that if the current screen direction (first screen direction) is greater than the direction (first window direction) of the camera's application window (first split-screen window), the calculated first direction correction value is the first correction value; if the current screen direction (first screen direction) is equal to the direction of the camera's application window, the calculated first direction correction value is the second correction value (first preset value); if the current screen direction (first screen direction) is less than the direction of the camera's application window (first window direction), the calculated first direction correction value is the third correction value.

[0047] In a second aspect, a screen splitting method is provided, which is applied to an electronic device, wherein the electronic device includes a display screen, and the method includes:

[0048] The display screen displays a third interface in portrait mode, wherein the third interface includes a third split-screen window of the second application, and the third split-screen window includes a second icon of the camera;

[0049] The electronic device receives a second operation of the user on the second icon, where the second operation is used to start the camera;

[0050] In response to the second operation, the electronic device displays a fourth interface, the fourth interface includes a fourth split-screen window of the camera, and the fourth split-screen window includes an image in a camera viewfinder;

[0051] The electronic device receives a third operation, where the third operation is used to rotate the display screen;

[0052] In response to the third operation, the electronic device adjusts a fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, wherein the fourth direction is the direction of the image in the camera viewfinder included in the fourth split-screen window relative to the display screen.

[0053] In an implementation of the second aspect, the electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, including:

[0054] The electronic device determines whether the fourth direction needs to be adjusted;

[0055] If the fourth direction needs to be adjusted, the electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen.

[0056] In an implementation of the second aspect, the electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, including:

[0057] The electronic device determines whether the second application is in a first preset list, wherein the first preset list is configured in a system component;

[0058] If the second application is in the first preset list, the electronic device determines that the fourth direction needs to be adjusted.

[0059] In an implementation of the second aspect, after the electronic device determines whether the second application is in the first preset list, the method further includes:

[0060] If the second application is not in the first preset list, the electronic device determines whether the second application is in a second preset list, wherein the second preset list is configured in the application component;

[0061] If the second application is in the second preset list, the electronic device determines that the fourth direction needs to be adjusted.

[0062] In the embodiment of the present application, the split-screen display method is executed when the split-screen mode has been entered. Therefore, the display mode of the current application window is set to the split-screen mode by default, and there is no need to determine the display mode of the current application window.

[0063] In an implementation of the second aspect, the electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, including:

[0064] The electronic device calculates a second direction correction value according to a third direction of the fourth split-screen window of the current camera relative to the display screen;

[0065] The electronic device adjusts the fourth direction according to the second direction correction value.

[0066] In an implementation of the second aspect, the electronic device calculates the second direction correction value according to a third direction of the fourth split-screen window of the current camera relative to the display screen, including:

[0067] If the second screen direction is greater than the second window direction, the second direction correction value is the difference between the second screen direction and the second window direction, wherein the second screen direction is the direction of the current display screen, and the second window direction is the direction of the current fourth split-screen window;

[0068] If the second screen direction is equal to the second window direction, the second direction correction value is a first preset value, and the first preset value indicates that there is no need to adjust the display direction of the camera;

[0069] If the second screen direction is smaller than the second window direction, the second direction correction value is the sum of the difference between the second screen direction and the second window direction and the second preset value.

[0070] In some embodiments, the algorithm for adjusting the display direction of the camera may include the following steps:

[0071] If the current screen direction is greater than the direction of the camera application window, the output result includes the package name of the application that calls the camera and a first correction value, wherein the first correction value is the difference between the screen direction and the direction of the camera application window.

[0072] If the current screen direction is equal to the direction of the camera application window, the output result includes a second correction value, and the second correction value indicates that there is no need to adjust the display direction of the camera. For example, the second correction value is -1.

[0073] If the current screen direction is smaller than the direction of the camera application window, the output result includes the registration number of the application that called the camera and a third correction value, wherein the third correction value is the sum of the difference between the screen direction and the direction of the camera application window and a second preset value. For example, the second preset value is 4.

[0074] It should be noted that if the current screen direction (second screen direction) is greater than the direction (second window direction) of the camera's application window (fourth split-screen window), the calculated second direction correction value is the first correction value; if the current screen direction (second screen direction) is equal to the direction of the camera's application window, the calculated second direction correction value is the second correction value (first preset value); if the current screen direction (second screen direction) is smaller than the direction of the camera's application window (second window direction), the calculated second direction correction value is the third correction value.

[0075] In a third aspect, an electronic device is provided, comprising a processor, wherein the processor is used to run a computer program stored in a memory to implement any method of the first aspect or any method of the second aspect of the present application.

[0076] In a fourth aspect, a chip system is provided, comprising a processor, wherein the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement any method of the first aspect or any method of the second aspect of the present application.

[0077] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, it implements any method of the first aspect or any method of the second aspect of the present application.

[0078] In a sixth aspect, the present application provides a computer program product, which, when executed on a device, enables the device to execute any method of the first aspect or any method of the second aspect of the present application.

[0079] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0081] Figure 2 It is a schematic diagram of the external structure of a folding screen mobile phone provided in an embodiment of the present application;

[0082] Figure 3 is a schematic diagram of a coordinate system provided in an embodiment of the present application;

[0083] Figure 4 is a schematic diagram of the angle area provided in the embodiment of the present application;

[0084] Figure 5 is a schematic diagram of the screen direction of the electronic device provided in an embodiment of the present application;

[0085] Figure 6 is a schematic diagram of the display direction provided in an embodiment of the present application;

[0086] Figure 7 is a schematic diagram of a user operation application scenario provided in an embodiment of the present application;

[0087] Figure 8 This is a schematic diagram of a process of forcibly rotating a window in horizontal or split-screen mode provided in an embodiment of the present application;

[0088] Fig. 9This is a schematic diagram of the process of forcibly rotating the window after rotating the screen in split-screen mode provided by this application;

[0089] Fig.10 is a schematic block diagram of the software architecture of the electronic device provided in the embodiment of the present application;

[0090] Fig.11 It is a timing diagram of the split-screen display method provided in an embodiment of the present application;

[0091] Fig.12 is a flow chart of a method for determining whether the display direction of the camera needs to be updated;

[0092] Fig.13 is a timing diagram of a split-screen display method provided by another embodiment of the present application;

[0093] Fig.14 It is a schematic diagram of an application scenario of the split-screen display method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] In the following description, specific details such as specific system structures and technologies are provided for illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details.

[0095] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0096] It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more than two; "and / or" describes the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0097] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", "fourth", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0098] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0099] The embodiment of the present application provides a split-screen display method, which can be applied to an electronic device with a screen, a display function and a camera function. The electronic device in the embodiment of the present application may also be referred to as a terminal device, a terminal, a user equipment (UE) or a user terminal, etc. For example, the electronic device may be a mobile phone, a tablet computer, a handheld computer, a wearable device (such as a smart watch, a smart bracelet, etc.), a laptop computer, a PDA, a personal computer, a smart home device (such as a TV, etc.), a smart screen, a game console, a mobile Internet device (MID), a smart sales terminal (POS), an augmented reality (AR) device, and a virtual reality (VR) device, etc. The embodiment of the present application does not specifically limit the specific form of the electronic device. For ease of explanation, the following embodiments are introduced by taking a mobile phone as an example.

[0100] For example, see Figure 1 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0101] like Figure 1As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 140A, a battery 140B, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0102] The processor 110 may include one or more processing units, wherein different processing units may be independent devices or may be integrated into one or more processors.

[0103] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0104] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge an electronic device, and can also be used to transfer data between an electronic device and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0105] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0106] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 140B, it may also power the electronic device through the power management module 140A.

[0107] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0108] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas.

[0109] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0110] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0111] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. In some embodiments, the electronic device 100 displays a user interface through the display screen 194.

[0112] When the electronic device 100 includes multiple display screens, it can be a folding screen device. The display screen of the electronic device 100 can be set to a foldable display screen, referred to as a folding screen. Among them, the folding screen can use an integrally formed flexible display screen, or a spliced ​​display screen composed of multiple flexible display screens and hinges between each two flexible display screens, or a spliced ​​display screen composed of multiple rigid screens and hinges between each two rigid screens. This embodiment of the application is not limited to this.

[0113] For example, the electronic device 100 may be a foldable screen mobile phone. Figure 2 , is a schematic diagram of the external structure of a folding screen mobile phone provided in an embodiment of the present application. The display screen of a folding screen mobile phone may include an inner screen and an outer screen. The inner screen is a folding screen, and the size of the inner screen is larger than the size of the outer screen. Figure 2 (a) in the figure shows the outer side of the foldable screen mobile phone when the inner screen is unfolded. Figure 2 As shown in (a), the first part 301 on the outside of the foldable screen mobile phone can be set as the outer screen of the foldable screen mobile phone, and the second part 302 can be set as the shell of the mobile phone. The first part 301 and the second part 302 on the outside of the foldable screen mobile phone can both be set as cameras. In other foldable screen mobile phone examples, the first part 301 and the second part 302 on the outside of the foldable screen mobile phone can both be set as the outer screen of the foldable screen mobile phone, or can both be set as the shell of the foldable screen mobile phone. Figure 2 (b) in the figure shows the inner side of the foldable screen mobile phone when the inner screen is unfolded. Figure 2As shown in (b), the inner screen (i.e., folding screen) of the folding screen mobile phone includes a first display unit 303 and a second display unit 304. The first display unit 303 and the second display unit 304 can display different display interfaces respectively, or they can display a display interface together. The posture or form of the folding screen can be determined according to the angle between the first display unit 303 and the second display unit 304. For example, when the angle between the first display unit 303 and the second display unit 304 is 0 degrees, the folding screen is in a fully folded state; when the angle between the first display unit 303 and the second display unit 304 is 180 degrees, the folding screen is in a fully unfolded state; when the angle between the first display unit 303 and the second display unit 304 is greater than 0 degrees and less than 180 degrees, the folding screen is in a semi-folded state. It should be noted that in the above example, the description of the size of the angle between the first display unit 303 and the second display unit 304 of the folding screen mobile phone in the fully folded state, the semi-folded state, and the fully unfolded state is only an example, and the embodiment of the present application does not limit this. In addition, the folding screen is in a fully unfolded state, a fully folded state or a half-folded state, which can also be referred to as the folding screen mobile phone being in a fully unfolded state, a fully folded state or a half-folded state.

[0114] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0115] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0116] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, the electronic device 100 saves the user's punch-in data in the external memory card.

[0117] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function, etc. The data storage area may store data created during the use of the electronic device 100 (such as interface information corresponding to the user interface, etc.). In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0118] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0119] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0120] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0121] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0122] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0123] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0124] Understandably, Figure 1 It is only an example of the structure of the electronic device 100, and some modules related to the split-screen display method described in the embodiment of the present application are shown. The structure shown in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0125] To facilitate understanding, the names involved in the embodiments of the present application are first explained.

[0126] 1. Define the coordinate system

[0127] See also Figure 3 , is a schematic diagram of a coordinate system provided in an embodiment of the present application. Figure 3As shown, in the embodiment of the present application, four direction values ​​0, 1, 2 and 3 can be defined. Among them, the direction of gravity is defined as 2, and the direction opposite to the direction of gravity is defined as 0. The direction perpendicular to the direction 0 and satisfying the right-hand rule is defined as 1, that is, extend the thumb and index finger of the right hand so that they are perpendicular to each other, and then point the thumb to the direction 0, and the direction pointed by the index finger is the direction 1. The direction opposite to the direction 1 is defined as 3.

[0128] It should be noted that the above is only an example of a coordinate system, and other coordinate systems or other direction definition methods can also be used. For example, using other digital definitions Figure 3 Or, a three-dimensional rectangular coordinate system, such as the xyz coordinate system, can be used. Figure 3 The coordinate system shown is further divided into more directions, such as 8 directions, 36 directions, etc. The embodiment of the present application does not specifically limit the coordinate system and direction, as long as the coordinate system or direction definition used in the process of executing the split-screen display method of the embodiment of the present application is consistent.

[0129] like Figure 3 As shown, four direction values ​​are defined in the embodiment of the present application. In practical applications, when a sensor (such as a gyroscope, etc.) of an electronic device senses that the electronic device is rotating, the electronic device obtains the direction after rotation and corresponds the direction after rotation to one of the four direction values, that is, one of the four direction values ​​is used to assign a value to the direction after rotation.

[0130] In some application scenarios, the rotation angle of the electronic device is not necessarily strictly 90° or its multiples. In this case, a possible implementation method is to use Figure 3 The coordinate system in the image is divided into multiple angle regions, each angle region corresponds to a direction value; the electronic device can determine the angle region to which the direction after rotation belongs according to the direction before rotation and the rotation angle, thereby determining the direction value after rotation.

[0131] For example, see Figure 4 , is a schematic diagram of the angle region provided in the embodiment of the present application. Figure 4 As shown, Figure 3The coordinate system shown is divided into 4 areas. Assuming that the 3 direction is the 0° direction and the 0 direction is the 90° direction, the 4 angle areas are (45°, 135°), (-135°, 135°), (-45°, -135°) and (-45°, 45°). The electronic device determines the angle area according to the direction before rotation and the rotation angle. If the angle area to which the direction after rotation belongs is (45°, 135°), the direction value after rotation is determined to be 0; if the angle area to which the direction after rotation belongs is (-135°, 135°), the direction value after rotation is determined to be 1; if the angle area to which the direction after rotation belongs is (-45°, -135°), the direction value after rotation is determined to be 2; if the angle area to which the direction after rotation belongs is (-45°, 45°), the direction value after rotation is determined to be 3. For example, the direction value before rotation is 0, and it rotates 60° clockwise. Then the angle range to which the direction after rotation belongs is (-45°, 45°), and the direction value after rotation is determined to be 3.

[0132] It should be noted that Figure 4 The division of the angle regions shown is only an example, and the angle range of each region can be adjusted according to the user's experience. For example, the angle region corresponding to the 0 direction can be adjusted from (45°, 135°) to (30°, 150°) according to the user's experience. The embodiment of the present application does not specifically limit the division of the angle regions.

[0133] For ease of explanation, in the embodiment of the present application, the direction the user is viewing is direction 0.

[0134] 2. Screen orientation

[0135] In the embodiment of the present application, the screen direction refers to the direction of the display screen of the electronic device. In order to meet the viewing habits of the user, the screen direction can be defined as the direction that meets the viewing habits of the user.

[0136] For example, see Figure 5 , is a schematic diagram of the screen direction of the electronic device provided in the embodiment of the present application. Figure 5 As shown, taking a mobile phone as an example, the position of the charging port 51 can be defined as the bottom of the mobile phone, and the position of the camera 52 on the display screen can be defined as the top of the mobile phone. Usually, the direction from the bottom to the top of the mobile phone is consistent with the viewing direction of the user, so the direction from the bottom to the top of the mobile phone can be defined as the screen direction of the mobile phone, such as Figure 5 The direction is indicated by arrow 53.

[0137] In the embodiment of the present application, when the electronic device does not rotate, it can be said that the electronic device is in a vertical screen (or portrait screen) state; when the electronic device rotates 90° counterclockwise or clockwise, it can be said that the electronic device is in a horizontal screen state; when the electronic device rotates 180° counterclockwise or clockwise, it can be said that the electronic device is in an inverted screen state. Figure 5 As shown in (a) in , the screen direction is 0, indicating that the electronic device has not rotated. Figure 5 As shown in (c) in FIG. 1 , the screen direction is 1, which means that the electronic device is rotated 90° counterclockwise, that is, the electronic device is in a horizontal screen state. Figure 5 As shown in (b), the screen direction is 2, indicating that the electronic device is rotated 180°, that is, the electronic device is in an inverted state.

[0138] 3. Display direction

[0139] In the embodiments of the present application, the display direction of the screen refers to the direction in which the content displayed on the screen of the electronic device is facing. In some application scenarios, after the screen of the electronic device is rotated, in order to ensure the user's viewing experience, the content displayed on the screen will be rotated accordingly to make it consistent with the user's viewing direction. Therefore, in the above scenario, the screen direction is inconsistent with the display direction.

[0140] For example, see Figure 6 , is a schematic diagram of the display direction provided by the embodiment of the present application. Figure 6 As shown, the screen direction is 1, and the display direction of the content displayed on the screen (the direction pointed by the thick arrow) is 0.

[0141] 4. Window direction

[0142] Taking the Android system as an example, in the Android system, the content displayed to the user by the application is the user interface, and Activity is a component of the application. Each Activity has an application window associated with it for displaying the user interface of the application. That is, in the embodiment of the present application, the application window refers to the window associated with Activity for displaying the user interface of the application.

[0143] In some application scenarios, after the electronic device is rotated, the direction of the application window may be different from the display direction.

[0144] With the development of terminal technology, users' functional requirements for electronic devices are becoming more and more diverse. For example, in order to meet users' needs to use multiple applications in the same time period, many electronic devices can support split-screen mode, in which the screen of the electronic device can display two or more windows so that users can view the contents displayed in multiple windows.

[0145] In some application scenarios, users can launch the camera in a split-screen window of a non-camera application. Figure 7 , is a schematic diagram of a user operation application scenario provided by an embodiment of the present application. Figure 7 As shown in (a) of FIG. 1 , a user interface 70 of an electronic device in a horizontal screen is divided into two display areas, a left display area and a right display area, by a split screen bar 701. An application window 702 of application A occupies the left display area, and an application window 703 of a calculator occupies the right display area.

[0146] When the user clicks / touches the camera icon 7021 in the application window 702 of application A, in response to the user operation, the electronic device displays the following Figure 7 The user interface 71 shown in (b) of FIG. In the user interface 71, the application window 702 of application A is replaced by the application window 704 of the camera, and the application window 704 of the camera is in the 0 direction (see FIG. Figure 7 (b) in the application window 704, the direction of the text and buttons displayed), the display direction of the image in the camera viewfinder (see Figure 7 The thick arrow 7041 shown in (b) in the figure rotates in 3 directions.

[0147] like Figure 7 As shown, when the electronic device is in landscape or split-screen mode and the camera is opened through application A, the user's intuitive feeling is that the image in the camera viewfinder is "crooked".

[0148] The reason for the above is that when the electronic device opens the camera through application A in landscape or split-screen mode, or after opening the camera through application A in split-screen mode and rotating the screen, in order to ensure that the display direction of buttons, text and other information in the camera's application window is consistent with the user's viewing direction, the camera's application window will be forcibly rotated back to the 0 direction.

[0149] As an example of a situation where an electronic device opens a camera through application A in landscape or split-screen mode, see Figure 8 , is a schematic diagram of the process of forced window rotation in landscape and split-screen modes provided by an embodiment of the present application. In an application scenario where the electronic device is in landscape mode, the camera in application A is opened in full screen, the direction of the application window is consistent with the screen direction, and the display direction is rotated to make it consistent with the user's viewing direction. In this case, the direction of the application window is different from the display direction. Figure 8 As shown in (a) of FIG. 8 , the screen direction and the direction of the application window 801 (see Figure 8 The directions of the text and buttons displayed in the application window 801 shown in (a) are all 1 direction, while the display direction (such as Figure 8In order to make the buttons, texts and other information in the application window conform to the user's viewing direction, the camera's application window 801 is forcibly rotated to the 0 direction. In this case, the display direction changes with the rotation of the application window and no longer conforms to the user's viewing direction. Figure 8 As shown in (b), the direction of the application window 801 after rotating 90° clockwise is the 0 direction. At this time, the direction of the application window 801 (see Figure 8 The direction of the text and buttons displayed in the application window 801 shown in (b) in FIG. 8 is consistent with the viewing direction of the user, and the display direction is 3 directions after the application window is rotated 90° clockwise (such as Figure 8 As shown in FIG. 1 , the display direction no longer matches the viewing direction of the user.

[0150] As an example of a case where the electronic device rotates the screen after opening the camera through application A in split screen mode, see Fig. 9 , is a schematic diagram of the process of forcibly rotating the window after rotating the screen in split-screen mode provided by this application. In the application scenario where the electronic device is in portrait mode, the camera is opened through application A in split-screen mode, and the direction of the application window, the screen direction and the display direction are all consistent. Fig. 9 As shown in (a) of FIG. 1 , the screen direction, the direction of the application window 901 and the display direction (eg Fig. 9 The direction of the thick arrow 9011 shown in (a) is 0. After the electronic device is rotated to landscape mode (rotating the electronic device counterclockwise 90°), the screen direction and the direction of the application window rotate accordingly, and the display direction is 0 direction to ensure that the displayed content conforms to the viewing direction of the user. Fig. 9 As shown in (b) of FIG. 1 , the screen direction and the application window 901 (see Fig. 9 The direction of the text and buttons displayed in the application window 901 shown in (b) is rotated to direction 1, and the display direction (such as Fig. 9 The direction pointed by the thick arrow 9011 shown in (b) is still the 0 direction. In order to ensure that the direction of the application window is consistent with the viewing direction of the user, the application window is forcibly rotated to the 0 direction. In this case, the display direction changes with the rotation of the application window and no longer conforms to the viewing direction of the user. Fig. 9 As shown in (c), the direction of the application window 901 after rotating 90° clockwise is the 0 direction. At this time, the direction of the application window 901 (see Fig. 9 The direction of the text and buttons displayed in the application window 901 shown in (c) in FIG. 1 is consistent with the viewing direction of the user, and the display direction is 3 after the application window is rotated 90° clockwise (as shown in FIG. 1 ). Fig. 9It can be seen that the display direction no longer matches the viewing direction of the user.

[0151] It should be noted that the user interface viewed by the user will not display Figure 8 The process from (a) to (b) in Fig. 9 The process shown in (a) to (c) in FIG. 1 is to force the application window of the camera to rotate to the 0 direction. The above process is only used to illustrate the cause Figure 7 The principle of the scenario shown in the embodiment. The user interface sees Figure 7 The scene shown in the embodiment, that is, the image in the camera viewfinder is "distorted".

[0152] As described above, when the electronic device opens the camera through a split-screen application in landscape mode, or when the screen is rotated after the camera is opened through a split-screen application, the display direction of the camera preview stream does not match the user's viewing direction, affecting the user experience.

[0153] Based on this, an embodiment of the present application provides a split-screen display method. Through the method in the embodiment of the present application, when the electronic device opens the camera through the split-screen application in the horizontal screen state, or when the screen is rotated after the camera is opened through the split-screen application, the electronic device can automatically rotate the display direction of the camera preview stream so that the displayed content conforms to the viewing direction of the user, thereby improving the user experience.

[0154] The following introduces the internal implementation process of the split-screen display method in the embodiment of the present application.

[0155] To facilitate understanding, the software architecture of the electronic device is first introduced.

[0156] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-server architecture or a cloud architecture. The software architecture of the electronic device is exemplarily described below.

[0157] See also Fig.10 , is a schematic block diagram of the software architecture of the electronic device provided in the embodiment of the present application. Fig.10 As shown, the software architecture of the electronic device may include an application layer and a system layer.

[0158] The application layer can include multiple applications, each of which corresponds to an Activity. Fig. 9 As shown, the application layer includes a camera application, and the camera application corresponds to an Activity, which is used to open the camera interface and display the picture in the camera viewfinder.

[0159] The system layer can include IBinder, Camera Framework, native code repository (Android Open Source Project, AOSP), self-developed code repository and component configuration management module (component configuration management, CCM).

[0160] IBinder is a basic interface for remote objects. It is the core part of a lightweight remote call mechanism designed for high performance. It is used not only for remote calls (cross-process calls) but also for intra-process calls. This interface defines the protocol for interacting with remote objects. IBinder is used for interaction between the application layer and the system layer.

[0161] AOSP may include an activity client control module (ActivityClientController), an activity task management service module (ActivityTaskManagerService, ATMS), an activity record module (ActivityRecord), a task module (Task) and a system parameter module (SystemProerties).

[0162] When the camera application's Activity opens the camera application's interface, the activity client control module (ActivityClientController) can be notified through the IBinder on the system side, and the activity client control unit notifies the activity task management service module (ActivityTaskManagerService).

[0163] The Activity Task Management Service (ATMS) module is responsible for the management of Activities, including the life cycle of applications, such as starting applications, switching applications, and managing task stacks. The Activity Record module is used to record the information of each Activity. Usually, each Activity started in an application has an ActivityRecord instance corresponding to it in the AMS module. This ActivityRecord is created when the Activity is started and destroyed when the Activity is terminated. In some operating systems, the Activity Manager Service (AMS) module can also be responsible for the management of Activities.

[0164] The task module is the container of the Activity. The activity task management service module is used to manage the Activity and its container (tasks, stacks, and displays, etc.). The activity task management service module and the task module are used to provide problem identification scenarios (such as Figure 7 and Figure 8 The scene shown) and Activity key process control the camera frame to rotate the camera preview stream.

[0165] The system parameter module is used to provide the camera framework with the relevant parameters for display direction rotation. Accordingly, the camera framework rotates the display direction according to the parameters provided by the system parameter module, generates a preview stream, and provides the preview stream to the camera application of the application layer.

[0166] The self-developed code repository can include a multi-window management module (HnMultiWindowManager), an activity task management service extension module (HnActivityTaskManagerService EX), and a multi-window configuration management module (HnMultiWinConfigManager).

[0167] The activity task management service extension module is used to manage the implementation and use of self-developed activity task related methods.

[0168] The multi-window management module is used to manage the implementation and use of self-developed multi-window related methods.

[0169] The multi-window configuration management module is used to manage the reading and use of self-developed multi-window configurations.

[0170] The CCM module stores a multi-window configuration file (hn_multi_window_config). The file can be an XML file, which can include a list that defines the split-screen applications at the Activity level, that is, when the application in the list calls the camera application in the split-screen mode, the display direction of the camera can be automatically rotated by the split-screen display method provided in the embodiment of the present application. The file can include the application package name and Activity name of each application in the list.

[0171] The methods in the following embodiments can all be carried out in a Figure 1 The hardware structure and Fig.10 The software architecture is implemented in an electronic device.

[0172] The following is a detailed introduction to the split-screen display method provided in the embodiment of the present application in conjunction with the software architecture of the electronic device.

[0173] like Figure 8 and Fig. 9 As shown in the embodiment, in the embodiment of the present application, the calling node of the split-screen display method is divided into two situations: in one situation, the electronic device is in a non-zero orientation state (such as horizontal screen or inverted screen), the split screen opens the camera application (such as Figure 8 In the application scenario shown in FIG. 1 , the electronic device uses Fig.11 In another case, after the electronic device opens the camera application in split screen, the screen is rotated (such as Fig. 9 In the application scenario shown in FIG. 1 , the electronic device uses Fig.13 The split-screen display method shown automatically rotates the display direction.

[0174] See also Fig.11 , is a timing diagram of the split-screen display method provided in the embodiment of the present application. Fig.11 As shown, it is a timing diagram when the electronic device turns on the automatic rotation function as the camera application is started.

[0175] S1101, the activity task management service module (ATMS) detects that the split-screen application opens the camera application and starts the camera Activity.

[0176] In some implementations, starting the camera through application A in split-screen mode includes starting the camera's Activity. Specifically, the activity task management service module (ATMS) calls the setResumeActivityUnchecklocked function, through which the state of the camera's Activity is set to resume, which indicates that the current Activity is in use.

[0177] In the embodiment of the present application, it is equivalent to setting an insertion point in the process of starting the camera's Activity, and when the program runs to the insertion point, the split-screen display method provided by the embodiment of the present application is executed. For example, after the state of the Activity is set to the resume state, the insertion point is set. Correspondingly, after the state of the Activity is set to the resume state, the following steps S1102-S1107 are executed.

[0178] After the activity task management service module (ATMS) starts the activity, it can send a notification to the activity task management service extension module (HnActivityTaskManagerService EX) (such as Fig.11 The first notification shown) is used to instruct the activity task management service extension module (HnActivityTaskManagerService EX) to execute S1102.

[0179] S1102, after starting the camera Activity, the activity task management service extension module (HnActivityTaskManagerService EX) determines whether the display direction of the camera needs to be updated.

[0180] After the activity task management service extension module (HnActivityTaskManagerService EX) detects the first notification, it confirms that the camera's Activity has been started, and then executes the step of determining whether the display direction needs to be updated. In this step, the display direction is the display direction of the image in the camera viewfinder.

[0181] In some embodiments, see Fig.12 , is a flow chart of a method for determining whether the camera display direction needs to be updated. Fig.12 As shown, the judgment process of S1102 includes the following steps:

[0182] S1201, the activity task management service extension module (HnActivityTaskManagerService EX) detects the display mode of the camera application window.

[0183] In one implementation, the activity task management service extension module (HnActivityTaskManagerServiceEX) may obtain the display mode of the camera application window through the activity record module (ActivityRecord).

[0184] S1202: If the display mode of the camera application window is split-screen mode, the activity task management service extension module (HnActivityTaskManagerService EX) checks whether the application A currently calling the camera is in the system list.

[0185] Among them, the system list can be configured in the system component, such as in the self-developed code repository.

[0186] S1203: If application A is in the system list, it is determined that the display direction of the camera needs to be updated.

[0187] S1203, if application A is not in the system list, the activity task management service extension module (HnActivityTaskManagerService EX) notifies the multi-window configuration management module (HnMultiWinConfigManager) to perform verification.

[0188] The activity task management service extension module (HnActivityTaskManagerService EX) can send notifications to the multi-window configuration management module (HnMultiWinConfigManager).

[0189] S1204, after receiving the notification (second notification) from the activity task management service extension module (HnActivityTaskManagerService EX), the multi-window configuration management module (HnMultiWinConfigManager) checks whether application A is in the list of configuration files of CCM and returns the judgment result.

[0190] Specifically, the multi-window configuration management module (HnMultiWinConfigManager) obtains the configuration file from CCM and searches for the package name of application A in the list of the configuration file; if it exists, it is determined that application A is in the list of configuration files of CCM, and true is returned to the activity task management service extension module (HnActivityTaskManagerService EX); if it does not exist, it is determined that application A is not in the list of configuration files of CCM, and false is returned to the activity task management service extension module (HnActivityTaskManagerService EX).

[0191] The activity task management service extension module (HnActivityTaskManagerService EX) determines whether the camera display direction needs to be updated according to the judgment result returned by the multi-window configuration management module (HnMultiWinConfigManager). For example, if the return result is true, it is determined that the camera display direction needs to be updated; if the return result is false, it is determined that the camera display direction does not need to be updated.

[0192] In the embodiment of the present application, the system list corresponds to the system component, and the list in the CCM corresponds to the product component (also called application component). The system component is effective for the entire system, while the product component is effective for a certain product. By adding a system-side list, if there is a problem common to all products, it can be solved uniformly through the system-side list, without the need for single configuration of each product.

[0193] In some other embodiments, when there is no system list, after executing S1201, S1203 is executed, that is, there is no need to determine whether the Activity is in the system list.

[0194] S1103: If the direction of the camera application window needs to be updated, the activity task management service extension module (HnActivityTaskManagerService EX) obtains the current screen direction.

[0195] As mentioned above Figure 8 and Fig. 9As shown in the embodiment, when the electronic device opens the camera through the split-screen application in the horizontal screen state, or when the screen is rotated after the camera is opened through the split-screen application, the direction of the camera application window will be forcibly rotated to the 0 direction. Therefore, in the embodiment of the present application, the direction of the camera application window is set to the 0 direction by default. Of course, in some implementations, in order to prevent accidental situations, the direction of the camera application window can also be obtained to confirm whether the direction is the 0 direction.

[0196] In one implementation of obtaining the current screen direction, the Task of the ActivityRecord of the Activity that carries the current camera is obtained, and the TaskDisplayArea to which the Task belongs, that is, the display area corresponding to the Task on the screen is found, so as to obtain the current screen direction.

[0197] S1104, the activity task management service extension module (HnActivityTaskManagerService EX) sends the acquired screen direction to the multi-window management module (HnMultiWindowManager), and notifies it to update the display direction of the camera.

[0198] In some implementations, the activity task management service extension module (HnActivityTaskManagerServiceEX) may notify the multi-window management module (HnMultiWindowManager) to update the display direction by calling updateCameraRotation.

[0199] S1105, after receiving the notification from the activity task management service extension module (HnActivityTaskManagerService EX), the multi-window management module (HnMultiWindowManager) executes the camera display direction adjustment algorithm to obtain an output result.

[0200] In some embodiments, the algorithm for adjusting the display direction of the camera may include the following steps:

[0201] If the current screen direction is greater than the direction of the camera application window, the output result includes the package name of the application that calls the camera and a first correction value, wherein the first correction value is the difference between the screen direction and the direction of the camera application window.

[0202] If the current screen direction is equal to the direction of the camera application window, the output result includes a second correction value (first preset value), and the second correction value indicates that the display direction of the camera does not need to be adjusted. For example, the second correction value is -1.

[0203] If the current screen direction (first screen direction) is smaller than the direction (first window direction) of the camera's application window (first split screen window), the output result includes the registration of the application that calls the camera and a third correction value, wherein the third correction value is the sum of the difference between the screen direction and the direction of the camera's application window and a second preset value. For example, the second preset value is 4.

[0204] It should be noted that if the current screen direction is greater than the direction of the camera's application window, the calculated first direction correction value is the first correction value; if the current screen direction is equal to the direction of the camera's application window, the calculated first direction correction value is the second correction value; if the current screen direction (first screen direction) is smaller than the direction (first window direction) of the camera's application window (first split-screen window), the calculated first direction correction value is the third correction value.

[0205] Exemplarily, the meaning of each correction value can be set, for example, correction value 1 means 90 degrees counterclockwise rotation, correction value 2 means 180 degrees counterclockwise rotation, correction value 3 means 270 degrees counterclockwise rotation, and correction value -1 means no correction is required. It should be noted that the above is only an example of the meaning of the correction value, and this is not specifically limited in the embodiments of the present application.

[0206] like Figure 8 In the application scenario shown in (b), the screen direction is 1, the direction of the camera's application window is 0, and the screen direction is greater than the direction of the camera's application window. The first direction difference is 1-0=1, that is, the first correction value obtained is 1, indicating a counterclockwise rotation of 90 degrees.

[0207] In the embodiment of the present application, the output result includes the package name of the application that calls the camera. In this way, the display direction of the application is adjusted specifically without affecting other applications.

[0208] S1106, the multi-window management module (HnMultiWindowManager) sends the output result to the system parameter module so that it sets the system parameters according to the output result.

[0209] In one implementation, the multi-window management module (HnMultiWindowManager) may set the system parameters by calling the set sys_hn_multiwin_for_camera function in the system parameter module.

[0210] S1107, the system parameter module sends the set system parameters to the camera framework, so that the camera framework rotates the display direction of the image in the camera viewfinder according to the system parameters, and generates a rotated preview stream.

[0211] See also Fig.13, is a timing diagram of a split-screen display method provided by another embodiment of the present application. Fig.13 As shown, it is a timing diagram when the automatic rotation function of the electronic device is turned on as the electronic device rotates after the camera is started.

[0212] S1301, the task module (Task) detects screen rotation.

[0213] In some implementations, the task module may monitor whether task parameters have changed by calling the onConfigrationChangeLinner function, and detect screen rotation by monitoring.

[0214] After detecting screen rotation, the task module may send a third notification to the activity task management service extension module (HnActivityTaskManagerService EX) to instruct it to execute S1302.

[0215] S1302: When screen rotation is detected, the activity task management service extension module (HnActivityTaskManagerService EX) determines whether the display direction of the camera needs to be updated.

[0216] In some embodiments, the determination process may include the following steps:

[0217] The activity task management service extension module (HnActivityTaskManagerService EX) checks whether the application A that currently calls the camera is in the system list; if application A is in the system list, it determines that the display direction of the camera needs to be updated; if application A is not in the system list, the activity task management service extension module (HnActivityTaskManagerServiceEX) notifies the multi-window configuration management module (HnMultiWinConfigManager) for verification; after receiving the notification from the activity task management service extension module (HnActivityTaskManagerService EX), the multi-window configuration management module (HnMultiWinConfigManager) verifies whether application A is in the list of CCM configuration files and returns the judgment result.

[0218] and Fig.12 The difference between the embodiments is that Fig.13 The split-screen display method is described as executing when the split-screen mode has been entered. Therefore, the display mode of the current application window is set to the split-screen mode by default, and no further judgment is required.

[0219] S1303: If the display direction of the camera needs to be updated, the activity task management service extension module (HnActivityTaskManagerService EX) obtains the current screen direction.

[0220] S1304, the activity task management service extension module (HnActivityTaskManagerService EX) sends the acquired screen direction to the multi-window management module (HnMultiWindowManager), and notifies it to update the display direction of the camera.

[0221] S1305, after receiving the notification from the activity task management service extension module (HnActivityTaskManagerService EX), the multi-window management module (HnMultiWindowManager) executes the camera display direction adjustment algorithm to obtain an output result.

[0222] S1306, the multi-window management module (HnMultiWindowManager) sends the output result to the system parameter module so that it sets the system parameters according to the output result.

[0223] S1307, the system parameter module sends the set system parameters to the camera framework, so that the camera framework rotates the display direction of the image in the camera viewfinder according to the system parameters, and generates a rotated preview stream.

[0224] Steps S1303-S1307 and the above-mentioned S1103-S1107, for details, please refer to the description of the above-mentioned S1103-S1107, which will not be repeated here.

[0225] It should be noted that Fig.11 In the embodiment, it is equivalent to setting the insertion point in the process of starting the camera through application A, executing the split-screen display method, and Fig.13 In the embodiment, it is equivalent to setting the insertion point after the camera is started through application A and during the rotation of the electronic device screen to execute the split-screen display method.

[0226] See also Fig.14 , is a schematic diagram of an application scenario of the split-screen display method provided in an embodiment of the present application. Fig.14 , is a schematic diagram of a user operation application scenario provided by an embodiment of the present application. Fig.14 As shown in (a), the user interface 72 (first interface) of the electronic device in landscape mode is divided into two display areas, left and right, by a split screen bar 701. Among them, the application window 702 (first split screen window) of application A (first application) occupies the left display area, and the application window 703 of the calculator occupies the right display area.

[0227] When the user clicks / touches the camera icon 7021 (first icon) in the application window 702 of application A, in response to the user operation, the electronic device displays the following Fig.14 In the user interface 73 shown in (b) of FIG. 73, the application window 702 of application A is replaced by the application window 704 of the camera (the second split screen window), the application window 704 of the camera is in the 0 direction, and the display direction of the image in the camera viewfinder (see FIG. 70) Fig.14 The direction pointed by the thick arrow 7041 shown in (b) is rotated to the 0 direction.

[0228] and Figure 7 Compared with the scene shown, Fig.14 In the illustrated scenario, the electronic device opens the camera through application A in landscape mode and split-screen mode, and the image in the camera viewfinder is "straightened" to match the user's viewing direction. Through the method in the embodiment of the present application, the electronic device opens the camera through a split-screen application in landscape mode, or when the screen is rotated after the camera is opened through a split-screen application, the electronic device can automatically rotate the display direction of the camera preview stream so that the displayed content matches the user's viewing direction, thereby improving the user experience.

[0229] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0230] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0231] An embodiment of the present application also provides a computer program product. When the computer program product runs on a first device, the first device can implement the steps in the above-mentioned method embodiments.

[0232] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0233] The embodiment of the present application also provides a chip system, the chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip system can be a single chip or a chip module composed of multiple chips.

[0234] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0235] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Finally, it should be noted that the above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of the present application shall be based on the scope of protection of the claims.

Claims

1. A split-screen display method, characterized in that: Applied to an electronic device, the electronic device includes a display screen, and the method includes: The display screen displays a first interface in horizontal mode, wherein the first interface includes a first split-screen window of a first application, and the first split-screen window includes a first icon of a camera; The electronic device receives a first operation of the user on the first icon, where the first operation is used to start a camera; In response to the first operation, the electronic device adjusts a second direction according to the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction, wherein the second direction is a direction of the image in the camera viewfinder relative to the display screen; The electronic device displays a second interface according to the adjusted second direction, where the second interface includes the second split-screen window, and the second split-screen window includes an image in a camera viewfinder.

2. The method according to claim 1, characterized in that The electronic device adjusts the second direction of the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction, including: The electronic device determines whether the second direction needs to be adjusted; If the second direction needs to be adjusted, the electronic device adjusts the second direction according to the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction.

3. The method according to claim 2, characterized in that The electronic device determines whether the second direction needs to be adjusted, including: The electronic device detects a display mode of an application window of a current camera; If the display mode of the current camera application window is a split-screen mode, the electronic device determines that the second direction needs to be adjusted.

4. The method according to claim 3, characterized in that After the electronic device detects the display mode of the application window of the current camera, the method further includes: If the display mode of the application window of the current camera is a split-screen mode, the electronic device determines whether the first application is in a first preset list, wherein the first preset list is configured in the system component; If the first application is in the first preset list, the electronic device determines that the second direction needs to be adjusted.

5. The method according to claim 4, characterized in that After the electronic device determines whether the first application is in a first preset list, the method further includes: If the application is not in the first preset list, the electronic device determines whether the first application is in a second preset list, wherein the second preset list is configured in the application component; If the first application is in the second preset list, the electronic device determines that the second direction needs to be adjusted.

6. The method according to any one of claims 1 to 5, characterized in that: The electronic device adjusts the second direction of the second split-screen window of the current camera relative to the first direction of the display screen to obtain the adjusted second direction, including: The electronic device calculates a first direction correction value according to a first direction of a second split-screen window of the current camera relative to the display screen; The electronic device adjusts the second direction according to the direction correction value to obtain the adjusted second direction.

7. The method according to claim 6, characterized in that The electronic device calculates a first direction correction value according to a first direction of a second split-screen window of a current camera relative to a display screen, including: If the first screen direction is greater than the first window direction, the first direction correction value is the difference between the first screen direction and the first window direction, wherein the first screen direction is the direction of the current display screen, and the first window direction is the direction of the current second split-screen window; If the first screen direction is equal to the first window direction, the first direction correction value is a first preset value, and the first preset value indicates that there is no need to adjust the display direction of the camera; If the first screen direction is smaller than the first window direction, the first direction correction value is the sum of the difference between the first screen direction and the first window direction and the second preset value.

8. A split-screen display method, characterized in that: Applied to an electronic device, the electronic device includes a display screen, and the method includes: The display screen displays a third interface in portrait mode, wherein the third interface includes a third split-screen window of the second application, and the third split-screen window includes a second icon of the camera; The electronic device receives a second operation of the user on the second icon, where the second operation is used to start the camera; In response to the second operation, the electronic device displays a fourth interface, the fourth interface includes a fourth split-screen window of the camera, and the fourth split-screen window includes an image in a camera viewfinder; The electronic device receives a third operation, where the third operation is used to rotate the display screen; In response to the third operation, the electronic device adjusts a fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, wherein the fourth direction is the direction of the image in the camera viewfinder included in the fourth split-screen window relative to the display screen.

9. The method according to claim 8, characterized in that The electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, including: The electronic device determines whether the fourth direction needs to be adjusted; If the fourth direction needs to be adjusted, the electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen.

10. The method according to claim 9, characterized in that The electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, including: The electronic device determines whether the second application is in a first preset list, wherein the first preset list is configured in a system component; If the second application is in the first preset list, the electronic device determines that the fourth direction needs to be adjusted.

11. The method according to claim 10, characterized in that After the electronic device determines whether the second application is in the first preset list, the method further includes: If the second application is not in the first preset list, the electronic device determines whether the second application is in a second preset list, wherein the second preset list is configured in the application component; If the second application is in the second preset list, the electronic device determines that the fourth direction needs to be adjusted.

12. The method according to any one of claims 8 to 11, characterized in that The electronic device adjusts the fourth direction according to the third direction of the current fourth split-screen window relative to the display screen, including: The electronic device calculates a second direction correction value according to a third direction of the fourth split-screen window of the current camera relative to the display screen; The electronic device adjusts the fourth direction according to the second direction correction value.

13. The method according to claim 12, characterized in that The electronic device calculates a second direction correction value according to a third direction of a fourth split-screen window of the current camera relative to the display screen, including: If the second screen direction is greater than the second window direction, the second direction correction value is the difference between the second screen direction and the second window direction, wherein the second screen direction is the direction of the current display screen, and the second window direction is the direction of the current fourth split-screen window; If the second screen direction is equal to the second window direction, the second direction correction value is a first preset value, and the first preset value indicates that there is no need to adjust the display direction of the camera; If the second screen direction is smaller than the second window direction, the second direction correction value is the sum of the difference between the second screen direction and the second window direction and the second preset value.

14. An electronic device, characterized in that: The electronic device comprises a processor, and the processor is used to run a computer program stored in a memory, so that the electronic device implements the method according to any one of claims 1 to 7, or any one of claims 8 to 13.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 7, or any one of claims 8 to 13 is implemented.

Images