System and method for providing elements in correct format on user equipment

Abstract

A system for providing an element on a UE including a collapsible portion is provided. The system includes a processor configured to: identify an angular orientation between the first foldable portion and the second foldable portion; detecting an element of an application running on the first foldable portion; identifying whether the first foldable portion includes a sensor based on the element being detected to run on the first foldable portion; determining whether an element is displayed on the first foldable portion based on identifying that the first foldable portion has no sensor; based on determining that the first foldable portion does not have the sensor, fusing data from the sensor with data corresponding to the element of the application; and control the display of the elements based on the fused data to provide a correct format of the at least one element on the first collapsible portion.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to display of location information, and more particularly, to a system and method for providing at least one element in a correct format on a user device. BACKGROUND

[0002] In recent years, electronic devices have evolved. These evolutions have made electronic devices compact, met the needs of users, and made the lives of users more convenient. For example, electronic devices such as smartphones have evolved into foldable phones. The foldable phones are compact in size and provide multiple foldable sections, which provide a larger operating space to the user as per the need. Further, the foldable phones can have inbuilt navigation applications, such as map services. Users can use map services in their day-to-day life to reach a destination, such as a shopping center, a theater, a home, and the like. This increases the dependency of users on navigation applications. This raises the necessity of providing a target determination location on the foldable phone.

[0003] The accurate determination of the target on the foldable phone (smartphone) is provided by multiple sensors, such as ultra-wideband (UWB) sensors, magnetometers, accelerometers, gyroscopes, and inertial measurement units (IMUs). The multiple sensors are used to determine the orientation and movement of the smartphone, thereby accurately determining the location and movement of the target. For example, UWB sensors can be used for indoor positioning. The distance measurement of the UWB sensor employs a time of flight method (time of flight, ToF) rather than measuring the signal strength (received signal strength indication, RSSI) as compared to low-power Bluetooth and Wi-Fi. Further, to position the target, two-way ranging (TWR) can be used to measure the time of flight of the UWB signal between the radio frequencies (RF) of the two smartphones. The TWR can be used in combination with the phase difference of arrival (PDoA) that provides the angle of arrival measurement, which can be used to position the location of the target. Further, the magnetometer can be used to measure the magnetic field of the earth. This sensor helps to provide the orientation of the foldable smartphone or any other device with respect to the magnetic poles of the earth. This sensor can be used with GPS for navigation. However, when the area of movement is small (like indoor navigation), the navigation applications, such as map services, can rely on other sensor data to navigate to the target.

[0004] However, the limitation of the foldable phone is that the multiple sensors responsible for providing the accurate location of the target, particularly the IMU sensors and the magnetometer, exist only on one of the multiple foldable sections. When the foldable phone is in a partially unfolded state, the location of the target displayed on the other foldable section, which is oriented differently from the foldable section having the multiple sensors, becomes inaccurate as the multiple sensors (e.g., accelerometer, gyroscope) consider the values from the IMU, magnetometer, and exist in the other of the multiple foldable sections.

[0005] In addition, due to limitations in space, power consumption, structure, and complexity of the foldable phone, it is not possible to embed multiple sensors into each foldable portion. SUMMARY

[0006] A system and method for providing at least one element in a correct format on a foldable user equipment are provided.

[0007] This summary is provided to introduce some concepts in a simplified format that are further described in the detailed description of the disclosure. This summary is neither intended nor is it to be construed to identify key or essential inventive concepts, nor is it to be used to determine the scope of the disclosure.

[0008] According to an aspect of the disclosure, a method of controlling a user equipment (UE) including a plurality of foldable portions includes identifying a current state of the UE, wherein the current state of the UE includes an angular orientation between a first foldable portion and a second foldable portion of the plurality of foldable portions; detecting at least one element of an application running on the first foldable portion; based on the detecting, identifying whether the first foldable portion includes a plurality of sensors; based on the identifying indicating that the first foldable portion does not have the plurality of sensors, determining whether the at least one element is displayed on the first foldable portion; fusing data from the plurality of sensors with data corresponding to the at least one element of the application; and controlling display of the at least one element based on the fused data to provide a correct format of the at least one element on the first foldable portion.

[0009] The angular orientation between the first foldable portion and the second foldable portion of the plurality of foldable portions can be less than 180 degrees.

[0010] The plurality of sensors can include any one or any combination of an accelerometer sensor, a gyroscope, a magnetometer, an inertial sensor, and an ultra-wideband positioning sensor.

[0011] The at least one element of the application can operate in a multi-window mode on the first foldable portion.

[0012] The method can include, before detecting the at least one element of the application operating on the first foldable portion of the plurality of foldable portions, determining a boundary of the application running on the UE.

[0013] The detecting of the at least one element can include determining a boundary of an application running on the UE, detecting whether the application is running in a multi-window mode or a full-screen mode on the UE based on the boundary of the application, detecting a first coordinate of the application on the UE based on the determined boundary of the application and adding the first coordinate to a second coordinate associated with a relative position of the application, wherein the second coordinate is determined with respect to a center position of a view of the at least one element, and detecting at least one element of the application on the first foldable portion based on a width of the plurality of foldable portions and a number of the plurality of foldable portions.

[0014] The fusing of the data can include determining a rotation matrix based on the angular orientation between the first foldable portion and the second foldable portion, and calculating the rotation matrix based on inputs from the plurality of sensors to fuse the data corresponding to the at least one element of the application.

[0015] According to an aspect of the disclosure, a system for providing at least one element on a UE including a plurality of foldable portions includes a memory, and at least one processor communicatively coupled with the memory and configured to identify a current state of the UE, wherein the current state of the UE includes an angular orientation between a first foldable portion and a second foldable portion of the plurality of foldable portions, detect at least one element of an application running on the first foldable portion, identify whether the first foldable portion includes a plurality of sensors based on the at least one element being detected as running on the first foldable portion, determine whether the at least one element is displayed on the first foldable portion based on identifying that the first foldable portion does not have the plurality of sensors, fuse data from the plurality of sensors with data corresponding to the at least one element of the application based on determining that the first foldable portion does not have the plurality of sensors, and control a display of the at least one element based on the fused data to provide a correct format of the at least one element on the first foldable portion.

[0016] The at least one processor can be further configured to detect the at least one element based on identifying that the angular orientation between the first foldable portion and the second foldable portion of the plurality of foldable portions is less than 180 degrees.

[0017] The plurality of sensors can include any one or any combination of an accelerometer sensor, a gyroscope, a magnetometer, an inertial sensor, and an ultra-wideband positioning sensor.

[0018] The at least one element of the application can operate in a multi-window mode on the first foldable portion.

[0019] The at least one processor can be further configured to determine a boundary of an application running on the UE before detecting the at least one element of the application operating on the first foldable portion of the plurality of foldable portions.

[0020] The at least one processor can be further configured to: determine a boundary of an application running on the UE; detect whether the application is running in a multi-window mode or a full-screen mode on the UE based on the boundary of the application; detect a first coordinate of the application on the UE based on the determined boundary of the application and add the first coordinate to a second coordinate associated with a relative position of the application, wherein the second coordinate is determined with respect to a center position of a view of the at least one element; and detect at least one element of the application on the first foldable portion based on a width of the plurality of foldable portions and a number of the plurality of foldable portions.

[0021] The at least one processor can be further configured to: determine a rotation matrix based on an angular orientation between the first foldable portion and the second foldable portion; and calculate the rotation matrix based on inputs from the plurality of sensors to fuse data corresponding to the at least one element of the application.

[0022] According to an aspect of the disclosure, an apparatus for controlling display of information on a user device including a plurality of foldable portions and a sensor disposed in a first foldable portion of the plurality of foldable portions, the apparatus comprising: a memory; and at least one processor communicatively coupled with the memory and configured to: identify, based on an output of the sensor, whether an application is running on the user device to display a first element; modify, based on the first element being displayed on a second foldable portion of the plurality of foldable portions, a display of the first element based on an angle between the first foldable portion and the second foldable portion.

[0023] The at least one processor can be further configured to: maintain, based on the output of the sensor, a display of a second element on the first foldable portion without modification.

[0024] The at least one processor can be further configured to: move the display of the first element from the second foldable portion to the first foldable portion, and control the display of the first element independently of the angle between the first foldable portion and the second foldable portion based on the first element being displayed on the first foldable portion.

[0025] The output of the sensor can indicate a motion of the user device.

[0026] The output of the sensor can indicate a first orientation corresponding to the first foldable portion.

[0027] The at least one processor can be further configured to: identify, based on the output of the sensor and the angle between the first foldable portion and the second foldable portion, a second orientation corresponding to the second foldable portion.

[0028] The at least one processor can be further configured to: modify the display of the first element continuously based on the second orientation.

[0029] The apparatus can comprise a user equipment. BRIEF DESCRIPTION OF DRAWINGS

[0030] The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0031] Figure 1A , Figure 1B , Figure 1C and Figure 1D depicts a scenario depicting inaccurate target location on a related user equipment (UE);

[0032] Figure 2 depicts a system communicatively coupled with a UE having a plurality of foldable portions, according to an embodiment;

[0033] Figure 3 depicts a block diagram of a system connected with a user equipment, according to an embodiment;

[0034] Figure 4 depicts detection of at least one element of an application on a first foldable portion of a plurality of foldable portions in a UE by a system, according to an embodiment;

[0035] Figure 5 depicts an angle between a first foldable portion of a plurality of foldable portions in a UE not having a plurality of sensors and a second foldable portion having a plurality of sensors, according to an embodiment;

[0036] Figure 6 depicts a rotation matrix based on an angle between a first foldable portion and a second foldable portion of a plurality of foldable portions, according to an embodiment;

[0037] Figure 7 depicts a representation of at least one element of a correct format of an application on a first foldable portion of a plurality of foldable portions of a UE, according to an embodiment;

[0038] Figure 8 depicts a method performed by a system, according to an embodiment; and

[0039] Figure 9A and Figure 9B depicts a plurality of use cases of a system, according to an embodiment. DETAILED DESCRIPTION

[0040] Example embodiments of the present disclosure are described below with reference to the accompanying drawings. It will be understood that such changes and further modifications in the illustrated system and such further applications of the principles of the present disclosure as illustrated therein are contemplated as being within the scope of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The systems and examples provided herein are illustrative only and not restrictive.

[0041] As used herein, the term "some" can be understood to mean "one" or "more than one" or "all". Accordingly, the terms "one", "more than one", "more than one but not all" or "all" will fall within the definition of "some". It will be understood by those within the art that the terms and expressions used herein are intended to describe, teach and illustrate some embodiments and are not meant to limit, restrict or narrow the scope of the present disclosure.

[0042] Unless otherwise stated, the use of terms herein such as "comprise", "comprising", "comprises", "include", "including", "includes", "have", "has", "having", or variants thereof, is not to be construed as limiting or restricting or excluding the possibility of adding one or more features or elements.

[0043] Unless otherwise defined, all terms used herein, including technical and / or scientific terms, can be interpreted as having the same meaning as commonly understood by one of ordinary skill in the art.

[0044] Reference is made herein to some "embodiments". It will be understood that the embodiments are examples of possible implementations of any feature and / or element of the present disclosure. Some embodiments are described in order to illustrate multiple potential ways in which a particular feature and / or element of the present disclosure can satisfy the requirements of novelty, utility, and non-obviousness.

[0045] The use of phrases and / or terminology including, but not limited to, “first embodiment,” “another embodiment,” “alternative embodiment,” “one embodiment,” “embodiment,” “embodiments,” “some embodiments,” “other embodiments,” “further embodiments,” “still further embodiments,” “additional embodiments,” or other variations thereof, does not necessarily refer to the same embodiment, unless otherwise noted. Unless otherwise stated, multiple specific features and / or elements described in connection with a number of embodiments can be present in one embodiment, or in more than one embodiment, or in all embodiments, or can be absent in any embodiment. While a number of features and / or elements can be described in the context of only a single embodiment or a number of embodiments, these need not all be present in only one embodiment or in more than one embodiment, or in all embodiments, or need not be absent in any embodiment. Conversely, any feature and / or element described in the context of separate embodiments can also be implemented together in a single embodiment.

[0046] Any specific details and all the details set forth herein are for some embodiments of the present disclosure and, therefore, should not be thought of as limiting the present disclosure.

[0047] Embodiments will be described in detail below with reference to the attached drawings.

[0048] Figure 1A , Figure 1B , Figure 1C And Figure 1D depicts a scenario where the location of a target is inaccurate on a related user equipment (UE). As multiple sensors exist on one of the foldable portions, the location of the target displayed on another foldable portion is shown in the wrong direction. That is, the location displayed on another foldable portion is an inaccurate location. For example, if a foldable phone is unfolded about 20 degrees, and a navigation application instructs the user to go north, north can be directly in front of the user. The multiple sensors on another foldable portion of the foldable phone would identify the direction of true north as 20 degrees to the left. As a result, the user would be directed to move diagonally to the left, rather than being instructed to proceed in a straight line, providing an inaccurate location. As Figure 1D depicted, this configuration causes additional drift in providing the location of the target to the user, which is inaccurate, misleading the user.

[0049] Figure 2 depicts an environment 200 of a system 204 communicatively coupled with a user equipment (UE) 202 having multiple foldable portions 206, according to an embodiment. Figure 3 depicts a block diagram 300 of a system 204 connected with a user equipment 202, according to an embodiment. Figure 4It is shown that at least one element of the application on the first foldable portion of the plurality of foldable portions 206 is detected by the system 204 in the UE 202, according to an embodiment. Figure 5 It is shown that the angle between the first foldable portion of the plurality of foldable portions 206 without a plurality of sensors and the second foldable portion with a plurality of sensors, according to an embodiment. Figure 6 It is shown that a rotation matrix based on the angle between the first foldable portion and the second foldable portion of the plurality of foldable portions 206, according to an embodiment. Figure 7 It is shown that at least one element of the application in the correct format on the first foldable portion of the plurality of foldable portions 206 of the UE 202, according to an embodiment.

[0050] In an embodiment, the user equipment 202 can be a foldable smartphone or any other foldable electronic device having a navigation application, a virtual reality application, or an augmented reality application, without departing from the scope of the present disclosure. In an embodiment, the user equipment 202 includes a plurality of foldable portions 206. In an embodiment, at least one of the plurality of foldable portions 206 can have a display portion, without departing from the scope of the present disclosure.

[0051] Further, the system 204 is configured to provide the at least one element in the correct format on the UE 202, without departing from the scope of the present disclosure.

[0052] In an embodiment, the system 204 can include, but is not limited to, at least one processor (herein referred to as one or more processors and / or processor) 304, a memory 308, and a plurality of modules (i.e., circuitry) 312, among other examples detailed in subsequent paragraphs.

[0053] The system 204 can include an input / output (I / O) interface 328, a transceiver (i.e., transmitter and receiver) 326, and a window manager (i.e., window management circuitry) 306. Further, in some embodiments, the system 204 can be implemented as a standalone entity in a server / cloud architecture, and the system 204 can communicate with a plurality of user equipment to receive data from each of the plurality of user equipment, the details provided below for the system 204 and the user equipment 202 also apply to the system 204 and the plurality of user equipment.

[0054] In an embodiment, the processor 304 can be operatively coupled to each of the I / O interface 328, the plurality of modules 312, the transceiver 326, and the memory 308. In one embodiment, the processor 304 can include a graphics processing unit (GPU) and / or an artificial intelligence (AI) engine (AIE). In one embodiment, the processor 304 can include at least one data processor for performing processing in a virtual storage area network. The processor 304 can include a special purpose processing unit (i.e., a special purpose processing circuit), such as an integrated system (bus) controller, a memory management control unit (i.e., a memory management controller), a floating point unit (i.e., a floating point circuit), a graphics processing unit (i.e., a graphics processor), a digital signal processing unit (i.e., a digital signal processor), etc. In one embodiment, the processor 304 can include a central processing unit (i.e., a central processor) (CPU), a graphics processing unit (i.e., a graphics processor) (GPU), or both. The processor 304 can be one or more general processors, digital signal processors, application specific integrated circuits, field programmable gate arrays, servers, networks, digital circuitry, analog circuitry, combinations thereof, or other devices for analyzing and processing data. The processor 304 can execute software programs, such as a manually authored (i.e., programmed) code, to perform desired operations.

[0055] The processor 304 can be configured to communicate with one or more input / output (I / O) devices through the I / O interface 328. In some embodiments, the processor 304 can communicate with the UE 202 using the I / O interface 328. In some embodiments, the I / O interface 328 can be implemented within the user equipment 202. The I / O interface 328 can employ communication technologies such as code division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long term evolution (LTE), WiMax, etc. In an embodiment, the I / O interface 328 can enable input to and output from the system 204 using suitable devices such as, but not limited to, a display, a keyboard, a mouse, a touchscreen, a microphone, a speaker, etc.

[0056] Using the I / O interface 328, the system 204 can communicate with one or more I / O devices (specifically, the user equipment 202) to which the system 204 provides at least one element in a correct format. For example, input devices can include an antenna, a microphone, a touchscreen, a touchpad, a storage device, a transceiver, a video device / source, etc. Output devices can include a video display (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), a light-emitting diode (LED), a plasma display, a plasma display panel (PDP), an organic light-emitting diode display (OLED), etc.), an audio speaker, etc.

[0057] The processor 304 can be configured to communicate with a communication network through a network interface. In an embodiment, the network interface can be the I / O interface 328. The network interface can be connected to a communication network to enable the connection of the system 204 with the UE 202. The network interface can employ a variety of connection protocols including, but not limited to, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.11a / b / g / n / x, etc. The communication network can include, but is not limited to, direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Using the network interface and the communication network, the system 204 can communicate with other devices. The network interface can employ a variety of connection protocols including, but not limited to, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.11a / b / g / n / x, etc.

[0058] The transceiver 326 can be configured to receive signals from and / or transmit signals to the UE 202. In an embodiment, the database can be configured to store information required by the plurality of modules 312 and the processor 304 to perform one or more functions for providing at least one element on the UE 202 in the correct format.

[0059] In some embodiments, the memory 308 can be communicatively coupled with the processor 304. The memory 308 can be configured to store data and instructions executable by the processor 304. In an embodiment, the memory 308 can be disposed within the UE 202. In another embodiment, the memory 308 can be disposed within the system 204 that is remote from the UE 202. In yet another embodiment, the memory 308 can be in communication with the processor 304 through a bus within the system 204. In yet another embodiment, the memory 308 can be located at a location remote from the processor 304 and can be in communication with the processor 304 through a network. The memory 308 can include, but is not limited to, non-transitory computer readable storage media such as various types of volatile and non-volatile storage media including, but not limited to, random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media, etc.

[0060] In one example, the memory 308 can include a cache or random access memory for the processor 304. In alternative examples, the memory 308 is separate from the processor 304, such as the cache memory of the processor, the system memory, or other memory. The memory 308 can be an external storage device or database for storing data. The memory 308 can operate as a database module for storing instructions executable by the processor 304. The functions, acts or tasks illustrated in the figures or described can be performed by the programmed processor 304 operating the instructions stored in the memory 308. The functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and can be performed by software, hardware, integrated circuits, firm ware, micro-code and the like, operating alone or in combination or conjunction with each other. Likewise, processing strategies can include multiprocessing, multitasking, parallel processing and the like.

[0061] In some embodiments, the plurality of modules 312 can be included within the memory 308. The memory 308 can also include a database to store data. The plurality of modules 312 can include a set of instructions that can be executed to cause the system 204, particularly the processor 304 of the system 204, to perform any one or more of the methods / processes disclosed herein. The plurality of modules 312 can be configured to perform the operations of the present disclosure using data stored in the database. For example, the plurality of modules 312 can be configured to perform the operations disclosed in the Figures 4 to 7

[0062] In embodiments, each of the plurality of modules 312 can be hardware that can be located outside of the memory 308. Further, the memory 308 can include an operating system for performing one or more tasks of the system 204, as performed by a general purpose operating system.

[0063] In one example, the modules 312 can include an identifying module (i.e., identifying circuitry) 314, a determining module (i.e., determining circuitry) 316, a detecting module (i.e., detecting circuitry) 318, a computing module (i.e., computing circuitry) 320, a fusing module (i.e., fusing circuitry) 322, and a modifying module (i.e., modifying circuitry) 324. The identifying module 314, the determining module 316, the detecting module 318, the computing module 320, the fusing module 322, and the modifying module 324 can each be in communication with each other. Further, the identifying module 314, the determining module 316, the detecting module 318, the computing module 320, the fusing module 322, and the modifying module 324 can each be in communication with the processor 304.

[0064] ​Further, the present disclosure relates to a computer readable medium comprising instructions or receiving and executing instructions in response to a propagated signal. Further, the instructions can be sent or received over a network using a communication port or interface or using a bus. The communication port or interface can be part of the processor 304 or can be a separate component. The communication port can be implemented using hardware.

[0065] The communication port can be configured to connect with a network, an external medium, a display, or any other component in the system or a combination thereof. The connection with the network can be a physical connection (e.g., wired Ethernet connection) or can be established wirelessly. Likewise, the additional connections with other components of the system 204 can be physical connections or can be established wirelessly. The network can alternatively be connected directly to the bus. The architecture and standard operation of the memory 308, the processor 304, the transceiver 326, and the I / O interface 328 are not discussed in detail for brevity.

[0066] Further, in an embodiment, the working of the system 204 to provide at least one element on the UE 202 in a correct format will be discussed in subsequent paragraphs in conjunction with Figures 3 to 7 described in detail.

[0067] In conjunction with the identification module 314, the determination module 316, the detection module 318, the computation module 320, the fusion module 322, and the modification module 324, the processor 304 can be configured to perform specific operations explained in subsequent paragraphs.

[0068] In an embodiment, the identification module 314 can be configured to identify a current state of the UE 202. In an embodiment, the current state of the UE 202 includes an angular orientation between the first foldable portion and the second foldable portion of the plurality of foldable portions 206. For example, without departing from the scope of the present disclosure, the angular orientation between the first foldable portion and the second foldable portion of the plurality of foldable portions 206 can be identified to be less than 180 degrees.

[0069] In an embodiment, upon identifying the current state, the window manager 306 can be configured to determine coordinates / boundaries of an application running on the UE 202. In another embodiment, without departing from the scope of the present disclosure, the window manager 306 can be configured to provide coordinates / boundaries of a view. Without departing from the scope of the present disclosure, the view is an area on the UE 202 within a window of the application where data from the application is projected. Further, in an embodiment, upon determining the coordinates / boundaries, based on the identification, the detection module 318 can be configured to determine at least one element of the application displayed on the first foldable portion from the plurality of foldable portions 206.

[0070] In this embodiment, the detection module 318 can be configured to detect that the application is running on the UE 202 in at least a multi-window mode or a full-screen mode, based on the application boundaries determined by the window manager 306. In this embodiment, without departing from the scope of this disclosure, detection of the application running in at least a multi-window mode or a full-screen mode occurs when multiple collapsible portions 206 are in an expanded state. In another embodiment, the detection module 318 can be configured to detect that the application is running on the UE 202 in at least a multi-window mode or a full-screen mode, based on the view boundaries determined by the window manager 306.

[0071] Furthermore, the detection module 318 can detect (e.g., based on the determined application boundaries) Figure 4 (As shown in the box in the image) The starting coordinates x1 and y1 applied to UE 202 (reference) Figure 4 In another embodiment, without departing from the scope of this disclosure, the detection module 318 detects the starting coordinates x1, y1 (see reference) applied to the UE 202 based on the determined boundary of the view. Figure 4 In an embodiment, without departing from the scope of this disclosure, coordinates x1 and y1 may be provided by a view attribute application. In an embodiment, without departing from the scope of this disclosure, x1 represents the left coordinate of the application, and y1 represents the top coordinate of the application. Furthermore, the detected coordinates x1 and y1 may be associated with coordinates x2 and y2 relative to the application's position (see reference). Figure 4 Add them together, such as ( The coordinates x2 and y2 relative to the application's position can be determined relative to the center position of the view of at least one element of the application. In embodiments, without departing from the scope of this disclosure, the coordinates x2 and y2 can also be determined based on the view's margin and padding.

[0072] The detection module 318 can also detect at least one element of the application on the first foldable portion of the plurality of foldable portions 206 by calculating the sum of the widths and the number of foldable portions together. This detection can be performed based on Formula 1 provided below:

[0073]

[0074] Formula 1

[0075] In an embodiment, the plurality of foldable portions 206 can also be referred to as a plurality of display portions without departing from the scope of the disclosure. In an embodiment, the width of the screen represents the total width of the plurality of foldable portions 206 without departing from the scope of the disclosure. In one example, the relationship between the coordinates and the detection of the plurality of foldable portions 206 is shown in Table 1:

[0076]

[0077] Table 1

[0078] Further, in an embodiment, at least one element of the application can operate in a multi-window mode on the first foldable portion.

[0079] In an embodiment, based on the detection result, the identification module 314 identifies whether the first foldable portion (i.e., the portion displaying at least one element of the application) includes a plurality of sensors. In an embodiment, the plurality of sensors includes at least one of an accelerometer sensor, a gyroscope, a magnetometer, an inertial measurement unit (IMU) sensor, and an ultra-wideband (UWB) sensor.

[0080] In an embodiment, the identification module 314 identifies that the first foldable portion does not have a plurality of sensors (as shown in FIG. 4B), and the determination module 316 determines that at least one element of the application is not operating (i.e., displayed) on the first foldable portion (as shown in FIG. 4C). Figure 4 Figure 4 In an embodiment, the determination module 316 determines that at least one element of the application is displayed on the first foldable portion (as shown in FIG. 4D), and the first foldable portion does not have a plurality of sensors (i.e., the plurality of sensors are disposed in a different foldable portion). Figure 4

[0081] Further, in an embodiment, each of the plurality of foldable portions 206 can include a hinge sensor configured to indicate the folding of each foldable portion, which can be used to identify one or more folding angles (as shown in FIG. 4E) without departing from the scope of the disclosure. Figure 5 For example, the plurality of foldable portions 206 can include N portions, and the plurality of sensors can be on the Nth foldable portion. The angle is the angle formed between the first foldable portion and the second foldable portion. Similarly, the angle is formed between the second foldable portion and the third foldable portion, and so on. Then, the angle between the Nth foldable portion and the Mth foldable portion (where M < N and M > N are included) will be:

[0082]

[0083] Equation 2​​

[0084] In an embodiment, M denotes the first foldable portion and N denotes the second foldable portion. In an embodiment, Θ denotes the hinge sensor angle between each of the plurality of foldable portions 206 shown in Table 2 below:

[0085]

[0086] Table 2

[0087] In an embodiment, a denotes the angle between the first foldable portion without the plurality of sensors and the second foldable portion with the plurality of sensors. In another embodiment, the above equation 2 can be used / applied in any order of the plurality of foldable portions 206 accordingly.

[0088] In an embodiment, based on the determination and identification of the angle between the first foldable portion without the plurality of sensors and the second foldable portion with the plurality of sensors, the fusion module 322 can fuse the data from the plurality of sensors to at least one element of the application displayed on the foldable portion, wherein the at least one element operates on the foldable portion. In an embodiment, the at least one element of the application can operate on the first foldable portion. Specifically, the determination module 316 determines the rotation matrix based on the identified angle orientation between the first foldable portion and the second foldable portion as shown in Figure 6 In addition, the computation module 320 can be configured to compute the rotation matrix with the input from the plurality of sensors. Subsequently, the fusion module 322 can fuse the data from the second foldable portion corresponding to the at least one element of the application operating on the first foldable portion based on the computation result. For better understanding of the determination of the rotation matrix, an example is provided in the subsequent paragraphs.

[0089] In one example, the plurality of foldable portions 206 rotates around the Y-axis, thus the rotation matrix is determined for the rotation around the Y-axis as well. In addition, according to equation 2, the determination / identification of is made, thus for the rotation matrix, the angle between the negative X-axis of the current foldable portion and the plurality of sensors is 180 - a. In addition, the rotation matrix can be given by equation 3 as follows:

[0090]

[0091] =

[0092] Equation 3

[0093] In addition, the updated value of the plurality of sensors can be provided by equation 4 as follows:

[0094]

[0095] as well as

[0096]

[0097] Formula 4

[0098] In addition, after determining the required value according to the above formula, the following relevant table is provided:

[0099]

[0100] Table 3

[0101] Determining the rotation matrix helps to determine and transform values ​​from the first collapsible portion to the second collapsible portion.

[0102] In this embodiment, based on the fused data, the modification module 324 modifies at least one element to provide the correct format of at least one element on the first foldable portion of the plurality of foldable portions 206 of the UE 202 in the current state of the UE 202, such as... Figure 7 As shown. In the embodiments, without departing from the scope of this disclosure, Figure 7 A representation of at least one element of the correct format applied on the first foldable portion of the plurality of foldable portions 206 of the UE 202 is shown.

[0103] Figure 8 A method 800 performed by system 204 according to an embodiment is shown.

[0104] Method 800 can be executed by a programmed computing device, for example, based on instructions retrieved from a non-transitory computer-readable medium. The computer-readable medium may include machine-executable instructions or computer-executable instructions to perform all or part of the described method. The computer-readable medium may be, for example, a digital storage device, a magnetic storage medium (e.g., disks and magnetic tapes), a hard disk drive, or an optically readable data storage medium.

[0105] Method 800 includes Figure 8 The series of operations shown in operations 802 to 812. Method 800 can be executed by system 204 in conjunction with module 312, the details of which are related to module 312. Figures 3 to 7 The details have been explained, and for the sake of brevity, will not be repeated here. Method 800 begins with operation 802.

[0106] At operation 802, the method 800 includes identifying a current state of the UE 202. The UE includes a plurality of foldable portions 206. Further, the current state of the UE 202 includes an angular orientation between a first foldable portion and a second foldable portion of the plurality of foldable portions 206. The angular orientation between the first foldable portion and the second foldable portion of the plurality of foldable portions 206 is identified to be less than 180 degrees.

[0107] The method 800 includes determining, by the window manager 306, coordinates / bounds of an application running on the UE 202, prior to detecting at least one element of the application operating on the first foldable portion of the plurality of foldable portions 206. Further, once the coordinates / bounds are determined, at operation 804, the method 800 includes detecting, based on the identification, at least one element of the application running on the first foldable portion of the plurality of foldable portions 206.

[0108] Further, the method 800 includes detecting, based on the bounds of the application determined by the window manager 306, that the application is running on the UE 202 in a multi-window mode or a full screen mode. For example, it can be detected that the at least one element of the application is operating on the first foldable portion in the multi-window mode. The method 800 includes detecting, based on the determined bounds of the application, a starting coordinate xl, yl of the application on the UE 202. Further, the detected coordinate xl, yl and a coordinate x2, y2 associated with a relative position of the application can be added as (x1+x2, y1+y2). The coordinate x2, y2 of the relative position of the application can be determined with respect to a center position of a view of the at least one element of the application. In an embodiment, the coordinate x2, y2 can also be determined based on an outer margin and an inner margin of the view, without departing from the scope of the present disclosure. The method 800 includes detecting the at least one element of the application on the first foldable portion of the plurality of foldable portions 206 by computing the above sum together with a width of the plurality of foldable portions 206 and a number of the plurality of foldable portions 206.

[0109] At operation 806, the method 800 includes identifying, based on the detection, whether the first foldable portion includes a plurality of sensors. In an embodiment, the plurality of sensors includes at least one of an accelerometer sensor, a gyroscope, a magnetometer, an inertial measurement unit (IMU) sensor, and an ultra-wideband (UWB) sensor.

[0110] At operation 808, the method 800 includes determining whether the at least one element of the application is operating on the first foldable portion, wherein, based on the identification, the first foldable portion of the plurality of foldable portions 206 does not have the plurality of sensors.

[0111] At operation 810, the method 800 includes fusing data from the plurality of sensors of the second foldable portion corresponding to the at least one element of the application operating on the first foldable portion. The method 800 includes determining a rotation matrix based on the identified angular orientation between the first foldable portion and the second foldable portion. The method 800 includes computing the rotation matrix using inputs from the plurality of sensors. The method 800 includes fusing the data from the second foldable portion corresponding to the at least one element of the application operating on the first foldable portion based on the computation.

[0112] At operation 812, the method 800 includes modifying the at least one element based on the fused data to provide a correct format of the at least one element on the first foldable portion of the plurality of foldable portions 206 of the UE 202 in the current state of the UE 202.

[0113] Figure 9A and Figure 9B A number of use cases are illustrated in which the system 204 provides the at least one element in a correct format on the UE 202 without deviating from the scope of the present disclosure, in accordance with an embodiment.

[0114] Referring to Figure 9A , the UE 202 can be adapted to accurately locate a lost item (e.g. a smart tag) within a venue, unlike related configurations, thereby providing convenience to a user using the UE 202. As Figure 9A illustrated, the UE 202 can display an image of the area in a correct format on one of the foldable units along with the at least one element 910 indicating the location of the smart tag. Further, unlike related configurations, the present subject matter also facilitates accurate location of a target within a venue such as a shopping mall on the UE 202, thereby ensuring convenience to the user.

[0115] Further, referring to Figure 9B , the at least one element displayed in a correct format on the UE 202 can be adapted to accurately locate a target in a map by displaying a correct path and eliminating drift, unlike related configurations that display an incorrect path, thereby ensuring convenience to the user.

[0116] As mentioned earlier, the disclosed system 204 and method 800 provide a comprehensive approach to provide the at least one element in a correct format on the UE 202. The present configuration provides the at least one element on the first foldable portion of the plurality of foldable portions 206 that does not have the plurality of sensors, thereby ensuring correct format of the at least one element on the plurality of foldable portions 206 irrespective of the presence of the plurality of sensors on one of the plurality of foldable portions 206. The configuration provides accurate location of a target to the user, thereby ensuring convenience to the user.

[0117] In some embodiments, each component represented by a block as Figure 3 shown can be implemented to perform the corresponding functions using one or more hardware and / or software structures in accordance with the exemplary embodiments. For example, at least one of these components can include various hardware components that can perform the corresponding functions under the control of one or more microprocessors or other control devices, including digital circuitry, programmable or non-programmable logic devices or arrays, application-specific integrated circuits (ASICs), transistors, capacitors, logic gates, or other circuitry employing direct circuitry structures, such as memory, processors, logic circuitry, look-up tables, etc. In addition, at least one of these components can also include or can be implemented as a processor, such as a central processing unit (CPU), microprocessor, etc., that performs the various functions. The functional aspects of the exemplary embodiments can be implemented as algorithms executed on one or more processors. Furthermore, the components, elements, modules or units represented by blocks or processing steps can employ any number of related art techniques for electronics configuration, signal processing and / or control, data processing, and the like.

[0118] While various aspects of embodiments have been particularly shown and described, it will be understood that various changes in form and details can be made therein without departing from the spirit and scope of the claims.

Claims

1. A method for controlling a user equipment (UE) comprising a plurality of foldable sections, the method comprising: Identify the current state of the UE, wherein the current state of the UE includes the angular orientation between the first foldable portion and the second foldable portion among the plurality of foldable portions; Detect at least one element of the application running on the first foldable portion; Based on the detection, it is determined whether the first foldable portion includes multiple sensors; Based on the identification indication that the first foldable portion does not have the plurality of sensors, it is determined whether the at least one element is displayed on the first foldable portion; Fusing data from the plurality of sensors with data corresponding to at least one element of the application; and The display of at least one element is controlled based on the fused data to provide the correct format of the at least one element on the first foldable portion.

2. The method according to claim 1, wherein, The angular orientation between the first foldable portion and the second foldable portion of the plurality of foldable portions is less than 180 degrees.

3. The method according to claim 1, wherein, The plurality of sensors include any one or any combination of accelerometer sensors, gyroscopes, magnetometers, inertial sensors, and ultra-wideband positioning sensors.

4. The method according to claim 1, wherein, The at least one element of the application operates in a multi-window mode on the first collapsible portion.

5. The method according to claim 1, further comprising: Before detecting at least one element of the application operating on the first foldable portion of the plurality of foldable portions, the boundaries of the application running on the UE are determined.

6. The method according to claim 1, wherein, Detecting the at least one element includes: Determine the boundaries of the application running on the UE; Based on the application's boundaries, detect whether the application is running in multi-window mode or full-screen mode on the UE; Based on the defined boundary of the application, a first coordinate of the application on the UE is detected, and the first coordinate is added to a second coordinate associated with the relative position of the application, wherein the second coordinate is determined relative to the center position of the view of the at least one element; and Based on the width of the plurality of foldable portions and the number of the plurality of foldable portions, the application detects at least one element on the first foldable portion.

7. The method according to claim 1, wherein, Data fusion includes: Based on the angular orientation between the first foldable portion and the second foldable portion, a rotation matrix is ​​determined; and The rotation matrix is ​​calculated based on inputs from the plurality of sensors to fuse data corresponding to at least one element of the application.

8. A system for providing at least one element on a user equipment (UE) comprising a plurality of foldable portions, the system comprising: Memory; as well as At least one processor, communicatively coupled to the memory, is configured to: Identify the current state of the UE, wherein the current state of the UE includes the angular orientation between the first foldable portion and the second foldable portion among the plurality of foldable portions; Detect at least one element of the application running on the first foldable portion; Based on the detection that at least one element is operating on the first foldable portion, it is identified whether the first foldable portion includes multiple sensors; Based on the identification that the first foldable portion does not have the plurality of sensors, it is determined whether the at least one element is displayed on the first foldable portion; Based on the determination that the first foldable portion does not have the plurality of sensors, data from the plurality of sensors are fused with data corresponding to at least one element of the application; and The display of at least one element is controlled based on the fused data to provide the correct format of the at least one element on the first foldable portion.

9. The system according to claim 8, wherein, The at least one processor is further configured to detect the at least one element based on the identification that the angular orientation between the first foldable portion and the second foldable portion of the plurality of foldable portions is less than 180 degrees.

10. The system according to claim 8, wherein, The plurality of sensors include any one or any combination of accelerometer sensors, gyroscopes, magnetometers, inertial sensors, and ultra-wideband positioning sensors.

11. The system according to claim 8, wherein, The at least one element of the application operates in a multi-window mode on the first collapsible portion.

12. The system according to claim 8, wherein, The at least one processor is further configured to: determine the boundaries of the application running on the UE before detecting at least one element of the application operating on the first foldable portion of the plurality of foldable portions.

13. The system according to claim 8, wherein, The at least one processor is further configured to: Determine the boundaries of the application running on the UE; Based on the application's boundaries, detect whether the application is running in multi-window mode or full-screen mode on the UE; Based on the defined boundary of the application, a first coordinate of the application on the UE is detected, and the first coordinate is added to a second coordinate associated with the relative position of the application, wherein the second coordinate is determined relative to the center position of the view of the at least one element; as well as Based on the width of the plurality of foldable portions and the number of the plurality of foldable portions, the application detects at least one element on the first foldable portion.

14. The system according to claim 8, wherein, The at least one processor is further configured to: A rotation matrix is ​​determined based on the angular orientation between the first foldable portion and the second foldable portion; The rotation matrix is ​​calculated based on inputs from the plurality of sensors to fuse data corresponding to at least one element of the application.

15. An apparatus for controlling the display of information on a user equipment, the user equipment comprising a plurality of foldable portions and a sensor disposed in a first foldable portion of the plurality of foldable portions, the apparatus comprising: Memory; as well as At least one processor, communicatively coupled to the memory, is configured to: Based on the output of the sensor, it is determined whether an application is running on the user device to display the first element; Based on the fact that the first element is displayed on the second foldable portion of the plurality of foldable portions, the display of the first element is modified based on the angle between the first foldable portion and the second foldable portion.

Images