Electronic apparatus

Abstract

To provide an electronic apparatus including a display surface with flexibility for displaying an object (body) on the display surface in accordance with a three-dimensional shape of the display surface.SOLUTION: An electronic apparatus includes a display unit including a display device with flexibility for displaying an object on a display surface, a detection unit that detects the positional information of a predetermined place on the display surface, and a calculation unit that calculates the three-dimensional shape of the display surface on the basis of the positional information, and calculates the operation of the object so that the object moves following a predetermined rule in accordance with the calculated three-dimensional shape of the display surface.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an electronic device having a flexible display surface, a recording medium having a program for displaying the electronic device recorded thereon, and a recording medium having a program for displaying the electronic device recorded thereon. The present invention relates to a recording medium, a program for displaying the medium, and a display method thereof. [Background technology]

[0002] In recent years, portable information terminals such as mobile phones and smartphones, portable music players, and portable games have become increasingly High-performance portable consumer electronic devices equipped with display devices, such as smartphones, have become widespread, and we modern people It has come to completely change people's lives.

[0003] The development of such portable electronic devices is progressing further, and in addition to the functionality of the devices such as multi-function, Developments in external dimensions, such as smaller size and lighter weight, are also active. As a result, sheet-like flexible electronic devices have been actively researched and developed.

[0004] The electronic book disclosed in Patent Document 1 has a flexible housing, and the electronic book can be easily bent. The position sensor and the bending sensor detect the part that is being By switching the display content on the display unit based on the detection results of the bending sensor, Taking advantage of the flexibility of the housing, the user can operate the display without using a mouse or buttons. This is what is done. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-157060 Summary of the Invention [Problem to be solved by the invention]

[0006] On the other hand, in such flexible portable electronic devices, Even if the operation is possible, the images or videos displayed there are flat, and the display surface and The flexibility of the electronic devices that incorporate it is not being fully utilized. Therefore, the display surface is flexible and the product is compatible with electronic devices that have various functions. There is a lack of synergy in terms of planning, and electronic devices with flexible displays lack appeal to users. can.

[0007] Therefore, one aspect of the present invention is to utilize the flexibility of the display surface to display an object on the display surface. To provide an electronic device that displays an object in accordance with the three-dimensional shape of a display surface. This will be the challenge.

[0008] Furthermore, one aspect of the present invention is to provide a display screen that is flexible, and thereby to allow objects displayed on the display screen to be displayed in a flexible manner. The object of the present invention is to provide a program that displays an object in accordance with the three-dimensional shape of the display surface. do. [Means for solving the problem]

[0009] Therefore, one aspect of the configuration of the invention disclosed in this specification is to display an object on a display screen, A display unit including a flexible display device and a detection unit for detecting position information of a predetermined location on the display surface. and a display unit that calculates the three-dimensional shape of the display surface based on the position information and displays the three-dimensional shape of the display surface in accordance with the calculated three-dimensional shape of the display surface. The object's movement is calculated so that it follows a predetermined law. and a calculation unit.

[0010] Furthermore, one embodiment of the configuration of the invention disclosed in this specification is a display device for displaying an object on a display surface. A first device for detecting position information of a predetermined point on a display surface of an electronic device including a flexible display device. a second step of calculating a three-dimensional shape of the display surface based on the position information; The object is controlled to move according to a predetermined rule according to the three-dimensional shape of the display surface. A third step of calculating the movement of the object, and displaying the calculated movement of the object on the display surface. a fourth step of displaying the program; and a computer readable medium recording a program for executing the program. It is a readable recording medium.

[0011] Furthermore, one embodiment of the configuration of the invention disclosed in this specification is a display device for displaying an object on a display surface. A first device for detecting position information of a predetermined point on a display surface of an electronic device including a flexible display device. a second step of calculating a three-dimensional shape of the display surface based on the position information; The object is controlled to move according to a predetermined rule according to the three-dimensional shape of the display surface. A third step of calculating the movement of the object, and displaying the calculated movement of the object on the display surface. The fourth step is to display the program.

[0012] Furthermore, one embodiment of the configuration of the invention disclosed in this specification is a display device for displaying an object on a display surface. Using an electronic device including a flexible display device, position information of a predetermined location on a display surface is detected. , calculates the three-dimensional shape of the display surface based on the position information, and , Calculate the motion of the object so that the object moves according to a predetermined law, and This is a display method for displaying the movement of a selected object on a display surface.

[0013] The electronic device according to the present invention is, for example, a mobile phone terminal, a PHS, a smartphone, a personal computer, a Computers, Personal Digital Assistants (PDAs), Tablet PCs, Laptops Laptops, minicomputers, electronic books (electronic paper), electronic dictionaries, electronic organizers, calculators, Navigation systems, digital photo frames, image playback devices, IC recorders, portable It is a portable electronic device such as a game console.

[0014] A flexible display surface and an electronic device equipped with the same. A detection unit in which a plurality of sensors are arranged in a matrix, for example, is provided directly below the display device. The position information detected by each sensor is combined to calculate the curved shape of the display. The object is displayed according to a predetermined rule according to the curved shape of the display surface (the three-dimensional shape of the display surface). It displays images so that they work.

[0015] The three-dimensional shape of the display surface refers to the shape of the electronic device or display device that is curved, bent, or the like when an external force is applied to the electronic device or display device. It refers to the physical shape of the deformed display surface, and is a shape that can be described in three-dimensional spatial coordinates. This includes the two-dimensional shape (i.e., flat, uncurved shape) before and after deformation. can be.

[0016] The object behaves in accordance with a predetermined rule according to the three-dimensional shape of the display surface, for example, When the display surface is bent downward, the object displayed on the display surface is displaced by gravity. Virtual forces move objects on the display surface in a way that mimics the dynamics of the natural world. The object is to make it appear to move. The object is a dice, a fallen leaf, a marble, etc. It can be a solid such as water, or a liquid such as water. It can also be gas, powder, or anything else in the universe. This allows the object to be defined as a target on the surface of an electronic device. This allows the user to intuitively experience the reality of the object as if it were actually there. do.

[0017] The object may also be a living thing. For example, in an electronic game, The characters can be humans, animals, plants, fictional or fantastical creatures, etc.

[0018] A predetermined law is a law that can be described by a predetermined equation. For example, it refers to the physical laws of nature. , especially the laws that govern the motion of objects. For example, gravity, gravitational force, friction force, air resistance, rigid body - Laws that can be described as equations of motion that allow the dynamics of fluids to be visually understood intuitively. However, the predetermined laws are not limited to the physical laws of nature. For example, For added effect, they may exaggerate deviations from the physical laws of nature, It may also be something that goes against the laws of nature, such as floating against gravity. The formula does not necessarily have to strictly embody the physical laws of nature, but can be a pseudo-formula. Formulas or simplified equations may be used.

[0019] The detection unit is configured to detect a plurality of predetermined positions in order to calculate the three-dimensional shape of the display screen of the electronic device. For example, a plurality of position sensors are provided near the display surface, and a matrix sensor is provided. Alternatively, the sensors may be arranged in a grid to detect relative position coordinates. As a sensor, multiple acceleration sensors are arranged in a matrix near the display surface. The sensor may be configured to detect the relative change in acceleration of each part that accompanies the movement. If it can detect the parameters necessary to calculate the three-dimensional shape of the display surface, Sensors that utilize mechanical, electromagnetic, thermal, acoustic, chemical, etc. may be used. For example, sensors include acceleration sensors, angular velocity sensors, vibration sensors, pressure sensors, and A color sensor or the like can be used. In addition, a combination of these sensors can also be used. good.

[0020] The present invention may be expressed as a method, hardware (electronic equipment, computer, semiconductor device, recording Any conversion between the above-mentioned media, systems, programs, software, etc. is also included in the present invention. This is an effective embodiment. [Effects of the Invention]

[0021] By using the display surface that is flexible, the image is displayed on the display surface in accordance with the three-dimensional shape of the display surface. An electronic device that displays an object can be provided.

[0022] In addition, since the display surface is flexible, the image can be displayed on the display surface in accordance with the three-dimensional shape of the display surface. A program can be provided that displays the objects displayed.

[0023] This allows the object displayed on the display surface to appear as if it were actually present on the surface of the electronic device. UI (User Interface) that allows users to intuitively experience reality as if they were (service) can be provided. [Brief explanation of the drawings]

[0024] [Figure 1] 1A to 1C illustrate one embodiment of an electronic device. [Figure 2] FIG. 1 is a block diagram illustrating one embodiment of a hardware configuration of an electronic device. [Figure 3]FIG. 2 is a block diagram illustrating the configuration of a memory. [Figure 4] FIG. 1 is a functional block diagram illustrating one embodiment of an electronic device. [Figure 5] FIG. 1 is a diagram illustrating a data structure. [Figure 6] 5A and 5B are diagrams illustrating the behavior of objects displayed on the display surface. [Figure 7] 10 is a flowchart showing a procedure for displaying an object. [Figure 8] 10 is a flowchart showing a procedure for displaying an object. [Figure 9] 10 is a flowchart showing a procedure for displaying an object. [Figure 10] 10 is a flowchart showing a procedure for displaying an object. [Figure 11] 10 is a flowchart showing a procedure for displaying an object. [Figure 12] 10 is a flowchart showing a procedure for displaying an object. [Figure 13] 10 is a flowchart showing a procedure for displaying an object. [Figure 14] 5A and 5B are diagrams illustrating the behavior of objects displayed on the display surface. [Figure 15] 5A and 5B are diagrams illustrating the behavior of objects displayed on the display surface. [Figure 16] 5A and 5B are diagrams illustrating the behavior of objects displayed on the display surface. [Figure 17] 5A and 5B are diagrams illustrating the behavior of objects displayed on the display surface. [Figure 18] 5A and 5B are diagrams illustrating the behavior of objects displayed on the display surface. [Figure 19] 10 is a flowchart showing a procedure for displaying an object. [Figure 20] 5A and 5B are diagrams illustrating the behavior of objects displayed on the display surface. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, embodiments of the invention disclosed in this specification will be described in detail with reference to the accompanying drawings. However, the invention disclosed in this specification is not limited to the following description, and various forms and details may be used. It will be readily understood by those skilled in the art that the invention disclosed in this specification can be modified in any manner. The present invention should not be construed as being limited to the description of the following embodiments.

[0026] (Embodiment 1) In this embodiment, the configuration of an electronic device having a flexible display surface and a display on the display surface are described. One embodiment of the display method will be described with reference to FIGS.

[0027] (Electronic device configuration) An example of the configuration of an electronic device having a flexible display screen according to the present invention will be described with reference to FIG. The electronic devices described in this embodiment are, for example, mobile phones, e-mails, Various applications for viewing and creating text, playing music, internet communication, computer games, etc. 1A is a top view of the electronic device 100. The electronic device 100 has a housing 101, a display screen 102, and a home button 103. do.

[0028] The display surface 102 is a part of a display device that displays images such as still images and moving images. As a display device including the above, a light-emitting element represented by an organic light-emitting diode (OLED) is provided in each pixel. Display is performed using a light-emitting device, a liquid crystal display device, an electrophoresis system, an electronic liquid powder system, etc. Electronic paper, DMD (Digital Micromirror Device), PDP (Plasma Display Panel), FED (Field Emis) sion Display), SED(Surface Conduction Display), SED(Surface Conduction Display) ctron-emitter Display), LED(Light Emittin) g Diode) display, carbon nanotube display, nanocrystal display The display surface of the present invention can be used with these display devices. A flexible display device is used as a part of the device and includes the display surface.

[0029] In this embodiment, the display surface 102 is provided with a pointing device such as a finger or a pen for inputting information. The touch panel is provided as an input means. Since there is no need to place a keyboard on the electronic device, the display surface can be placed in a large area. In addition, since information can be input using a pen or finger, it is user-friendly. The touch panel can be a resistive type, a capacitive type, or a Various methods can be used, such as the quantum method, infrared method, electromagnetic induction method, and surface acoustic wave method. However, since the display surface 102 according to the present invention is curved, it is particularly suitable for resistive film type and capacitive type. It is preferable to use the volume method.

[0030] Since the display surface 102 is flexible, the housing 101 must also be able to deform. The housing 101 is made of an elastic resin material, a metal material that can be plastically deformed, or a combination of these. For example, the four corners of the housing 101 may be made of pressed metal plates. Other parts may be made of plastic moldings. Only the display device including the display surface 102 is made flexible, and a gap is formed between the display device and the housing 101. By providing this, the housing 101 can be made of a material that does not have flexibility. In this case, the bellows structure or the like is provided so that the housing 101 expands and contracts in accordance with the degree of curvature of the display surface 102. It may be provided in a part of the housing 101 .

[0031] As shown in FIG. 1A, the electronic device 100 shown in this embodiment has a rectangular shape. The long side is longer than the short side. This is because the user can more easily understand the flexible feature of the electronic device 100. In order to make the product more flexible, the shape is designed to be easily bent, especially in the direction perpendicular to the long side. The electronic device 100 can also be rotated 90° with the short side facing downwards to be used as a portrait display device. To achieve this, an acceleration sensor or the like is provided inside the electronic device 100, The rotation of the device 100 is detected, and the image display on the display surface 102 is switched from landscape to portrait. .

[0032] The four corners of the electronic device 100 are rounded. By wrapping the material, stress caused by bending or twisting is prevented from concentrating on the four corners. This allows the durability of the display device and the electronic device 100 to be improved.

[0033] In addition, the thickness of the electronic device 100 is set to a certain value so that the electronic device 100 can be easily bent. It is preferable to make it as thin as possible while still maintaining a certain strength.

[0034] However, the shape of the electronic device 100 shown in FIG. 1A is an example and is not limited to this. The shape can be any shape such as square, circle, oval, etc. according to the demand.

[0035] The home button 103 is provided on the housing part at the center of the bottom top surface of the electronic device 100. When the home button 103 is pressed, the home screen is displayed on the display surface 102. By pressing and holding the button 103 for a predetermined time, the main power of the electronic device 100 is turned off. If the device has entered the sleep mode, pressing the home button 103 Alternatively, you can set the period of time you press the button to wake up the device from sleep mode. It can be used as a switch to activate various functions by pressing it simultaneously with other buttons. In this way, by providing various functions to the home button 103, 1, the number of actual buttons provided on the electronic device 100 can be reduced, and the structure of the electronic device 100 can be simplified. In addition, the design of the electronic device 100 can be simplified. Since the electronic device 100 is flexible, the failure of the electronic device 100 is reduced and the reliability is improved. To achieve this, the fewer buttons mounted on the housing 101, the better.

[0036] FIG. 1B shows the lower surface (bottom surface) of the electronic device 100. The lower surface of the electronic device 100 is In other words, the side surface of the electronic device 100 is inclined from the top to the bottom. As shown in FIG. 1B, the electronic device 100 has a plurality of input / output devices such as buttons on the side surface. There is a Noh performance.

[0037] The volume adjustment button 104 and the mute button 105 are provided in the upper right corner of the side of the electronic device 100. A speaker 107 for sound output is provided in the lower left corner of the side of the electronic device 100. The speaker 107 emits specific sounds such as the operating system (OS) startup sound. The sound set during operation, music from music playback applications, and other applications It outputs various sounds such as sounds from sound files executed in the program, and email ringtones. In the electronic device 100 according to the present invention, when the display surface 102 is curved, the speaker 10 7 to output a sound according to the curvature, or according to the movement of the object, which will be described later. Sound may be output.

[0038] In addition, by using the volume adjustment button 104, the volume of the sound output from the speaker 107 can be adjusted. You can adjust the volume. You can also instantly mute the volume by pressing the mute button 105. Although not shown, a speaker 10 for sound output can be provided. 7 with or instead of the Speaker 107 headphones, earphones, or headset A connector for outputting sound to a device such as the above may be provided.

[0039] A microphone 106 is provided in the lower right corner of the side of the electronic device 100. The electronic device 100 also has a sleep button 1 at the top left corner of its side. When the sleep button 108 is pressed, the electronic device 100 goes to sleep. The main power supply remains on, but any functions, such as the display on the display surface 102, are stopped. This allows the battery built into the electronic device 100 to be turned off, thereby saving power. This can reduce Lee's power consumption.

[0040] In this way, by concentrating various input / output devices on the four corners of the electronic device 100, The components that do not have flexibility are gathered at the corners, and the electronic device 100 as a whole is made flexible. Furthermore, by using non-flexible members at the four corners, the structural strength of the electronic device 100 can be increased. At the same time, the operability of the electronic device 100 is improved. It is preferable to use non-flexible members at the four corners of 101 that are different from those in other parts. stomach.

[0041] On the other hand, the camera 109 is provided at the center of the bottom surface of the electronic device 100. The image acquired by the camera 109 can be displayed on the display surface 102. By providing the camera 1, even when the user operates the electronic device 100 while looking at the top surface of the electronic device 100, The electronic device 100 according to the present invention is flexible. Therefore, by bending the electronic device 100 into a U-shape and placing it on a desk or the like, it can be fixed to a tripod or the like. This allows you to take pictures without using any equipment and without causing camera shake.

[0042] Although not shown, the electronic device 100 has a terminal for connecting an external memory drive. The external memory drive may be, for example, an external HDD (hard disk drive). drives, flash memory drives, and DVDs (Digital Versatile Disks). sk) drive, DVD-R (DVD-Recordable) drive, DVD-RW (DVD-ReWritable) drive, CD (Compact Disc) drive CD-R (Compact Disc Recordable) drive, CD-R W (Compact Disc ReWritable) drive, MO (Magnetic Optical Disk Drive, FDD (Floppy Disk Drive) e) or non-volatile solid state drives other than the flash memory drives mentioned above. Recording media drives such as Solid State Drives (SSD) devices In addition, the electronic device 100 has a touch panel on the display surface 102. Alternatively, a keyboard may be provided on the housing 101, or an external keyboard may be attached. good.

[0043] The electronic device 100 described above is a flexible device as shown in FIGS. 1(C) and 1(D). For example, Fig. 1(D) shows an electronic device 100 whose left and right ends are curved upward. Therefore, the housing 101 and the display surface 102 can be curved. The flexibility of the electronic device means that at least some of the effects of the invention described in this specification can be obtained. It is not intended to impose any specific restrictions on the direction or degree of bending of electronic devices. stomach.

[0044] FIG. 2 shows the hardware configuration of a flexible electronic device 100 according to this embodiment. 1 is an example of a block diagram. The electronic device 100 includes a processor 151 and a main memory 152. a memory controller 153, an auxiliary memory 154, a sensor controller 155, A sensor 156, a display controller 157, a display device 158, and a power controller A controller 159, a power supply 160, a communication controller 161, and a communication I / F (interface A sound controller 163, a speaker 164, and a sound output connector 16 5, a microphone 166, an input interface 167, and a housing switch 168. , a touch panel 169, a keyboard 170, a camera 171, an external port 172, The processor 172 includes an output interface 173 and a vibration motor 174. 151, main memory 152, memory controller 153, sensor controller 155, A display controller 157, a power supply controller 159, a communication controller 161, Sound controller 163, input interface 167, output interface 17 3 are connected to each other via one or more system buses 150 and can communicate with each other. This can be done.

[0045] The above-described configuration of the electronic device 100 is an example, and may be, for example, a touch panel without the keyboard 170. A virtual keyboard is realized by software using the touch panel 169 and the display device 158. It is also possible to omit some components, such as adding other components in addition to the above configuration. It may also be configured to include additional components.

[0046] The processor 151 includes a CPU (Central Processing Unit) Others include DSP (Digital Signal Processor) and GPU (Graphics Processing Unit). It is used in conjunction with other microprocessors such as the Intel 802.11b / g (Intel 802.11b / g) and Intel 802.11b / g (Intel 802.11b / g) The processor 151 can interpret and execute instructions from various programs. This is used to process various data and control programs.

[0047] The processor 151 includes a thin-film transistor using an oxide semiconductor in a channel formation region. Since the off-state current of the transistor is extremely small, the transistor Using a resistor as a switch to hold the charge (data) that has flowed into the memory element This allows data to be retained for a long period of time. By using it as a register etc. of 1, the processor 151 operates only when necessary, and in other cases By saving the immediately preceding processing contents in the memory element, normally-off computing is realized. This makes it possible to reduce the power consumption of electronic devices.

[0048] The main memory 152 is used as a main storage device. FIG. 3 shows the configuration of the main memory 152. The main memory 152 is a RAM (Random Access Memory). Memory 180 and ROM (Read Only Memory) 181. It has volatile and non-volatile memory.

[0049] The RAM 180 may be, for example, a DRAM (Dynamic Random Access Memory). emory) is used, and memory space is virtually allocated as the working space of the processor 151. The operating system stored in the auxiliary memory 154 such as the HDD is used. program 182, application program 183, program module 184, program The program data 185 and the like are loaded into RAM 180 for execution, as shown in FIG. These data, programs, and program modules loaded into the AM180 are In FIG. 3, the RAM 180 is directly accessed and operated by the processor 151. The memory controller for controlling the stored data is built into the processor 151. Although the description is omitted here, the memory controller for controlling the main memory 152 A separate controller may be provided.

[0050] On the other hand, ROM181 does not require rewriting, and it contains BIOS (Basic Input / Output) The ROM 186 and firmware are stored in the ROM 186. The physical properties of the display unit are configured by physical parameters related to the flexible display surface 102. data 187, and sensor characteristic data 18 relating to the characteristics of the sensor 156 that detects the position information. 8 can be stored in advance in ROM 181. ROM and OTPROM (One Time Programmable Read Only Memory) Only Memory), EPROM (Erasable Programmable EPROM can be used. UV-EPROM (Ultra-Vision) that allows erasing of stored data by exposure to ultraviolet light et Erasable Programmable Read Only Memor y), EEPROM (Electrically Erasable Program Examples include DDR3 memory, DDR4 memory, and flash memory.

[0051] The auxiliary memory 154 built into the electronic device 100 functions as an auxiliary storage device. The memory 154 is a storage medium with a larger capacity than the main memory 152, and is controlled by the memory controller The memory controller 153 is connected to the system bus 150 via the auxiliary memory. It functions as an interface that controls the reading and writing of data from memory 154. The auxiliary memory 154 may be a hard disk drive (HDD) or a nonvolatile solid state drive. Recording on solid state drives (SSD) devices A media drive or the like can be used.

[0052] The auxiliary memory 154 shown in FIG. 2 is configured to be built into the electronic device 100. However, an external storage device located outside the electronic device 100 is connected via an external port 172. This may be a device, and may be used in combination with the auxiliary memory 154 as an auxiliary storage device. That's fine.

[0053] The sensor 156 receives parameters necessary for calculating the three-dimensional shape of the display screen of the electronic device 100. For example, the sensor 156 may be used to determine the relative position of the meters. A plurality of position sensors capable of detecting the position of the object are arranged in a matrix near the display surface. Alternatively, the sensor 156 may be an acceleration sensor. Multiple sensors are placed in a matrix near the display surface, and the acceleration of each sensor changes with the deformation of the display surface. The sensor 156 is not limited to these, and may also be used to calculate the three-dimensional shape of the display surface. Anything that can detect the parameters necessary to detect the Sensors that utilize thermal, acoustic, chemical, etc. can be used. Acceleration sensors, angular velocity sensors, vibration sensors, pressure sensors, gyro sensors, etc. In addition, a combination of these sensors may be used. The sensor 156 may be incorporated into a touch sensor provided on the display surface 102. By combining the switch sensor and position sensor into one component, the number of parts can be reduced. This can contribute to making the electronic device 100 thinner.

[0054] The sensor controller 155 is an interface that controls a plurality of sensors 156. The sensor controller 155 supplies power from a power source 160 to a plurality of sensors 156. The system also receives input from the sensor 156, converts it into a control signal, and transmits it to the system bus 15 The sensor controller 155 performs error management for the sensor 156. Alternatively, a calibration process for the sensor 156 may be performed.

[0055] A display device 158 is connected to the system bus 150 via a display controller 157. The display device 158 has a light emitting element, typically an organic light emitting diode (OLED), at each pixel. Light-emitting devices, liquid crystal display devices, electrophoresis systems, electronic liquid powder systems, etc. Display electronic paper, DMD (Digital Micromirror Device) e), PDP (Plasma Display Panel), FED (Field E Mission Display), SED(Surface Conduction) Electron-emitter Display), LED(Light Emit Display) ting Diode display, carbon nanotube display, nanocrystal display The display device used is a flexible display such as a display panel, a quantum dot display, etc. The display controller 157 receives input from the processor 151 via the system bus 150. In response to the input drawing instruction, the display device 158 is controlled to display the image on the display surface 102 of the display device 158. A predetermined image is displayed.

[0056] The power supply 160 supplies power to the various components of the electronic device 100. 60 has, for example, one or more primary batteries or secondary batteries. In this case, an alternating current (AC) power supply may be used as the external power supply. When using the device separately from the external power supply, do not use it for a long time as it has a large charge / discharge capacity. It is desirable that the power supply 160 can be used. A charger may be used separately from the electronic device 100. Therefore, it is preferable to use a power supply 160 that is also flexible. Examples of batteries include lithium ion secondary batteries and lithium ion polymer secondary batteries. In addition, to give these batteries flexibility, a laminated bag is used for the battery outer container. It is good to use.

[0057] Although not shown, the power supply 160 is connected to a power supply management device (battery management unit: B The BMU may also include a battery monitor (BMU). The BMU collects data on the battery cell voltage and cell temperature, and performs overcharging. and over-discharge monitoring, cell balancer control, battery deterioration status management, remaining battery capacity (State It calculates the State of Charge (SOC) and controls fault detection.

[0058] The power supply controller 159 supplies power from the power supply 160 to the system bus 150 and other power supply lines. The power supply controller 159 controls the power supply to each component via the It has a power converter, inverter, and protection circuit for each channel. The electronic device 100 has a low power consumption function. Detects that there is no input for a certain period of time, and reduces the clock frequency of the processor 151 or or the operation of the processor 151 itself is stopped, or the rotation of the HDD is stopped. This function reduces power consumption by stopping the power supply. It may be performed by the controller 159 alone or in conjunction with the processor 151 .

[0059] A communication I / F (communication interface) 162 communicates with the system via a communication controller 161. The communication controller 161 and the communication I / F 162 are connected to the processor. In response to a command from the server 151, the electronic device 100 is connected to a computer network. The control unit 100 controls the connection signal for the communication and transmits the signal to the computer network. The Internet and Intranet, which are the foundations of the World Wide Web (WWW) , extranet, PAN (Personal Area Network), LAN (Local Area Network), CAN (Campus Area Network) work), MAN (Metropolitan Area Network), WAN (Wide Area Network), GAN (Global Area Network) .ork) and the electronic device 100 to communicate with each other. can be done.

[0060] When the electronic device 100 communicates with other devices wirelessly without using a transmission path, a high frequency circuit ( The communication I / F (communication interface) 162 is provided with an RF circuit for transmitting and receiving RF signals. High frequency circuits are electromagnetic signals and electrical signals in the frequency bands defined by the laws of each country. and a circuit for wirelessly communicating with other communication devices using the electromagnetic signals. The practical frequency band generally used is from several tens of kilohertz to several tens of gigahertz. The high frequency circuit has a high frequency circuit section and an antenna that are compatible with multiple frequency bands. The high frequency circuit section consists of an amplifier, mixer, filter, DSP (Digital Signal Processor), It may have a wireless signal processor, an RF transceiver, etc. When transmitting, GSM (Global System for Mobile Communications) is used as the communication protocol or technology. m for Mobile Communication: registered trademark), EDGE (En hanced Data Rates for GSM Evolution), CDM A2000(Code Division Multiple Access 2000 ), W-CDMA (Wideband Code Division Multiple Communication standards such as Wi-Fi (Wireless Fidelity Access) and : registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), etc. E can use the specifications standardized for communication.

[0061] When the electronic device 100 is used as a telephone, the communication controller 161 and the communication The I / F 162 connects the electronic device 100 to a telephone line in response to an instruction from the processor 151. The control unit 100 controls a connection signal for connecting to the telephone line and transmits the signal to the telephone line.

[0062] The acoustic speaker 164, the sound output connector 165, and the microphone 166 are 163 and is connected to the processor 151 via the system bus 150. In response to a command from the processor 151, the sound controller 163 An audible analog sound signal is generated and output to a speaker 164 or a sound output connector 165 . On the other hand, sound data input to the microphone 166 is input to the sound controller 163. The sound is converted into a digital signal by the sound controller 163 or the processor 151. The sound output connector 165 is connected to a headphone, an earphone, a headset, or the like. A sound output device is connected, and the sound generated by the sound controller 163 is output to the device. .

[0063] One or more switches (hereinafter referred to as "casing switches 168") provided on the casing, A touch panel 169 provided near the display surface 102, a keyboard provided on the housing 101, 170, a camera 171 provided on the underside of the housing 101, and other input components are connected. The connectable external port 172 is controlled by the input interface 167. The interface 167 is connected to the processor 151 and the like via the system bus 150 .

[0064] The housing switch 168 is, for example, the home button 103 and the volume control button 104 described in FIG. These include the button 04, mute button 105, and sleep button 108. Switch 168, touch panel 169, keyboard 170, camera 171, external port 17 2 is a microphone 166 for sound input, a sensor for detecting changes in the shape of the display surface 102 156 together form an interface between the electronic device 100 and the user.

[0065] The touch panel 169 is provided on the display surface 102 and allows information to be input by a pointing means such as a finger or a pen. The touch panel 169 can be used as an input device for inputting the following: Since there is no need for an area on the electronic device to place a keyboard, the display surface can be placed in a large area. In addition, since information can be input using a pen or finger, it is user-friendly. The touch panel 169 can be a resistive type. Various methods are used, including capacitance, infrared, electromagnetic induction, and surface acoustic wave. However, since the display surface 102 according to the present invention is curved, it is particularly suitable for a resistive film type. It is preferable to use a capacitance type. The number of parts can be reduced, and the electronic device 100 can be Contributes to thinner designs.

[0066] The vibration motor 174 is connected to the system bus 150 via the output interface 173. In response to an instruction from the processor 151, the output interface 173 etc. to vibrate the vibration motor 174. This causes the electronic device 100 to vibrate. The vibration can be used to notify you of incoming emails, or to alert you to computer games and other applications. This can be used as a haptic effect for the user. The vibration motor 174 may vibrate depending on the flexibility of the display surface 102, for example, when a predetermined limit is reached. If there is a threshold, it can be used to vibrate or warn when the threshold is exceeded. Although not shown, the output interface 173 can receive the vibration motor 174. In addition, various output devices that users can perceive through their five senses can be connected. For example, the output interface 173 may include a light-emitting device for indicating the operating status of the electronic device 100. It is also possible to connect an aroma diffuser that disperses fragrance through vibration.

[0067] (Functions of electronic devices) Next, FIG. 4 shows the main functions of the electronic device 100 according to this embodiment. The electronic device 100 includes at least a display unit 201, a detection unit 202, The system has four functional blocks: an arithmetic unit 203, and a storage unit 204. 5. The output unit 206 may be included.

[0068] The display unit 201 includes the display device 158 and the display controller 157 described in FIG. The display device 158 displays fields, objects, etc. on its display surface. The display device 158 is flexible and can be deformed. On the display surface 102, an object that moves in accordance with the change in the shape of the display surface 102 is displayed. Here, the object is an object displayed on the display surface 102. A field is an object that moves according to the deformation of the object. It is the background that influences the movement of objects. The graphic expression can be a series of textures that represent the material, or a transparent It may be clear.

[0069] The detection unit 202 includes the sensor 156 and the sensor controller 155 described in FIG. For example, the sensor 156 may be a matrix type. A plurality of sensors 156 are arranged at the same location, and the sensors 156 are arranged relative to each other at their respective locations. The position information acquired by the sensor 156 is transmitted to the sensor controller 15 5 to the calculation unit 203.

[0070] The calculation unit 203 includes the processor 151 and the like described in FIG. The position information of each point on the display surface 102 output from 202 is input, and the display is The three-dimensional shape of the display surface 102 is calculated. The calculation of the three-dimensional shape of the display surface 102 can be performed as appropriate according to the sensor 156 and the calculation method. At this time, the data relating to the three-dimensional shape of the display surface 102 calculated previously is compared with the latest data. The processor 151 calculates the amount of change and obtains a new three-dimensional shape of the display surface 102. The calculation load on the display surface 102 can be reduced. At least the most recent data is stored in the storage unit 204. Alternatively, the position information previously acquired by the sensor 156 and the position information newly acquired by the sensor 156 may be used. The amount of change is compared with the position information, and the amount of change is integrated into the three-dimensional shape of the display surface 102. In this case, the three-dimensional shape of the object 02 can be calculated using the data previously acquired by the sensor 156. At least the most recent data of the position information is stored in the storage unit 204 .

[0071] After calculating the three-dimensional shape of the display surface 102, the calculation unit 203 calculates the three-dimensional shape of the display surface 102. Move the object on the field according to the body shape.

[0072] The storage unit 204 includes the main memory 152, auxiliary memory 154, and memory controller 156 described in FIG. The storage unit 204 stores data related to objects, fields, etc. At least such data, data relating to rules that regulate the movement of objects, is stored. These data may be hard-coded into the operating system or may be stored in the application. The memory unit 2 stores the application program, program module, or program data. These data may be stored in the auxiliary memory 154. When the electronic device 100 is started, the data is written to the HDD as needed by the DRAM or the like. The program is loaded into the main memory 152, which is configured.

[0073] The data related to the object (object data 250) is as shown in FIG. Object ID 251, object shape 252, mass to be set, surface condition (friction coefficient The object physical quantity 253, such as the number of objects, the object image 254, and the object initial position, etc. These are structured lists, tables, or databases. In the example shown in FIG. 5(A), the object ID 2 The data is linked to each of the 51 categories and structured hierarchically.

[0074] The data related to the field (field data 260) is the data related to the field as shown in FIG. Field ID 261, field shape 262, field physical quantity 263, field image 2 64, and field initialization 265, which are structured lists, tables, or data The database can be stored in the storage unit 204. In the example shown in FIG. The data is linked to each field ID 261, creating a hierarchical structure.

[0075] The laws governing the movement of an object are the predetermined equations that govern the movement of the object. It refers to laws that can be described. For example, it refers to the physical laws of nature, especially the laws that govern the movement of objects. For example, gravity, attraction, friction, air resistance, the motion of rigid bodies and fluids, etc. can be visualized. This refers to the laws that can be described as equations of motion that allow us to intuitively understand mechanics. For example, to enhance the visual effect, the physical laws of nature are used. It may be something that emphasizes deviation from the law, or something that floats in opposition to gravity. The above equations may be in strict accordance with the laws of nature. It does not necessarily have to be a concrete expression, but it can be a pseudo-equation or a simplified equation. The data relating to the rules governing the movement of the object may be the data relating to the rules governing the movement of the object. These are the basic equations for simulating the motion of the object. The data relating to the rules governing the movement of objects is also stored in a structured list, table, etc. The information can be stored in the storage unit 204 as a file or a database.

[0076] The calculation unit 203 calculates the IDs of the objects and fields stored in the storage unit 204. Display calculated based on the attached physical quantity, position information, etc. and data received from the detection unit 202 The object is moved in accordance with the above-mentioned rules by referring to the three-dimensional shape of the surface 102. Simulates the movement of an object based on data about the laws that govern its movement. That is, when the calculation unit 203 receives a change in the shape of the display surface 102, the calculation unit 203 calculates the change in the shape. Object data, field data, object movement along with information about changes The data related to the rules that govern the movement of the object is read into the workspace. Calculate the object's motion by inserting each parameter into the equations contained in the law-related data. The calculation of the object's motion is performed until the object's motion converges. In both cases, further changes in the shape of the display surface 102 are monitored by the detection unit 202, and if there is a change, If so, immediately modify the object's motion.

[0077] In this way, the object's movement is converged by repeating the correction according to the change in the shape of the display surface 102. By continuing the simulation until the display surface 102 reaches a state of bunching (i.e., stationary), This allows the user to perceive realistic movement of the object in response to the deformation of the object. The motion of the object simulated by the calculation unit 203 is output to the display unit 201. , is displayed on the display surface 102. Note that a certain threshold is set for the change in the shape of the display surface 102, It is preferable that the calculation unit 203 executes the calculation only when the threshold value is exceeded. The increase in the amount of calculation by the calculation unit 203 is suppressed, and the motion of the object is stabilized at a predetermined level. It can be stopped.

[0078] The input unit 205 includes the microphone 166, the housing switch 168, the touch panel, and the like, which are described in FIG. Panel 169, Keyboard 170, Camera 171, External Port 172, Sound Controller For example, the input from the microphone 166 is The object displayed on the display surface 102 can be moved in response to the input voice. Also, the sensor 156 is a sensor for calculating the shape of the display surface 102. Apart from this, sensors can be used as input interfaces. For example, an acceleration sensor By using a sensor in the electronic device 100, an object can be moved according to the tilt of the electronic device 100. As a sensor for such input, the sensor 156 described above can be used. It may be used.

[0079] The output unit 206 includes the speaker 164, the sound output connector 165, and the sound controller 166 described in FIG. 163, a vibration motor 174, an output interface 173, etc. The object whose motion has been determined as described above moves, and the vibration motor 174 is turned on. The electronic device 100 can be controlled to vibrate, providing a sense of reality to the user through tactile sensation. can.

[0080] (Example of electronic device operation) Next, an example of the operation of the electronic device 100 will be described with reference to FIG.

[0081] FIG. 6A is a diagram showing the top surface of the electronic device 100, and shows the display screen 10 of the electronic device 100. 2, an object 301 is displayed. In the figure, the display of the object 301 In order to explain the operation, other displays are omitted. In addition to 301, background images, icons, toolbars, pointers, windows, text, and videos Anything that can be displayed, such as a web browser, can be displayed at the same time as object 301. do.

[0082] In FIG. 6A, an object 301 is a spherical solid object. The moving range of the object 301 is defined as a field. , which acts as a foundation for influencing the motion of object 301.

[0083] FIG. 6B is a diagram showing the state in which the right end of the electronic device 100 is lifted and bent. When the device 100 is deformed, the display surface 102 is also deformed. The calculation unit 203 calculates the deformed shape of the display surface 102 by combining the information from the sensor 156. Furthermore, various data is read from the storage unit 204 and the object is generated in the manner described above. The simulated result is displayed on the display surface 102 as an object. In FIG. 6B, the right side of the display surface 102 is lifted up, Therefore, the object 301 moves to the center of the display surface 102 as if being attracted by gravity. (Direction indicated by the arrow in the diagram).

[0084] In the case of FIG. 6(B), object 301 is a solid sphere. 01 rolls and moves to the center of the display surface 102 according to Newton's classical mechanics. The object 301 has a predetermined physical quantity. Therefore, the moving speed is determined according to the mass, etc. The degree and rotation speed of the field are determined. The physical quantities of the field are also set in advance. For example, object 301 is subjected to air resistance set in the field, and It moves under the specified gravitational acceleration and friction set in the field.

[0085] In this way, by applying various rules to the behavior of the object 301, it is possible to provide a realistic experience to the user. You can experience the tea.

[0086] It should be noted that the physical quantities and the like assigned to the object 301 and the field are not necessarily all referenced. It is not necessary to refer to only a part of it, and it may be set to refer to only a part of it. This reduces the load on the calculation unit 203. The equations may be set so that only a part of the equations stored in the storage unit 204 is used. In addition, in FIG. 6(B), only one object 301 is displayed on the field. In this case, the object 301 displayed on the display surface 102 is Depending on the number of data, the number of parameters or data such as physical quantities and equations to be referenced can be adjusted appropriately. It can also be done as follows.

[0087] FIG. 6C shows an object when the electronic device 100 is further bent and deformed into a downward convex shape. 6(B) shows the movement of the spherical object 301. The object 301 falls into a depression formed by the curvature of the display surface 102. The object 301 is blocked on both sides by high walls of the display surface 102, so it is difficult to move left and right. It cannot move, but moves along the bottom of the depression and eventually comes to rest.

[0088] In this way, information relating to the three-dimensional shape of the display surface 102 is transmitted to the sensor 156 and the processor 151. By calculating the shape of the object 301, it is possible to display the object 301 moving according to the shape of the object. As a result, the object 301 displayed on the display surface 102 can be displayed on the The user can intuitively sense the reality as if the object were actually on the surface.

[0089] In this case, the deformation of the display surface 102 is used as a start condition for the movement of the object 301. However, the condition for starting the movement of the object 301 is not limited to this. When an acceleration sensor is provided in the electronic device 100 as the interface 167, The detection of acceleration may be used as a condition for starting the movement of the object 301. The object 301 can be moved in a direction according to the tilt of the input. When a microphone 166 is used as the interface 167, for example, The object 301 can be moved depending on the size. By inputting a direction by the touch panel, the object 301 is moved in a direction corresponding to the input direction. Alternatively, when a home button 103 provided on the housing 101 of the electronic device 100 is pressed, This may return the object 301 to its initial position.

[0090] (Object display procedure) Next, the processing procedure for displaying an object in the electronic device 100 according to the present invention will be described with reference to FIG. 7 to 13 will be used to explain this.

[0091] FIG. 7 shows a processing procedure 500 for displaying an object in the electronic device 100 according to the present invention. This is a flowchart showing the procedure for displaying an object. This section explains the case of an application that displays a project. An example of the application is the background of the home screen of the electronic device 100.

[0092] Therefore, the procedure for displaying objects described below is performed by the application program. The program is executed by the auxiliary memory 154, the main memory 152, etc. It is recorded on a computer-readable recording medium.

[0093] In addition, the program corresponds to software here, but such processing procedures The stages can also be implemented as electronic circuits or mechanical hardware.

[0094] First, an application for displaying an object in the electronic device 100 according to the present invention will be described. The application program stored in the auxiliary memory 154 is started (S001). The program etc. is loaded into the main memory 152. The application is started by Depending on the settings, it may be linked to the operating system and started at the same time.

[0095] Next, initial conditions for the objects and fields are set (S002, S003). The initial conditions for objects and fields are determined from the previous use of the application. You can set it to inherit the conditions at the time of termination, or reset it to the default conditions each time. Alternatively, the user may be allowed to select one of these.

[0096] The initial condition of the object is the object data 250 stored in the storage unit 204. It sets the object ID 251 and display position etc. of the field. The initial condition is the field I in the field data 260 stored in the storage unit 204. Specifically, the setting is to set the object ID 251 and The object physical quantity 253 and the like associated with the object are read from the auxiliary memory 154, and the main The field data 260 is also stored in the memory 152. Set it up.

[0097] Setting initial conditions for objects (S002) and fields The setting (S003) is not limited to the processes executed in this order, but also includes the setting of initial conditions for fields. Alternatively, you can set the initial conditions for the object (S002) and the Setting of initial conditions for the field (S003) may be performed at the same time.

[0098] Next, a plurality of sensors 156 measure position information at each position installed near the display surface 102. Then, based on the position information acquired by the plurality of sensors 156, The calculation unit 203 calculates the three-dimensional shape of the display surface 102 (S005). After starting the application, if other systems, applications, etc. have already started before starting this application, Therefore, when the shape of the display surface 102 is acquired, the display is displayed using the information related to the already acquired shape. The three-dimensional shape of the display surface 102 may be calculated.

[0099] Next, the shape of the display surface 102 calculated in step S005 and the initial state of the object are and determine whether to move the object (S006). The initial position of the object set in S002 is determined based on the curved shape of the display surface 102. If the position corresponds to a position having an inclination exceeding a predetermined threshold value with respect to the horizontal plane, the calculation unit 20 3 determines that the object needs to be moved. On the other hand, if the shape of the display surface 102 is curved, Even if the initial position of the object corresponds to a horizontal position relative to the horizontal plane, Since the object is in a stationary state, the calculation unit 203 determines that the object does not need to move. If the calculation unit 203 determines that the object needs to be moved, the process transition to object movement mode 510 (or object movement mode 520, shown as another example) On the other hand, if it is determined that the object does not need to be moved, the object is moved. The system transitions to standby mode 530 (S008).

[0100] When the object waiting mode 530 is completed, the completion determination is made in step S009. If the condition is met (if the application termination flag is "1" as described below), The application is terminated (S010), and if the condition is not satisfied (see below, the application If the end flag is "0", the process returns to step S004.

[0101] When the application is terminated, various data stored in the main memory 152 is It is preferable to store the file in the auxiliary memory 154. Next, start the application. When the user selects the device, these data can be used as the initial settings.

[0102] FIG. 8 shows the procedure for displaying an object in the electronic device 100 according to the present invention. 10 is a flowchart showing the object movement mode 510. 10 repeatedly simulates the movement of the object and displays the results on the display surface 102. This mode processes the object's movement one by one. This is done in units of time (hereinafter referred to as Δt).

[0103] After the object movement mode starts (S020), according to the shape of the display surface that has already been calculated, The motion of the object is determined (S021). Here, the motion of the object is determined by Δ This is done for the period up to t.

[0104] The movement of the object is calculated by the calculation unit 203 based on a predetermined rule. The simulation is performed using the parameters set in steps S002 and S003. This simulation is performed using the physical quantities of the specified objects, the physical quantities of the field, etc. The direction and speed of the object's movement are determined by the For simulation of direction, speed, etc., the field related to the associated field ID 261 is used. The field data 260 is also referenced. For example, the field of the selected field ID 261 If the data for the physical quantity 263 includes data for elastic materials such as rubber, When an object falls into a depression caused by the curvature of the display surface 102, it moves in a bouncy motion. The information relating to the simulated object's behavior is stored in the storage unit 20 4.

[0105] Furthermore, to make the user perceive reality, object rotation and object deformation are However, a simulation may be performed to display the above. If this is done, a load is placed on the calculation unit 203, and a delay in the processing of the calculation unit 203 makes it difficult to display accurately. This makes it difficult to realize the reality that the user experiences. To avoid this slowdown, it is recommended to simplify the calculations and create a database of the calculation results. I wish.

[0106] Then, the object's motion is determined by the simulation. The motion is displayed on the display screen 102 for a period of Δt (S022). Depending on the model, the influence of factors such as friction coefficient, air resistance, and gravity is added to the object's movement. Note that the motion of an object also includes the object being stationary. In step S023, the motion sequence from rest to motion and from motion to rest of the object is Included.

[0107] Next, the three-dimensional shape of the display surface 102 is recalculated (step S023). In step 204, the three-dimensional shape of the display surface 102 calculated previously and stored in the storage unit 204 is The data relating to the three-dimensional shape of the display surface is recalculated in a sequence 511. The data is rewritten into data relating to the three-dimensional shape of the display surface 102.

[0108] Here, the three-dimensional shape recalculation sequence 511 for the display surface will be described with reference to FIG. When the three-dimensional shape recalculation sequence 511 of the display surface is started (S040), Then, the position information of the predetermined location on the display surface 102 is acquired again (S041). The three-dimensional shape of the display surface 102 is recalculated based on the position information (S042). The data relating to the three-dimensional shape of the display surface 102 is stored in the storage unit 204. The body shape recalculation sequence 511 ends (S043).

[0109] The recalculation sequence 511 of the three-dimensional shape of the display surface calculates the data relating to the three-dimensional shape of the display surface 102. The updated data is overwritten with the previously calculated data relating to the three-dimensional shape of the display surface 102. Furthermore, the previously calculated data relating to the three-dimensional shape of the display surface 102 is stored without being overwritten. The data may be stored by specifying an address different from the address of the storage unit 204 .

[0110] Simultaneously with the recalculation sequence 511 of the three-dimensional shape of the display surface, in step S023 The input information is read from the input interface 167 by an input device such as a touch panel. It may be included.

[0111] Next, in order to update the information related to the operation of the object stored in the main memory 152, The object's movement is simulated again (S024). The operation of the display screen 102 may be simulated, or the previously calculated data relating to the three-dimensional shape of the display screen 102 may be used. and the data relating to the three-dimensional shape of the display surface 102 recalculated in step S023. By comparing the previously calculated data relating to the three-dimensional shape of the display surface 102, the recalculated data Only when there is a change in the data relating to the three-dimensional shape of 102, the behavior of the object is simulated. In this case, a threshold value for the amount of change may be set for comparison of data relating to the three-dimensional shape. By setting a value, it is possible to prevent noise and unnecessary operations of the calculation unit 203 in an area that cannot be perceived by the user. In other words, the load of calculation can be reduced. The amount of change in the recalculated data relating to the three-dimensional shape of the display surface 102 relative to the data relating to the shape is determined. Only when the threshold value is exceeded, it is determined that there has been a change in the three-dimensional shape of the display surface 102, and the step The process proceeds to step S024, and if there is no change, the process proceeds to step S025.

[0112] Next, in step S025, it is determined whether the object is in motion or stationary. In the object movement mode 510, the object is moved every Δt. Therefore, the simulation in step S021 In the simulation results, the object may be in motion after Δt. By recalculating the three-dimensional shape of the display surface 102 in step S023, the stain in step S024 is In such cases, the object may behave differently due to the simulation. , it is determined that the object is not stationary and is in motion, and the process returns to step S022. .

[0113] On the other hand, if the object is determined to be stationary, object movement mode 5 is used. 10 ends (S026).

[0114] As described above, the period of the object's movement is displayed in Δt intervals, and the shape of the display surface is displayed. By making changes and repeating the simulation, the object's behavior is corrected and displayed one by one. This allows the user to perceive reality.

[0115] FIG. 10 shows a process for another object movement mode 520, which is different from the process procedure described in FIG. 1 is a flowchart showing a processing procedure.

[0116] The object movement mode 510 described above divides the display time of the object movement and After the display of a certain item is completed, a simulation for the next display is performed. In the object movement mode 520, the movement of the object is displayed for a predetermined period of time. During this period, a simulation for the next display is performed.

[0117] When the object movement mode 520 is started (S050), the already calculated three-dimensional image of the display surface 102 is The motion of the object is determined by simulation according to the information about the shape (S0 51).

[0118] Next, the motion determined by the simulation is displayed in a table for a predetermined period (here, Δt). During a period of Δt, the object's motion is simulated for the next period of Δt. The object motion is displayed according to the object motion display sequence 521. The next object motion is calculated in the object motion calculation sequence 522. Therefore, it is executed (S052).

[0119] Here, the display sequence 521 of the object's movement will be explained with reference to FIG. After the start of this sequence (S060), the time t is initialized to "0" (S061). In steps S062 to S065, the movement of the object is displayed. That is, after displaying the object's motion for each time t (S063), t is added and updated. New (S064), the object's motion is displayed again until the time t becomes Δt. This is repeated (S065). As a result, the motion of the object is displayed until the time t becomes Δt. When the time t reaches Δt, the object motion display sequence 521 ends. Complete (S066).

[0120] Next, the object motion calculation sequence 522 will be described with reference to FIG. As with the object movement display sequence 521, after the sequence starts (S070), The time t is initialized (S071). Next, in steps S072 to S077, , Δt, the motion of the object according to the three-dimensional shape of the display surface 102 is simulated. To do.

[0121] First, the sensor 156 acquires position information of a predetermined location on the display surface 102 (S073 Next, the three-dimensional shape of the display surface 102 is calculated based on the acquired position information (S074). Then, the motion of the object is calculated based on the calculated data relating to the three-dimensional shape of the display surface 102. The simulation is performed for a period of Δt and the value is determined (S075). After that, t is added and the value is updated. (S076).

[0122] The above steps S073 to S076 are repeated until t becomes Δt (S 077), where the period Δt is the period determined by the display sequence 521 of the object's motion. By synchronizing both sequences, both sequences are synchronized during the same period (Δt). The sequences are processed in parallel, i.e., the object motion display sequence 521 While displaying the object's motion, the calculation sequence of the object's motion is The process 522 allows the behavior of the object for the next period to be predetermined. Cut.

[0123] In the object motion calculation sequence 522, the shape of the display surface 102 during the period Δt is calculated. If there is a change in the state of the object, the determined movement of the object is overwritten and corrected. In FIG. 12, the object is displayed regardless of whether the shape of the display surface 102 changes. The simulation of the robot's movement is repeated, but the change in the shape of the display surface 102 is not judged. It may be provided midway, and the simulation may be omitted if there is no change in shape. The calculation of the three-dimensional shape of the display surface 102 is performed when the position information acquired by the sensor is changed by more than a specified value. It may also be the case that there was

[0124] When t becomes Δt by repeating the above operation, the calculation of the object's movement is The sequence 522 ends (S078).

[0125] After step S052, as shown in FIG. 10, the object is in motion or stationary. If it is in the motion state, the process returns to step S052. If the object is in a stationary state, the object movement mode 520 ends (S054). The system then transitions to a project standby mode 530.

[0126] As described above, the object movement mode 520 shown in FIG. 10 moves the object within a predetermined period. Simultaneous display of the object's motion and simulation of the object's motion for the next period This type of processing allows the user to intuitively experience reality.

[0127] Next, the object waiting mode 530 will be described. 5 is a flowchart showing the processing procedure of the object waiting mode 530. When the application is running, the object displayed on the display surface 102 is stationary. 1 shows the internal processing of the electronic device 100 during the period.

[0128] When the object standby mode is started (S030), in step S031, The application termination flag indicates whether the application has terminated or not, and the object wait mode is The mode end flag indicating whether the mode 530 has ended is initialized. For example, both flags are set to "0". Let's say.

[0129] Next, the deformation of the display surface 102 is confirmed using the sensor 156 (S032). , which has already been calculated in step S005, step S042 or step S074 The data relating to the three-dimensional shape of the display surface 102 and the display surface 1 recalculated in step S032 The data relating to the three-dimensional shape of the object is compared with the data relating to the three-dimensional shape of the object (S032). For comparison, if a threshold value is set for the amount of change, noise and areas that cannot be perceived by the user can be eliminated. This can reduce the load of unnecessary calculations on the calculation unit 203 in the previous region. The recalculated orientation of the display surface 102 for the data relating to the three-dimensional shape of the display surface 102 calculated once Only when the amount of change in the data relating to the body shape exceeds a predetermined threshold, the three-dimensional shape of the display surface 102 is displayed. It is preferable to determine that a change has occurred and proceed to step S034.

[0130] If it is determined that there is a change in the three-dimensional shape of the display surface 102, the mode end flag is set to "1". If there is no change in the shape of the display surface 102, the input interface 106 is set (S034). Check the input from the input interface 167. The inputs are limited to those that affect the behavior of the object. The mode end flag is set to "1" (S034). If there is no input from any of the interfaces 167, the mode end flag remains at "0". Well, let's move on to the next step.

[0131] In step S035, it is confirmed whether an instruction to terminate the application has been notified. If it is known, the application termination flag for this application is set to "1" " is set (S036). The application termination command is, for example, The user may issue an application termination command via 1, or other applications or programs may This includes instructions from the operating system to terminate the application. If the application has not been notified, the application termination flag remains at the initial setting of "0". become.

[0132] In step S037, the flag is checked. If either or both of the application end flags are set to "1", the On the other hand, if both flags indicate "0", the object standby mode is terminated (S038). If so, the object standby mode continues, i.e., the process returns to step S032. .

[0133] As described above, in the object waiting mode 530, confirmation of a predetermined input is looped. When any of the inputs is received, the object The standby mode 530 is ended, and the process proceeds to step S009.

[0134] This embodiment mode can be implemented in appropriate combination with other embodiment modes. The expression of clarity can be applied to methods, hardware (electronic devices, computers, semiconductor devices, recording media, etc.), Conversions between systems, programs, software, etc. are also within the scope of the present invention. It is effective.

[0135] (Embodiment 2) In the first embodiment, a spherical solid object 301 is shown as an example of an object. However, the object is not limited to this. In this embodiment, the object is a liquid. The case where the above-mentioned is used will be described with reference to FIG.

[0136] FIG. 14A shows an electronic device 100 that displays a liquid object 303 on the display surface 102. 6. As in the electronic device 100 shown in FIG. In order to explain the operation, other displays are omitted. 303, background images, icons, toolbars, pointers, windows, text, videos Anything that can be displayed, such as a web browser, can be displayed at the same time as object 303. do.

[0137] In FIG. 14(A), an object 303 is a liquid object, and is positioned at a predetermined position. The moving range of the object 303 is defined as a field. This acts as a foundation that influences the movement of the object 303 .

[0138] FIG. 14(B) is a diagram showing the case where the right end of the electronic device 100 is lifted and bent. When the child device 100 is deformed, the display surface 102 is also deformed. The movement is controlled by the calculation unit 203 so that the movement conforms to the shape of the display surface 102. In FIG. 14B, the right side of the display surface 102 is raised. The liquid object 303 moves to the center of the display surface 102 as if being attracted by gravity. (in the direction indicated by the arrow in the figure).

[0139] The fact that the object 303 is a liquid is indicated by the object name in the object data 250. The calculation unit 203 calculates the object physical quantity stored in the object physical quantity 253. The simulated object is then simulated by referring to data or parameters that indicate that the object is a liquid. The object 303 simulates the motion of a real fluid that obeys the laws of fluid dynamics (hydraulics). By accelerating the speed, the movement of the object is expressed as if the liquid is flowing from high to low. As data or parameters that indicate that an object is a liquid, For example, you may prepare numerical values ​​for physical quantities such as density, viscosity, compressibility, and surface tension. For simplicity, the degree of physical quantities such as viscosity is divided into several stages and prepared as several data. In addition, in order to allow the user to visually perceive the state of the liquid flowing, various states of the liquid may be displayed. A set of body images may be pre-stored in object images 254 .

[0140] The field is also set with physical quantities in advance, for example, friction determined by the field. The liquid object moves in response to the force.

[0141] In this way, by applying various rules to the behavior of the liquid object 303, You can experience the reality.

[0142] It should be noted that the physical quantities and the like assigned to the object 303 and the field are not necessarily all referenced. It is not necessary to refer to only a part of it, and it may be set to refer to only a part of it. This reduces the load on the calculation unit 203. The equations may be set so that only a part of the equations stored in the storage unit 204 is used. In addition, in FIG. 14(B), only one object 303 is displayed on the field. However, multiple displays are acceptable. For example, if more complex calculations are possible, In this example, the liquid object 303 is split into multiple objects by movement, and multiple You can also combine multiple objects into one object. When an object is created or destroyed, it is assigned an object ID of 251. You can define it as a single object with one object ID, and then define the object type. The state may be stored as a divided object in the state 252, etc. For simplification, an image showing droplets may be displayed around the liquid object 303 as appropriate. .

[0143] FIG. 14(C) shows the liquid crystal display when the electronic device 100 is further bent and deformed into a downward convex shape. 14(B) is a diagram showing the movement of a liquid object 303. The object 303 flows into the depression formed by the curvature of the display surface 102. The object 303 that fell was blocked on both sides by high walls made up of the display surface 102. Therefore, it cannot move left or right, and instead accumulates at the bottom of the depression and eventually comes to a standstill.

[0144] In this way, information relating to the three-dimensional shape of the display surface 102 is transmitted to the sensor 156 and the processor 151. By calculating the shape of the object, a liquid object 303 is displayed. This allows the liquid object 303 displayed on the display surface 102 to be The user can intuitively sense the reality of the device as if it were actually present on the surface of the device 100. This can be done.

[0145] This embodiment mode can be implemented in appropriate combination with other embodiment modes. The expression of clarity can be applied to methods, hardware (electronic devices, computers, semiconductor devices, recording media, etc.), Conversions between systems, programs, software, etc. are also within the scope of the present invention. It is effective.

[0146] (Embodiment 3) In this embodiment, a touch panel is used as an input device included in the input unit. Figure 1 shows an electronic device that can operate objects by inputting from a touch panel. 5 will be used to explain.

[0147] FIG. 15(A) is a bird's-eye view of the electronic device 100 that has already been bent. A spherical solid object 302 is displayed on the right edge of the display surface 102. For example, if the user touches the object 302 (actually, the object 302 is displayed), (a) touching the touch panel provided on the display surface 102 in the state shown in FIG. 15(A) When a finger is flicked on the display surface 102 in the direction of the arrow in FIG. In FIG. 15(B), the display surface 102 is in the center, and the ball rolls in the direction of the bounce. The object 302 is curved so that it drops sharply from the center to the left. It appears to fall and roll.

[0148] Even if the electronic device 100 has a configuration in which a touch panel is used as an input unit, In principle, the procedure for displaying the object is the same as that described in the first embodiment. The condition for starting the object's movement is the detection of the deformation of the display surface 102 as described in the first embodiment. In addition to the above, access to objects from the touch panel is also included. Among the processing procedures shown in the first embodiment, the step of determining whether an object has moved (S006, Steps S025 and S053 include confirmation of detection of input by the touch panel.

[0149] Similarly, in the object movement mode 510 shown in FIG. 8, the three-dimensional shape of the display surface In the change update step, the input information from the touch panel is also updated (S 023). In the object movement mode 520 shown in FIG. A step of acquiring position information (S073) and a step of calculating the three-dimensional shape of the display surface (S07 In 4), the input information from the touch panel is also confirmed. In the object waiting mode 530 shown in FIG. In the input confirmation step (S033), the information input from the touch panel is confirmed. .

[0150] In this embodiment, a touch panel is used to operate the object 302. However, the input unit 205 may be configured as a touch panel or a A housing switch, a keyboard built into the electronic device 100 or an external keyboard, a keyboard connected to an external port, Pointing devices such as a mouse or controller, and a microphone for sound input. Various input devices can be used to operate the object 302. For example, a microphone When using a microphone, the behavior of the object 302 is adjusted appropriately depending on the volume of the input. (See FIG. 16(A)). Also, for example, when the user says "move left," The voice input is analyzed to recognize its meaning, and the object is moved to the left according to the meaning. The user may move the device in the opposite direction (see Figure 16(B)). The electronic device 10 includes a gesture device that converts the gesture into an input signal as a configuration of the input unit 205. 0 to operate the object 302.

[0151] In this way, various input devices are used as components of the input unit 205, and the display surface 10 2, the object 302 displayed on the electronic device 100 can be moved. It allows the user to intuitively experience the reality as if it were actually on the surface of the .

[0152] This embodiment mode can be implemented in appropriate combination with other embodiment modes. The expression of clarity can be applied to methods, hardware (electronic devices, computers, semiconductor devices, recording media, etc.), Conversions between systems, programs, software, etc. are also within the scope of the present invention. It is effective.

[0153] (Fourth embodiment) In this embodiment, the display process of an object on the display screen is performed in conjunction with other applications. The case where they are executed simultaneously will be described with reference to FIG.

[0154] FIG. 17A shows another application software being started and the application software is displayed on the display surface 102. The electronic document is displayed in a window 305 of the application software. The display surface 102 displays a home screen displayed by the operating system of the electronic device 100. A plurality of icons 304 in the image are displayed.

[0155] In FIG. 17(A), a word processing application is used as another application software. This includes, but is not limited to, software such as spreadsheets, database management, telephone, email, etc. Web browser, blog, video conferencing, music player, video player, digital camera Various applications such as digital cameras, electronic books, and computer games Includes software.

[0156] On the display surface 102, an object 306a and an object 306b according to the present invention are displayed. The object 306a and the object 306b are the same as those in the first to third embodiments. The object moves according to the input from the sensors and input unit as explained above. The object 306a and the object 306b are restricted to moving in the entire area of ​​the display surface 102. It is set so that it cannot enter into a predetermined area. In this case, a plurality of icons 304 and a window 305 of application software are displayed. The displayed area is set as an area that objects cannot enter, and object 3 Object 306a and object 306b move to avoid the area. By regarding the area where the object cannot be detected as an object with a certain physical quantity, the object can be detected according to the physical quantity. The motion of the object is set to be prescribed.

[0157] 17B, ​​the object 30 that moves in accordance with the deformation of the display surface 102 is 6 is also affected by the icon 304 and moves accordingly. The shape of the object 306 and the shape of the virtual icon 304 are the same. This affects the image quality, allowing the user to perceive it as more realistic.

[0158] In this way, not only objects and fields, but also icons and application software Various images displayed on the display surface 102, such as windows of clothing, are treated as objects. By adding physical quantities, you can make the movement of objects more realistic. .

[0159] Note that the objects to which physical quantities are assigned and treated as objects are not limited to images displayed on the display screen. It may be the edge (four sides) of the device, or the user's finger detected by the touch panel.

[0160] To achieve the above display, images such as icons and windows are also displayed in the second It is considered as an object, and the object ID, object shape, and object Physical quantities can be set in advance. The motion of the object is simulated by the calculation unit. In this case, the conditions of the second object may also be taken into account.

[0161] Alternatively, you can define an image, such as an icon or window, as part of the field and then The information may be incorporated into the object's motion information, so that it can be reflected in the object's motion simulation.

[0162] It is not necessary to imagine all images such as icons and windows as objects. You can also limit it to computers and windows. The objects are then moved over these images so that they are treated as a background for the object. In addition, the user may be prompted to select whether or not to virtualize these images as objects. It may be set to

[0163] Images such as icons and application software windows, or displays When processing the motion of an object that captures the edge of the display screen as an object, The display of the operating system is managed by the operating system. Therefore, the processing procedure for displaying the object is By incorporating it into the operating system as a It can be carried out efficiently.

[0164] This embodiment mode can be implemented in appropriate combination with other embodiment modes. The expression of clarity can be applied to methods, hardware (electronic devices, computers, semiconductor devices, recording media, etc.), Conversions between systems, programs, software, etc. are also within the scope of the present invention. It is effective.

[0165] (Embodiment 5) In this embodiment, a predetermined area on the curved display surface is selected, and only the area is displayed as an object. The display process for moving the object will be described with reference to FIG.

[0166] 18(A) and (B) show an object 307 displayed on a portion of the curved display surface 102. 1 is a top view of the electronic device 100 showing the state of the display screen 102. It curves so that it drops sharply towards the top.

[0167] In FIG. 18(A), the object is displayed on the high right side portion (area 308a) of the curved display surface 102. The object 307 is displayed on the curved display surface 102. That is, the area 308a is determined and the robot moves selectively within the area 308a.

[0168] On the other hand, in FIG. 18(B), the left lower portion (region 308b) of the curved display surface 102 is covered with an on-screen image. The object 307 is displayed on the curved display surface 102. The target portion, i.e., region 308b, is determined and the target portion is selectively moved in region 308b.

[0169] The display of such objects can be controlled by the operating system's home screen or screen. It can be used for various applications such as a server and other software.

[0170] To display the above objects, the processing procedure described in the first embodiment can be used. However, in the object movement mode in this processing procedure, the three-dimensional It is necessary to appropriately define the area in which the object can move based on its shape. Therefore, for the object movement mode, the processing procedure shown in FIG. 19 is followed, for example.

[0171] FIG. 19 is a flowchart showing the processing procedure of the object movement mode 540 according to this embodiment. This is a chart, and is a configuration in which a step is added to the object movement mode 510 already explained. After the object movement mode is started (S080), the calculation unit 203 has already calculated In step S081, based on the information relating to the three-dimensional shape of the display surface 102, Determine the object's range of motion.

[0172] In this embodiment, the area in which the object can move is a specific height on the display surface 102. For example, if the height of the center of the display surface is the reference position, The two-dimensional space at a position higher (or lower) by a predetermined height is defined as the object's possible movement area. Alternatively, the position may be higher (or lower) by a predetermined height range than the reference position. The three-dimensional space can be defined as the object's possible motion area. The three-dimensional space can be defined numerically in advance, or the object can be displayed A certain space including the location may be defined as the exercise area.

[0173] After determining the area in which the object can move, the object's movement within that area is controlled in steps. The object motion is determined in step S082. The information on the area determined in step S081 is used as the boundary condition. (S083 to S089) are the same as the object movement mode 510 shown in FIG. If the shape of the display surface 102 changes while the object is moving, In step S086, the area in which the object can move is determined again, and the object's movement is The display is overwritten.

[0174] This embodiment mode can be implemented in appropriate combination with other embodiment modes. The expression of clarity can be applied to methods, hardware (electronic devices, computers, semiconductor devices, recording media, etc.), Conversions between systems, programs, software, etc. are also within the scope of the present invention. It is effective.

[0175] (Embodiment 6) In this embodiment, a display in which an object moves according to the shape of the display surface is used. The computer game will be explained with reference to FIG.

[0176] FIG. 20 shows an electronic device 100 with a computer game image displayed on a display surface 102. In FIG. 20(A), the shape of the display surface 102 is flat, and the display surface 102 A river 309 flows in the center, and on the left bank of the river 309 are characters 310a and 310b that can be operated by the user. In this computer game, the character 31 Character 310a and character 310b are blocked by the river 309 flowing in the center and cannot move to the right bank. I can't come.

[0177] Therefore, as shown in FIG. 20(B), the user may change the display surface 102 so that the center of the display surface 102 is convex downward. (approaching the direction of the arrow in Figure 20(B)) and bending the river 309 downward. By moving the left bank and the right bank closer to each other, the character 310a and the character 310b are 10b can be moved to the right bank.

[0178] Here, by setting the river 309 as the object described above, the flow of water in the river 309 can be By linking this to the deformation of the display surface 102, it is possible to give the user a sense of reality. By calculating the shape of the display surface 102 using the arithmetic unit 156, the character's The range of movement can be controlled.

[0179] In order to realize such a computer game, the objects described in the first to fifth embodiments are The object display processing procedure is used for objects including characters that appear in computer games. The procedure for displaying an object can be written as a module in a computer program. The display procedure of the object may be incorporated into the operating system. Such programs, modules, etc. may be executed in conjunction with the system. The data is recorded on a computer-readable recording medium such as the main memory 152 or the memory 154. Such processing means may be realized as electronic circuits or mechanical hardware. You may do so.

[0180] As described above, the game can be progressed in accordance with the change in the shape of the display surface 102. This makes it possible to provide users with a realistic computer game.

[0181] This embodiment mode can be implemented in appropriate combination with other embodiment modes. The expression of clarity can be applied to methods, hardware (electronic devices, computers, semiconductor devices, recording media, etc.), Conversions between systems, programs, software, etc. are also within the scope of the present invention. It is effective. [Explanation of symbols]

[0182] 100 Electronic equipment 101 Case 102 Display surface 103 Home button 104 Volume adjustment button 105 Mute button 106 microphones 107 Speaker 108 Sleep button 109 Camera 150 System Bus 151 processors 152 main memory 153 Memory Controller 154 Auxiliary Memory 155 Sensor Controller 156 Sensors 157 Display Controller 158 Display device 159 Power Controller 160 Power supply 161 Communication Controller 162 Communication I / F 163 Sound Controller 164 speakers 165 Sound output connector 166 microphones 167 Input Interface 168 Case Switch 169 Touch Panel 170 keyboards 171 Camera 172 external port 173 Output Interface 174 Vibration Motor 180 RAM 181 ROM 182 Operating Systems 183 Application Programs 184 Program Modules 185 Program Data 186 BIOS 187 Display Physical Property Data 188 Sensor characteristic data 201 Display section 202 Detection unit 203 Arithmetic section 204 Storage section 205 Input section 206 Output section 250 object data 251 Object ID 252 object shapes 253 Object Physical Quantity 254 object images 255 Object Initialization 260 Field Data 261 Field ID 262 Field Shape 263 Field Physical Quantities 264 Field Images 265 Field Initial Settings 301 Object 302 Objects 303 Objects 304 Icons 305 Window 306 objects 306a Object 306b Object 307 Objects 308a area 308b area 309 River 310a Character 310b character

Claims

1. a first state in which the display surface is flat and a second state in which a portion of the display surface is bent to have a recessed region recessed downward are selectable; an electronic device having a function of controlling display so that, in the second state, when an object displayed on the display surface moves from outside the recessed area into the recessed area, the object makes a bouncing motion.

2. A display device having a display surface and a housing, a first state in which the display surface is flat and a second state in which a portion of the display surface is bent to have a recessed region recessed downward are selectable; an electronic device having a function of controlling display so that, in the second state, when an object displayed on the display surface moves from outside the recessed area into the recessed area, the object makes a bouncing motion.

Images