Electronic device and control method thereof

Abstract

To enable viewing video data that is desired to be viewed, in a desired screen layout with ease.SOLUTION: An electronic apparatus includes: acquisition means for acquiring a plurality of videos obtained from a plurality of different video sources; display means; storage means for storing information in which an orientation of the display means is associated with at least either one of the plurality of videos based on operations by operation means; detection means for detecting the orientation of the display means; and control means for controlling the display means in such a way to display a video obtained from a same video source as that of the video associated with the detected orientation.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an electronic device and a control method thereof, and more particularly to a display control technique for images captured by a plurality of imaging devices. [Background technology]

[0002] Conventional methods for viewing video content include editing videos shot with one or more cameras and distributing them as a single piece of content, or live broadcasting in which videos shot with multiple cameras are switched between. In particular, in recent years, not only are horizontal videos suitable for television screens being distributed, but vertical videos adapted to portrait-oriented devices such as smartphones and tablets are also being distributed on social media and video distribution services.

[0003] Furthermore, there are services that allow users to simultaneously view not only video data from a single viewpoint captured by one camera, but also video data from multiple viewpoints captured by multiple cameras. These services include free viewpoint video, which allows users to view the video from any viewpoint by synthesizing the videos, and services that deliver video from multiple viewpoints as is without synthesizing the videos (see, for example, Patent Document 1). In addition, horizontally long videos showing the entire field of view and vertically long videos focusing on people are simultaneously delivered, allowing users to select the video they want to view and enjoy it at the field of view they want. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2019-057291 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when multiple viewpoints are being distributed, such as horizontally long videos showing the entire video or vertically long videos focusing on people, it is time-consuming to select the video data you want to watch and change the screen layout to match the size of the video data. For example, it is necessary to change the display orientation settings of the viewing device or change the orientation of the device itself, and there is a problem that important scenes may be missed while performing such operations.

[0006] The present invention has been made in consideration of the above problems, and has as its object to enable a user to easily view video data that the user desires to view in a desired screen layout. [Means for solving the problem]

[0007] In order to achieve the above object, an electronic device of the present invention includes: an acquisition means for acquiring a plurality of images obtained from a plurality of different image sources; a display means; an operation unit that receives an operation to instruct that an image displayed on the display unit among the plurality of images be associated with an orientation of the display unit; The display device includes a storage means for storing information relating the orientation of the display means to at least one of the plurality of images based on operation by an operation means, a detection means for detecting the orientation of the display means, and a control means for controlling the display means to display an image obtained from the same image source as the image associated with the detected orientation. [Effects of the Invention]

[0008] According to the first aspect of the present invention, desired video data can be easily viewed in a desired screen layout. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing an example of the configuration of a video distribution system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram showing an example of the configuration of an electronic device according to an embodiment. [Figure 3] 10 is a flowchart of a priority display setting process according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a priority display setting screen according to the first embodiment. [Figure 5] 4 is a flowchart of a video display process according to the first embodiment. [Figure 6] FIG. 3 is a diagram showing an example of a display screen in the first embodiment. [Figure 7] 10 is a flowchart of a priority display setting process according to the second embodiment. [Figure 8] FIG. 10 is a diagram showing an example of the priority of video data in the second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a priority display setting screen according to the third embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a display layout setting screen according to the third embodiment. [Figure 11] 10 is a flowchart of a video display process according to the third embodiment. [Figure 12] FIG. 11 is a diagram showing an example of a display screen in the third embodiment. [Figure 13] FIG. 13 is a diagram showing an example of the priority of video data in a modified example of the third embodiment. [Figure 14] FIG. 13 is a diagram showing an example of a display screen in the fourth embodiment. [Figure 15] FIG. 10 is a diagram for explaining a method of displaying video data in the fourth embodiment. [Figure 16] FIG. 10 is a diagram for explaining a method of displaying video data in the fourth embodiment. [Figure 17] FIG. 13 is a diagram showing an example of audio information linked to video data and the priority for each mode in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] 1 is a diagram showing an example of the configuration of a video distribution system according to this embodiment. Video cameras 101-104 record video signals and output the video signals to electronic device 200, which is connected wirelessly or via a wire. Electronic device 200 edits the video signals received from video cameras 101-104, improving image quality, audio, adding subtitles, etc., and distributes the created video data via communication network 201, such as the Internet. Note that electronic device 200 may distribute the video signals received from video cameras 101-104 as is, without editing them.

[0012] The electronic device 300 displays and plays back the video data received via the communication network 201 on the screen of a display unit. Note that although the electronic device 300 is taken as an example of an information processing device here, it may be any terminal having a display unit on which video content can be viewed, such as a digital camera with a wireless function, a tablet device, or a personal computer.

[0013] FIG. 2 is a block diagram showing an example of the configuration of an electronic device 300 that constitutes the system of FIG. 1 and is an example of an information processing device according to this embodiment. The control unit 11 controls each unit of the electronic device 300 in accordance with input signals and programs stored in the non-volatile memory 13 described below. Note that instead of the control unit 11 controlling the entire device, the entire device may be controlled by a plurality of hardware components sharing the processing.

[0014] The imaging unit 12 converts the subject light imaged by the lens included in the imaging unit 12 into an electrical signal, performs noise reduction processing, etc., and outputs the digital data as image data. The captured image data is stored in a buffer memory, and then the control unit 11 performs a predetermined calculation on the data and records it on the recording medium 17.

[0015] The nonvolatile memory 13 is an electrically erasable and recordable nonvolatile memory. The nonvolatile memory 13 stores an OS (operating system), which is basic software executed by the control unit 11, and applications that cooperate with the OS to realize applied functions. In this embodiment, the nonvolatile memory 13 also stores an application for communicating with the electronic device 200. The work memory 14 is used as an image display memory for the display unit 16, a work area for the control unit 11, and the like.

[0016] The operation unit 15 is used to receive instructions from a user for the electronic device 300. The operation unit 15 includes operation members such as a power button that the user uses to instruct the electronic device 300 to power on / off, and a touch panel 16a.

[0017] The touch panel 16a may be any of a variety of touch panel types, including resistive film type, capacitive type, surface acoustic wave type, infrared type, electromagnetic induction type, image recognition type, optical sensor type, etc. Depending on the type, there are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of a finger or pen to the touch panel, and either type is acceptable.

[0018] The touch panel 16a and the display unit 16 can be configured as an integrated unit. For example, the touch panel 16a is configured so that its light transmittance does not interfere with the display of the display unit 16, and is attached to the upper layer of the display surface of the display unit 16. Then, input coordinates on the touch panel 16a are associated with display coordinates on the display screen of the display unit 16. This makes it possible to provide a GUI (Graphical User Interface) that allows the user to directly operate the screen displayed on the display unit 16. The control unit 11 can detect operations or states on the touch panel 16a.

[0019] The display unit 16 displays image data and text for interactive operations. The recording medium 17 can record image data output from the imaging unit 12. The recording medium 17 may be configured to be detachable from the electronic device 300, or may be built into the electronic device 300. In other words, the electronic device 300 only needs to have a means for accessing the recording medium 17.

[0020] The communication unit 18 is an interface for connecting to an external device. The electronic device 300 of this embodiment can exchange data with the electronic device 200 via the communication unit 18. In this embodiment, the communication unit 18 is an antenna, and the control unit 11 can connect to the electronic device 200 via the antenna. Note that the connection with the electronic device 200 may be direct or via an access point. A protocol for data communication may be, for example, PTP / IP (Picture Transfer Protocol over Internet Protocol) via a wireless LAN. Note that communication with the electronic device 200 is not limited to this. For example, the communication unit 18 may include a wireless communication module such as an infrared communication module, a Bluetooth (registered trademark) communication module, or a Wireless USB. Furthermore, a wired connection such as a USB cable, HDMI (registered trademark), or IEEE1394 may also be employed.

[0021] The short-range wireless communication unit 19 is a communication unit for realizing contactless short-range communication with other devices. The short-range wireless communication unit 19 is composed of an antenna for wireless communication, a modulation / demodulation circuit for processing wireless signals, and a communication controller. The short-range wireless communication unit 19 realizes contactless short-range communication by outputting modulated wireless signals from the antenna and demodulating wireless signals received by the antenna. Here, contactless communication is realized in accordance with the ISO / IEC 18092 standard (so-called NFC (registered trademark)).

[0022] When the short-range wireless communication unit 19 receives a data read request from another device, it outputs response data based on the data stored in the nonvolatile memory 13. In this embodiment, the electronic device 300 operates in a card reader mode, a card writer mode, and a P2P mode defined by the NFC (registered trademark) standard through the short-range wireless communication unit 19, and mainly behaves as an initiator. On the other hand, in the card emulation mode, the electronic device 300 mainly behaves as a target through the short-range wireless communication unit 19.

[0023] Alternatively, the short-distance wireless communication unit 19 outputs a modulated wireless signal from an antenna and demodulates a wireless signal received by the antenna to realize short-distance wireless communication in accordance with the IEEE802.15 standard. In this embodiment, the short-distance wireless communication unit 19 realizes short-distance wireless communication in accordance with the IEEE802.15.1 standard (so-called Bl ue In this embodiment, the Bluetooth communication uses the low-power consumption Bluetooth Low Energy version 4.0 (BLE).

[0024] The public network communication unit 20 is an interface used when performing public wireless communication. The electronic device 300 can communicate with other devices via the public network communication unit 20. In this case, the control unit 11 inputs and outputs audio signals via a microphone 21 and a speaker 22 to realize the communication. In this embodiment, the public network communication unit 20 is an antenna, and the control unit 11 can connect to the public network via the antenna. It is also possible for the communication unit 18 and the public network communication unit 20 to share a single antenna.

[0025] The orientation detection unit 23 detects the orientation of the electronic device 300 with respect to the direction of gravity. Based on the orientation detected by the orientation detection unit 23, it is possible to determine whether the image captured by the imaging unit 12 was captured with the electronic device 300 held horizontally or vertically. The control unit 11 can add orientation information corresponding to the orientation detected by the orientation detection unit 23 to the image file of the image captured by the imaging unit 12, or rotate and record the image.

[0026] An acceleration sensor, a gyro sensor, or the like can be used as the attitude detection unit 23. The attitude detection unit 23 can also detect the movement of the electronic device 300 (pan, tilt, roll, lift, whether or not it is stationary, etc.) using an acceleration sensor or a gyro sensor.

[0027] First Embodiment ●Setting the priority display image 3 is a flowchart of the priority display setting process performed in the electronic device 300 in the first embodiment, and each process is executed by the control unit 11 by loading a program stored in the nonvolatile memory 13 into the work memory 14. Note that this priority display setting process is executed when, for example, "priority display setting" is displayed as one of the options in a menu and this option is selected via the operation unit 15.

[0028] In S301, the control unit 11 receives one or more pieces of video data captured by at least one of the video cameras 101 to 104 and distributed from the electronic device 200 via the communication network 201, and stores the data in the working memory 14 within the main body. The video data stored in the working memory 14 is not limited to the video distributed and received by the electronic device 200. Specifically, the video data may be video data captured by an in-camera or an out-camera built into the electronic device 300.

[0029] In S302, the control unit 11 acquires the orientation of the display unit 16 for which the priority display setting is selected via the operation unit 15, and stores the acquired information in the working memory 14. Specifically, this information is information on orientation such as portrait position, landscape position, etc., and will be hereinafter referred to as portrait display mode and landscape display mode. In S303, the control unit 11 reads out the video data stored in the work memory 14, displays it on the display unit 16, and displays a screen for setting the video data to be preferentially displayed in each display mode.

[0030] FIG. 4 shows an example of a priority display setting screen for setting video data to be preferentially displayed in portrait and landscape display modes. Window 401 is a window for landscape display mode, and window 402 is a window for portrait display mode. When electronic device 300 is determined to be in landscape display mode or portrait display mode, video data to be preferentially displayed on display unit 16 is displayed. Images 403, 404, and 405 are images of video data captured by different video cameras 101-104 or a camera built into electronic device 300, acquired in S301. The user first selects an area of window 401 or 402 via operation unit 15, and then reselects one of images 403, 404, or 405. This allows the user to specify the video data to be preferentially displayed in landscape or portrait display mode corresponding to selected window 401 or 402.

[0031] At this time, the video data displayed as images 403, 404, and 405 is not limited to the video data acquired in S301, and previously acquired video data stored in working memory 14 may be displayed, for example, in chronological order. In this way, video data captured by a video camera viewed in the past can be looked back on and set, making it easier for the user to fix video data from the video camera intended in the display area.

[0032] In S304, the control unit 11 displays the video data selected in S303 in the window 401 or 402, and also stores information about the selected video data in the working memory 14 in association with the display mode.

[0033] In S305, the control unit 11 determines whether to end the priority display setting process. Here, it determines to end the process if the "Cancel" button 406 or the "Determine" button 407 is pressed. If the "Cancel" button 406 is pressed, the setting contents are discarded, and if the "Determine" button 407 is pressed, the setting contents are saved and the process ends. If neither the "Cancel" button 406 nor the "Determine" button 407 has been pressed, the priority display setting process is continued, and the process returns to S301.

[0034] ● Display of priority display image Next, the display process of the video data set to be preferentially displayed for each display mode as described above will be described with reference to Fig. 5. Each process is executed by the control unit 11 by loading a program stored in the non-volatile memory 13 into the work memory 14.

[0035] In S501, the control unit 11 acquires one or more pieces of video data captured by at least one of the video cameras 101 to 104 and distributed from the electronic device 200 via the communication network 201, video data stored in the electronic device 300, etc. Then, the acquired video data is used to update the video data stored in the working memory 14 in the main body.

[0036] In S502, the control unit 11 acquires the value of the attitude information from the attitude detection unit 23 and stores it in the work memory 14. In S503, the control unit 11 acquires the value of the attitude information stored in the working memory 14 and determines the attitude of the electronic device 300. Table 1 shows an example of the criteria for determining the attitude of the electronic device 300 from the attitude information.

[0037] TIFF0007719622000001.tif26152 When the orientation information of the electronic device 300 is between -100 and 100, it is determined to be in a vertical position, and when it is between 101 and 300, it is determined to be in a horizontal position (clockwise). 0If it is between -101 and -300, it is determined to be in a vertical position (upside down), and if it is between -101 and -300, it is determined to be in a horizontal position (counterclockwise). The determination result is then stored in the working memory 14 as the latest orientation information of the electronic device 300. Note that the orientation information is not limited to vertical and horizontal, and may be for multiple directions.

[0038] Next, in S504, the control unit 11 compares the past orientation information of the electronic device 300 with the latest orientation information of the electronic device 300 stored in the working memory 14 in the immediately preceding S503, and if they are different, proceed to S505, and if they are the same, proceed to S506. 7 Proceed to.

[0039] In S505, the control unit 11 overwrites the previous orientation information of the electronic device 300 stored in the work memory 14 with the latest orientation information of the electronic device 300 stored in the work memory 14 in S503.

[0040] In S506, the control unit 11 displays on the display unit 16 the video data obtained from the same video source as the video data set by the processing described in Figure 3, in accordance with the latest orientation information of the electronic device 300 stored in the working memory 14.

[0041] In S507, the control unit 11 determines whether or not to end viewing. The determination of whether or not to end viewing is notified by the user via, for example, a touch operation on the touch panel 16a of the operation unit 15.

[0042] An example of displaying video data according to the priority display process shown in FIG. 5 will be described with reference to FIG. As shown in FIG. 6(a), when the orientation information of electronic device 300 is set to portrait, video data obtained from the same video source as the video data of image 404 set in portrait display mode in FIG. 4 is displayed.

[0043] If the orientation of the electronic device 300 is changed from this state to the landscape position (YES in S504), 300 The orientation information of is changed (S505), and 6As shown in (b), the display is changed to video data obtained from the same video source as the video data of image 403 set in landscape display mode in FIG. 4 (S506).

[0044] As described above, according to the first embodiment, it is possible to switch to desired video data that has been set in advance in conjunction with a change in the orientation of the terminal screen.

[0045] Furthermore, the video data displayed on the screen of the terminal may correspond to the orientation information of electronic device 300, and automatically display landscape video data when display unit 16 is in landscape orientation, and portrait video data when in portrait orientation. In this way, the video data to be displayed in the display area can be displayed without trimming or resizing, which would significantly change the video data, and therefore the video data can be displayed in the display area of display unit 16 without losing any information about the video data.

[0046] <Second embodiment> Next, a second embodiment of the present invention will be described. In the first embodiment, a case where one piece of video data is set for each of the portrait and landscape display modes has been described, whereas in the second embodiment, a case where multiple pieces of video data are set with priorities assigned for each display mode will be described.

[0047] ●Setting the priority display image 7 is a flowchart of the priority display setting process performed by the electronic device 300 in the second embodiment, and each process is executed by the control unit 11 by loading a program stored in the non-volatile memory 13 into the work memory 14. As in the first embodiment, this priority display setting process is executed when, for example, "Priority Display Setting" is displayed as one of the options in a menu and this option is selected via the operation unit 15. In FIG. 7, the processes of S301 to S303 and S305 are the same as those shown in FIG. 3, and therefore the same reference numerals are used and their description will be omitted.

[0048] When a display mode is selected in S302, the process proceeds to S303, where one of the video data stored in the working memory 14 is selected. Then, in S701, the video data selected first is displayed, and the information of the selected video data is stored in the working memory 14 with priority 1, associated with the display mode.

[0049] Then, in S702, it is determined whether or not to select video data with the next highest priority, and if so, the process returns to S303 and repeats the above processing. Note that each time the process of S303 is repeated, the priority is lowered in order.

[0050] When the selection of video data is completed, in S703 it is determined whether or not to change the display mode, and if so, the process returns to S303 and the above processing is performed. For example, in the example shown in Fig. 4, when video data is being selected for the landscape display mode of window 401, if portrait display mode is selected by touching window 402, the display mode is changed.

[0051] If the display mode is not changed in S703, the process proceeds to S305, where the control unit 11 determines whether to end the priority display setting process. If an instruction to end has not been given, the process returns to S702 and is repeated; if an instruction to end has been given, the process ends.

[0052] Fig. 8 is a table showing an example of the setting contents by the process shown in Fig. 7. In Fig. 8, it is assumed that video data A to E are held in the working memory 14 as video data that can be set.

[0053] Note that the display process of the priority display image in the second embodiment is the same as the process described above with reference to Figure 5, except that in S506, video data obtained from the same video source as the video data with a higher priority among the video data that can be displayed is displayed, so explanation will be omitted.

[0054] As described above, according to the second embodiment, multiple pieces of video data are stored with priority assigned for each display mode. This makes it possible to display video data obtained from the same video source as the video data with the next highest priority, even if the video data cannot be obtained from the same video source as the video data with the highest priority.

[0055] The priority may be set by the user, or may be automatically assigned based on the angle of view taken. For example, based on the size of the distributed video and the attached information about the posture at the time of shooting, video data taken with a vertically long angle of view may be assigned a high priority in portrait display mode, and video data taken with a horizontally long angle of view may be assigned a high priority in landscape display mode.

[0056] <Third embodiment> Next, a third embodiment of the present invention will be described. In each display mode, the number of pieces of video data that can be displayed on one screen is not limited to one, and multiple pieces of video data may be displayed simultaneously. In the third embodiment, the display layout is set following the priority display setting process shown in Fig. 7 described in the second embodiment. Note that this display layout setting process is executed, for example, when a "display layout setting" button 901 is pressed via the operation unit 15, as shown in Fig. 9, and the screen shown in Fig. 10 is displayed.

[0057] ● Display layout settings Fig. 10 shows a screen for selecting a display layout. As an example, Fig. 9 shows a screen for setting the layout when multiple video data are displayed on one screen while the landscape display mode is selected.

[0058] Reference numeral 501 denotes the display layout currently being selected. The user can select from layout candidates 502, 503, and 504 via the operation unit 15 (for example, by touching the touch panel 16a). The example shown in FIG. 10(a) shows a case where layout candidate 503 is selected. The display layout may be a layout in which one screen is divided, as in layout candidates 503 and 504, or a layout in which a display area for displaying one piece of video data, as in layout candidate 502, has a second display area 505 for displaying other video data.

[0059] In the third embodiment, the display areas are prioritized in descending order of size in each layout candidate, and whether the display area is in portrait or landscape orientation is automatically determined according to the aspect ratio of each display area. However, this may also be configured to be manually set by the user.

[0060] 10(b) is a table showing, as an example, the distinction between vertical and horizontal positions of the display areas 5011 to 5013 of the selected display layout 501 and an example of the priority order of each display area. In the third embodiment, when video data is displayed, the video data is selected and displayed according to the priority order shown in FIG. 8 by a video data display process described later with reference to FIG. 11.

[0061] For example, display area 5011 has the first priority in portrait orientation, and therefore video data D is displayed. Display area 5012 has the first priority in landscape orientation, and therefore video data C is displayed. Display area 5013 is in landscape orientation and is smaller than display area 5012, and therefore has the second priority in landscape orientation, and therefore video data B is displayed.

[0062] When a layout candidate is selected and the "OK" button 507 in FIG. 10(a) is pressed, the selected layout is retained, and when the "Cancel" button 506 is pressed, the display layout settings are discarded and the screen returns to that of FIG. 9.

[0063] If the portrait display mode is selected in FIG. 9, layout candidates for dividing the window 402 are displayed.

[0064] Furthermore, the layout candidates are not limited to the three displayed layout candidates 502, 503, and 504, and there may be four or more candidates. In this case, for example, a simplified display such as 508 is displayed to indicate that there are other layout candidates, and by performing a swipe operation or the like, the other hidden layout candidates are displayed in the area where the layout candidates 502, 503, and 504 were displayed.

[0065] ● Display of priority display image Next, the display process of video data set to be preferentially displayed in each display area as described above will be described with reference to Fig. 11. Each process is executed by control unit 11 expanding a program stored in non-volatile memory 13 into working memory 14. In the process shown in Fig. 11, the same processes as those shown in Fig. 5 are denoted by the same reference numerals, and their description will be omitted where appropriate.

[0066] In S505, the previous orientation information of the electronic device 300 stored in the work memory 14 is overwritten with the latest orientation information of the electronic device 300 stored in the work memory 14 in S503, and then the process proceeds to S1101. In S1101, the control unit 11 changes the display layout in accordance with the latest orientation information of the electronic device 300 stored in the work memory 14.

[0067] In S1102, the control unit 11 obtains the number of display areas from the display layout information stored in the work memory 14, stores it in the loop variable Num, and initializes the variable i to 0. Thereafter, the processes from S1103 to S1106 are repeated as long as the variable i is smaller than the loop variable Num.

[0068] S110 3Then, the control unit 11 increments the variable i. Then, in S1104, the control unit 11 selects the display area with the highest priority among the display areas in the display layout in which no video data is displayed. Here, the larger the area of the display area, the higher the priority.

[0069] In S1105, the control unit 11 selects the video data with the highest priority from among the video data not displayed in the display area according to the priority shown in FIG. 8, in accordance with the type of vertical / horizontal position shown in FIG. 10(b) and the priority for each position for the selected display area. In S1106, the control unit 11 displays the video data obtained from the same video source as the video data selected in S1105 in the display area selected in S1104.

[0070] Then, in S1107, if the variable i is equal to or greater than the loop variable Num, the process proceeds to S507, and if it is smaller than the loop variable Num, the process returns to S1103 and repeats the above processing.

[0071] An example of displaying video data according to the priority display process shown in FIG. 11 will be described with reference to FIG. FIG. 12 is a diagram showing an example of video data displayed when a display layout conforming to layout candidate 502 is selected in portrait display mode and landscape display mode in the display layout setting described in the third embodiment.

[0072] 12(a), when portrait orientation is set as orientation information of electronic device 300, loop variable Num is set to 2 in S1102. Then, by the processes of S1103 to S1106, video data obtained from the same video source as video data D with priority 1 set in portrait display mode in Fig. 8 is displayed in display area 1201. Also, video data obtained from the same video source as video data C with priority 1 set in landscape display mode in Fig. 8 is displayed in display area 1202.

[0073] If the orientation of the electronic device 300 is changed from this state to the landscape position (YES in S504), 300 The orientation information is changed (S505), and the display layout selected in landscape display mode is obtained and changed (S1101). Then, in S1102, the variable Num is set to 2, and by the processes of S1103 to S1106, video data obtained from the same video source as video data C with priority 1 set in landscape display mode in Fig. 8 is displayed in display area 1203. Also, video data obtained from the same video source as video data D with priority 1 set in portrait display mode in Fig. 8 is displayed in display area 1204.

[0074] As described above, according to the third embodiment, even when one screen is divided into a plurality of display areas, it is possible to switch to desired video data in conjunction with a change in the orientation of the terminal screen.

[0075] <Variation 1> Instead of using the priority order of video data for the display modes shown in FIG. 8, the priority order of video data may be set for each layout candidate.

[0076] Fig. 13 is a table showing the priority of video data in portrait / landscape orientation for each layout candidate, and is used instead of the priority shown in Fig. 8. Note that in order to assign a priority to video data for each layout candidate, for example, this can be done by selecting each layout candidate shown in Fig. 10(a) instead of selecting a display mode in S302 of the process shown in Fig. 7.

[0077] In this way, by setting the video data to be displayed preferentially for each layout candidate, it becomes possible to display different video data preferentially when the screen is not divided and when it is divided into multiple display areas.

[0078] <Fourth embodiment> Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, various control methods for setting a display area for displaying other video data while video data is being displayed will be described.

[0079] Fig. 14 is a diagram showing an example of a display screen in this embodiment. When video data set in portrait display mode is displayed in main display area 1401, a "KEEP" button 1402 is displayed as shown in Fig. 14(a). When this "KEEP" button 1402 is pressed, a sub-display area 1403 is set within main display area 1401 as shown in Fig. 14(b), and the video data displayed in main display area 1401 is displayed in sub-display area 1403.

[0080] In this state, if the orientation of electronic device 300 is changed to landscape, the display state will be as shown in Figure 12(b), and the main display area 1203 will display video data set to landscape display mode, and sub-display area 1204 will continue to display the video data that was displayed in sub-display area 1403.

[0081] In this way, in video content such as a television program in which images from a plurality of cameras are switched and distributed, the user can easily specify the content distributed from a favorite video camera and continue viewing it.

[0082] The video data to be displayed in the sub-display area may be set by inputting a fixed instruction or touching the operation unit 15, or by inputting a voice instruction from the microphone 21.

[0083] Fig. 15(a) is a schematic diagram showing the state when sub-display area 1502 is selected by a selection operation via operation unit 15. When sub-display area 1502 is selected by an input operation via operation unit 15 in this way, sub-display area 1502 may be hidden as shown in Fig. 15(b), changed to display a small icon 1503 as shown in Fig. 15(c), or moved to another area as shown in Fig. 15(d). Note that when icon 1503 is selected, it returns to the display of sub-display area 1502 of Fig. 15(a).

[0084] Furthermore, the iconization and movement of the sub-display area 1502 may be performed automatically based on the subject information of the display area 1501. Specifically, the control unit 11 refers to the subject information of the video data stored in the working memory 14 and determines whether the sub-display area 1502 overlaps with the subject displayed in the display area 1501. If they overlap, the sub-display area 1502 is hidden as shown in FIG. 15(b), iconized as shown in FIG. 15(c), or moved to another area as shown in FIG. 15(d). If they do not overlap, the display continues as is. This makes it possible to avoid a situation where the sub-display area 1502 overlaps with the subject and the desired video cannot be viewed.

[0085] 16 is a diagram showing an example of the process of changing the video data displayed in each of multiple display areas, and the control shown below can be performed when multiple video data are set with priority. The user can change the video data displayed in the main display area 1601 by operating the operation unit 15 or by touching the main display area 1601 with a swipe or other touch operation.

[0086] Specifically, when a rightward swipe operation is input while video data obtained from the same video source as the video data with priority 1 is displayed in the main display area 1601, the video data displayed in the main display area 1601 is changed to video data obtained from the same video source as the video data with priority 2, which has the next lowest priority in portrait display mode. In the example shown in FIG. 8, the video data obtained from the same video source as the video data D is changed to video data obtained from the same video source as the video data C. At this time, if the swipe operation is performed on the sub-display area 1602, it may be processed as an instruction to change the video data displayed in the sub-display area 1602. Furthermore, the processing may be changed depending on the swipe direction, for example, by increasing the priority of the video data to be displayed if the swipe operation is to the right and decreasing the priority if the swipe operation is to the left.

[0087] Although the method for switching the video data displayed on the display unit 16 based on the posture information of the electronic device 300 has been described so far, it is also possible to determine the subject of the video data and switch the audio data simultaneously with the video.

[0088] 17 is a table relating to audio information when the video data displayed in the main display area 1201 and the sub-display area 1202 as shown in FIG. 12(a) is changed to the main display area 1203 and the sub-display area 1204 as shown in FIG. 12(b) based on a change in the orientation of the display unit 16. The audio information output from the speaker 22 built into the electronic device 300 is set so that a higher proportion of the audio information is given to the video data displayed in the display area with a higher priority in the current orientation stored in the working memory 14.

[0089] Note that audio information is not limited to information linked to video data, and may be determined for each subject. Specifically, audio data for each subject linked to video data may be received, and a setting may be made to output more of the audio of the subject appearing in the video data displayed in the display area with priority 1. In this way, when video data showing the same subject is displayed preferentially, the audio of the subject can be heard louder, allowing the user to selectively listen to the audio they want to hear.

[0090] As described above, according to the fourth embodiment, video data can be switched in various ways, and desired video data can be easily switched to without complicated key operations or the like.

[0091] In the above-described embodiment, the present invention has been described as being applied to an electronic device such as a smartphone, but the present invention is not limited to this example and can be applied to any electronic device that has a display unit and can determine the orientation of the device itself based on information acquired from an orientation detection unit. In other words, the present invention can also be applied to cameras, personal computers, PDAs, and the like that have an imaging unit and an image playback function.

[0092] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0093] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0094] 11: System control unit, 12: Imaging unit, 13: Non-volatile memory, 14 Working memory, 15: Operation unit, 16: Display unit, 16a: Touch panel, 17: Recording medium, 18: Communication unit, 19: Short-range wireless communication unit, 20: Public network communication unit, 21: Microphone, 22: Speaker, 23: Attitude detection unit, 101 to 104: Video camera, 200: Electronic device, 201: Communication network, 300: Electronic device

Claims

1. acquisition means for acquiring a plurality of images obtained from a plurality of different image sources; A display means; an operation means for receiving an operation to instruct associating an image displayed on the display means, among the plurality of images, with an orientation of the display means; a storage means for storing information associating an orientation of the display means with at least one of the plurality of images based on an operation by the operation means; a detection means for detecting the orientation of the display means; a control means for controlling the display means to display an image obtained from the same image source as the image associated with the detected orientation; An electronic device comprising:

2. 2. The electronic device according to claim 1, wherein the storage means stores one of the plurality of images in association with each orientation of the display means.

3. the storage means stores one or more images from among the plurality of images in association with each orientation of the display means, with priority assigned to each image; 2. The electronic device according to claim 1, wherein the control means controls the display of images obtained from the same image source as the image associated with the detected orientation, by selecting and displaying images that can be displayed in order of higher priority.

4. 4. The electronic device according to claim 3, wherein the control means controls the display means to simultaneously display a plurality of images on the screen, and sets priorities for the display areas displaying the plurality of images.

5. The electronic device according to claim 4, characterized in that the control means determines the orientation of each display area in order of the highest priority, and controls the display means to select and display, from among images obtained from the same image source as the image associated with that orientation as the orientation of the display means, images that can be displayed in order of the highest priority.

6. the control means controls the display means to simultaneously display a plurality of images on the screen, and sets priorities for the display areas displaying the plurality of images; 2. The electronic device according to claim 1, wherein the storage means stores information that associates the orientation of the display area with at least one of the plurality of images based on an operation by an operation means.

7. The electronic device described in any one of claims 1 to 3 and 6, characterized in that when an instruction is given to retain the image displayed in a first display area of the display means while the control means is displaying an image in the first display area, the control means sets a second display area within the first display area and displays the same image as the first display area in the second display area.

8. 8. The electronic device according to claim 7, wherein the control means controls the display means to maintain the second display area and display the image that was displayed in the first display area when the orientation of the display means is changed.

9. 8. The electronic device according to claim 7, wherein when the second display area is designated by the operation means, the control means changes the second display area to an icon or makes the second display area invisible.

10. The electronic device according to claim 7, characterized in that the control means detects the position of a subject in the image displayed in the first display area, and when the second display area overlaps the subject, changes the second display area to an icon, makes it invisible, or moves it to a position where it does not overlap the subject.

11. Further, a touch panel is provided on the surface of the display means, The electronic device according to any one of claims 3 to 6, characterized in that when a swipe operation is performed on the touch panel, the control means changes the image displayed in the swipe area to an image with a higher or lower priority depending on the direction of the swipe.

12. Further comprising a speaker; The electronic device described in any one of claims 3 to 11, characterized in that when multiple images are displayed on the display means, the control means sets the proportion of audio information associated with each of the multiple images in accordance with the priority set for each display area displaying the multiple images, and controls the audio information to be output from the speaker.

13. 13. The electronic device according to claim 1, wherein the operation means includes means for accepting at least one of a fixed instruction input and a voice instruction.

14. The electronic device according to any one of claims 1 to 13, characterized in that the control means displays the plurality of images obtained by the acquisition means on the display means, and the operation means selects an image to be associated with each orientation of the display means from the plurality of displayed images.

15. acquisition means for acquiring a plurality of images obtained from a plurality of different image sources; A display means; an operation means for receiving an operation to instruct associating an image displayed on the display means, among the plurality of images, with an orientation of the display means; a detection means for detecting the orientation of the display means; a control means for controlling the display means to display an image in the same orientation as the detected orientation; An electronic device comprising:

16. acquiring a plurality of images obtained from a plurality of different image sources; an operation step of receiving an operation by an operation means for instructing that an image displayed on a display means among the plurality of images be associated with an orientation of the display means; a storage step of storing information in storage means that associates an orientation of the display means with at least one of the plurality of images based on an operation by the operation means; a detection step of detecting the orientation of the display means; a display step of displaying, on the display means, an image obtained from the same image source as the image associated with the detected orientation; 1. A method for controlling an electronic device, comprising:

17. acquiring a plurality of images obtained from a plurality of different image sources; an operation step of receiving an operation to instruct associating an image displayed on a display means, among the plurality of images, with an orientation of the display means; a detection step of detecting the orientation of the display means; a display step of displaying an image in the same orientation as the detected orientation on the display means; 1. A method for controlling an electronic device, comprising:

18. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 15.

19. A computer-readable storage medium storing the program according to claim 18.

Images