Information processing device, information processing method, and program

Abstract

The invention relates to an information processing apparatus, an information processing method, and a program. [Problem] To add an extended function that enables a more intuitive operation without hindering the behavior of an external application. [Solution] Provided is an information processing device provided with a control unit for controlling an extended function of an external application. And a control unit that displays an extended menu icon whose display position is adjustable on a display window of an external application by superimposing, and controls the display position of the extended menu according to the display position of the extended menu icon, and a control unit configured to control the display position of the sub-icon associated with the extended function icon according to the display position of the extended function icon such that the extended function icon belonging to the extended function is selected from the extended menu to be displayed on the display window by superimposing in a manner that the display position thereof is adjustable, and control the display position of the sub-icon associated with the extended function icon according to the display position of the extended function icon.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201980092167.6, titled "Information Processing Apparatus, Information Processing Method, and Program", filed on August 16, 2021. Technical Field

[0002] The present disclosure relates to an information processing apparatus, an information processing method, and a program. Background Art

[0003] In recent years, various applications with high entertainment value have been developed. In addition, techniques have been proposed that can create various additional values by providing extended functions in cooperation with such applications as described above. For example, PTL 1 discloses a technique for synthesizing an image of a user who is playing a game application onto a game screen and for distributing the resulting screen.

[0004] [Citation List]

[0005] [Patent Document]

[0006] [PTL 1]

[0007] JP 2017-188833A Summary of the Invention

[0008] [Technical Problem]

[0009] As described above, in the case of providing an extended function to an external application, cooperation is important so as not to cause any interference with the original behavior of such an external application and the user operations of the application.

[0010] [Solution to the Problem]

[0011] According to the present disclosure, there is provided an information processing apparatus including a controller configured to control the extension of functions of an external application. The controller is configured to cause an extension menu icon whose display position is adjustable to be superimposed and displayed on a display window of the external application, and to control the display position of the extension menu based on the display position of the extension menu icon; and to cause an extended function icon whose display position is adjustable to be superimposed and displayed on the display window, where the extended function icon is associated with an extended function selected on the extension menu, and to control the display position of a sub-icon accompanying the extended function icon based on the display position of the extended function icon.

[0012] In addition, according to the present disclosure, an information processing method is provided, which includes controlling the function expansion of an external application by a processor. The control includes causing an expansion menu icon whose display position is adjustable to be superimposed and displayed on the display window of the external application, and controlling the display position of the expansion menu according to the display position of the expansion menu icon; and causing an expansion function icon to be superimposed and displayed on the display window, where the display position of the expansion function icon is adjustable, the expansion function icon is associated with an expansion function selected on the expansion menu, and controlling the display position of a sub-icon accompanying the expansion function icon according to the display position of the expansion function icon.

[0013] In addition, according to the present disclosure, a program is provided, which causes a computer to function as an information processing device, and the information processing device includes a controller configured to control the function expansion of an external application. The controller is configured to cause an expansion menu icon whose display position is adjustable to be superimposed and displayed on the display window of the external application, and controlling the display position of the expansion menu according to the display position of the expansion menu icon; and causing an expansion function icon to be superimposed and displayed on the display window, where the display position of the expansion function icon is adjustable, the expansion function icon is associated with an expansion function selected on the expansion menu, and controlling the display position of a sub-icon accompanying the expansion function icon according to the display position of the expansion function icon. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A block diagram is depicted showing an example of the functional configuration of an information processing device according to an embodiment of the present disclosure.

[0015] Figure 2 A diagram is depicted for illustrating the cooperation between an extended application and an external application according to an embodiment.

[0016] Figure 3A A diagram is depicted for illustrating the display of an expansion menu based on the display position of an expansion menu icon according to an embodiment.

[0017] Figure 3B A diagram is depicted for illustrating the display of an expansion menu based on the display position of an expansion menu icon according to an embodiment.

[0018] Figure 4 A diagram is depicted for illustrating the display control of an expansion menu icon MI based on non-operation time according to an embodiment.

[0019] Figure 5 A diagram is depicted for illustrating an image recording function according to an embodiment.

[0020] Figure 6 A diagram is depicted for illustrating an image recording function according to an embodiment.

[0021] Figure 7 A diagram for illustrating an image recording function according to an embodiment is depicted.

[0022] Figure 8 A diagram for illustrating an example of the hardware configuration of an information processing apparatus according to an embodiment of the present disclosure is depicted. DETAILED DESCRIPTION

[0023] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that, in this specification and the accompanying drawings, elements having substantially the same function and structure are denoted by the same reference numerals, and redundant description is omitted.

[0024] Note that the description will be made in the following order.

[0025] 1. Embodiment

[0026] 1.1. Example of Functional Configuration

[0027] 1.2. Details of Functions

[0028] 2. Example of Hardware Configuration

[0029] 3. Summary

[0030] <1. Embodiment>

[0031] <<1.1. Example of Functional Configuration>>

[0032] First, an example of the functional configuration of an information processing apparatus 10 according to an embodiment of the present disclosure will be described. As described above, in recent years, various applications having high entertainment value have been developed. Examples of such applications as described above include game applications. Users can easily play various game applications by using a terminal such as a smart phone.

[0033] In addition, in recent years, the fun of users is not limited to playing game applications. Sharing the game state of a user in a game application with others has become popular. For example, a user can transmit a moving image generated by synthesizing the user's face image on a recording screen of the play state of a game application to an acquaintance, or can distribute the moving image to an unspecified large number of viewers / listeners via a moving image distribution service or other service. Such sharing of the moving image enables viewers / listeners to appreciate the play state of the game application and the reaction of the distributor, etc., and thus enables sharing of the game experience among multiple users.

[0034] Here, in the case of generating a moving image as described above, generally, after recording an image of the game state of a game application and simultaneously taking a picture of the user's image, editing work is required to synthesize the user's own image. However, such editing work requires knowledge of moving image editing and the like in addition to time and labor.

[0035] On the other hand, as described above, in order to alleviate such a burden on the distributor, as described in, for example, PTL 1, there is also known a technique for automatically generating a game-playing moving image in which the distributor's own image is synthesized by collaborating with a game application. By using such a technique, even a distributor who does not have knowledge of moving image editing can easily generate a game-playing moving image of a game application, distribute the obtained image, and perform any other related actions.

[0036] However, in the case of providing an extended function to an external application using such a technique as described above, cooperation is important so as not to cause any interference with the original behavior of such an external application and the user's operation of the external application. In addition, it is also desired that the user can operate the external application more intuitively according to the situation of the external application and flexibly control the extended function.

[0037] In view of the above circumstances, the technical idea of the present disclosure has been conceived, and an extended function that allows more intuitive operation without interfering with the behavior of an external application can be provided. To this end, an information processing apparatus 10 according to an embodiment of the present disclosure includes a controller 140 that controls function extension for an external application. Further, one of the features of the controller 140 according to an embodiment of the present disclosure is that an extended menu icon whose display position is adjustable is displayed and superimposed on a display window of the external application, and the display position of the extended menu is controlled according to the display position of the superimposed extended menu icon that is displayed. In addition, one of the features of the controller 140 according to an embodiment of the present disclosure is that the controller 140 causes an extended function icon associated with an extended function selected on the above-mentioned extended menu to be displayed and superimposed on the above-mentioned display window, wherein the display position of the extended function icon is adjustable, and the controller 140 controls the display position of a sub-icon accompanying the displayed superimposed extended function icon according to the display position of the extended function icon.

[0038] Hereinafter, an example of the functional configuration of the information processing apparatus 10 having the above features according to the present embodiment will be described in detail. The information processing apparatus 10 according to the present embodiment can be a smart phone, a tablet computer, a PC (personal computer), etc. that can execute various applications.

[0039] Figure 1 A block diagram illustrating an example of the functional configuration of the information processing apparatus 10 according to the present embodiment is depicted. As Figure 1As shown, the information processing apparatus 10 according to the present embodiment includes an operation unit 110, a voice input unit 120, an image capture unit 130, a controller 140, a voice output unit 150, and a display unit 160.

[0040] (Operation unit 110)

[0041] The operation unit 110 according to the present embodiment detects various operations of the user, such as operations of applications. For this function, the operation unit 110 according to the present embodiment includes, for example, a touch panel, buttons, a keyboard, a mouse, etc. The operation unit 110 according to the present embodiment inputs information associated with the detected user operation to the controller 140.

[0042] (Voice input unit 120)

[0043] The voice input unit 120 according to the present embodiment collects voices and the like emitted by the user based on the control of the controller 140. For this function, the voice input unit 120 according to the present embodiment includes a microphone and the like.

[0044] (Image capture unit 130)

[0045] The image capture unit 130 according to the present embodiment captures moving images based on the control of the controller 140. The image capture unit 130 can capture, for example, the face of the user operating the information processing apparatus 10. For this function, the image capture unit 130 according to the present embodiment includes an imaging element. A smart phone as an example of the information processing apparatus 10 includes an inward-facing camera that captures the face of the user and the like and is arranged on the side of the display unit 160, and a main camera that captures landscapes and the like and is arranged on the back side of the display unit 160. In the present embodiment, as an example, the shooting performed by using the inward-facing camera is controlled.

[0046] (Controller 140)

[0047] The controller 140 according to the present embodiment controls each component included in the information processing apparatus 10. In addition, one of the features of the controller 140 according to the present embodiment is to control the extension of the functions of external applications. The details of the functions of the controller 140 according to the present embodiment will be described separately later.

[0048] (Voice output unit 150)

[0049] The voice output unit 150 according to the present embodiment outputs various voices. For example, the voice output unit 150 according to the present embodiment outputs voices according to the situation of the application based on the control of, for example, the controller 140. For this function, the voice output unit 150 according to the present embodiment includes a speaker and an amplifier.

[0050] (Display Unit 160)

[0051] The display unit 160 according to the present embodiment displays various visual information based on the control of the controller 140. The display unit 160 according to the present embodiment can display, for example, images and characters associated with applications. For this function, the display unit 160 according to the present embodiment includes various display devices.

[0052] So far, an example of the functional configuration of the information processing apparatus 10 according to the present embodiment has been described. It should be noted that the configuration described above is only an example, and the functional configuration of the information processing apparatus 10 according to the present embodiment is not limited to this example. The functional configuration of the information processing apparatus 10 according to the present embodiment can be flexibly modified according to specifications and operations. Figure 1 The functional configuration of the information processing apparatus 10 according to the present embodiment can be flexibly modified according to specifications and operations.

[0053] <<1.2. Details of Functions>>

[0054] Next, the functions of the information processing apparatus 10 according to the present embodiment will be described in detail. One of the features of the controller 140 of the information processing apparatus 10 according to the present embodiment is to control an extended application for providing extended functions to various external applications. Examples of the above external applications may include game applications. The controller 140 according to the present embodiment can, for example, provide extended functions such as an image recording function to an external game application produced by a different company by making the above extended application cooperate with the external application.

[0055] Note that hereinafter, the case where the external application associated with the present embodiment is a game application will be mainly described as an example; however, the external application associated with the present embodiment is not limited to such an example. Examples of the external application associated with the present embodiment may include a drawing application, various editing applications, and applications for viewing / listening to moving images, music, etc.

[0056] In addition, the external application associated with the present embodiment is not limited to applications produced by different companies, and it can be an internally produced application. By using the extended application according to the present embodiment, regardless of who the producer of the existing application is, it is possible to easily provide extended functions to the existing application without editing the source code, etc.

[0057] Figure 2 A diagram for illustrating the cooperation between the extended application and the external application according to the present embodiment is depicted. The extended application and the external application according to the present embodiment both operate by being installed in the information processing apparatus 10.

[0058] In Figure 2On the left side, the main screen of the OS (operating system) displayed on the display unit 160 of the information processing apparatus 10 is depicted. Here, when the user selects the icon Ia associated with the extended application, the controller 140 starts the extended application and, as shown in the center of the figure, causes the display unit 160 to display the start screen of the extended application.

[0059] In Figure 2 In the case of one example depicted, on the above-mentioned start screen, icons Ig and Ih associated with the installed external applications available for cooperation, and icons Ii to Ik associated with the uninstalled external applications recommended to the user are displayed. The controller 140 can automatically generate an application list including the installed external applications and exemplary applications based on the recommendation information or other information. In addition, such external applications displayed on the start screen may include another external application optionally added by the user.

[0060] Here, when the user selects the icon associated with the expected external application, as depicted on the right side of the figure, the controller 140 starts the external application corresponding to the selected icon and causes the display unit 160 to display the display window of the expected external application. In Figure 2 In the figure, an example of this case is depicted, where the user selects the icon Ig associated with the racing game on the above-mentioned start screen.

[0061] In addition, at this time, the controller 140 causes the extended menu icon MI to be superimposed and displayed on the above-mentioned display window. The extended menu icon MI may be an icon for invoking a menu for extended functions to be provided to the external application. Additionally, for example, by dragging the extended menu icon MI, the user can be allowed to move the extended menu icon MI to any position on the display window. By appropriately moving the extended menu icon MI according to the situation of the display screen of the external application, it is possible to avoid interfering with the user's operation of the external application and the display of various visual information displayed by the external application.

[0062] Note that the above description has been made regarding an exemplary case where the user starts an external application via the start screen of the extended application to cooperate with the extended application. However, for example, the user may be able to directly select the icon associated with the expected external application on the main screen in order to start the expected external application in a state of cooperation with the extended application.

[0063] Next, the display control of the extended menu according to the present embodiment will be described. As described above, the user can invoke the extended function menu by selecting the extended menu icon MI superimposed and displayed on the display window of the external application. In this case, the controller 140 according to the present embodiment controls the display position of the extended menu based on the display position of the extended menu icon MI on the display window.

[0064] For example, the controller 140 according to the present embodiment can control the display position of the extended menu based on both the display position of the extended menu icon MI and the aspect ratio of the display window. Figure 3A And 3B is a diagram for illustrating the display of the extended menu based on the display position of the extended menu icon according to the present embodiment.

[0065] Generally, there are external applications that operate together with the information processing apparatus 10 held in the vertical direction, external applications that operate together with the information processing apparatus 10 held in the horizontal direction, and external applications that operate when the display of the information processing apparatus 10 turns according to the posture of the information processing apparatus 10. For this reason, the controller 140 according to the present embodiment can display the extended menu at a position where the menu is easier to operate by detecting the aspect ratio of the display window of the external application.

[0066] For example, in Figure 3A In the case of an example depicted in, the display unit 160 is displaying a display window suitable for operating the information processing apparatus 10 in a vertically held state. In this way, when the display window is in a state where its side along the height direction is longer than its side along the width direction, the controller 140 can cause the extended menu M to be superimposed and displayed in either the upper region or the lower region of the display window that contains the display position of the extended menu icon MI.

[0067] For example, in Figure 3A In the case of an example depicted in the upper part of, the extended menu icon MI is located in the upper region of the display window. Here, when the extended menu icon MI is selected, similar to the display position of the extended menu icon MI, the controller 140 can cause the extended menu M to be superimposed and displayed in the upper region of the display window.

[0068] On the other hand, in Figure 3A In the case of an example depicted in the lower part of, the extended menu icon MI is located in the lower region of the display window. Here, when the extended menu icon MI is selected, similar to the display position of the extended menu icon MI, the controller 140 can cause the extended menu M to be superimposed and displayed in the lower region of the display window.

[0069] Furthermore, for example, in Figure 3BIn the case of an example depicted, the display unit 160 is displaying a display window suitable for operating the information processing apparatus 10 while being held horizontally. In this way, when the display window is in a state where its side along the width direction is longer than its side along the height direction, the controller 140 can cause the extended menu M to be superimposed and displayed in either the left region or the right region of the display position of the display window that includes the extended menu icon MI.

[0070] For example, in Figure 3B In the case of an example depicted in the upper part of, the extended menu icon MI is located in the left region of the display window. Here, when the extended menu icon MI is selected, similar to the display position of the extended menu icon MI, the controller 140 can cause the extended menu M to be superimposed and displayed in the left region of the display window.

[0071] On the other hand, in Figure 3B In the case of an example depicted in the lower part of, the extended menu icon MI is located in the right region of the display window. Here, when the extended menu icon MI is selected, similar to the display position of the extended menu icon MI, the controller 140 can cause the extended menu M to be superimposed and displayed in the right region of the display window.

[0072] So far, the display control of the extended menu M based on the display position of the extended menu icon MI according to the present embodiment has been described. According to the display control as described above, a user who has performed an operation on the extended menu icon MI can continuously select any extended function in the same area, and thus an extended menu display with easier operation can be achieved. In addition, according to such display control as described above, the extended menu M can be displayed in the same area as the display position of the extended menu icon MI where the user has moved to a position where the icon does not block the user's operation. This can effectively reduce the possibility that important visual information output by an external application is obscured from view by the extended menu M.

[0073] Note that in Figure 3A and 3B In the examples depicted, a case is illustrated where functions such as setting a game mode, setting a focus, searching, taking a screen snapshot, and recording an image are provided as extended functions.

[0074] The above-described function of setting a game mode can be an extended function that can set a mode for each external application, such as a mode that gives priority to energy saving or a mode that gives priority to performance.

[0075] In addition, the above-described function of setting the focus can be an extended function that can set various conditions, which can, for example, help the user to further focus on an external application. The user can be able to set functions such as: releasing the RAM, locking the navigation bar, not displaying notifications from other applications (such as a messaging application), and turning off the function of automatically adjusting the brightness of the display unit 160. The details of the game mode settings and focus settings can be stored after the external application ends, thereby allowing the settings to be automatically reflected when starting the next operation.

[0076] In addition, the above-described search function can be an extended function that can perform various information retrievals on the display window of an external application without the user separately starting a browser application. By using the search function, the user can, for example, play a game while referring to policy information on the display window.

[0077] In addition, the above-described function of taking a screen shot can be an extended function that can store the playing state of an external application as a still image. Additionally, the above-described function of recording an image can be an extended function that can store the playing state of an external application as a moving image. The details of the image recording function according to the present embodiment will be described separately later.

[0078] So far, examples of the extended menu M and extended functions according to the present embodiment have been described. Note that such extended functions as described above are used by the user according to circumstances, and thus it is assumed that the extended functions are not generally frequently operated. For this reason, it is preferable to display an extended menu icon MI for calling up the extended menu M as described above so as not to interfere with playing an external application as much as possible without a user operation.

[0079] For this reason, the controller 140 according to the present embodiment can control the display position and display manner of the extended menu icon MI such that the extended menu icon MI becomes less noticeable when no operation by the user on the extended menu icon MI has been performed for a predetermined period of time.

[0080] Figure 4 is a diagram for illustrating the display control of the extended menu icon MI based on the non-operation time according to an embodiment. In Figure 4 an example depicted on the left side, the extended menu icon MI is deployed at the lower right corner of the display window. In this way, when the extended menu icon MI in the edge area of the display window has not been used for more than a predetermined period of time, the controller 140 can move the extended menu icon MI to a position where at least a part of the extended menu icon MI is hidden outside the display window, as Figure 4 described on the right side.

[0081] In addition to the above control of the display position, the controller 140 according to the present embodiment can also control the display mode so that the extended menu icon MI becomes less noticeable. For example, when the extended menu icon MI has not been used for more than a predetermined period of time, the controller 140 can control the overlay display to reduce the prominence of the extended menu icon MI. For example, the controller 140 can gray out or make the extended menu icon MI translucent.

[0082] According to the control described above, when the extended function is not operated for a long time, the extended menu icon MI automatically moves to a position where the extended menu icon MI is not likely to interfere with playing the external application, and the degree of highlighting is reduced, so that the user can focus on playing the external application.

[0083] Note that when the extended menu icon MI is deployed in the central area of the display window, it is assumed that the user is likely to perform an operation of the extended function from now on. For this reason, when the extended menu icon MI is deployed in the central area, as described above, the controller 140 does not need to perform the change of the display position and the control of the display mode.

[0084] Next, the image recording function, which is one of the extended functions according to the present embodiment, will be described in detail. The user can record an image of the playing state of any external application by using the extended application according to the present embodiment and store it as a moving image. In this case, the controller 140 according to the present embodiment can cause the image captured by the image capturing unit 130 to be overlaid and displayed on the display window, and can record the overlaid image together with the playing state of the external application.

[0085] Figures 5 to 7 A diagram for illustrating the image recording function according to the present embodiment is depicted. When the image recording function is selected on the extended menu M, the controller 140 causes the extended function icon EI associated with the image recording function and a plurality of accompanying sub-icons SI to be overlaid and displayed on the display window of the external application, as Figure 5 drawn in.

[0086] In this case, as shown in the figure, the controller 140 according to the present embodiment can cause a preview of the image captured by the image capturing unit 130 to be overlaid and displayed as the extended function icon EI. According to this control, the user can check his / her own image before starting the image recording, so that the user can prepare for tidying himself / herself up, etc.

[0087] Note that the sub-icon SI1 can be an icon for the user to issue an instruction to end the image recording function. In the case where the sub-icon SI1 is selected, the controller 140 can perform control such that when the extended menu icon MI is superimposed and displayed again, the extended function icon EI and all sub-icons SI are not displayed.

[0088] In addition, the sub-icon SI2 can be an icon for the user to issue an instruction to control the sound recording mode associated with the image recording of an external application. The controller 140 according to the present embodiment can dynamically control the above-mentioned sound recording mode based on the user operation of the sub-icon SI2. For example, the controller 140 can dynamically switch between the on and off states of the above-mentioned sound recording based on the operation of the sub-icon SI2.

[0089] In addition, the sub-icon SI3 can be an icon for the user to issue an instruction to start and end the shooting of the image capture unit 130. The controller 140 according to the present embodiment can dynamically control the shooting of the image capture unit 130 and the superimposition and display of the captured images based on the user operation on the sub-icon SI3. The sub-icon SI3 for starting and ending shooting is deployed separately from the icon for starting and ending the image recording function. Therefore, the user can check his / her own image before starting shooting, so that the user can prepare for tidying himself / herself up, etc.

[0090] In addition, the sub-icon SI4 can be an icon for the user to issue an instruction to start and end the image recording function. The controller 140 according to the present embodiment can control the start and end of the image recording function based on the user operation on the sub-icon SI4.

[0091] So far, examples of the sub-icon SI according to the present embodiment have been described. In this way, with the extended application according to the present embodiment, the user can flexibly and dynamically set the superimposition of the captured image, the sound recording mode, and any other related operations, and then perform the recording of the image in the play state.

[0092] Note that similar to the extended menu icon MI, the extended function icon EI according to the present embodiment can be optionally adjusted with respect to its display position. In this case, the controller 140 according to the present embodiment controls the display position of the sub-icon SI based on the display position of the extended function icon EI.

[0093] More specifically, based on the display position of the extended function icon EI, the controller 140 according to the present embodiment can cause the sub-icon SI to be displayed around the extended function icon, so that the sub-icon SI can fit the display window.

[0094] For example, in Figure 5In the case of an example depicted in the upper left side of , the extended function icon EI is deployed in the upper left corner of the display window. In this case, the controller 140 can cause the sub-icons SI1 to SI4 to be displayed on the right side and the lower side of the extended function icon EI.

[0095] In addition, for example, in Figure 5 the case of an example depicted in the upper right side of , the extended function icon EI is deployed in the upper right corner of the display window. In this case, the controller 140 can cause the sub-icons SI1 to SI4 to be displayed on the left side and the lower side of the extended function icon EI.

[0096] In addition, for example, in Figure 5 the case of an example shown in the lower left side of , the extended function icon EI is arranged in the lower left corner of the display window. In this case, the controller 140 can cause the sub-icons SI1 to SI4 to be displayed on the upper side and the right side of the extended function icon EI.

[0097] In addition, for example, in Figure 5 the case of an example depicted in the lower right side of , the extended function icon EI is deployed in the lower right corner of the display window. In this case, the controller 140 can cause the sub-icons SI1 to SI4 to be displayed on the upper side and the left side of the extended function icon EI. Note that Figure 5 depicts display examples of the sub-icons SI1 to SI4 in the case where the extended function icon EI is deployed in any one of the four corners of the display window. On the other hand, the display of such sub-icons SI1 to SI4 as described above is not limited to the case where the extended function icon EI is deployed in one of the four corners of the display window. That is, in the case where the display window is divided into four parts, that is, in two vertical columns and two horizontal rows, the display of the sub-icons is controlled according to the area where the extended function icon EI is deployed. For example, in Figure 5 the case depicted, when the extended function icon EI is in a state of being deployed in the upper left area of the display window, the sub-icons SI1 to SI4 are displayed on the right side and the lower side of the extended function icon EI. In the case where the user drags the extended function icon EI and moves the extended function icon EI to the upper right area of the display window, the sub-icons SI1 to SI4 are displayed on the left side and the lower side of the extended function icon EI.

[0098] According to the control described above, the following situation can be avoided: when the sub-icon SI is moved along with the extended function icon EI, the sub-icon is partially hidden outside the display window, and thus, some functions are unavailable. Note that during the process of moving the extended function icon EI to any position on the display window by dragging the extended function icon EI, a display state can be generated such that the sub-icons SI1 to SI4 are not displayed, and only the extended function icon EI is displayed. With such a configuration, the amount of information displayed on the screen is reduced, enabling the user to easily focus on moving the icon. In this case, after the dragging operation for moving is completed and the user releases the user's finger from the display unit 160, which is set to be used as a touch panel for example, the controller 140 can cause the sub-icons SI1 to SI4 to be displayed again.

[0099] Next, the control executed after the start of image recording will be described. As described above, the controller 140 according to the present embodiment is capable of starting the image recording function based on the user's operation on the sub-icon SI4. In this case, as Figure 6 depicted, the controller 140 can cause the sub-icons SI1 to SI4 not to be displayed, and can provide the extended function icon EI with a visual effect indicating that image recording is being performed. According to this control, unnecessary information such as the sub-icon SI can be prevented from being recorded together. In addition, in the state where shooting has been started by the operation of the sub-icon SI3, the display state changes such that an image of the user's face or the like captured by the image capturing unit 130 is superimposed and displayed as the extended function icon EI, as Figure 6 depicted, and thus, an image of the user's facial expression while playing an external application can be recorded.

[0100] In addition, even after the start of image recording, the extended function icon EI can be moved to any position. For example, the user can perform a dragging operation or the like according to the situation of the external application, thereby moving the extended function icon EI to a position where the icon does not interfere with the user's operation.

[0101] Note that such a movement of the extended function icon EI described above can be automatically executed by the controller 140. By analyzing the image associated with the external application, the controller 140 can automatically move the extended function icon EI to an area on the screen where there is less movement or no UI such as buttons for user operation is deployed. In addition, by learning the user's movement operation, the controller 140 can move the extended function icon EI to the optimal display position according to the state of the game, for example. The controller 140 can also perform such automatic position control as described above for the extended menu icon MI.

[0102] In addition, even after the start of image recording, the settings related to the superimposed display of sound recording and the captured images can be changed at any time.

[0103] When the user presses and selects the extended function icon EI after the start of image recording, the controller 140 causes the sub-icon SI to be superimposed and redisplayed on the display window, as Figure 7 depicted therein, thereby providing a state in which the user can perform various settings. At this time, the controller 140 can change the display from the display associated with the sub-icon SI4 to a visual effect for issuing an instruction to end image recording. The user can press the sub-icon SI4 to issue an instruction to end image recording. Note that the user can press the extended function icon EI again, thereby again causing the sub-icon SI not to be displayed. In addition, even in a state where the sub-icon SI is displayed together with the extended function icon E1, during the process of moving the extended function icon EI to any position on the display window by dragging the extended function icon EI, the display state can be such that only the extended function icon EI is displayed and the sub-icon SI is not displayed.

[0104] In addition, for example, the user can press the sub-icon SI3 to change the operation, that is, the presence or absence of image capture by the image capture unit 130 and the superimposed display of the captured images. With the extended application according to the present embodiment, even after the start of image recording, the presence or absence of the superimposed display of the captured images can be dynamically changed according to circumstances.

[0105] In addition, for example, in a similar manner, the user can press the sub-icon SI2 to change the sound recording method. In Figure 7 an example of a situation is depicted, in which each time the sub-icon SI2 is pressed, the display of the sub-icon SI2 changes according to the mode.

[0106] For example, the sub-icon SI2a indicates a mode in which sound recording is not performed. In addition, the sub-icon SI2b indicates a mode in which sound recording is performed only on the sound output by an external application. In addition, the sub-icon SI2c indicates a mode in which sound recording is performed on both the sound output by an external application and the sound emitted by the user. In addition, the sub-icon SI2d indicates a mode in which sound recording is performed only on the sound emitted by the user. In this way, the sound recording method according to the present embodiment can include the on and off of sound recording and the mode of sound recording.

[0107] The controller 140 according to the present embodiment controls the mode change selected by the user's operation on the sub-icon SI2 as described above, and controls the sound recording according to each mode. With this control, the on and off of sound recording and the sound to be recorded can be dynamically selected according to circumstances, so that a more flexible generation of moving images can be achieved.

[0108] Note that, in Figure 7 , an example of this situation is described by using an example in which the state of the sub-icon SI2 changes each time the sub-icon SI2 is pressed; however, the controller 140 may cause a plurality of additional icons for specifying a sound recording mode to be superimposed and displayed around, for example, the sub-icon SI2.

[0109] So far, the dynamic changes in the settings according to the present embodiment after the start of image recording have been described. By using the above functions of the extended application according to the present embodiment, the content to be recorded can be dynamically and flexibly changed according to the playing state of an external application, etc., so that the production of a moving image can be realized, in which the user's intention is more reflected.

[0110] Note that the setting of the above image recording function is only an example. The image recording function according to the present embodiment can perform various settings other than the superimposed display of captured images and sound recording. For example, the controller 140 according to the present embodiment can control the posting of the recorded image to an external service.

[0111] In Figure 7 In the example illustrated, the sub-icon SI5 may be an icon for the user to issue an instruction to post the recorded moving image to an external service. The controller 140 according to the present embodiment can also dynamically control the posting of the recorded moving image to various external services based on the user's operation on the sub-icon SI5.

[0112] In this case, the user can be able to pre-set an external service as the posting destination, or can optionally select an external service as the posting destination by the user's operation on the sub-icon SI5. In this way, the moving image recorded together with the recorded image of the playing state can be distributed in real time via one or more external services.

[0113] In addition, in the above description, an example of the case where the image captured by the image capturing unit 130 is superimposed and displayed on the display window and recorded together with the playing state has been described; however, the image to be superimposed and displayed is not limited to the image captured by the image capturing unit 130.

[0114] The controller 140 can, for example, acquire a moving image associated with the playing state of another user who is playing a game with the user via a network, an image generated by photographing the other user, and any other relevant images, and then can cause the acquired images to be superimposed and displayed. In addition, the controller 140 can cause an avatar or the like to be superimposed and displayed in place of the user's captured image by animation.

[0115] In this way, the image to be superimposed and displayed on the display window can be optionally set by the user. In this case, the sub-icon SI2 associated with the superimposed display can be an icon for the user to issue an instruction to change the presence or absence of the superimposed display of various acquired images and to determine the type of image to be superimposed and displayed.

[0116] <2. Hardware Configuration Example>

[0117] Next, an example of the hardware configuration of the information processing apparatus 10 according to an embodiment of the present disclosure will be described. Figure 8 A diagram for illustrating an example of the hardware configuration of the information processing apparatus 10 according to an embodiment of the present disclosure is depicted. As Figure 8 shown, the information processing apparatus 10 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a storage device 880, a drive 881, a connection port 882, and a communication device 883. Note that the hardware configuration described herein is merely an example; thus, some constituent elements may be omitted. In addition, constituent elements other than those described herein may also be included.

[0118] (Processor 871)

[0119] The processor 871 serves as, for example, an arithmetic processing device or a control device, and controls all or part of the operations of the respective constituent elements based on various programs stored in the ROM 872, the RAM 873, the storage device 880, or the removable recording medium 901.

[0120] (ROM 872 and RAM 873)

[0121] The ROM 872 is a component that stores programs to be loaded into the processor 871, data to be used for calculations, etc. In the RAM 873, for example, programs to be loaded into the processor 871, various parameters that are appropriately changed when the programs are executed, etc. are stored temporarily or permanently.

[0122] (Host Bus 874, Bridge 875, External Bus 876, and Interface 877)

[0123] The processor 871, the ROM 872, and the RAM 873 are interconnected via, for example, a host bus 874 capable of high-speed data transfer. On the other hand, the host bus 874 is connected to an external bus 876 capable of relatively low-speed data transfer via a bridge 875. In addition, the external bus 876 is connected to various constituent elements via an interface 877.

[0124] (Input Device 878)

[0125] The input device 878 includes, for example, a mouse, a keyboard, a touchpad, buttons, switches, and a joystick. Additionally, in some cases, the input device 878 includes a remote controller (hereinafter referred to as a remote) capable of transmitting control signals by using infrared rays or any other radio waves. Further, the input device 878 includes a voice input device such as a microphone.

[0126] (Output device 879)

[0127] The output device 879 is a device capable of notifying a user of acquired information visually or auditorily. Examples of such a device include display devices such as a CRT (cathode ray tube), an LCD, or an organic EL, audio output devices such as speakers or headphones, printers, mobile phones, facsimile machines, and any other terminal. Further, the output device 879 according to the present disclosure includes various vibration devices capable of outputting a tactile stimulus.

[0128] (Storage device 880)

[0129] The storage device 880 is a device that stores various data. The storage device 880 includes, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or any other storage device.

[0130] (Drive 881)

[0131] The drive 881 is, for example, a device that reads information stored in a removable recording medium 901 (such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or any other recording medium) and writes information to the removable recording medium 901.

[0132] (Removable recording medium 901)

[0133] The removable recording medium 901 includes, for example, DVD media, Blu-ray (registered trademark) media, HD DVD media, various semiconductor storage media, and any other recording medium. Needless to say, the removable recording medium 901 may also be, for example, an IC card equipped with a non-contact IC chip, an electronic device, or the like.

[0134] (Connection port 882)

[0135] The connection port 882 is a port for connecting an external connection device 902. Examples of the port include a USB (universal serial bus) port, an IEEE 1394 port, a SCSI (small computer system interface), an RS-232C port, an optical audio terminal, and any other connection port.

[0136] (External connection device 902)

[0137] The external connection device 902 includes, for example, a printer, a portable music player, a digital camera, a digital video camera, an IC recorder, etc.

[0138] (Communication device 883)

[0139] The communication device 883 is a communication device for connecting to a network. Examples of communication devices include communication cards for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), routers for optical communication, routers for ADSL (Asymmetric Digital Subscriber Line), modems for various communications, and any other communication devices.

[0140] <3. Abstract>

[0141] As described above, the information processing apparatus 10 according to an embodiment of the present disclosure includes a controller 140 that controls the function expansion of an external application. In addition, one of the features of the controller 140 according to an embodiment of the present disclosure is that an extended menu icon whose display position is adjustable is superimposed and displayed on the display window of the external application, and the display position of the extended menu is controlled according to the display position of the superimposed extended menu icon. Further, one of the features of the controller 140 according to an embodiment of the present disclosure is that an extended function icon associated with the extended function selected on the above extended menu is superimposed and displayed on the above display window, wherein the display position of the extended function icon is adjustable, and the display position of the sub-icon accompanying the superimposed display of the extended function icon is controlled according to the display position of the extended function icon. According to this configuration, an extended function can be provided that allows for more intuitive operation without disturbing the behavior of the external application.

[0142] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to the above examples. Obviously, those skilled in the art can find various alternatives and modifications within the scope of the appended claims, and it should be understood that such alternatives and modifications will naturally fall within the technical scope of the present disclosure.

[0143] In addition, the effects described in this specification are merely illustrative or exemplary effects and are not restrictive. That is, together with or instead of the above effects, other effects clear to those skilled in the art from the description of this specification can be achieved according to the technology of the present disclosure.

[0144] In addition, a program can also be created that causes hardware such as a CPU, ROM, and RAM built into a computer to perform functions equivalent to the configuration of the information processing apparatus 10, and a computer-readable non-transitory recording medium recording such a program can also be provided.

[0145] Note that the following configurations also fall within the technical scope of the present disclosure. (1)

[0147] An information processing apparatus, comprising:

[0148] a controller configured to control function expansion of an external application, wherein

[0149] the controller is configured to:

[0150] overlay and display an extension menu icon with an adjustable display position on a display window of the external application, and control a display position of the extension menu according to the display position of the extension menu icon, and

[0151] overlay and display an extension function icon on the display window, wherein the display position of the extension function icon is adjustable, the extension function icon is associated with an extension function selected on the extension menu, and control a display position of a sub-icon accompanying the extension function icon according to the display position of the extension function icon. (2)

[0153] The information processing apparatus according to (1), wherein the controller controls the display position of the extension menu based on the display position of the extension menu icon and an aspect ratio of the display window. (3)

[0155] The information processing apparatus according to (2), wherein

[0156] when a side along a height direction of the display window is longer than a side along a width direction of the display window, the controller overlays and displays the extension menu in an upper region or a lower region of the display window including the display position of the extension menu icon, and

[0157] when a side along the width direction of the display window is longer than a side along the height direction of the display window, the controller overlays and displays the extension menu in a left region or a right region of the display window including the display position of the extension menu icon. (4)

[0159] The information processing apparatus according to any one of (1) to (3), wherein the controller causes the sub-icon to be displayed around the extension function icon according to the display position of the extension function icon such that the sub-icon fits inside the display window. (5)

[0161] The information processing apparatus according to (4), wherein the extended function icon includes an icon associated with an image recording function for recording an image of a playing state of the external application. (6)

[0163] The information processing apparatus according to (5), wherein

[0164] the sub-icon includes an icon for the user to issue instructions to start and end the image recording function, and

[0165] the controller controls the start and end of the image recording function based on the user's operation on the icon included in the sub-icon. (7)

[0167] The information processing apparatus according to (5) or (6), wherein the controller controls the superimposed display of the acquired image on the display window and records the acquired superimposed and displayed image together with the playing state of the external application. (8)

[0169] The information processing apparatus according to (7), wherein

[0170] the sub-icon includes an icon for the user to issue an instruction to select whether to superimpose and display the acquired image, and

[0171] the controller dynamically changes between the presence and absence of the superimposed display of the acquired image based on the user's operation on the icon included in the sub-icon. (9)

[0173] The information processing apparatus according to (7) or (8), wherein the acquired image includes an image captured by the terminal that displays the display window. (10)

[0175] The information processing apparatus according to (9), wherein the controller causes the preview of the captured image to be superimposed and displayed as the extended function icon. (11)

[0177] The information processing apparatus according to (9) or (10), wherein

[0178] the sub-icon includes an icon for the user to issue instructions to start and end the shooting of the terminal, and

[0179] the controller dynamically controls the shooting of the terminal and the superimposed display of the captured image based on the user's operation on the icon included in the sub-icon. (12)

[0181] The information processing apparatus according to any one of (5) to (11), wherein

[0182] The sub-icon includes an icon for the user to issue an instruction to select a sound recording method associated with the image recording of the external application, and

[0183] The controller dynamically controls the sound recording method based on the user's operation on the icon included in the sub-icon. (13)

[0185] The information processing apparatus according to (12), wherein

[0186] The sound recording method includes a sound recording mode, and

[0187] The mode includes a mode of only sound recording the sound output by the external application and a mode of sound recording both the sound output by the external application and the sound made by the user. (14)

[0189] The information processing apparatus according to any one of (5) to (13), wherein

[0190] The sub-icon includes an icon for the user to issue an instruction to post the recorded moving image to an external service, and

[0191] The controller dynamically controls posting the recorded moving image to an external service based on the user's operation on the icon included in the sub-icon. (15)

[0193] The information processing apparatus according to any one of (1) to (14), wherein when the extended menu icon in the edge area of the display window has not been used for more than a predetermined period of time, the controller moves the extended menu icon to a position where at least a part of the extended menu icon is hidden outside the display window. (16)

[0195] The information processing apparatus according to any one of (1) to (15), wherein when the extended menu icon has not been used for more than a predetermined period of time, the controller controls the superimposed display of the extended menu icon so that the degree of highlighting of the extended menu icon is reduced. (17)

[0197] The information processing apparatus according to any one of (1) to (16), wherein the external application includes a game application. (18)

[0199] An information processing method, comprising:

[0200] Function expansion of an external application controlled by a processor, where

[0201] The control includes:

[0202] An extended menu icon with an adjustable display position is superimposed and displayed on the display window of the external application, and the display position of the extended menu is controlled according to the display position of the extended menu icon, and

[0203] An extended function icon is superimposed and displayed on the display window, where the display position of the extended function icon is adjustable, the extended function icon is associated with an extended function selected on the extended menu, and the display position of a sub-icon accompanying the extended function icon is controlled according to the display position of the extended function icon. (19)

[0205] A program for making a computer function as an information processing device,

[0206] The information processing device includes a controller configured to control function expansion of an external application, where

[0207] The controller is configured to:

[0208] An extended menu icon with an adjustable display position is superimposed and displayed on the display window of the external application, and the display position of the extended menu is controlled according to the display position of the extended menu icon, and

[0209] An extended function icon is superimposed and displayed on the display window, where the display position of the extended function icon is adjustable, the extended function icon is associated with an extended function selected on the extended menu, and the display position of a sub-icon accompanying the extended function icon is controlled according to the display position of the extended function icon.

[0210] [List of reference symbols]

[0211] 10: Information processing device

[0212] 110: Operation unit

[0213] 120: Sound input unit

[0214] 130: Image capture unit

[0215] 140: Controller

[0216] 150: Sound output unit

[0217] 160: Display unit

Claims

1. An information processing apparatus, comprising: A controller, configured to control the functional expansion of an external application, wherein the controller is configured to: overlay and display an expansion menu icon with an adjustable display position on the display window of the external application, and control the display position of the expansion menu according to the display position of the expansion menu icon, and overlay and display an expansion function icon on the display window, wherein the display position of the expansion function icon is adjustable, the expansion function icon is associated with the expansion function selected on the expansion menu, and control the display position of the sub-icon accompanying the expansion function icon according to the display position of the expansion function icon.

2. The information processing apparatus according to claim 1, wherein the controller controls the display position of the extended menu based on the display position of the extended menu icon and the aspect ratio of the display window.

3. The information processing apparatus according to claim 2, wherein, When the side along the height direction of the display window is longer than the side along the width direction of the display window, the controller overlays and displays the expansion menu in either the upper area or the lower area of the display window including the display position of the expansion menu icon, and When the side along the width direction of the display window is longer than the side along the height direction of the display window, the controller overlays and displays the expansion menu in either the left area or the right area of the display window including the display position of the expansion menu icon.

4. The information processing apparatus according to claim 1, wherein the controller causes the sub - icons to be displayed around the extended function icon according to the display position of the extended function icon, such that the sub - icons fit inside the display window.

5. The information processing apparatus according to claim 4, wherein the extended function icon includes an icon associated with an image recording function for recording an image of the playing state of the external application.

6. The information processing apparatus according to claim 5, wherein the sub - icons include icons for the user to issue instructions to start and end the image recording function, and the controller controls the start and end of the image recording function based on the user's operation of the icons included in the sub - icons.

7. The information processing apparatus according to claim 5, wherein the controller controls to superimpose and display the acquired image on the display window, and records the acquired superimposed and displayed image together with the playing state of the external application.

8. The information processing apparatus according to claim 7, wherein the sub - icons include icons for the user to issue instructions to select whether to superimpose and display the acquired image, and the controller dynamically changes between the presence and absence of the superimposed display of the acquired image based on the user's operation of the icons included in the sub - icons.

9. The information processing apparatus according to claim 7, wherein the acquired image includes an image captured by a terminal that displays the display window.

10. The information processing apparatus according to claim 9, wherein the controller causes a preview of the captured image to be superimposed and displayed as the extended function icon.

11. The information processing apparatus according to claim 9, wherein The sub-icon includes an icon for a user to issue instructions for starting and ending shooting of the terminal, and the controller dynamically controls the shooting of the terminal and the superimposed display of the captured image based on the user's operation on the icon included in the sub-icon.

12. The information processing apparatus according to claim 5, wherein the sub-icon includes an icon for a user to issue an instruction to select a sound recording method associated with image recording of the external application, and the controller dynamically controls the sound recording method based on the user's operation on the icon included in the sub-icon.

13. The information processing apparatus according to claim 12, wherein the sound recording method includes a sound recording mode, and the mode includes a mode of only sound recording the sound output by the external application and a mode of sound recording both the sound output by the external application and the sound emitted by the user.

14. The information processing apparatus according to claim 5, wherein the sub-icon includes an icon for a user to issue an instruction to post the recorded moving image to an external service, and the controller dynamically controls posting the recorded moving image to an external service based on the user's operation on the icon included in the sub-icon.

15. The information processing apparatus according to claim 1, wherein when the extended menu icon in the edge area of the display window has not been used for more than a predetermined period of time, the controller moves the extended menu icon to a position where at least a part of the extended menu icon is hidden outside the display window.

16. The information processing apparatus according to claim 1, wherein when the extended menu icon has not been used for more than a predetermined period of time, the controller controls the superimposed display of the extended menu icon such that the degree of highlighting of the extended menu icon is reduced.

17. The information processing apparatus according to claim 1, wherein the external application includes a game application.

18. An information processing method, comprising: The functional expansion of an external application is controlled by a processor, wherein the control includes: overlay and display an expansion menu icon with an adjustable display position on the display window of the external application, and control the display position of the expansion menu according to the display position of the expansion menu icon, and overlay and display an expansion function icon on the display window, wherein the display position of the expansion function icon is adjustable, the expansion function icon is associated with the expansion function selected on the expansion menu, and control the display position of the sub-icon accompanying the expansion function icon according to the display position of the expansion function icon.

19. A program for causing a computer to function as an information processing apparatus, The information processing apparatus includes a controller configured to control function expansion of an external application, wherein the controller is configured to: superimpose and display an expansion menu icon with an adjustable display position on a display window of the external application, and control a display position of an expansion menu according to the display position of the expansion menu icon, and superimpose and display an expansion function icon on the display window, wherein a display position of the expansion function icon is adjustable, the expansion function icon is associated with an expansion function selected on the expansion menu, and control a display position of a sub-icon accompanying the expansion function icon according to the display position of the expansion function icon.

Images