Information processing device, information processing method, and program
The information processing device addresses the limitation of fixed webcam placement by allowing the webcam image to follow the cursor, improving the explainer's ability to convey explanations and enhancing viewer understanding by dynamically positioning the image relative to the explanation target.
Patent Information
- Application Number
- PCT/JP2024/017960
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-17
- Filing Date
- 2024-05-15
- Publication Date
- 2024-11-21
AI Technical Summary
Conventional technologies restrict the explainer's ability to move freely while explaining, as webcam videos are fixedly placed, making it difficult for the audience to understand the explanation target on the screen.
An information processing device with object movement control, image display control, and image movement control means that allows the webcam image to follow the movement of a cursor, enabling the explainer to position and move the image relative to the explanation target, facilitating easier understanding.
Enables the explainer to convey explanations more effectively by allowing the webcam image to follow the cursor, reducing the viewer's need to constantly shift focus between the explainer and the explanation target, enhancing the clarity and engagement of the presentation.
Smart Images

Figure JP2024017960_21112024_PF_FP_ABST
Abstract
Description
Information processing device, information processing method, and program
[0001] The present invention relates to an information processing device, an information processing method, and a program.
[0002] There has been a technology for displaying an image of a web camera on a screen while displaying an object to be explained (see, for example, Patent Document 1).
[0003] Japanese Patent Application Laid-Open No. 2021-061527
[0004] In the conventional technology described above, the webcam image was fixedly positioned at the four corners or bottom of the screen, which meant that the presenter had to be positioned far away from the subject (e.g., a graph) displayed on the screen, and the presenter's explanation was often not conveyed to the viewer.
[0005] The present invention has been made in consideration of the above circumstances, and aims to provide a technique that allows a presenter to provide an easy-to-understand explanation to viewers about the subject of explanation displayed on a screen.
[0006] In order to achieve the above object, an information processing device according to one embodiment of the present invention comprises: an object movement control means for controlling an object that points to a predetermined position on a screen to move within the screen in response to a user's operation; an image display control means for controlling the display of an image to be displayed on the screen; and an image movement control means for performing tracking control, in which the image is positioned at a relative position based on the object, and the image is moved to follow the movement of the object.
[0007] An information processing method and a program according to one aspect of the present invention are respectively an information processing method and a program corresponding to the information processing device according to the above-described one aspect of the present invention.
[0008] According to the present invention, it is possible to provide a technique that allows an presenter to provide an easy-to-understand explanation to a viewer about an explanation subject displayed on a screen.
[0009] 1 is a diagram showing an example of a system configuration of an information processing system including a server of an embodiment of an information processing device according to the present invention. FIG. 1 is a diagram showing an example of a hardware configuration of a server in the information processing system of FIG. 1. FIG. 2 is a functional block diagram showing an example of a functional configuration of a server having the hardware configuration shown in FIG. 2. FIG. 3 is a flowchart showing the operation of a server having the hardware configuration of FIG. 2 and the functional configuration of FIG. 3. FIG. 1 is a diagram showing an example screen (example of shortcut icon display) of a server having the functional configuration of FIG. 3. FIG. 1 is a diagram showing an example screen when a controller appears near the mouse cursor immediately after launching this application from the screen of FIG. 5. FIG. 2 is a diagram showing an example screen when a camera function is activated with the controller on the screen of FIG. 6. FIG. 3 is a diagram showing an example screen when a camera function menu is displayed with the controller on the screen of FIG. 7. FIG. 4 is a diagram showing an example screen when mouse tracking is enabled in the camera function menu of the controller on the screen of FIG. 8. FIG. 4 is a diagram showing an example screen showing a state in which a camera image follows and moves the cursor when mouse tracking is enabled and the mouse is moved on the screen of FIG. 9. FIG. 5 is a diagram showing an example screen when selecting a shooting area to be recorded on the presenter terminal of FIG. 1. FIG. 6 is a diagram showing an example screen when a camera image follows the movement of the cursor while selecting a shooting area and shooting on the screen of FIG. 11. 13 is a diagram showing an example of a screen showing how facial expressions and emotions are conveyed by the camera image following the cursor movement on the screen of FIG. 12. FIG. 14 is a diagram showing an example of a screen when uploading the captured content using the controller on the screen of FIG. 13 after completing the capture.
[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, the system configuration of an information processing system including a server according to an embodiment of the information processing device of the present invention will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the system configuration of an information processing system including a server according to an embodiment of the information processing device of the present invention.
[0011] 1 is configured to include a server 1, a presenter terminal 2, viewer terminals 3-1 to 3-n, and a web camera 4. The server 1, the presenter terminal 2, and the viewer terminals 3-1 to 3-n are connected to each other so as to be able to communicate with each other via a predetermined network NW such as the Internet. Note that the form of the network NW is not particularly limited, and examples thereof include Bluetooth (registered trademark), Wi-Fi, a LAN (Local Area Network), and the Internet.
[0012] The presenter terminal 2 is an information processing device, such as a smartphone, tablet terminal, or personal computer (PC), managed by the presenter, and has an app for the presenter provided by the service provider installed. The presenter terminal 2 is equipped with a webcam 4 on the display surface side and functions as a display device. For example, when operated by the presenter, the presenter terminal 2 can explain the contents of the materials while displaying explanatory materials such as graphs and the presenter's own camera image on the screen of the launched app. The presenter terminal 2 communicates with the viewer terminals 3-1 to 3-n as appropriate and executes various control processes, including processes such as disclosing content to each of the viewer terminals 3-1 to 3-n.
[0013] The viewer terminals 3-1 to 3-n are terminals that accept operations by viewers, such as doctors, who are member users of the service, and are information processing devices such as smartphones, tablet terminals, and personal computers (PCs). A viewer app provided by the service provider is installed on the viewer terminals 3-1 to 3-n. During a webinar broadcast live on the Internet by a presenter, the viewer terminals 3-1 to 3-n can display explanatory materials such as graphs provided on the app screen and view an explanation of the graphs while the presenter moves a cursor on the screen using a mouse. Hereinafter, when there is no need to distinguish between the viewer terminals 3-1 to 3-n, they will be collectively referred to as the "viewer terminal 3."
[0014] The webcam 4 is placed where the presenter is present and captures an image of the presenter. Specifically, the webcam 4 captures a facial image of the presenter as he or she gives a presentation facing the display screen of the presenter terminal 2. The image of the presenter captured by the webcam 4 (camera image) is distributed from the presenter terminal 2 to each of the viewer terminals 3-1 to 3-n via the network NW and the server 1. In this example, the webcam 4 is connected to the presenter terminal 2, but a network-compatible camera may be directly connected to the network NW. Furthermore, if the presenter terminal 2 itself has a built-in camera function, the webcam 4 is not necessary.
[0015] In this embodiment, as an example of collaboration between the viewer terminal 3, the presenter terminal 2, and the server 1, a first method is adopted in which a dedicated app provided by a service provider is installed on the viewer terminal 3 and the presenter terminal 2, and the viewer terminal 3 and the presenter terminal 2 execute the processing of the app.However, in addition to this, a second method may also be adopted in which the viewer terminal 3 and the presenter terminal 2 access a website disclosed by the server 1 and display the website on each terminal.
[0016] According to this first method, the viewer terminal 3 and the presenter terminal 2 communicate with the server 1 as needed, and the server 1 executes the main processing. The method for realizing this service through collaboration between the server 1, the presenter terminal 2, and the viewer terminal 3 is not limited to the first and second methods. For example, a method that appropriately combines the first and second methods can also be adopted as a "collaboration" method.
[0017] However, for the sake of convenience, the following examples will be described assuming that the first method is adopted as the "collaboration" method. In other words, in the following examples, the server 1 is the subject of the processing, but this is merely an example, and in actual implementation, the presenter terminal 2, viewer terminal 3, or other information processing device may be the subject of the processing, as appropriate.
[0018] In this example, the server 1 executes the main processing, so the hardware configuration and functional configuration of the server 1 will be described below, focusing on the server 1 in the information processing system of FIG.
[0019] First, an example of the hardware configuration of the server 1 in the information processing system of Fig. 1 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the hardware configuration of the server according to an embodiment of the information processing device of the present invention.
[0020] The server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.
[0021] The CPU 11 executes various processes according to programs recorded in the ROM 12 or programs loaded from the storage unit 18 into the RAM 13. The RAM 13 also stores data and the like necessary for the CPU 11 to execute various processes.
[0022] The CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output interface 15 is also connected to this bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.
[0023] The output unit 16 is composed of a display such as a liquid crystal display device, a printer, a speaker, etc., and outputs various information. The input unit 17 is composed of input devices such as a keyboard and a mouse, and inputs various information. The storage unit 18 is composed of a DRAM (Dynamic Random Access Memory) etc., and stores various data. The communication unit 19 communicates with other devices via a network NW including the Internet.
[0024] Removable media 21, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately loaded into drive 20. Programs read from removable media 21 by drive 20 are installed in storage unit 18 as needed. The removable media 21 can also store various data stored in storage unit 18 in the same way as storage unit 18. The hardware configuration of server 1 has been described above, but the hardware configurations of presenter terminal 2 and viewer terminal 3 are similar, and therefore description thereof will be omitted.
[0025] Next, with reference to Fig. 3, the functional configuration of the server 1, the presenter terminal 2, and the viewer terminal 3 having the hardware configuration of Fig. 2 will be described. Fig. 3 is a functional block diagram showing an example of the functional configuration of the server, the presenter terminal, and the viewer terminal having the hardware configuration of Fig. 2. In the following description, in order to distinguish between the hardware of the server 1, the hardware of the presenter terminal 2, and the hardware of the viewer terminal 3, the hardware elements of the server 1 will be marked with k, the hardware elements of the presenter terminal 2 will be marked with s, and the hardware elements of the viewer terminal 3 will be marked with r.
[0026] 3, a display instruction receiving unit 51 and a display control unit 52 function in the CPU 11s of the presenter terminal 2. A presentation application program (hereinafter referred to as the "app") provided by the service provider is installed in the presenter terminal 2, and a shortcut icon for launching the app is arranged in the menu bar of the screen. Alternatively, when the shortcut icon is clicked, a browser pre-installed on the terminal may be launched, and the URL of a predetermined page on the server 1 may be entered from the browser to access the server 1 and display the presentation screen.
[0027] The display instruction receiving unit 51 receives operations by the presenter, such as receiving an instruction to start an application (such as clicking an icon on the menu bar) by operating the mouse, instructions such as the position and size of content (files such as documents, graphs, and slides) and camera images to be opened and arranged on the screen, and instructions such as selection operations on the controls of the controller 83 (see FIG. 6 ) displayed on the screen.
[0028] The display control unit 52 executes control to display the processing results according to the received instruction on the screen. For example, when a click on an icon is received, an application is launched and the results are displayed on the application's presentation screen. Alternatively, when a click on a shortcut icon is received, a browser pre-installed on the terminal may be launched, and the URL of a predetermined page on the server 1 may be entered from the browser to access the server 1 and display the presentation screen.
[0029] In this service, separate apps are installed for the presenter and the viewer, with the appropriate app installed on each device. However, it is also possible to install the same app on both the presenter's and viewer's devices, and vary the app or the operating functions within the presentation screen displayed depending on the user's authority. For example, the presenter's authority may allow the user to operate all functions within the presentation screen, while the viewer's authority may only allow the user to display content on the screen.
[0030] A display instruction receiving unit 61 and a display control unit 62 function in the CPU 11r of the viewer terminal 3. An application provided by the service provider is installed in the viewer terminal 3, and a shortcut icon for launching the application is arranged in the menu bar on the screen.
[0031] The display instruction receiving unit 61 receives operations by the viewer, such as an instruction to start an application by operating a mouse (such as clicking an icon on a menu bar).
[0032] The display control unit 62 executes control to display on the screen the results of processing in accordance with the instructions received by the display instruction receiving unit 61. For example, when a click on an icon is received, an app is launched and the results are displayed on the presentation screen of the app. Alternatively, when a shortcut icon is clicked, a browser pre-installed on the terminal may be launched, and the URL of a specific page on the server 1 may be entered from the browser to access the server 1 and display the presentation screen. In this service, the functions that can be operated within the app or the displayed presentation screen vary depending on the user's authority. The viewer's authority is limited to launching the app and viewing the contents of the displayed presentation screen.
[0033] Furthermore, in the CPU 11k of the server 1, a cursor movement control unit 41, a camera image display control unit 42, an image movement control unit 43, an operation acceptance unit 44, and a controller display control unit 45 function.
[0034] The cursor movement control unit 41 executes control to move a cursor indicating a predetermined position on the screen within the screen in response to a user operation. Specifically, the cursor movement control unit 41 executes control to move the cursor using the display (a display device having a camera on the display surface side) of the presenter terminal 2 as a screen.
[0035] The camera image display control unit 42 executes control to display the camera image captured by the web camera 4 on the screen of each of the presenter terminal 2 and the viewer terminal 3. Note that the control of the camera image includes not only control of the image captured by the camera, but also control of the camera itself (control of the angle of view and control of the orientation). The camera image display control unit 42 displays the facial image of the presenter obtained as a result of capturing an image by the web camera 4 (captured image) on the screen as an image. As a result, when the cursor is moved, the facial image of the presenter follows, making it possible to include the presenter's facial expression in the explanation, making the explanation easier to understand.
[0036] The image movement control unit 43 executes tracking control, for example, by positioning the camera image at a position near the cursor on the lower right and moving the camera image to follow the movement of the cursor. As a result, when the cursor is moved to the position of an object to be explained, such as a graph, with the camera image positioned on the lower right of the cursor, the camera image also moves to follow the movement, allowing the presenter to explain the graph to the viewer in an easy-to-understand manner.
[0037] The operation acceptance unit 44 accepts a mouse operation (predetermined operation) at a predetermined position indicated by the cursor while the camera image is being tracked. This allows the mouse to be operated as usual even when the webcam 4 image is being tracked to the lower right of the cursor. The operation acceptance unit 44 permits execution of the tracking control when a user, for example, a presenter, instructs the user to start tracking, and prohibits execution of the tracking control when the presenter instructs the user to stop tracking. Specifically, the operation acceptance unit 44 permits execution of the tracking control when an operator (for example, a mouse tracking item 87 among the items in the menu 86) located on the controller 83 (see FIG. 9 , etc.) displayed on the screen is selected and activated, and prohibits execution of the tracking control when the operator 87 is selected and deactivated again.
[0038] The controller display control unit 45 displays on the screen a controller 83 including one or more operators (six function icons in this example) that each perform one or more instructions to at least one of the webcam 4 and the image, and accepts an operation when the presenter (user) operates any of the six function icons arranged in the controller 83.
[0039] 9 , one of the six function icons included in the controller 83 is, for example, a camera-related function icon, and clicking on this function icon displays a menu 86. The menu 86 includes function items such as "Camera Position," "Recording Area Corner," "Follow Mouse," "Camera List," "Anmaze Virtual Camera," and "Integrated Camera." The display of this controller 83 allows the presenter, when it becomes necessary to switch functions during a presentation, to smoothly switch functions by operating the controller 83 located on the screen, without having to perform unnecessary operations such as moving the cursor to the task bar or start button at the bottom of the screen.
[0040] Next, the operation of the server 1 having the functional configuration of FIG. 3 will be described with reference to FIG. 4. FIG. 4 is a flowchart showing the operation of the server having the functional configuration of FIG. 3. When an online briefing is held, the presenter and viewers start an application installed on their respective terminals, display a presentation screen (hereinafter referred to as "screen") on each terminal, and start communication between the terminals via the server 1. When the presenter displays the content to be presented on the screen of the presenter terminal 2 and operates the mouse to move the cursor (object), in step S1, the cursor movement control unit 41 of the server 1 executes control to move the cursor, which indicates a predetermined position on the screen, within the screen in accordance with the presenter's operation.
[0041] Furthermore, when the presenter operates the mouse to display a camera image (image to be displayed) on the screen, in step S2, the camera image display control unit 42 executes control to display the camera image on the screen.
[0042] In step S3, the image movement control unit 43 performs tracking control, for example, by placing the camera image at a position near the cursor (relative positioning based on the cursor) at the bottom right, and moving the camera image to follow the movement of the cursor.
[0043] As described above, according to the information processing system of this embodiment, by positioning the camera image at the lower right position near the cursor and controlling the camera image to move in accordance with the movement of the cursor, the presenter can provide viewers with an easy-to-understand explanation of graphs and other items displayed on the screen. In addition, the server 1 of this embodiment can achieve the following effects: By having the camera image 85 follow the cursor 82, viewers can easily understand which part of the screen 80 the presenter is explaining. The presenter can record and play back the elements he or she wants to explain and his or her own facial expressions in adjacent positions on the screen 80. Viewers watching (viewing) the explanation can simultaneously check the subject of the explanation and the other person's facial expressions with minimal eye movement. Since the movement of the camera image 85 can be easily controlled with mouse operation, the presenter can easily express and convey his or her own emotions to viewers. For example, by moving the mouse slowly, the camera image 85 also moves slowly, emphasizing and conveying key points. By moving the mouse quickly and vigorously, the camera image 85 also moves quickly and vigorously, conveying the presenter's excitement to viewers.
[0044] Next, a specific example of operation in an online briefing will be described with reference to FIGS. 5 to 10. FIG. 5 is a diagram showing an example of a screen when the presenter terminal of FIG. 1 is started up. FIG. 6 is a diagram showing a state in which a cursor and a controller are displayed on the screen of FIG. 5. FIG. 7 is a diagram showing a state in which the cursor is moved to the position of the camera function icon in the controller of FIG. 6. FIG. 8 is a diagram showing an example of a menu displayed when the icon of FIG. 7 is clicked. FIG. 9 is a diagram showing a state in which the "mouse tracking" function item in the menu of FIG. 8 is selected with the cursor. FIG. 10 is a diagram showing a state in which the camera image follows when the cursor is moved with the "mouse tracking" function of FIG. 9 enabled.
[0045] In this information processing system, when the presenter terminal 2 is powered on, a home screen of the operating system (OS) is displayed as a screen 80, as shown in FIG. 5 . A task bar is provided at the bottom of the screen 80. Shortcut icons 81 are arranged on the task bar. The shortcut icons 81 are provided by a service provider and are used to launch an app installed on the presenter terminal 2.
[0046] When the presenter uses the mouse to click on a shortcut icon 81 located on the task bar of the screen 80, the app launches, and a cursor 82 and a controller 83 are displayed on the screen 80, as shown in FIG. 6 . Six function icons (operators) are arranged on the controller 83. One of the six function icons is, for example, a camera-related function icon 84 shown in FIG. 7 . When the presenter uses the mouse to move the cursor 82 to the position of the function icon 84 and click it, an image of the presenter's face captured by the webcam 4 (hereinafter referred to as a "camera image") is displayed in a circular window. Note that the concept of a camera image also includes still images and videos. To enable the mouse tracking function of the camera image 85, the presenter moves the cursor 82 to the position of the upward arrow symbol immediately to the right of the function icon 84 and clicks it, as shown in FIG. 8 . A menu 86 pops up. The menu 86 contains function items such as "Camera Position," "Corner of Recording Area," "Follow Mouse," "Camera List," "Anmaze Virtual Camera," and "Integrated Camera." By displaying this menu 86, when it becomes necessary to switch functions during the explanation, the presenter can smoothly switch functions (e.g., enable / disable) by displaying the menu 86 as a pop-up near the cursor 82 and selecting the desired function item.
[0047] Here, the presenter operates the mouse to move the cursor 82 to the position of the function item "Follow Mouse" 87 in the menu 86, as shown in Figure 9, and clicks, thereby enabling the mouse-following function of the camera image 85.
[0048] As a result, when the presenter operates the mouse to move the cursor 82 on the screen 80 along the arrow line Y, as shown in Figure 10, the camera image 85, which was positioned immediately to the lower right of the cursor 82, will follow the cursor 82 and move along the same trajectory as the cursor 82.
[0049] Next, the operation of the screen recording function of the server 1 of the embodiment will be described with reference to FIGS. 11 to 14. FIG. 11 is a diagram illustrating an operation for selecting a shooting area to be recorded on the presenter terminal of FIG. 1. FIG. 12 is a diagram illustrating how the camera image follows the cursor when the cursor is pointed at a graph displayed in the shooting area while the presenter is shooting the shooting area of FIG. 11. FIG. 13 is a diagram illustrating how the camera image follows the cursor as it moves within the shooting area while the presenter is shooting the shooting area of FIG. 11. FIG. 14 is a diagram illustrating how the presenter completes shooting using an icon in the controller and uploads the captured content. The server 1 of the embodiment has a function for recording a specified area of the screen being presented by the presenter. In this case, clicking the "Corner of Recording Area" function item located in the menu 86 shown in FIG. 9 activates the screen recording function, and a cross cursor 88 is displayed on the screen 80 as shown in FIG. 11. Here, the presenter operates the mouse to specify the area to be recorded (shooting area) displayed on the screen 80.
[0050] Specifically, the presenter operates the mouse to move the cross cursor 88 to the upper left corner of the desired shooting area and clicks it, and then drags it in the direction of arrow T to move the cross cursor 88 to the lower right corner of the desired shooting area and clicks it, thereby finalizing the shooting area 89 and starting screen recording of the shooting area 89. At the same time, the pause button on the controller 83 changes to a button indicating that recording is in progress, for example, a red "circle" mark 90, making it visible that the device is in recording mode.
[0051] As shown in Figure 12, when content such as a graph 91 is displayed within the shooting area 89 of the screen 80 and an explanatory video is shot, the presenter operates the mouse to move the cursor 82 to an important point on the graph 91 while giving the explanation, and the camera image 85 following the cursor 82 also moves close to the important point, so that the presenter's facial expression shown in the camera image 85 can also be conveyed to the viewer at the same time.
[0052] Furthermore, as shown in FIG. 13, the camera image 85 moves as indicated by the arrow P, following the cursor 82 which moves to the location within the cross cursor 89 where the presenter wants to explain, so the expressions and emotions of the presenter in the camera image 85 can be conveyed well to the viewer.
[0053] When filming is finished, the presenter operates the mouse to move cursor 82 to the position of check mark button 92 on controller 83 displayed on screen 80, as shown in Fig. 14, and clicks check mark button 92, whereby the recorded (filmed) explanatory video is uploaded to server 1, and a dialog box 93 indicating that uploading is in progress is displayed. In this way, the explanatory video uploaded to server 1 is stored in storage unit 18 (see Fig. 2), so that the recorded explanatory video can be effectively utilized, for example, by reading out the explanatory video from storage unit 18 and playing it again during a briefing session to provide an explanation, or by making it available (distributed) to viewers who were unable to attend the briefing at a later date.
[0054] While one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and modifications, improvements, and the like within the scope of achieving the object of the present invention are included in the present invention. In the above embodiment, the function of the "Follow Mouse" item among the functional items arranged on the controller 83 has been mainly described. However, the server 1 also has the following functions. For example, the following functions are provided as setting functions related to the camera image 85. A function to adjust the size of the camera image 85 that follows the cursor 82. A function to adjust the transparency of the camera image 85 that follows the cursor 82. A function to switch the window frame of the camera image 85 that follows the cursor 82 to a specified shape, such as a circle or a square. A function to add a border of any color and thickness to the camera image 85 that follows the cursor 82. A function to set a virtual background for the camera image 85 that follows the cursor 82. A function to display the contents of a voice transcript near the camera image 85 that follows the cursor 82. A speed adjustment function to adjust the tracking speed of the camera image 85 that follows the cursor 82. This speed adjustment function prevents the camera image 85 from moving violently by making it follow the cursor 82 after a short time has passed (delayed following) rather than immediately following the cursor 82, making it easier for the viewer to view the facial expression of the presenter in the camera image 85.
[0055] In the above-described embodiment, an example in which the shooting area is specified as a rectangle (see FIG. 11 ) is shown. However, other shapes such as a circle or other shapes may be used, as long as the area is a predetermined area. In the above-described embodiment, an example in which the camera image 85 is positioned in the lower right position near the cursor 82 is shown. However, the relative positional relationship between the cursor 82 and the camera image 85 can be changed by setting conditions. In addition, in the above-described embodiment, the camera video is shown in the shape of a circular window (see FIG. 7 , etc.). However, other shapes (e.g., rectangular, elliptical, or other three-dimensional shapes) may be used. In the above-described embodiment, a graph (see FIG. 12 ) is shown as an example of content. However, other shapes may be used as long as the content is sufficient. In the above-described embodiment, the server 1, the presenter terminal 2, and the viewer terminal 3 are connected via a network NW. However, other configurations may be used. For example, some of the components provided in the server 1 may be provided in the presenter terminal 2 or the viewer terminal 3, and the functions of the present invention may be realized between these two.
[0056] The functional block diagram shown in Fig. 3 is merely an example and is not particularly limited. That is, it is sufficient if the information processing system in Fig. 4 is provided with a function that can execute the above-described series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the example in Fig. 3.
[0057] Furthermore, the locations of the functional blocks are not limited to the example in Fig. 3 and may be arbitrary. Furthermore, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination thereof.
[0058] When the processing of each functional block is performed by software, the programs constituting the software are installed onto a computer or the like from a network or a recording medium. The computer may be a computer incorporated into dedicated hardware. The computer may also be a computer capable of performing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.
[0059] The recording medium containing such a program may not only be composed of removable media that is distributed separately from the device itself in order to provide the program to each user, but may also be composed of recording media that are provided to each user in a state where they are pre-installed in the device itself.
[0060] In other words, the information processing device to which the present invention is applied is sufficient as long as it has the following configuration, and can take on a variety of different embodiments. (1) That is, an information processing device (e.g., server 1 in FIG. 3 ) according to one aspect of the present invention includes: object movement control means (e.g., cursor movement control unit 41 in FIG. 3 ) that controls the movement of an object (e.g., cursor 82 in FIG. 10 ) indicating a predetermined position within a screen, within the screen in response to a user's operation; image display control means (e.g., camera image display control unit 42 in FIG. 3 ) that controls the display of an image to be displayed (here, the image refers to a camera image (including both still images and moving images) captured by a webcam 4) within the screen; and image movement control means (e.g., image movement control unit 43 in FIG. 3 ) that controls the movement of the image (e.g., camera image 85 in FIG. 10 ) by positioning the image (e.g., camera image 85 in FIG. 10 ) at a relative position (e.g., a position near the cursor 82 at the bottom right) based on the object (e.g., cursor 82 in FIG. 10 ), and by moving the image (e.g., camera image 85 in FIG. 10 ) in accordance with the movement of the object (e.g., cursor 82 in FIG. 10 ). As a result, when an image (e.g., camera image 85 in Figure 10) is placed to the lower right of an object (e.g., cursor 82 in Figure 10), and the cursor is moved to the position of the object to be explained (e.g., content such as a graph, slide, or document), the image (e.g., camera image 85 in Figure 10) also moves in accordance with the movement, allowing the presenter to explain the object to be explained (e.g., content such as a graph, slide, or document) in an easy-to-understand manner to the viewer, while incorporating the presenter's own facial expressions.
[0061] (2) An information processing device (e.g., server 1 in FIG. 3) according to one embodiment of the present invention further comprises: operation receiving means (e.g., operation receiving unit 44 in FIG. 3) for receiving a predetermined operation (mouse operation) on the predetermined position indicated by the object (e.g., cursor 82 in FIG. 11) in a state where the image (e.g., camera image 85 in FIG. 11) is following; and thereby, even if an image (e.g., camera image 85 in FIG. 11) is following the object (e.g., cursor 82 in FIG. 11) at the lower right, the mouse can be operated as usual.
[0062] (3) In the information processing device (e.g., server 1 in FIG. 3), the operation reception unit (e.g., operation reception unit 44 in FIG. 3) permits execution of the tracking control when a tracking instruction is given by the user's operation, and prohibits execution of the tracking control when a tracking stop instruction is given by the user's operation, so that cursor tracking can be quickly enabled and disabled using a controller.
[0063] (4) In the information processing device (for example, server 1 in FIG. 3), the object movement control means (for example, cursor movement control section 41 in FIG. 3) controls the movement of the object (for example, cursor) using the display surface of a display device (for example, presenter terminal 2) equipped with a camera (for example, webcam 4 in FIG. 3) on the side of the display surface as the screen, and the image display control means (for example, camera image display control section 42 in FIG. 3) displays the captured image (for example, facial image of the presenter, etc.) obtained as a result of being captured by the camera as the image on the screen. Therefore, when the object (for example, cursor 82 in FIG. 10) is moved, the captured image (for example, facial image of the presenter, etc.) follows, making it possible to provide an explanation that includes the presenter's facial expression, and the explanation can be made easier for the viewer to understand by using voice and facial expressions to explain.
[0064] (5) The information processing device (e.g., server 1 in FIG. 3) further comprises a controller display control means (e.g., controller 83 in FIG. 6) that displays on the screen a controller including at least one operator that performs one or more instructions for at least one of the camera and the image, and that accepts an operation when the user operates any of the one or more operators (e.g., controller display control unit 45 in FIG. 3). This makes it possible to smoothly give instructions for explanation without performing unnecessary operations such as returning to the menu bar during explanation.
[0065] (6) An information processing method executed by an information processing device according to one embodiment of the present invention can include: an object movement control step (e.g., image display step S1 in FIG. 4) that executes control to move an object (e.g., cursor 82 in FIG. 10) pointing to a predetermined position on a screen within the screen in response to a user's operation; an image display control step (e.g., image display control step S2 in FIG. 4) that executes control to display an image to be displayed (e.g., camera image 85 in FIG. 10) within the screen; and an image movement control step (e.g., image movement control step S3 in FIG. 4) that executes control as a tracking control to place the image (e.g., camera image 85 in FIG. 10) at a relative position (e.g., a position near the cursor 82 at the bottom right) based on the object (e.g., cursor 82 in FIG. 10) and move the image (e.g., camera image 85 in FIG. 10) in accordance with the movement of the object (e.g., cursor 82 in FIG. 10).
[0066] (7) A program according to one aspect of the present invention can cause a computer to execute control processing including: an object movement control step (e.g., image display step S1 in FIG. 4 ) for controlling an object (e.g., cursor 82 in FIG. 10 ) pointing to a predetermined position on a screen to be moved within the screen in response to a user's operation; an image display control step (e.g., image display control step S2 in FIG. 4 ) for controlling an image to be displayed (e.g., camera image 85 in FIG. 10 ) on the screen; and an image movement control step (e.g., image movement control step S3 in FIG. 4 ) for performing tracking control to position the image (e.g., camera image 85 in FIG. 10 ) at a relative position (e.g., a position near the cursor 82 at the bottom right) based on the object (e.g., cursor 82 in FIG. 10 ) and move the image (e.g., camera image 85 in FIG. 10 ) in accordance with the movement of the object (e.g., cursor 82 in FIG. 10 ).
[0067] 1...server, 2...presenter terminal, 3, 3-1 to 3-n...viewer terminal, 11...CPU, 12...ROM, 13...RAM, 14...bus, 15...input / output interface, 16...output unit, 17...input unit, 18...storage unit, 19...communication unit, 20...drive, 41...cursor movement control unit, 42...camera image display control unit, 43...image movement control unit, 44...operation reception unit, 45...controller display control unit, 51, 61...display instruction reception unit, 52, 62...display control unit, NW...network
Claims
1. An information processing device comprising: an object movement control means for executing control to move an object indicating a predetermined position within a screen within the screen in response to a user's operation; an image display control means for executing control to display an image to be displayed within the screen; and an image movement control means for executing tracking control to position the image at a relative position based on the object, and to move the image to follow the movement of the object.
2. The information processing device according to claim 1, further comprising: an operation receiving means for receiving a predetermined operation for the predetermined position pointed to by the object in a state where the image is being followed.
3. The information processing device according to claim 1, wherein the image movement control means permits execution of the tracking control when a tracking instruction is given by the user's operation, and prohibits execution of the tracking control when a tracking stop instruction is given by the user's operation.
4. The information processing device according to claim 1, wherein the object movement control means controls the movement of the object using a display surface of a display device having a camera on the side of the display surface as the screen, and the image display control means causes an image captured by the camera to be displayed on the screen as the image.
5. An information processing device as described in claim 4, further comprising a controller display control means for displaying on the screen a controller including at least one or more operators each performing one or more instructions for at least one of the camera and the image, and for accepting an operation when the user operates any of the one or more operators.
6. An information processing method executed by an information processing device, comprising: an object movement control step of executing control to move an object indicating a predetermined position within a screen, within the screen in response to a user's operation; an image display control step of executing control to display an image to be displayed within the screen; and an image movement control step of executing control to position the image at a relative position based on the object, and to move the image to follow the movement of the object, as a tracking control.
7. A program for causing a computer to execute control processing including: an object movement control step for executing control to move an object indicating a predetermined position within a screen within the screen in response to a user's operation; an image display control step for executing control to display an image to be displayed within the screen; and an image movement control step for executing control to position the image at a relative position based on the object and move the image to follow the movement of the object as a tracking control.
Citation Information
Patent Citations
Cursor display method
JP1992257046A
Image processing method
JP2000331180A
Item management in the user interface
JP2013542514A
Web conference system, processing method, and program
JP2022021122A
System and method for accessing remote screen content
US20040150664A1
