Information processing device and information processing program

Abstract

To provide an information processing device and an information processing program which can display a video at a time point at which a user wants a video to be stopped from a time point at which a stopping instruction of the video is actually received as a static image.SOLUTION: An information processing device comprises a processor. The processor records a movement of a user to a time point at which a video stopping instruction for stopping a displayed video is performed, and displays a static image of the video at a second time point that is before the first time point at which the video stopping instruction is received, that is when acceleration following the movement of the user exceeds a predetermined reference for the first time and that is immediately after the first time point.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Patent Document 1 describes a video recording system including a recording device that records video data and a remote device that is communicably connected to the recording device via a network, wherein the recording device includes a recording means for recording video data, a video transmission means for converting the video data recorded in the recording means into low-quality video data with reduced image quality and transmitting the converted video data to the remote device, and a still image transmission means for generating requested still image data of higher image quality than the low-quality video data from the video data recorded in the recording means in response to an image request from the remote device and transmitting the generated still image data to the remote device, and the remote device includes a display means for displaying images, and a display means for displaying the low-quality video data on the display screen of the display means while receiving the low-quality video data transmitted from the recording device. A video recording system is disclosed that includes a video playback means for playing and displaying a video indicated by video data; a frame image display means for, in response to receiving a user's instruction input for the video being played, displaying on the display surface of the display means a plurality of frame images that constitute the video played between the time the instruction input was received and a predetermined time prior; an image request means for, in response to a user selection of one of the plurality of frame images, sending an image request to the recording device requesting a high-quality image of the selected frame image; and a still image display means for receiving still image data sent from the recording device in response to the image request and displaying the high-quality still image indicated by the still image data on the display surface of the display means.

[0003] Patent document 2 discloses an annotation device that is characterized by having a display control means for displaying a moving image and symbols corresponding to people appearing in the moving image, and an annotation means for adding annotations to the moving image based on operations on the symbols. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-047447 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-252208 Summary of the Invention [Problem to be solved by the invention]

[0005] Advances in network technology and devices have made it possible for users in remote locations to communicate with each other via voice and video. Voice and video communication is also useful for remote maintenance, where support is provided from a remote location while viewing video of a specific object on-site. In remote maintenance, the main focus of communication is how to operate which part of the specific object displayed on the screen.

[0006] In remote maintenance, there are cases where it is desired to display a still image at a point when the object or a specific part of the object that is the target of support is clearly visible. However, when a user freezes the image to provide support, there is a discrepancy between the point when the user attempts to freeze the image and the point when the instruction to freeze the image is actually accepted, and the image may not be frozen at the point the user intended.

[0007] The present invention aims to provide an information processing device and an information processing program that can display, as a still image, the video at the time the user intended to freeze the video, going back from the time the instruction to freeze the video was actually accepted. [Means for solving the problem]

[0008] An information processing device according to a first aspect includes a processor that records the user's actions up until a freeze instruction is given to freeze the displayed image, and displays a still image of the image at a second point in time immediately before the first point in time at which the freeze instruction is accepted and at which the acceleration associated with the user's actions exceeds a predetermined standard for the first time.

[0009] In the information processing device of the second aspect, the processor displays the still image of the video, setting the second point in time to the point at which the standard is first exceeded within the range from the first point in time to a third point in time that is a predetermined time back.

[0010] In the information processing device of the third aspect, the processor displays a still image of the video at the first time point, then accepts a still image display instruction to display a still image of the video at the second time point, and displays the still image of the video at the second time point.

[0011] In the information processing device according to a fourth aspect, the processor displays a plurality of still image candidates cut out from video from the second time point to the first time point, including the second time point.

[0012] In the information processing device according to a fifth aspect, the processor accepts a selection instruction for selecting at least one still image candidate from the still image candidates, and displays the selected still image candidate as a still image.

[0013] In the information processing device according to the sixth aspect, the processor displays still images of the video at a fourth point in time that is a predetermined time before the second point in time, and a fifth point in time that is a predetermined time ahead of the second point in time, as the still image candidates.

[0014] In the information processing device according to a seventh aspect, the processor recognizes an object captured in the video at the second time point, and displays a plurality of the still image candidates including the object.

[0015] In the information processing device according to an eighth aspect, the processor displays, from among the still image candidates, still image candidates in which the object is shown in the center of the screen.

[0016] In the information processing device according to a ninth aspect, when there are a plurality of objects, the processor displays the still image candidates in which each of the objects is captured.

[0017] In the information processing device according to a tenth aspect, the processor plays back the video from the point in time when the still image was displayed.

[0018] An information processing program according to an eleventh aspect causes a computer to record a user's actions up until a freeze instruction is given to freeze the displayed image, and displays a still image of the image at a second point in time immediately before a first point in time when the freeze instruction is accepted and at which the acceleration associated with the user's actions exceeds a predetermined standard for the first time. [Effects of the Invention]

[0019] According to the information processing device of the first aspect, it is possible to display, as a still image, the video at the time when the user intended to freeze the video, going back from the time when the instruction to freeze the video was actually accepted.

[0020] According to the information processing device of the second aspect, it is possible to limit the time point at which a still image is displayed.

[0021] According to the information processing device of the third aspect, it is possible to visually compare still images of the video at both the time when the instruction to freeze the video is accepted and the time when the user's intention to freeze the video is recognized.

[0022] According to the information processing device of the fourth aspect, it is possible to visually compare, from among the plurality of displayed still image candidates, a still image of a video image that is closest to the time when the user intended to freeze the video.

[0023] According to the information processing device of the fifth aspect, it is possible to display a still image of a video that is closest to the time when the user intended to freeze the video.

[0024] According to the information processing device of the sixth aspect, it is possible to make a comparison between candidates for still images taken at a time close to the user's intention to freeze the video.

[0025] According to the information processing device of the seventh aspect, it is possible to display a plurality of still image candidates in which an object that the user is particularly paying attention to in the video is displayed within the screen.

[0026] According to the information processing device of the eighth aspect, it is possible to display a plurality of still image candidates of a video at a time when an object that the user is particularly paying attention to is easily visible.

[0027] According to the information processing device of the ninth aspect, it is possible to display a plurality of still image candidates of video at times when the objects that the user is paying attention to are each easily visible.

[0028] According to the information processing device of the tenth aspect, it is possible to continue playing the video from the point at which the user intended to stop the video.

[0029] According to the information processing program of the eleventh aspect, it is possible to display, as a still image, the video at the time when the user intended to freeze the video, going back from the time when the instruction to freeze the video was actually accepted. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 1 is a diagram illustrating an outline of an entire information processing system. [Figure 2] FIG. 10 is a diagram showing an example of video exchange between the engineer side and the expert side. [Figure 3] FIG. 2 is a diagram illustrating an example of a display of the information processing device according to the first embodiment. [Figure 4]FIG. 10 is a diagram showing an example of changes in video at multiple points in time. [Figure 5] FIG. 2 is a block diagram showing a hardware configuration of the information processing device. [Figure 6] FIG. 2 is a block diagram showing a functional configuration of the information processing device. [Figure 7] FIG. 10 is a diagram illustrating an example of a method for detecting a point in time when a user intends to stand still. [Figure 8] FIG. 2 is a flowchart illustrating processing by the information processing device according to the first embodiment. [Figure 9] FIG. 10 is a diagram showing an example of a display of an information processing device according to the present embodiment that has a "slight rewind button" implemented therein. [Figure 10] FIG. 10 is a flowchart illustrating another example of the process of the information processing device according to the first embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of a display of an information processing device according to a second embodiment. [Figure 12] FIG. 10 is a diagram showing an example of displaying candidates for still images of a video at the time when a specific object is placed in the center of the screen. [Figure 13] FIG. 10 is a diagram showing an example of displaying candidates for still images of a video at the time when each of a plurality of objects is positioned at the center of the screen. [Figure 14] FIG. 10 is a flowchart illustrating processing by an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0031] Hereinafter, an example of an embodiment of the disclosed technology will be described in detail with reference to the drawings.

[0032] First Embodiment Fig. 1 is a diagram showing an outline of an entire information processing system including an information processing device 10 according to this embodiment. As shown in Fig. 1, the information processing system according to this embodiment is configured to include the information processing device 10 and an engineer terminal 20, and the information processing device 10 and the engineer terminal 20 are connected to each other. Although not shown in the figure, the information processing device 10 and the engineer terminal 20 are connected to a network and can communicate with each other via the network. As an example of this network, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), etc. are applied.

[0033] The information processing device 10 according to this embodiment is used for remote maintenance, for example, when an expert gives instructions regarding a specific object to an engineer at a remote location. FIG. 2 is a diagram showing an example of video exchange between the engineer side and the expert side using the information processing device 10. As shown in FIG. 2, the engineer uses the engineer side terminal 20 to capture an image of the object 30 that requires instructions from the expert. The image captured by the engineer side terminal 20 is transmitted to the information processing device 10 via the above-mentioned network. The image displayed on the display 21 or the like of the engineer side terminal 20 that captured the image of the object 30 is also displayed on the monitor 16 of the information processing device 10. The expert gives instructions to the engineer while visually checking the image of the object 30 displayed on the monitor 16 of the information processing terminal 10.

[0034] In FIG. 2 , the engineer's terminal 20 is depicted as a smartphone. However, the engineer's terminal 20 is not limited to a tablet-type terminal. Furthermore, the information processing device 10 is depicted as a general computer terminal. However, the information processing device 10 may be a tablet-type terminal such as a smartphone, and is not limited to a general computer terminal. Both the information processing device 10 and the engineer's terminal 20 are equipped with a monitor 16, a display 21, etc., as output devices for displaying videos or images. The monitor 16 and the display 21, which serve as output devices, may also function as input devices such as touch panels. Furthermore, the output device and the input device do not necessarily need to be built into the information processing device 10 and the engineer's terminal 20, but may be connected to each as external devices. Furthermore, in both devices and terminals, the output of screen display and the like is not limited to a display, but may be VR (Virtual Reality) glasses, etc.

[0035] FIG. 3 is a diagram illustrating an example of a display of the information processing device 10 according to the first embodiment. The monitor 16 of the information processing device 10 displays the video received from the engineer terminal 20. A video freeze button 41 is provided at the bottom of the display portion 40 that displays the video received from the engineer terminal 20. The expert visually examines the object 30 shown in the video received from the engineer terminal 20 and freezes the video temporarily when, for example, the expert wants to add annotations such as text or symbols to the video as supplemental information to more accurately convey instructions. The expert can also freeze the video by pressing or tapping the video freeze button 41 when the expert wants to carefully examine the object 30 or when the expert wants to see the object 30 more clearly at the moment it is captured.

[0036] However, because the image changes sequentially, there may be a discrepancy between the time when the expert intends to freeze the image and the time when the information processing device 10 actually receives a press of the image freeze button 41, etc. FIG. 4 is a diagram illustrating an example of changes in the image at multiple points in time. FIG. 4 shows an image that changes over time from point C to point A. Although the object 30 is shown in all of the images at points C, B, and A shown in FIG. 4, the position and angle of the object 30 change over time. In the image at point B shown in FIG. 4, the object 30 is shown near the center of the screen, but at point C the object 30 is shown at the bottom left corner of the screen, and at point A the object 30 is shown at the top right corner of the screen. Even if the expert intends to freeze the image at point B, when the object 30 is more easily visible, depending on the time when the image freeze button 41 is actually pressed, the image at point A, which is slightly later than point B, may be displayed as a still image. The information processing device 10 in this embodiment actually displays, as a still image, the video at the time when the expert intended to freeze the video, going back from the time when the instruction to freeze the video from the expert was actually received. In the following description, the expert will be referred to as the user.

[0037] Next, the configuration of the information processing device 10 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing an example of the hardware configuration of the information processing device 10 according to this embodiment. As an example, the information processing device 10 according to this embodiment will be described as a terminal such as a personal computer. However, the present invention is not limited to this.

[0038] 5, an information processing device 10 according to this embodiment includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a monitor 16, and a communication interface (communication I / F) 17. The CPU 11, the ROM 12, the RAM 13, the storage 14, the input unit 15, the monitor 16, and the communication I / F 17 are interconnected by a bus 18. Here, the CPU 11 is an example of a processor.

[0039] The CPU 11 supervises and controls the entire information processing device 10. The ROM 12 stores various programs, including the information processing program used in this embodiment, and data. The RAM 13 is a memory used as a work area when various programs are executed. The CPU 11 performs processing to display each image and character string by expanding the program stored in the ROM 12 into the RAM 13 and executing it. The storage 14 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory. The storage 14 may store the information processing program, etc. The input unit 15 is, for example, a mouse and a keyboard that accept character input, etc. The monitor 16 displays each screen. The communication I / F 17 transmits and receives data.

[0040] Next, the functional configuration of the information processing device 10 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the functional configuration of the information processing device 10 according to this embodiment.

[0041] 6, the information processing device 10 has, as its functional components, a recording unit 101 and a display unit 102. The CPU 11 executes an information processing program to function as the recording unit 101 and the display unit 102.

[0042] The recording unit 101 records the user's actions up to the time when an instruction to freeze the image displayed on the display portion 40 of the monitor 16 is given.

[0043] In this embodiment, the video freeze instruction for freezing the video is an operation such as the user pressing or tapping the video freeze button 41.

[0044] The user's actions refer to operations such as pressing or tapping the video freeze button 41 using a peripheral device such as a touch pen used by the user. The user's actions are recorded, for example, using a motion sensor pre-installed in the peripheral device such as a touch pen used by the user. Recording of the user's actions is not limited to recording three-dimensional movement changes using a motion sensor installed in the touch pen itself. If the monitor 16 is a touch panel, recording may be performed using an electrostatic contact method in which the monitor 16 itself reacts when the touch pen approaches the monitor 16, or a method in which movement changes of a mouse pointer are detected by operating a mouse. If the information processing terminal 10 is a tablet terminal, the user's actions may be recorded based on movement changes using a motion sensor installed in the information processing device 10 itself. Furthermore, the user's actions may be recorded using a motion sensor installed in a wearable terminal such as a watch-type terminal or a ring-type terminal worn by the user.

[0045] Furthermore, the display unit 102 displays a still image of the video at a second point in time immediately preceding the first point in time at which the video freeze instruction was accepted and at which the acceleration associated with the user's movement exceeded a predetermined standard for the first time.

[0046] Based on the change in acceleration accompanying the user's movements recorded by the recording unit 101, the display unit 102 displays the video at the time the user intended to freeze the video as a still image, going back from the time the instruction to freeze the video was actually accepted.

[0047] The first time point is the time point when an instruction to freeze the video is accepted, that is, the time point when the information processing device 10 accepts that the user has pressed the video freeze button 41. The second time point is the time point immediately before the first time point when the acceleration accompanying the user's movement exceeds a predetermined standard for the first time, and is the time point when it is presumed that the user actually intended to freeze the video.

[0048] 7 is a graph showing an example of a change in acceleration accompanying a user's movement recorded by the recording unit 101, and is a diagram showing an example of a method for detecting the point in time when the user intends to stand still. In FIG. 7, the X axis represents the magnitude of the acceleration accompanying the detected user's movement, and the Y axis represents the passage of time.

[0049] As shown in the graph in Figure 7, when we start recording the user's movements, there is almost no change in the acceleration associated with the user's movements. However, at a certain point, the acceleration associated with the user's movements suddenly starts to increase, and as soon as it reaches its maximum, the acceleration drops again.

[0050] The following describes an example of a series of actions that occur when a user presses image freeze button 41 with a touch pen. First, the user picks up the touch pen. Next, the user lifts the touch pen and moves it close to image freeze button 41. After moving the touch pen close to image freeze button 41 and accurately pressing image freeze button 41, the user puts the touch pen down.

[0051] The above-described series of actions for pressing the image freeze button 41 will be described in relation to the changes in acceleration accompanying a series of user actions shown in Fig. 7. The point in time when the stylus is lifted and moved close to the image freeze button 41 is the point in time when it is assumed that the user actually attempted to freeze the image; in other words, the second point in time is point B when the acceleration accompanying the user's action begins to rise sharply. Furthermore, as shown in Fig. 7, there is a time lag of time t from point B to point A when the user presses the image freeze button 41, the information processing device 10 accepts the request to freeze the image, and actually freezes the image, that is, point in time 1.

[0052] However, regardless of the occurrence of the time lag t, the display unit 102 displays a still image of the video at a second time point (time point B) immediately before the first time point (time point A) when the acceleration caused by the user's movement exceeded the threshold for the first time. The threshold value of the acceleration caused by the user's movement for detecting time point B, i.e., the second time point, can be set arbitrarily.

[0053] Furthermore, the display unit 102 displays a still image of the video at a second time point that is the first time point that exceeds the reference within a range from the first time point to a third time point that is a predetermined time back.

[0054] Some users may constantly move peripheral devices or wearable devices without intending to freeze the video even while viewing the video as a moving image, which may result in frequent user actions with accelerations exceeding the threshold. Therefore, based on the user actions recorded by the recording unit 101, the display unit 102 detects accelerations associated with the user actions that exceed the threshold within a limited range from the first time point to a third time point, a predetermined time prior to the first time point. In the example of FIG. 7 , for example, if the third time point is time point C, then the time at which the acceleration associated with the user actions first exceeds the threshold from time point C prior to time point A is time point B. The predetermined time prior to time point C from the first time point to the third time point can also be set arbitrarily.

[0055] Furthermore, the display unit 102 may play back the video from the point of the displayed still image. The display unit 102 may not only display the video as a still image at a predetermined point in time, but may also continue to play back the video from the predetermined point in time at which it was displayed as a still image. When playing back the video from the point of the displayed still image, a video play button or the like provided near the video freeze button 41 is displayed.

[0056] Next, the operation of the information processing device 10 according to this embodiment will be described. Fig. 8 is a flowchart showing the processing of the information processing device according to the first embodiment. The processing of the information processing device 10 according to this embodiment is performed by the CPU 11 reading and executing an information processing program stored in the RAM 13 or the like.

[0057] In step S101, the CPU 11 causes the display unit 102 to display the video received from the engineer's terminal 20.

[0058] In step S102, the CPU 11 functions as the recording unit 101 to record the user's actions.

[0059] In step S103, the CPU 11 determines whether or not an image freeze instruction has been received by pressing the image freeze button 41. If an image freeze instruction has not been received, the process returns to step S101, and if an image freeze instruction has been received, the process proceeds to step S104.

[0060] In step S104, the CPU 11 causes the display unit 102 to display a still image at the second time point.

[0061] As described above, according to the information processing device 10 of this embodiment, it is possible to display the video at the time when the user intended to freeze the video as a still image, going back from the time when the instruction to freeze the video was actually accepted.

[0062] In this embodiment, when the video freeze button 41 is pressed, the display unit 102 automatically displays a still image of the video at the second time point, instead of the first time point. However, the display unit 102 may temporarily display a still image of the video at the first time point. As a variation of this embodiment, a "slight rewind button" may be additionally implemented in addition to the video freeze button 41. FIG. 9 is a diagram showing an example of a display of the information processing device 10 according to this embodiment, which has the "slight rewind button" implemented.

[0063] Furthermore, after displaying a still image of the video at the first time point, the display unit 102 receives a still image display instruction to display a still image of the video at the second time point, and displays the still image of the video at the second time point.

[0064] 9 is additionally implemented, display unit 102 first displays a still image of the video at the first time point when video freeze button 41 is pressed. After displaying the still image of the video at the first time point, when the user presses "slight rewind button" 42, the display unit 102 goes back from the first time point and displays a still image of the video at the second time point, which is the time point when it is presumed that the user intended to freeze the video.

[0065] Next, the operation of the information processing device 10 according to this embodiment when a "slight rewind button" 42 is additionally implemented will be described. Fig. 10 is a flowchart illustrating another example of the processing of the information processing device according to the first embodiment. As with the above-described flowchart, the processing of the information processing device 10 according to this embodiment is also performed by the CPU 11 reading and executing an information processing program stored in the RAM 13 or the like.

[0066] Steps S201 to S203 have the same functions as steps S101 to S103 in the above-described flowchart, and therefore their explanation will be omitted.

[0067] In step S204, the CPU 11 causes the display unit 102 to freeze the image that is being displayed at the time when the image freeze instruction is received.

[0068] In step S205, the CPU 11 determines whether or not the "slight rewind button" 42 has been pressed. If the "slight rewind button" 42 has not been pressed, the process proceeds to step S207. If the "slight rewind button" 42 has been pressed, the process proceeds to step S206.

[0069] In step S206, the CPU 11 causes the display unit 102 to display a still image at the second time point.

[0070] In step S207, the CPU 11 causes the display unit 102 to display a still image at the first time point.

[0071] As described above, by additionally implementing the "slight rewind button" 42, the user can compare both still images of the video at the first point in time and still images of the video at both points in time, the second point in time.

[0072] <Second embodiment> Next, a second embodiment will be described. In this embodiment, a case where a plurality of still image candidates of a video are displayed will be described in detail. Note that, since the information processing device 50 according to this embodiment is based on the information processing device 10 according to the first embodiment, the same reference numerals as in the first embodiment are used to designate the same configurations, functions, actions, etc. as the information processing device 10 according to the first embodiment, and detailed descriptions thereof will be omitted.

[0073] The hardware configuration of the information processing device 50 according to this embodiment is common to the hardware configuration of the information processing device 10 according to the first embodiment shown in FIG. 5, and therefore description thereof will be omitted.

[0074] The functional configuration of the information processing device 50 according to the present embodiment is also common to the functional configuration of the information processing device 10 according to the first embodiment, as shown in Fig. 6. The information processing device 50 also includes, as functional units, a recording unit 101 and a display unit 202. Here, only the functions that differ from the first embodiment will be described, and a description of the functions that are common to the first embodiment will be omitted.

[0075] The display unit 202 displays a plurality of still image candidates cut out from video from a second time point including the second time point to the first time point.

[0076] FIG. 11 is a diagram showing an example of a display of an information processing device 50 according to the second embodiment. Three still image candidates, display portions 40A to 40B, are displayed on the monitor 16 of the information processing device 50. The display unit 202 displays a plurality of still image candidates, cut out from a video from a second time point, including the second time point, to a first time point, on the monitor 16. In the example shown in FIG. 11, three still image candidates, display portions 40A to 40C, are displayed. However, the number of still image candidates to be displayed does not need to be limited to three. Cutting out still images from a video from a second time point, including the second time point, to a first time point may be performed at predetermined intervals.

[0077] Furthermore, display unit 202 accepts a selection instruction to select at least one still image candidate from among the still image candidates, and displays the selected still image candidate as a still image. The user compares the still image candidates displayed in display portions 40A to 40C shown in Fig. 11, and selects one of the still images displayed at the time that best reflects the user's still image intention. Display unit 202 accepts a still image selection instruction from the user, and displays the still image selected by the user.

[0078] In the case of monitor 16 having a touch panel function, the user's instruction to select a still image candidate is by tapping or the like. However, the user's instruction to select a still image candidate is not limited to a tap operation. The user's instruction to select a still image candidate may also be by operating a mouse pointer or by keyboard input. Furthermore, as shown in FIG. 11 , the still image selected by the user may be displayed in display portion 40D while still displaying the still image candidate including the selected still image on monitor 16, or only the still image selected by the user may be displayed on monitor 16.

[0079] In addition, the display unit 202 displays still images of the video at a fourth time point that is a predetermined time before the second time point and a fifth time point that is a predetermined time before the second time point as still image candidates.

[0080] The fourth time point is a time point that is a predetermined time before the second time point, and the fifth time point is a time point that is a predetermined time before the second time point. For example, if the predetermined time is set to 2 seconds, the display unit 202 displays, as still image candidates, a still image of the video at a time point that is 2 seconds before the second time point and a still image of the video at a time point 2 seconds after the second time point.

[0081] Furthermore, the display unit 202 recognizes the object 30 appearing in the video at the second time point, and displays a plurality of still image candidates including the object 30.

[0082] The display unit 202 recognizes the object 30 on the video that the user is particularly paying attention to, using image recognition technology, etc. The object 30 may be recognized automatically using image recognition technology, or may be recognized by having the user arbitrarily select an object 30 from among the objects on the video.

[0083] Furthermore, the display unit 202 displays, from among the still image candidates, the still image candidate in which the object 30 is shown in the center of the screen.

[0084] Fig. 12 is a diagram showing an example of displaying candidate still images of a video at the time when a specific object is positioned at the center of the screen. In the example shown in Fig. 12, multiple objects in addition to object 30 are displayed in display portion 40D. Display unit 202 recognizes object 30 displayed in the video displayed on display portion 40D of monitor 16, and displays multiple candidate still images in which object 30 is displayed at the center of the screen in display portions 40A to 40C. In display portions 40A and 40C, which are candidate still images shown in Fig. 12, multiple objects other than object 30 are partially visible on the screen, but object 30 is displayed near the center of the screen. In display portion 40B, object 30 and multiple other objects are all displayed within the screen, with object 30 in particular being displayed near the center of the screen.

[0085] The state in which the object 30 is displayed in the center of the screen means that when the entire screen is recognized as multiple sections, a specific object 30 is displayed superimposed within the section located in the center of the screen. The user can arbitrarily change the extent of the section located in the center of the screen.

[0086] Furthermore, when there are multiple objects 30, the display unit 202 displays still image candidates in which each of the objects 30 appears.

[0087] Fig. 13 is a diagram showing an example of displaying candidate still images of a video at the time when multiple objects are each positioned at the center of the screen. The video displayed on display portion 40A of monitor 16 in Fig. 13 shows three types of objects: a triangular object 30a, a circular object 30b, and a star-shaped object 30c. As shown in Fig. 13, when there are multiple objects 30, display unit 202 displays candidate still images showing each of objects 30a to 30c.

[0088] 13, display portion 40A is a still image in which object 30b is clearly displayed. In display portion 40B, although objects 30b and 30c do not entirely fit within the screen, object 30a is displayed in a manner that makes it easy to view the entire object 30a. In display portion 40C, although objects 30a and 30b do not entirely fit within the screen, object 30c is displayed in a manner that makes it easy to view the entire object 30c.

[0089] 13, when there are three types of objects 30, objects 30a to 30c, the display unit 202 displays, as still image candidates, still images of video at a time when each of the three types of objects 30a to 30c is displayed in a state where it is easily visible. As with the above, when recognizing multiple objects 30, the display unit 202 recognizes each object 30 using video recognition technology. The number of objects 30 to be recognized does not have to be three.

[0090] Next, the operation of the information processing device 50 according to the present embodiment will be described. Fig. 14 is a flowchart showing the processing of the information processing device according to the second embodiment. The processing of the information processing device 50 according to the present embodiment is performed by the CPU 11 reading and executing an information processing program stored in the RAM 13 or the like.

[0091] Steps S301 to S303 are the same as steps S101 to S103 of the information processing device 10 in the first embodiment, and therefore a description thereof will be omitted.

[0092] In step S304, CPU 11 determines whether or not the image shows multiple objects 30. If the image shows multiple objects 30, the process proceeds to step S305. If the image shows multiple objects 30, the process proceeds to step S306.

[0093] In step S305, the CPU 11 causes the display unit 202 to display still image candidates in which each of the plurality of objects is contained within the screen.

[0094] In step S306, if there is a single object 30, the CPU 11 causes the display unit 202 to display a still image candidate in which the object 30 is shown near the center of the screen.

[0095] In step S307, the CPU 11 causes the display unit 202 to display the multiple still image candidates that were displayed in the previous step S305 or step S306.

[0096] In step S308, the CPU 11, functioning as the display unit 202, accepts that an arbitrary still image candidate has been selected by the user from among the multiple still image candidates displayed in the previous step S307.

[0097] In step S309, the CPU 11 causes the display unit 202 to display, as a still image, any still image candidate selected by the user in the previous step S308.

[0098] As described above, by display unit 202 recognizing object 30 in the video, it is possible to display a still image of the video at a point in time when object 30 that the user is paying attention to is more easily visible. Furthermore, information processing device 50 according to the present embodiment displays a plurality of still image candidates, and can display, from the displayed plurality of candidates, a still image of the video that is closest to the point in time when the user intended to freeze.

[0099] In the first and second embodiments, an example has been described in which an engineer transmits video to an expert, and the expert visually checks the video while having the engineer perform remote maintenance on the target object 30. However, the information processing device 10 and the information processing device 50 may be applied to video such as videos that are widely and generally distributed. Furthermore, the application is not limited to video received in real time, and may also be applied to video that has been recorded in advance. [Explanation of symbols]

[0100] 10, 50 Information processing equipment 11 CPU 12 ROM 13 RAM 14. Storage 15 Input section 16 monitors 17 Communication I / F 18 Bus 20 Engineer's terminal 21 Engineer's terminal display 30 Objects 40 Display area 41 Freeze button 42 Rewind button 101 Recording section 102, 202 Display section

Claims

1. a processor; The processor: The video received via the network is displayed on the screen. recording in a memory, using a sensor that measures acceleration, the acceleration of the user's movement until the image freeze instruction is given to freeze the image displayed on the screen; A still image of the video at a second time point immediately before the first time point when the video freeze instruction was accepted and when acceleration accompanying the user's movement exceeded a predetermined standard stored in a nonvolatile memory for the first time is displayed on the screen. Information processing device.

2. The processor: The still image of the video is displayed at a time point that exceeds the reference value for the first time within a range from the first time point to a third time point that is a predetermined time back, as the second time point. The information processing device according to claim 1 .

3. The processor: After displaying a still image of the video at the first time point, a still image display instruction for displaying a still image of the video at the second time point is accepted through a user pressing an operation button provided on the screen, and the still image of the video at the second time point is displayed.

3. The information processing device according to claim 1.

4. The processor: a plurality of still image candidates extracted from video from the second time point including the second time point to the first time point are displayed; 4. The information processing device according to claim 1.

5. The processor: When the user selects one of the still image candidates through the screen, a selection instruction for selecting at least one still image candidate from the still image candidates is accepted, and the selected still image candidate is displayed on the screen as a still image. The information processing device according to claim 4 .

6. The processor: and displaying still images of the video at a fourth time point that is a predetermined time before the second time point and a fifth time point that is a predetermined time before the second time point as the still image candidates.

6. The information processing device according to claim 4 or claim 5.

7. The processor: Recognizing an object appearing in the video at the second time point and displaying a plurality of still image candidates including the object.

7. The information processing device according to claim 4.

8. The processor: Among the still image candidates, a still image candidate in which the object is displayed at the center of the screen is displayed. The information processing device according to claim 7 .

9. The processor: When there are a plurality of objects, the still image candidates in which each object is captured are displayed. The information processing device according to claim 7 .

10. The processor: The video is played back from the point of the still image being displayed. The information processing device according to any one of claims 1 to 9.

11. On the computer, The video received via the network is displayed on the screen. recording in a memory, using a sensor that measures acceleration, the acceleration of the user's movement until the user issues an instruction to freeze the image displayed on the screen; A still image of the video at a second time point immediately before the first time point when the video freeze instruction was accepted and when acceleration accompanying the user's movement exceeded a predetermined standard stored in a nonvolatile memory for the first time is displayed on the screen. Information processing program.

Images