Imaging apparatus, control method, and program

Abstract

To provide a technique capable of determining the end of online communication at a proper timing.SOLUTION: An imaging apparatus includes connection means for connecting to an external device, imaging means for imaging a video, and control means for performing processing for transmitting the video imaged by the imaging means to the external device according to the reception of the transmission request of the video from the external device connected by the connection means. The control means starts first time measurement according to the completion of the reception of the video by the external device, starts second time measurement according to the transmission of the video to the external device, and releases a state for performing the processing, when a first time elapses from the start of the first time measurement or when a second time elapses from the start of the second time measurement.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system in which an external device and an imaging device are connected so as to be able to communicate with each other. [Background technology]

[0002] Online communication applications that allow users to communicate face-to-face online using video and audio (video calls, video conferences, web conferences, etc.) are becoming popular. Methods for determining the end of a video call in such online communication applications are described in Patent Documents 1 and 2. Patent Document 1 describes a method for determining the end of a call when a certain period of time has passed without any calls since the previous call, and Patent Document 2 describes a method for switching the icon display depending on whether the icon of the other party currently in a call is active or not. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-199851 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-138972 Summary of the Invention [Problem to be solved by the invention]

[0004] An imaging device that captures video of the user is connected to a user device on which an online communication application runs, but it may be difficult for the imaging device to determine the end of a video call at an appropriate timing. Furthermore, Patent Documents 1 and 2 do not take into consideration processing according to the characteristics of the user device on which the online communication application runs.

[0005] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to realize a technology that can determine the end of online communication at an appropriate timing. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, the imaging device of the present invention has a connection means for connecting to an external device, an imaging means for capturing images, and a control means for performing a process of transmitting the image captured by the imaging means to the external device in response to receiving a request to send the image from the external device connected by the connection means, wherein the control means starts a first timekeeping operation in response to the external device completing reception of the image, and starts a second timekeeping operation in response to transmitting the image to the external device, and cancels the state of performing the process when a first time has elapsed since the start of the first timekeeping operation or a second time has elapsed since the start of the second timekeeping operation. [Effects of the Invention]

[0007] According to the present invention, it becomes possible to determine at an appropriate timing that online communication has ended. [Brief explanation of the drawings]

[0008] [Figure 1] 1A is a block diagram showing the configuration of an imaging device according to an embodiment of the present invention, and FIGS. 1B and 1C are external views thereof. [Figure 2] FIG. 1 is a block diagram showing the configuration of an information processing apparatus according to an embodiment of the present invention. [Figure 3A] 5A and 5B are diagrams illustrating the states of the imaging device and the information processing device when the video call application is on standby according to the embodiment. [Figure 3B] 5A and 5B are diagrams illustrating the states of the imaging device and the information processing device at the start of a video call using the video call application of the present embodiment. [Figure 3C] 5A and 5B are diagrams illustrating the states of the imaging device and the information processing device during a video call using the video call application of the present embodiment. [Figure 3D]10A and 10B are diagrams illustrating the states of the imaging device and the information processing device when a video call of the video call application of the present embodiment ends. [Figure 4] FIG. 4 is a sequence diagram illustrating a first video call end determination process during a video call according to the present embodiment. [Figure 5] FIG. 10 is a sequence diagram illustrating a second video call end determination process during a video call according to the present embodiment. [Figure 6] FIG. 10 is a sequence diagram illustrating a third video call end determination process during a video call according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] Below, we will explain in detail, with reference to the accompanying drawings, an embodiment in which the imaging device of the present invention is applied to an image generating device such as a digital camera capable of capturing images such as still images and videos, and the information processing device of the present invention is applied to an information processing device such as a personal computer (PC).

[0011] First, the configuration of the system and device of this embodiment will be described with reference to FIGS.

[0012] <System configuration> The system of this embodiment includes an imaging device 100 and an information processing device 200. The imaging device 100 and the information processing device 200 are connected so that they can communicate with each other. The imaging device 100 operates as a client device, and the information processing device 200 operates as a host device. In this embodiment, the imaging device 100 transmits video data captured by the imaging unit 102 to the information processing device 200 via the connection unit 109 in response to receiving a video data transmission request from the information processing device 200 operating as the host device. Some information processing devices 200 have a feature of transmitting a video data reception completion notification to the imaging device 100 in response to receiving video data from the imaging device 100 operating as the client device, and some do not.

[0013] Furthermore, the imaging device 100 repeatedly performs an operation of starting a first time count by a timer in response to transmitting video data to the information processing device 200, starting a second time count by a timer in response to receiving a video data reception completion notification from the information processing device 200, and resetting the first time count and the second time count in response to receiving a video data transmission request from the information processing device 200. In this case, the imaging device 100 displays first information (an icon indicating that the imaging device 100 is in a live mode) indicating that the mode state for performing a process for transmitting video data to the information processing device 200 is continuing.

[0014] Furthermore, if the first time has elapsed since the start of the first timing or the second time has elapsed since the start of the second timing without receiving a request to transmit video data from the information processing device 200, the imaging device 100 determines that the video call has ended and cancels the mode state in which the processing for transmitting video data is performed. In this case, the imaging device 100 displays second information (an icon indicating that the device is in standby) indicating that the mode state in which the processing for transmitting video data is performed has been canceled to the information processing device 200.

[0015] The video data includes both a form including audio data and a form not including audio data.

[0016] <Configuration of Image Capturing Apparatus 100> Next, the configuration and functions of the image capturing apparatus 100 of this embodiment will be described with reference to FIG. 1(a).

[0017] In this embodiment, an imaging device such as a digital camera will be described, but the present invention is not limited to this and may also be an information and communication terminal such as a smart device, which is a type of mobile phone, or a tablet device with a camera function, a media player, a surveillance camera, a medical camera, etc.

[0018] The control unit 101 is a central processing unit (CPU) that controls the entire imaging device 100, and performs communication processing and control processing, which will be described later, by executing programs stored in a nonvolatile memory 103, which will be described later. Note that instead of the control unit 101 controlling the entire device, the entire device may be controlled by having multiple pieces of hardware share the processing. The control unit 101 also includes a first timer 101a and a second timer 101b that measure the time used for communication processing and control processing, which will be described later in this embodiment.

[0019] The imaging unit 102 includes a group of lenses including a zoom lens and a focus lens, and a shutter with an aperture function. The imaging unit 102 also includes an imaging element configured with a CCD, CMOS element, or the like that converts a subject image into an electrical signal, and an A / D converter that converts an analog image signal output from the imaging element into a digital signal. Under the control of the control unit 101, the imaging unit 102 converts the subject image light formed by the lens included in the imaging unit 102 into an electrical signal using the imaging element, performs noise reduction processing, and outputs image data consisting of a digital signal.

[0020] The control unit 101 performs resizing processes such as pixel interpolation and reduction, and color conversion processes on image data captured by the imaging unit 102. The control unit 101 also compresses and encodes the processed still image data in a format such as JPEG, and encodes the moving image data in a moving image compression format such as MP4, to generate an image file, which is then recorded on the recording medium 108. In the imaging device 100 of this embodiment, the image data is recorded on the recording medium 108 in accordance with the DCF (Design Rule for Camera File system) standard. The control unit 101 also performs predetermined arithmetic processing using the captured image data, and controls the focus lens, aperture, and shutter of the imaging unit 102 based on the obtained arithmetic results, thereby performing AF (autofocus) processing and AE (autoexposure) processing.

[0021] The nonvolatile memory 103 is an electrically erasable and recordable memory, such as an EEPROM. Constants and programs for the operation of the control unit 101 are recorded in the nonvolatile memory 103. The programs referred to here are programs for executing communication processing and control processing, which will be described later in this embodiment.

[0022] The working memory 104 is used as a working area for expanding constants and variables for the operation of the control unit 101, programs read from the nonvolatile memory 103, etc. The working memory 104 is also used as a buffer memory for temporarily storing image data captured by the imaging unit 102, and as an image display memory for the display unit 106.

[0023] The operation unit 105 is made up of operation members such as various switches, buttons, and a touch panel that accept various operations from the user. For example, as shown in FIGS. 1(b) and 1(c), the operation unit 105 includes a shutter button 105a for capturing an image, a playback button 105b for playing back a captured image, and a four-way key 105c consisting of up, down, left, and right buttons for configuring various camera settings. The operation unit 105 also includes a touch panel 105d that is integrally formed with a display unit 106 (described later). The operation unit 105 also includes operation members such as a dedicated connection button for starting communication with an external device via a connection unit 109 (described later).

[0024] The display unit 106 displays a viewfinder image during shooting, displays captured images, and displays text for interactive operations. The display unit 106 is, for example, a display device such as a liquid crystal display or an organic EL display. The display unit 106 may be integrated with the imaging device 100 or may be an external device connected to the imaging device 100. The imaging device 100 only needs to be able to connect to the display unit 106 and have a function for controlling the display on the display unit 106.

[0025] The audio input unit 107 is one or more microphones built into the imaging device 100 or connected via an audio terminal, and converts analog audio signals generated by collecting audio around the imaging device 100 into digital signals and outputs them to the control unit 101. The control unit 101 performs various types of audio processing on the digital signals generated by the audio input unit 107 to generate audio data.

[0026] The recording medium 108 records image data output from the imaging unit 102 and audio data output from the audio input unit 107. Furthermore, the control unit 101 reads out image files and audio files stored on the recording medium 108 in playback mode. The recording medium 108 may be a memory card or hard disk drive that is attached to the imaging device 100, or may be a flash memory or hard disk drive built into the imaging device 100. It is sufficient that the imaging device 100 has at least a means for accessing the recording medium 108.

[0027] The connection unit 109 is an interface for connecting to an external device such as the information processing device 200 so as to be able to communicate with the external device. The imaging device 100 of this embodiment can exchange data with the external device via the connection unit 109. For example, image data generated by the imaging unit 102 and audio data generated by the audio input unit 107 can be transmitted to the external device via the connection unit 109. In this embodiment, the connection unit 109 is a wired communication interface such as USB (Universal Serial Bus) or HDMI (registered trademark). The control unit 101 realizes communication with the external device by controlling the connection unit 109. The communication method is not limited to wired communication, and can include wireless communication interfaces such as a wireless LAN conforming to the IEEE802.11 standard, an infrared communication interface, and Wireless USB.

[0028] <Configuration of Information Processing Device 200> Next, the configuration and functions of the information processing device 200 of this embodiment will be described with reference to FIG.

[0029] In this embodiment, an information processing device such as a PC will be described, but the present invention is not limited to this and may also be an information communication terminal such as a smart device, which is a type of mobile phone, or a tablet device with communication functions, or a wearable computer such as a wristwatch-type smart watch or eyeglass-type smart glasses.

[0030] The information processing device 200 of this embodiment includes a control unit 201, an imaging unit 202, a nonvolatile memory 203, a working memory 204, an operation unit 205, a display unit 206, an audio input unit 207, and a recording medium 208. The basic functions of these elements are the same as those of the imaging device 100, and detailed descriptions thereof will be omitted.

[0031] The nonvolatile memory 203 stores an OS (operating system), which is basic software executed by the control unit 201, and applications that cooperate with the OS to realize applied functions. The nonvolatile memory 203 also stores an application for communicating with the imaging device 100. The nonvolatile memory 203 also stores an online communication application (hereinafter, video call application) that allows users to make video calls while face-to-face online.

[0032] The processing of the information processing device 200 of this embodiment is realized by loading software provided by a video call application. The video call application has software for utilizing the basic functions of the OS installed in the information processing device 200. The OS of the information processing device 200 may have software for realizing the processing of this embodiment.

[0033] The connection unit 209 includes a communication interface for wired or wireless communication with an external device such as the imaging device 100. The control unit 201 controls the connection unit 209 to realize communication with the external device. The connection unit 209 may be directly connected to the imaging device 100 or may be connected via an access point or the like. Protocols for data communication include, for example, UVC (USB Video Class), which is a type of USB standard, and UAC (USB Audio Class). Communication with the imaging device 100 is not limited to USB. For example, wireless communication interfaces such as an infrared communication interface, Bluetooth (registered trademark), and Wireless USB may be included. Furthermore, wired connection means such as HDMI (registered trademark) and IEEE1394 may also be used.

[0034] The audio output unit 211 is a speaker that converts the analog audio signal generated by the audio input unit 207 into sound and outputs it to the outside.

[0035] The public network connection unit 212 is an interface used when performing wireless communication via a wide area network (WAN) such as 4G / LTE or 5G. The information processing device 200 can make calls and perform data communication with other devices via the public network connection unit 212. When making a call, the control unit 201 inputs and outputs audio signals via the audio input unit 207 and the audio output unit 211. In this embodiment, the public network connection unit 212 is not limited to 4G / LTE or 5G, and other communication methods such as local 5G, WiMAX, ADSL, and FTTH may also be used.

[0036] <Operations of the imaging device and the information processing device during a video call> Next, with reference to FIG. 3, the operations of the imaging device 100 and the information processing device 200 during a video call according to this embodiment will be described.

[0037] The imaging device 100 and information processing device 200 of this embodiment are connected by a connection cable 306 such as a USB, and communication processing and control processing are performed between the imaging device 100 and the information processing device 200 by a video call application running on the information processing device 200. Furthermore, the information processing device 200 performs two-way communication of video data and audio data with other user devices by the video call application, enabling users to have face-to-face online video calls with each other.

[0038] The imaging device 100 is fixed to a tripod or the like, and is adjusted in advance to a position and angle of view that allows the imaging device 100 to capture an image of a user 308 of the information processing device 200 .

[0039] FIG. 3A illustrates the states of the imaging device and the information processing device when the video call application of this embodiment is on standby.

[0040] An image of a user 308 captured by the imaging device 100 is displayed on the display unit 106 of the imaging device 100 .

[0041] The video call application displays a GUI 301 on the display unit 206 of the information processing device 200, which includes a screen 305 displaying a video of the user of the other party 309, a video button 302 for transmitting the video captured by the imaging device 100 to the user equipment of the other party 309, an audio button 303 for transmitting the audio of the user 308, and a call button 304 for starting or ending the video call.

[0042] FIG. 3B illustrates the states of the imaging device and the information processing device at the start of a video call in the video call application of this embodiment.

[0043] The information processing device 200 of the user 308 and the user device of the other party 309 are connected via a video call application, and two-way communication of video data and audio data is performed between the user device of the other party 309, enabling a video call.

[0044] When a user 308 receives a request to start a video call from the user equipment of a call partner 309, the user 308 can respond to the request by operating the call button 304.

[0045] When the user 308 responds to a request from the other party 309 to start a video call, the image of the other party 309 is displayed on the screen 305 of the GUI 301 .

[0046] In the state shown in Figure 3B, the image capturing device 100 is not transmitting video data and / or audio data of the user 308 to the information processing device 200, and therefore the image of the user 308 is only displayed on the display unit 106 of the image capturing device 100.

[0047] FIG. 3C illustrates the states of the imaging device and the information processing device during a video call using the video call application of this embodiment.

[0048] In response to a request from the user 308 to start a video call from the call partner 309, the image capturing device 100 starts transmitting video data and / or audio data of the user 308 to the information processing device 200, thus entering a live state.

[0049] The information processing device 200 transmits the video data and audio data of user 308 sequentially received from the imaging device 100 to the user equipment of the call partner 309 using a video call application, and also receives the video data and audio data of the user of call partner 309 from the user equipment of call partner 309. In the example of Fig. 3C, on the display unit 206 of the information processing device 200, on screen 305 of GUI 301, an image of the user of call partner 309 is displayed large in the center of the screen, and an image 310 of user 308 is displayed small in the lower right corner of the screen.

[0050] When the video button 302 is operated by the user 308, video data captured by the imaging unit 102 of the imaging device 100 is stored in the working memory 104, converted by the control unit 101 into video data suitable for display by a video call application, and transmitted from the connection unit 109 to the information processing device 200 via the connection cable 306.

[0051] The display unit 206 of the imaging device 100 displays first information 311 indicating that a process of transmitting video data and audio data to the information processing device 200 is in progress. The first information 311 is, for example, an icon indicating that a live broadcast is in progress.

[0052] Furthermore, when the user 308 operates the audio button 303, audio data collected by the audio input unit 107 of the imaging device 100 is transmitted from the connection unit 109 via the connection cable 306 to the information processing device 200. Furthermore, when the user 308 operates the audio button 303 again, transmission of the audio data collected by the audio input unit 107 of the imaging device 100 to the information processing device 200 is stopped. In this way, by operating the audio button 303, the user 308 can set the audio to be transmitted to the information processing device 200 or to be muted.

[0053] The information processing device 200 stores video data of a user 308 received from the imaging device 100 via a connection cable 306 by a connection unit 209 in a working memory 204, and the control unit 201 performs a process of transmitting the video data to the user equipment of the other party 309 using a video call application, and a process of displaying the video data on a screen 305 of a GUI 301.

[0054] FIG. 3D illustrates the states of the imaging device and the information processing device at the end of a video call in the video call application of this embodiment.

[0055] When a user 308 is in a video call with a call partner 309, the user operates the call button 304 to end the video call.

[0056] When the call button 304 is operated during a video call, the video call application of the information processing device 200 changes the screen 305 of the GUI 301 that displays the video of the user of the other party 309 to a non-display state.

[0057] The display unit 106 of the imaging device 100 displays second information 312 indicating that the process of transmitting video data and audio data to the information processing device 200 is not being performed. The second information 312 is, for example, an icon indicating that the device is in standby mode.

[0058] Here, depending on the characteristics of the OS of the information processing device 200 and the load status, it may take time to determine whether the video call has ended, and the second information (standby icon) 312 may not be displayed at an appropriate time on the display unit 106 of the imaging device 100. Therefore, in this embodiment, by using the processing described later with reference to Figures 4, 5, and 6, the end of the video call is determined quickly and accurately, and the second information 312 is displayed at an appropriate time, regardless of the characteristics of the OS of the information processing device 200 and the load status.

[0059] <First video call end determination process> FIG. 4 is a sequence diagram illustrating the first video call end determination process during a video call.

[0060] In the first video call end determination process, after transmitting video data to the information processing device 200, the imaging device 100 starts a first time measurement by the first timer 101a in response to receiving a video data reception completion notification from the information processing device 200. The imaging device 100 determines that the video call has ended when the first time period has elapsed without receiving the video data reception completion notification. Then, the imaging device 100 controls the display unit 106 to display second information (a standby icon).

[0061] In the following, it is assumed that a video call application is started in the information processing device 200, and an image of the user of the communication partner 309 is displayed on the screen 305 of the GUI 301 as shown in FIG. 3B.

[0062] In S401, in the state of FIG. 3A, in response to the user 308 operating the call button 304 on the screen 305 of the GUI 301, the state becomes one in which a video call is possible, as shown in FIG. 3B.

[0063] In S402, in the state of FIG. 3B, when the user 308 operates the video button 302 on the screen 305 of the GUI 301, the control unit 201 sends a request to the imaging device 100 via the connection unit 209 to send the first frame of video data.

[0064] In S403, when the imaging device 100 receives a request to send video data from the information processing device 200, the control unit 101 reads out the first information (live icon) 311 from the non-volatile memory 103, draws the display data in the working memory 104, and displays it on the display unit 106.

[0065] In S404, the imaging device 100 stores video data of the user 308 captured by the imaging unit 102 in the working memory 104, and the control unit 101 converts the video data into video data suitable for display by a video call application and transmits it from the connection unit 109 to the information processing device 200.

[0066] In S405, when the information processing device 200 receives the video data from the imaging device 100, the control unit 201 transmits a video data reception completion notification to the imaging device 100 via the connection unit 209.

[0067] In S406, when the imaging device 100 receives a video data reception completion notification from the information processing device 200, the control unit 101 starts a first time measurement by the first timer 101a and stores the first time measurement time t1 in the work memory 104. The first timer 101a resets the first time measurement when a first time period (for example, a short period of about 100 ms) has elapsed since the start of the first time measurement, or when the first timer 101a receives a transmission request for the second frame of video data from the information processing device 200 before the first time period has elapsed.

[0068] 3C, the control unit 201 of the information processing device 200 displays the video 310 of the user 308 received from the imaging device 100 on the screen 305 of the GUI 301 displayed on the display unit 206. Note that in S404, the imaging device 100 may transmit the video data to the information processing device 200 without converting it into video data suitable for display by a video call application, and the information processing device 200 may convert the video data received from the imaging device 100 into video data suitable for display by a video call application and display it on the display unit 206.

[0069] In S408, the information processing device 200 performs the same process as in S402 on the video data of the second frame.

[0070] In S409, the imaging device 100 resets the first time counting by the first timer 101a that the control unit 101 started in S406.

[0071] In S410, the image capturing apparatus 100 performs the same process as in S404 on the video data of the second frame.

[0072] In S411, the information processing device 200 performs the same process as in S405 on the video data of the second frame.

[0073] In S412, the image capturing apparatus 100 performs the same process as in S406 on the video data of the second frame.

[0074] In S413, the information processing device 200 performs the same process as in S407 on the video data of the second frame.

[0075] In this way, the same processes as those of S402 to S407 are repeatedly performed for the Nth (N is a natural number equal to or greater than 3) frame of video data from the third frame onwards. This enables the information processing device 200 to update the video on the screen 305 of the GUI 301 displayed on the display unit 206 every time it receives the video 310 of the user 308 from the imaging device 100.

[0076] The processing of S414 to S420 indicates processing in which the imaging device 100 determines whether to end the video call when the user operates the call button 304 of the GUI 301. The following describes an example in which the imaging device 100 determines whether to end the video call when the call button 304 is operated immediately after receiving a request to transmit video data from the information processing device 200.

[0077] In S414, the information processing device 200 performs the same process as in S402 on the Nth frame of video data.

[0078] In S415, the image capturing apparatus 100 resets the first time measurement by the first timer 101a that was started in response to the control unit 101 receiving a notification that reception of the (N-1)th frame of video data has been completed.

[0079] In S416, when the user operates the call button 304 of the GUI 301, the information processing device 200 performs processing to end the video call.

[0080] In S417, the control unit 101 of the imaging device 100 transmits the N-th frame of video data to the information processing device 200 via the connection unit 209 in response to the video data transmission request received in S414.

[0081] In S418, the information processing device 200 performs the same process as in S405 on the Nth frame of video data.

[0082] In S419, in the imaging device 100, the control unit 101 starts a first timer 101a counting the Nth frame of video data, as in S406. Here, the information processing device 200 has ended the video call in SS416, and therefore does not transmit a request to transmit the N+1th frame of video data. Therefore, in the imaging device 100, a first time has elapsed since the first timer 101a started the first counting, and the first counting is reset.

[0083] In S420, the control unit 101 of the imaging device 100 determines that the video call has ended in response to the elapse of the first time period since the first timer 101a started the first time measurement in S419. Then, the imaging device 100 displays second information (a standby icon) 312 on the display unit 106 as shown in FIG. 3D.

[0084] In the first video call end determination process described above, if the information processing device 200 has the property of transmitting a reception completion notification of video data, the imaging device 100 can determine whether the video call has ended in a short time after the video call ends. However, if the information processing device 200 does not have the property of transmitting a reception completion notification of video data, the imaging device 100 cannot determine whether the video call has ended. Therefore, hereinafter, a configuration will be described that enables the imaging device 100 to determine whether the video call has ended even if the information processing device 200 does not have the property of transmitting a reception completion notification of video data.

[0085] <Second Video Call End Determination Process> FIG. 5 is a sequence diagram illustrating the second video call end determination process during a video call.

[0086] In the second video call end determination process, the imaging device 100 receives a video data transmission request from the information processing device 200, starts a second timekeeping by the second timer 101b in response to transmitting the video data to the information processing device 200, and then determines that the video call has ended if the second time has elapsed without receiving a next video data transmission request. Then, the imaging device 100 controls the display unit 106 to display second information (a standby icon).

[0087] In the following, it is assumed that a video call application is started in the information processing device 200, and an image of the user of the communication partner 309 is displayed on the screen 305 of the GUI 301 as shown in FIG. 3B.

[0088] In S501, the state is the same as in S401 in FIG.

[0089] In S502, the information processing device 200 performs the same process as in S402 of FIG.

[0090] In S503, the imaging apparatus 100 performs the same process as in S403 in FIG.

[0091] In S504, the imaging apparatus 100 performs the same process as in S404 in FIG.

[0092] In S505, immediately after transmitting the video data to the information processing device 200 in S504, the control unit 101 of the imaging device 100 starts second timing using a second timer 101b different from the first timer 101a, and stores the second timing time t2 in the work memory 104. The second timer 101b resets the second timing when a second time (for example, a period of about 1 to 2 seconds) has elapsed since the start of the second timing, or when the second timer 101b receives a request to transmit video data of a second frame from the information processing device 200 before the second time has elapsed.

[0093] In S506, similarly to S405 in Fig. 4, the control unit 201 of the information processing device 200 transmits a reception completion notification of the first frame of video data to the imaging device 100. In this case, unlike the first video call end determination process in Fig. 4, the second timekeeping by the second timer 101b is still ongoing when the information processing device 200 transmits the reception completion notification of the video data to the imaging device 100.

[0094] In S507, the information processing device 200 performs the same process as in S407 of FIG.

[0095] In S508, the information processing device 200 performs the same process as in S502 on the video data of the second frame.

[0096] In S509, the imaging device 100 resets the second time measurement by the second timer 101b that the control unit 101 started in S505.

[0097] In S510, the image capturing apparatus 100 performs the same process as in S504 on the video data of the second frame.

[0098] In S511, the image capturing apparatus 100 performs the same process as in S505 on the video data of the second frame.

[0099] In S512, the information processing device 200 performs the same process as in S506 on the video data of the second frame.

[0100] In S513, the information processing device 200 performs the same process as in S507 on the video data of the second frame.

[0101] In this way, the same processes as those of S502 to S507 are repeatedly performed for the Nth (N is a natural number equal to or greater than 3) frame of video data from the third frame onwards. This enables the information processing device 200 to update the video on the screen 305 of the GUI 301 displayed on the display unit 206 every time it receives the video 310 of the user 308 from the imaging device 100.

[0102] The processing of S514 to S519 indicates processing in which the imaging device 100 determines whether to end the video call when the user operates the call button 304 of the GUI 301. The following describes an example in which the imaging device 100 determines whether to end the video call when the call button 304 is operated immediately after receiving a request to transmit video data from the information processing device 200.

[0103] In S514, the information processing device 200 performs the same process as in S502 on the Nth frame of video data.

[0104] In S515, the image capturing apparatus 100 resets the second timekeeping by the second timer 101b that was started in response to the control unit 101 transmitting the (N-1)th frame of video data.

[0105] In S516, when the user operates the call button 304 of the GUI 301, the information processing device 200 performs processing to end the video call.

[0106] In S517, the control unit 101 of the imaging device 100 transmits the N-th frame of video data to the information processing device 200 via the connection unit 109 in response to the video data transmission request received in S514.

[0107] In S518, in the imaging device 100, the control unit 101 starts the second timer 101b to count the second time for the Nth frame of video data, as in S505. Here, the information processing device 200 has ended the video call in S516, and therefore does not transmit a request to transmit the N+1th frame of video data. Therefore, in the imaging device 100, the second time has elapsed since the second timer 101b started the second time, and the second time is reset.

[0108] In S519, the control unit 101 of the imaging device 100 determines that the video call has ended in response to the elapse of the second time since the second timer 101b started the second timing in S518. Then, the imaging device 100 displays second information (a standby icon) 312 on the display unit 106 as shown in FIG. 3D.

[0109] The second video call end determination process described above assumes that the information processing device 200 does not have the characteristic of transmitting a video data reception completion notification. In this case, the imaging device 100 cannot determine that the video call has ended, and therefore determines that the video call has ended on the condition that the second time t2 measured immediately after the video data transmission has elapsed. As a result, even if the information processing device 200 does not have the characteristic of transmitting a video data reception completion notification, the imaging device 100 can determine that the video call has ended within several seconds after the video call has ended. On the other hand, if the information processing device 200 has the characteristic of transmitting a video data reception completion notification, it will take unnecessary time for the imaging device 100 to determine that the video call has ended.

[0110] Therefore, the following describes a configuration that solves both the issues with the first video call end determination process (when the information processing device 200 does not have the characteristic of sending a reception completion notification of video data) and the issues with the second video call end determination process (when the information processing device 200 has the characteristic of sending a reception completion notification of video data), and that combines the first video call end determination process using the first timer 101a and the second video call end determination process using the second timer 101b, allowing the imaging device 100 to perform video call end determination.

[0111] <Third Video Call End Determination Process> FIG. 6 is a sequence diagram illustrating the third video call end determination process during a video call.

[0112] In the third video call end determination process, after transmitting video data to the information processing device 200, the imaging device 100 starts a first time measurement by the first timer 101a in response to receiving a video data reception completion notification from the information processing device 200. The imaging device 100 determines that the video call has ended when the first time period has elapsed without receiving a video data reception completion notification. Then, the imaging device 100 controls the display unit 106 to display second information (a standby icon).

[0113] Furthermore, the imaging device 100 receives a transmission request for video data from the information processing device 200, and starts a second time measurement by the second timer 101b in response to the imaging device 100 transmitting the video data to the information processing device 200. The imaging device 100 determines that the video call has ended when the second time period has elapsed without receiving a transmission request for the next video data. Then, the imaging device 100 controls the display unit 106 to display second information (a standby icon).

[0114] In the following, it is assumed that a video call application is started in the information processing device 200, and an image of the user of the communication partner 309 is displayed on the screen 305 of the GUI 301 as shown in FIG. 3B.

[0115] In S601, the state is the same as in S401 in FIG.

[0116] In S602, the information processing device 200 performs the same process as in S402 of FIG.

[0117] In S603, the imaging device 100 performs the same process as in S403 in FIG.

[0118] In S604, the imaging apparatus 100 performs the same process as in S404 in FIG.

[0119] In S605, the imaging device 100 performs the same process as in S505 of FIG.

[0120] In S606, similarly to S405 in Fig. 4, the control unit 201 of the information processing device 200 transmits a reception completion notification of the first frame of video data to the imaging device 100. In this case, unlike the first video call end determination process in Fig. 4, the second timekeeping by the second timer 101b is still ongoing when the information processing device 200 transmits the reception completion notification of the video data to the imaging device 100.

[0121] In S607, the imaging apparatus 100 performs the same process as in S406 in FIG.

[0122] In S608, the information processing device 200 performs the same process as in S407 of FIG.

[0123] In S609, the information processing device 200 performs the same process as in S602 on the video data of the second frame.

[0124] In S610, the imaging device 100 resets the second timekeeping by the second timer 101b that the control unit 101 started in S605. Also, the imaging device 100 resets the first timekeeping by the first timer 101a that the control unit 101 started in S607.

[0125] In S611, the image capturing apparatus 100 performs the same process as in S604 on the video data of the second frame.

[0126] In S612, the image capturing apparatus 100 performs the same process as in S605 on the video data of the second frame.

[0127] In S613, the information processing device 200 performs the same process as in S606 on the video data of the second frame.

[0128] In S614, the image capturing apparatus 100 performs the same process as in S607 on the video data of the second frame.

[0129] In S615, the information processing device 200 performs the same process as in S608 on the video data of the second frame.

[0130] In this way, the same processes as those of S602 to S608 are repeatedly performed for the Nth (N is a natural number equal to or greater than 3) frame of video data from the third frame onwards. This enables the information processing device 200 to update the video on the screen 305 of the GUI 301 displayed on the display unit 206 every time it receives the video 310 of the user 308 from the imaging device 100.

[0131] The processing of S616 to S624 indicates processing in which the imaging device 100 determines whether to end the video call when the user operates the call button 304 of the GUI 301. The following describes an example in which the imaging device 100 determines whether to end the video call when the call button 304 is operated immediately after receiving a request to transmit video data from the information processing device 200.

[0132] In S616, the information processing device 200 performs the same process as in S602 on the Nth frame of video data.

[0133] In S617, the image capturing device 100 resets the second time counting by the second timer 101b, which was started in response to the control unit 101 transmitting the (N-1)th frame of video data. The image capturing device 100 also resets the first time counting by the first timer 101a, which was started in response to the control unit 101 receiving a reception completion notification of the (N-1)th frame of video data.

[0134] In S619, the image capturing apparatus 100 performs the same process as in S604 on the Nth frame of video data.

[0135] In S620, the image capturing apparatus 100 performs the same process as in S605 on the Nth frame of video data.

[0136] Here, if the information processing device 200 has the characteristic of sending a notification that reception of video data has been completed, in S621, the information processing device 200 sends a notification that reception of video data has been completed to the imaging device 100 for the Nth frame of video data, as in S606.

[0137] In S622, the image capturing apparatus 100 performs the same process as in S607 on the Nth frame of video data.

[0138] In S623, the control unit 101 of the imaging device 100 determines that the video call has ended in response to the elapse of the first time period since the first timer 101a started the first time measurement in S622. Then, the imaging device 100 displays second information (a standby icon) 312 on the display unit 106 as shown in FIG. 3D.

[0139] If the information processing device 200 does not have the characteristic of transmitting a video data reception completion notification, in S624, the control unit 101 of the imaging device 100 determines that the video call has ended in response to the elapse of the second time since the second timer 101b started the second clocking in S620. Then, the imaging device 100 displays second information (a standby icon) 312 on the display unit 106 as shown in FIG. 3D.

[0140] In the third video call end determination process described above, if the information processing device 200 has the property of transmitting a video data reception completion notification, the imaging device 100 can determine whether the video call has ended in a short time after the video call has ended. On the other hand, if the information processing device 200 does not have the property of transmitting a video data reception completion notification, the imaging device 100 can determine whether the video call has ended in a few seconds after the video call has ended.

[0141] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of each embodiment to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0142] The disclosure of this specification includes the following imaging device, control method, and program. [Configuration 1] a connection means for connecting to an external device; An imaging means for capturing an image; a control means for performing a process of transmitting the image captured by the imaging means to the external device in response to receiving a request to transmit the image from the external device connected by the connection means, The control means a first timer is started in response to the external device completing reception of the video, and a second timer is started in response to transmission of the video to the external device; an imaging device that cancels the state in which the processing is performed when a first time has elapsed since the start of the first timekeeping or when a second time has elapsed since the start of the second timekeeping. [Configuration 2] The imaging device described in configuration 1, characterized in that the control means resets the first clock and the second clock in response to receiving a request to send video from the external device before the first time has elapsed since starting the first clock or before the second time has elapsed since starting the second clock. [Configuration 3] 3. The imaging device according to configuration 1 or 2, wherein the first time period is shorter than the second time period. [Configuration 4] 4. The imaging device according to any one of configurations 1 to 3, wherein the control means has a first timing means for performing the first timekeeping and a second timing means for performing the second timekeeping. [Configuration 5] The imaging device described in any one of configurations 1 to 3, characterized in that the control means resets the first clock and the second clock in response to receiving a request to send video from the external device, and displays first information on the display means indicating that the state in which the processing is being performed is continuing. [Configuration 6] 6. The imaging device according to configuration 5, wherein the first information is an icon indicating that a live broadcast is being performed. [Configuration 7] The imaging device described in any one of configurations 1 to 6, characterized in that the control means displays second information on the display means indicating that the state of performing the processing has been canceled when the first time has elapsed since the start of the first timing or when the second time has elapsed since the start of the second timing. [Configuration 8] 8. The imaging device according to configuration 7, wherein the second information is an icon indicating that the device is in standby mode. [Configuration 9] 9. The imaging device according to any one of configurations 1 to 8, wherein the image is an image of a user operating the external device captured by the imaging means. [Configuration 10] 10. The imaging device according to any one of configurations 1 to 9, wherein the control means converts the image captured by the imaging means into an image suitable for display by the external device and transmits the converted image to the external device. [Configuration 11] the external device has a display means for displaying a GUI provided by an application running on the external device, the application enabling a user to make a video call while viewing a video of another user received from another user device connected to the external device; the GUI includes an operation means for starting or ending a video call with the other user; The imaging device described in any one of configurations 1 to 10, characterized in that the external device stops requesting the imaging device to transmit the video when an operation to end the video call with the other user is performed using the operating means. [Configuration 12] The imaging device described in configuration 11, wherein the GUI further includes a screen for displaying an image of the other user, an operation means for transmitting an image of the user operating the external device to the other user device, and an operation means for transmitting a voice of the user operating the external device to the other user device. [Configuration 13] a connection means for connecting to an external device; A control method for an imaging device having an imaging means for capturing an image, comprising: a control step of transmitting the image captured by the imaging means to the external device in response to receiving a request to transmit the image from the external device connected by the connection means; In the control step, a first timer is started in response to the external device completing reception of the video, and a second timer is started in response to transmission of the video to the external device; A control method characterized by canceling the state in which the processing is performed when a first time has elapsed since the start of the first timekeeping or when a second time has elapsed since the start of the second timekeeping. [Configuration 14] A program for causing a computer to function as a control means for the imaging device according to any one of configurations 1 to 12.

[0143] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0144] 100...imaging device, 101...control unit, 102...imaging unit, 106...display unit, 109...connection unit, 200...information processing device, 201...control unit, 205...operation unit, 206...display unit, 209...connection unit

Claims

1. a connection means for connecting to an external device; An imaging means for capturing an image; a control means for performing a process of transmitting the image captured by the imaging means to the external device in response to receiving a request to transmit the image from the external device connected by the connection means, The control means a first timer is started in response to the external device completing reception of the video, and a second timer is started in response to transmission of the video to the external device; an imaging device that cancels the state in which the processing is performed when a first time has elapsed since the start of the first timekeeping or when a second time has elapsed since the start of the second timekeeping.

2. The imaging device according to claim 1, characterized in that the control means resets the first timer and the second timer in response to receiving a request to send video from the external device before the first time has elapsed since the start of the first timer or before the second time has elapsed since the start of the second timer.

3. The imaging device according to claim 1 , wherein the first time period is shorter than the second time period.

4. 2. The imaging device according to claim 1, wherein the control means comprises a first clocking means for performing the first time measurement and a second clocking means for performing the second time measurement.

5. The imaging device according to claim 1, characterized in that the control means resets the first timer and the second timer in response to receiving a request to transmit video from the external device, and displays first information on the display means indicating that the state in which the processing is being performed is continuing.

6. 6. The imaging device according to claim 5, wherein the first information is an icon indicating that a live broadcast is being performed.

7. The imaging device according to claim 1, characterized in that the control means displays second information on the display means indicating that the state in which the processing is to be performed has been canceled when the first time has elapsed since the start of the first timekeeping or when the second time has elapsed since the start of the second timekeeping.

8. 8. The imaging device according to claim 7, wherein the second information is an icon indicating that the device is in standby mode.

9. 2. The imaging device according to claim 1, wherein the image is an image of a user operating the external device captured by the imaging means.

10. 2. The imaging device according to claim 1, wherein the control means converts the image captured by the imaging means into an image suitable for display by the external device and transmits the converted image to the external device.

11. the external device has a display means for displaying a GUI provided by an application running on the external device, the application enabling a user to make a video call while viewing a video of another user received from another user equipment connected to the external device; the GUI includes an operation means for starting or ending a video call with the other user; 2. The imaging device according to claim 1, wherein the external device stops requesting the imaging device to transmit the video when an operation to end the video call with the other user is performed by the operating means.

12. The imaging device of claim 11, characterized in that the GUI further includes a screen for displaying an image of the other user, an operation means for transmitting an image of the user operating the external device to the other user equipment, and an operation means for transmitting a voice of the user operating the external device to the other user equipment.

13. a connection means for connecting to an external device; A control method for an imaging device having an imaging means for capturing an image, comprising: a control step of transmitting the image captured by the imaging means to the external device in response to receiving a request to transmit the image from the external device connected by the connection means; In the control step, a first timer is started in response to the external device completing reception of the video, and a second timer is started in response to transmission of the video to the external device; A control method characterized by canceling the state in which the processing is performed when a first time has elapsed since the start of the first timekeeping or when a second time has elapsed since the start of the second timekeeping.

14. A program for causing a computer to function as a control unit of the imaging device according to any one of claims 1 to 12.

Images