Imaging apparatus, control method of the same, and program

Abstract

An imaging device capable of voice communication with a client device in an established session with the client device establishes a session with the client device, receives a request from a user to establish the session, and when the request to establish the session is received, transmits a notification in accordance with the Open Network Video Interface Forum (ONVIF) standard to the client device as a notification requesting the establishment of the session.

Description

Technology Field

[0001] The present invention relates to an imaging device, a control method thereof, and a program. Background Technology

[0002] Conventionally, it is common for a camera and a video receiving client device connected via a network to use the function of bidirectional voice communication implemented using voice distribution and voice transmission from the client device to the camera (back channel).

[0003] The Open Network Video Interface Forum (ONVIF) standard is a common standard for the connection between surveillance cameras and client devices for receiving video, and it standardizes functions regarding video and audio distribution and back channels.

[0004] Devices compliant with the ONVIF specification use the Real Time Streaming Protocol (RTSP) to establish a session for two-way voice communication. Two-way voice communication using RTSP requires the client device to send an RTSP request to establish a session (ONVIF Streaming Specification (https: / / www.onvulif.org / specs / stream / ONVIF-Streaming-Spec.pdf)).

[0005] For example, in the case of surveillance cameras, it is common for a user of a client device monitoring a subject to speak to the subject using a back channel. The back channel can also be used for voice calls in combination with voice distribution from the camera to the client device. In this case, there may be instances where a request to establish a session is desired from the camera user, in addition to the user of the client device mentioned above. An example of a configuration including voice calls is the use of a camera that is wearable by the user and supports two-way voice communication with the user of the client device.

[0006] However, standard RTSP does not support the initiation of session establishment based on a request from the camera. Furthermore, if the session is disconnected by the client device, the session cannot be re-established through operations performed on the camera. Therefore, it is necessary to transmit a request for a voice call from the camera to the client device.

[0007] However, the ONVIF standard does not define a function to generate a notification requesting the establishment of a session. Therefore, devices compliant with the ONVIF standard cannot establish a session between an imaging device and a client device based on a request from the camera. means of solving the problem

[0008] According to one aspect of the present disclosure, an imaging device capable of voice communication with a client device in an established session with the client device comprises: an establishing unit configured to establish a session with the client device; a receiving unit configured to receive a request for the establishment of the session from a user; and a transmitting unit configured to transmit a notification in accordance with the Open Network Video Interface Forum (ONVIF) standard to the client device as a notification requesting the establishment of the session when the receiving unit receives the request for the establishment of the session.

[0009] According to another aspect of the present disclosure, a method for controlling an imaging device comprises the steps of establishing a session with a client device, receiving a request from a user to establish the session, and, when the request to establish the session is received, transmitting a notification in accordance with the ONVIF standard to the client device as a notification requesting the establishment of the session.

[0010] Another feature of the present disclosure will become apparent from the description of the following embodiments with reference to the accompanying drawings. Brief explanation of the drawing

[0011] FIG. 1 is a diagram showing a network configuration according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing the internal configuration of a camera according to an embodiment of the present disclosure. FIG. 3 is a block diagram showing the internal configuration of a client device according to an embodiment of the present disclosure. FIG. 4 is a sequence diagram showing a method for establishing an image and voice transmission and reception session according to an embodiment of the present disclosure. FIG. 5 is a flowchart showing the operation during a voice call input response according to an embodiment of the present disclosure. Specific details for implementing the invention

[0012] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0013] Hereinafter, an imaging device according to an embodiment of the present disclosure will be described with reference to FIGS. 1 to 5.

[0014] FIG. 1 is a network configuration diagram including a camera (1000). A client device (2000) is an external device in the present disclosure. The camera (1000) is an imaging device that can be worn by a user, such as a necklace-type camera that can be worn around the user's neck. The camera (1000) and the client device (2000) are connected via a network (3000) and communicate with each other. The client device (2000) transmits various control commands, for example, for image settings, to the camera (1000). The camera (1000) transmits a response to the command to the client device (2000). When a session (Real Time Streaming Protocol (RTSP) session) is established between the camera (1000) and the client device (2000), the camera (1000) (imaging device) can have a voice call with the client device (2000).

[0015] FIG. 2 is a block diagram showing the internal configuration of a camera (1000).

[0016] In FIG. 2, the control unit (1001) controls the operation of the camera (1000) (information processing device). The control unit (1001) is, for example, a central processing unit (CPU).

[0017] The memory unit (1002) is mainly used as a storage area for various data, such as a program storage area for a program executed by the control unit (1001), a work area during program execution, and a temporary storage area for image data generated by the imaging unit (1003) described later.

[0018] The imaging unit (1003) captures an image of a subject formed by the imaging system (1004) described later of the camera (1000), converts the acquired analog signal into digital data, and outputs the digital signal as an image to the memory unit (1002).

[0019] The imaging system (1004) is composed of an imaging optical system consisting of a lens, an imaging element, etc., and a pan-tilt-zoom mechanism that controls the imaging direction and angle of view.

[0020] The voice input unit (input unit) (1005) is composed of a voice input device such as a microphone or a terminal capable of connecting a voice input device, and acquires voice. The acquired voice is encoded and distributed to a client device (2000) by a communication unit (1007) described later.

[0021] The voice output unit (1006) is composed of a voice output device such as a speaker or a terminal capable of connecting a voice output device, and performs voice output. The voice output unit (1006) decodes encoded voice received from a client device (2000) by a communication unit (1007) described later, and outputs the decoded voice.

[0022] The communication unit (1007) is used to receive each setting value change and control command from the client device (2000) via the network (3000), transmit a response to each command, image data stored in the memory unit (1002), and voice acquired by the voice input unit (1005) to the client device (2000), and transmit various types of data, such as notification events generated by the event generation unit (1009) described later, to the client device (2000). The communication unit (1007) is also used when receiving voice from the client device (2000).

[0023] The voice call request input unit (1008) is composed of a button or a switch. This button or switch receives a request for the establishment of a session. In response to the reception of input through the voice call request input unit (1008), that is, in response to the reception of a request for the establishment of a session for a switch or button operated by a user of the camera (1000), the event generation unit (1009), described later, generates an event.

[0024] The event generation unit (1009) generates an event defined in the Open Network Video Interface Forum (ONVIF) standard based on input through the voice call request input unit (1008). Then, the event generation unit (1009) transmits the generated event to the client device (2000) via the communication unit (1007).

[0025] Although the internal configuration of the camera (1000) has been described above with reference to FIG. 2, the processing block shown in FIG. 2 is an example of a preferred embodiment of the information processing device in the present disclosure, and the present disclosure is not limited to this example. The present disclosure can be modified and changed in various ways within the scope of the gist of the present disclosure, such as by not installing a voice call request input unit (1008) and using voice input or captured image as a trigger for event generation.

[0026] FIG. 3 is a block diagram showing the internal configuration of a client device (2000).

[0027] The control unit (2001) is composed of, for example, a CPU and controls the operation of the client device (2000).

[0028] The memory unit (2002) is used as a storage area for various types of data, such as a storage area for a program executed by the control unit (2001), a work area during program execution, and information on a connectable camera currently existing on the network (3000).

[0029] The display unit (2003) is composed of, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display, and displays various setting screens, data acquisition and display screens, a viewer of images received from the camera (1000), events received from the camera (1000), and various messages.

[0030] The input unit (2004) is composed of, for example, a button, a four-way key, a touch panel, a mouse, etc., and notifies the control unit (2001) of the content of the screen operation performed by the user.

[0031] The voice input unit (2005) is composed of a voice input device such as a microphone or a terminal capable of connecting a voice input device, and acquires voice. The acquired voice is encoded and distributed to the camera (1000) by the communication unit (2007) described later.

[0032] The voice output unit (2006) is composed of a voice output device such as a speaker or a terminal capable of connecting a voice output device, and performs voice output. The voice output unit (2006) decodes the encoded voice from the camera (1000) received by the communication unit (2007) described later, and outputs the decoded voice.

[0033] The communication unit (2007) is used to transmit various change commands, including changing the imaging area, to the camera (1000) via the network (3000), receive a response to the change command, and receive a video stream from the surveillance camera (1000).

[0034] Although the internal configuration of the client device (2000) has been described above with reference to FIG. 3, the processing block shown in FIG. 3 is an example of a preferred embodiment of the information processing device in the present disclosure, and the present disclosure is not limited to this example. The client device (200) can be modified and changed in various ways within the scope of the gist of the present disclosure, such as by further installing an image analysis processing unit and an image accumulation unit.

[0035] Next, with reference to the sequence diagram of FIG. 4, a method for establishing a video and audio transmission and reception session using ONVIF commands according to the present embodiment will be described.

[0036] The processing in steps S1001, S1002, each Response, and S1008 of FIG. 4 is realized by being executed by the control unit (1001) of the camera (1000).

[0037] The processing of steps S1003, S1004, S1005, and S1007 of FIG. 4 is realized by being executed by the control unit (2001) of the client device (2000).

[0038] The image distribution of step S1006 of FIG. 4 is realized by being executed by the control unit (1001) of the camera (1000), and the bidirectional voice communication of step S1006 is realized by being executed by the control unit (1001) of the camera (1000) and the control unit (2001) of the client device (2000).

[0039] In step S1001, the control unit (1001) receives input of a voice call request from a user. In step S1002, the control unit (1001) transmits an event based on the input of the voice call request. The processing of step S1001 is an example of application of the receiving unit of the present disclosure.

[0040] The detailed processing of steps S1001 and S1002 is described with reference to the flowchart shown in Fig. 5.

[0041] In step S1011 (S1001), the camera (1000) receives a request from a user of the camera (1000) to establish a session between the camera (1000) and the client device (2000). Specifically, a user of the camera (1000) who wishes to initiate a voice call with a user of the client device (2000) presses a button provided in the voice call request input section (1008). In response to the button provided in the voice call request input section (1008) being pressed by a user of the camera (1000), the camera (1000) receives a voice call request (a request to establish a session). In this embodiment, as an example, the voice call request (a request to establish a session) is received in response to a pressing operation performed on the button. Alternatively, the request to establish a session may be received in response to a predetermined voice input. In response to the detection of a specific motion in a video (video) captured by the camera (1000), a request for the establishment of a session may be received.

[0042] In response to the camera (1000) receiving a voice call request (a request to establish a session) from the user of the camera (1000), processing proceeds to step S1012. In step S1012, the camera (1000) generates an event.

[0043] In step S1012 (S1002), a digital input event defined in the ONVIF standard is generated.

[0044] The camera (1000) generates an event indicating that the digital input state has changed in XML format as defined in the ONVIF standard.

[0045] A digital input event is generally used to notify a client device of a detection result (information digitally input from an external device) detected by a sensor (external device) connected to the external contact input terminal of a camera. However, in this embodiment, a digital input event is generated in response to the receipt of a voice call request (a request for session establishment).

[0046] As described above, in step S1012, the camera (1000) generates a notification (digital input event) in accordance with the ONVIF standard as a notification to request the establishment of a session.

[0047] In step S1013, the digital input event generated in step S1012 is transmitted to the client device (2000) as a notification to request the establishment of a session.

[0048] When this event is transmitted from the camera (1000) to the client device (2000), the client device (2000) needs to be in an initial configuration state to establish an ONVIF connection between the camera (1000) and the client device (2000), or in a state where the configuration of the client device (2000) is changed so that the client device (2000) is ready to receive the corresponding event from the camera (1000).

[0049] The camera (1000) determines whether the client device (2000) is ready to receive the event. If the client device (2000) is ready to receive the event, in step S1013 (S1002), the camera (1000) transmits a digital input event to the client device (2000) in step S1002. The processing of step S1013 (S1002) is an example of application of the transmission unit of the present disclosure.

[0050] Steps S1003, S1004, and S1005 are a sequence for establishing a session for video distribution and two-way voice communication. The processing of steps S1003, S1004, and S1005 is an example of application of the establishment part of the present disclosure.

[0051] In step S1003, the DESCRIBE method of the RTSP request is transmitted.

[0052] The camera (1000) receives information (DESCRIBE method) to be used to establish a session, which is transmitted by the client device (2000) in response to the reception of a digital input event (notification) transmitted in step S1002. Then, the camera (1000) establishes a session with the client device (2000) by executing a sequence for establishing a session described below.

[0053] In response to the reception of a digital input event in step S1002, the client device (2000) outputs a display or voice indicating the reception of the event and transmits a DESCRIBE method to the camera (1000) by means of an operation performed by the user of the client device (2000). Alternatively, the client device (2000) automatically transmits a DESCRIBE method in response to the reception of a digital input event.

[0054] The DESCRIBE method retrieves information regarding video and audio distributed by the camera (1000).

[0055] Next, the camera (1000) that received the DESCRIBE method describes information indicating that both voice reception and transmission are supported in a session description in the format of Session Description Protocol (SDP), and transmits this session description to the client device (2000) as a response.

[0056] In step S1004, the SETUP method of the RTSP request is transmitted.

[0057] The SETUP method requires the establishment of an RTSP session (hereinafter simply referred to as a session) to perform distribution.

[0058] The client device (2000) transmits a Uniform Resource Identifier (URI) containing information of a stream for which a session is to be established to the camera (1000) based on the information obtained in step S1003.

[0059] In this process, at step S1004, the client device (2000) transmits a SETUP method to the camera (1000) that enables both voice transmission and voice reception.

[0060] In response to the reception of the SETUP method, the camera (1000) establishes a session for video distribution and two-way voice communication and transmits session information to the client device (2000) as a response.

[0061] In step S1005, the PLAY method of the RTSP request is transmitted.

[0062] The PLAY method requests the camera (1000) to start video distribution and two-way voice communication.

[0063] In response to the reception of the PLAY method, the camera (1000) initiates video distribution and two-way voice communication, and transmits information regarding the success or failure of the distribution initiation to the client device (2000) as a response.

[0064] If the distribution start is successful, in step S1006, the image is distributed by the camera (1000), and voice is transmitted and received between the camera (1000) and the client device (2000).

[0065] In this process, the camera (1000) can notify the user of the camera (1000) that a session has been established and that video distribution and audio transmission and reception are possible. Examples of means for notifying the user include playing a specific audio through an audio output unit. This process is an example of application of the notification unit of the present disclosure and is performed to notify the user of the camera (1000) (imaging device) (1000) that a session has been established when a session with the client device (2000) is established.

[0066] In step S1007, the TEARDOWN method of the RTSP request is transmitted. The TEARDOWN method is a request to stop the transmission and reception of video and audio.

[0067] The user of the client device (2000) specifies a session to be stopped, and the client device (2000) sends a TEARDOWN method for this specified session.

[0068] In response to the reception of the TEARDOWN method, the camera (1000) performs the stopping of video and audio transmission and reception and the session disconnection according to the request, and transmits information on the success or failure of the processing to the client device (2000) as a response.

[0069] In step S1008, the user wearing the camera (1000) is notified that the session has been severed. The processing of step S1008 is an example of the application of the notification unit of the present disclosure and is performed to notify the user of the camera (1000) (imaging device) that the session has been severed when the session with the client device (2000) is severed.

[0070] Examples of means of notifying the user include outputting a voice indicating session termination from a voice output unit (1006) or illuminating a separately installed LED to indicate session termination.

[0071] Conventionally, the disconnection of the session was not notified to the user wearing the camera (1000). However, through the processing of step S1008, the user recognizes that the two-way voice communication has been disconnected and can request the establishment of a session by inputting a request for a voice call again.

[0072] For the time being, with reference to FIGS. 4 and 5, a method for establishing a video and audio transmission and reception session using ONVIF commands according to the present embodiment has been described. The steps described in FIGS. 4 and 5 are examples of preferred embodiments of internal processing in the present disclosure, and the present disclosure is not limited to these examples.

[0073] The processing of the voice call request input of step S1001 can be modified so that the camera (1000) generates a voice detection event in step D1012 in response to the receipt of a specific string of characters from the user via the voice input unit (1005), or the camera (1000) interprets the image generated by the imaging unit (1003) to detect the start of operation from a stationary state and generates an image change detection event.

[0074] In this embodiment, the camera (1000) responds to the receipt of a request for session establishment and transmits a digital input event defined in the ONVIF standard to the client device (200) as a notification for requesting session establishment. Instead of a digital input event, the camera (1000) may detect at least one of a predetermined change in the voice input by the voice input unit (1005) and the image captured by the voice input unit (1005) (by a detection unit), and when the predetermined change is detected, a notification may be made in response to an event in which the camera (1000) notifies the client device (2000).

[0075] Instead of using the TEARDOWN method of step S1007 as a trigger, the transmission and reception of video and audio can be stopped in response to a request to stop distribution input by the camera.

[0076] The present disclosure may be modified and altered in various ways within the scope of the gist of the present disclosure, such as not having the user wear the camera.

[0077] As described above, in this embodiment, a session between an imaging device (camera) and a client device can be established based on a request from the imaging device by using a notification in accordance with the ONVIF standard.

[0078] In addition, it becomes possible to notify the client device of a request to re-establish a session by utilizing ONVIF functions.

[0079] Preferred embodiments of the present disclosure have been described above. However, the present disclosure is not limited to these embodiments and various modifications and changes are possible within the scope of the gist thereof.

[0080] (Other embodiments)

[0081] Embodiments of the present invention may be implemented by a method performed by a computer of a system or device comprising one or more circuits (e.g., application-specific integrated circuits (ASICs)) that perform one or more functions of the aforementioned embodiment(s) of the present invention, by reading and executing computer-executable instructions (e.g., one or more programs) recorded in a storage medium (which may be referred to more specifically as a 'non-transient computer-readable storage medium') to perform one or more functions of the aforementioned embodiment(s), or by reading and executing computer-executable instructions from a storage medium to perform one or more functions of the aforementioned embodiment(s). The computer may have one or more central processing units (CPUs), microprocessors (MPUs), or other circuits, and may have a network of separate computers or separate computer processors. Computer-executable instructions may be given to the computer, for example, from a network of storage media. The storage medium may be, for example, one or more hard disks, random access memory (RAM), read-only memory (ROM), or a distributed computing system. Storage, optical disc (Compact Disc (CD), Digital Multifunction Disc (DVD), or Blu-ray Disc (BD) TM You may also provide flash memory devices, memory cards, etc.), etc.

[0082] The present invention can be executed in a process in which a program realizing one or more functions of the above-described embodiment is supplied to a system or device via a network or storage medium, and one or more processors in a computer of the system or device read and execute the program. In addition, it can also be executed by a circuit (e.g., an ASIC) realizing one or more functions.

[0083] Although the present invention has been described with reference to exemplary embodiments, it is obvious that the invention is not limited to these embodiments. The scope of protection of the following claims shall be interpreted as broadly as possible to encompass all variations, equivalent structures, and functions.

Claims

Claim 1 An imaging device capable of voice communication with a client device in an established session with said client device, comprising: an establishing unit configured to establish a session with said client device by initializing a request for the establishment of said session from a user wearing said imaging device; a receiving unit configured to receive a request for the establishment of said session from the user; a generating unit configured to generate a notification compliant with said ONVIF standard as a digital input event defined in said ONVIF standard at said imaging device upon receipt of said request; a determining unit configured to determine whether said client device is ready to receive said digital input event—said determination is performed at said imaging device—and a transmitting unit configured to transmit said digital input event to said client device as a notification requesting the establishment of said session when said receiving unit receives a request for the establishment of said session; wherein the establishing unit establishes a session with said client device by receiving information transmitted by said client device and used for the establishment of said session upon receipt of said notification transmitted by said transmitting unit, and the information used for the establishment of said session of the Real Time Streaming Protocol (RTSP) Required imaging device. Claim 2 delete Claim 3 In claim 1, the digital input event is an imaging device that notifies the client device of information digitally input from an external device. Claim 4 delete Claim 5 In claim 1, the information used to establish the session is an imaging device including information of the DESCRIBE method. Claim 6 An imaging device according to claim 1, further comprising an imaging unit, an input unit configured to input voice, and a detection unit configured to detect at least one predetermined change among the voice input by the input unit and the image captured by the imaging unit, wherein the notification conforming to the ONVIF standard is an event in which a notification is made to the client device when the detection unit detects the predetermined change. Claim 7 In claim 1, the receiving unit is equipped with a switch or button for receiving a request to establish the session from the user, and the receiving unit is an imaging device that receives a request to establish the session according to an operation performed on the switch or button by the user. Claim 8 In claim 1, the receiving unit is an imaging device that receives a request for the establishment of the session according to a predetermined voice input. Claim 9 In claim 1, the receiving unit is an imaging device that receives a request for the establishment of the session upon detection of a predetermined operation. Claim 10 An imaging device according to claim 1, further comprising a notification unit configured to notify the user of the imaging device of the establishment of the session when the session with the client device is established. Claim 11 In claim 10, when the session with the client device is severed, the notification unit is an imaging device that notifies the user of the imaging device of the severance of the session. Claim 12 In claim 1, the imaging device is an imaging device that is a necklace-type camera wearable by a user. Claim 13 A control method for an imaging device, comprising: a step of establishing a session with a client device by initializing a request for session establishment from a user wearing the imaging device; a step of receiving the request for session establishment from the user; a step of generating a notification compliant with the Open Network Video Interface Forum (ONVIF) standard as a digital input event defined in the ONVIF standard at the imaging device upon receiving the request; a step of determining whether the client device is ready to receive the digital input event - the determination is made at the imaging device - and, when the request for session establishment is received, a step of transmitting the digital input event to the client device as a notification requesting the session establishment, wherein the establishing step establishes a session with the client device by receiving information transmitted by the client device and used for session establishment upon receiving the notification transmitted in the transmitting step, and the control method in which information used for session establishment of the Real Time Streaming Protocol (RTSP) is required. Claim 14 A program stored in a computer-readable storage medium for executing a control method of an imaging device, comprising: a step of establishing a session with a client device by initializing a request for session establishment from a user wearing the imaging device; a step of receiving the request for session establishment from the user; a step of generating a notification compliant with the Open Network Video Interface Forum (ONVIF) standard as a digital input event defined in the ONVIF standard at the imaging device upon receiving the request; a step of determining whether the client device is ready to receive the digital input event - said determination is made at the imaging device - and, when the request for session establishment is received, a step of transmitting the digital input event to the client device as a notification requesting the session establishment, wherein the establishing step establishes a session with the client device by receiving information transmitted by the client device and used for session establishment upon receiving the notification transmitted in the transmitting step, and information used for session establishment of the Real Time Streaming Protocol (RTSP) is required.

Images