Control unit, imaging apparatus, control method, and program
The control device manages imaging device status to extend operational time and prevent interruptions by switching devices based on battery, heat, and storage capacity, ensuring continuous media data distribution.
Patent Information
- Application Number
- JP2023078697
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-11-21
AI Technical Summary
Existing systems fail to adequately manage the status of imaging devices, such as battery power, heat generation, and storage capacity, leading to potential blackouts and interruptions in media data distribution.
A control device that communicates with multiple imaging devices, receives status information, and generates control information to switch devices for optimal operation, extending their operational time and ensuring continuous media data distribution.
The system effectively extends the operational time of imaging devices by appropriately switching between them, minimizing interruptions in media data transmission.
Smart Images

Figure 2024162791000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a system for transmitting media data received from a plurality of imaging devices to a distribution device. [Background technology]
[0002] In a system that distributes images from multiple different viewpoints captured by multiple imaging devices, multi-angle distribution is known in which images from multiple viewpoints are distributed by switching between the imaging devices that capture the images to be distributed from among the multiple imaging devices.
[0003] Switching of imaging devices that capture images for distribution is performed by a dedicated operator or automatically in a preset order, but if some error occurs in an imaging device and images cannot be supplied, a blacked-out image may be distributed. In view of this background, Patent Document 1 describes a method for switching an imaging device that records images to another imaging device when the remaining battery power of an imaging device that records images becomes low in a system in which multiple imaging devices transmit and receive images via a network. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2019-9651 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, Patent Document 1 does not mention monitoring the status of the imaging device, such as remaining battery power, heat generation, and remaining storage capacity, or controlling the usage time, etc., of each imaging device so that the imaging device does not reach a state in which it is unable to transmit images.
[0006] The present invention has been made in consideration of the above-mentioned problems, and its object is to realize a technology that appropriately switches between multiple imaging devices to extend the operating time of the imaging devices and reduce the situation in which media data for distribution is not transmitted from the imaging devices. [Means for solving the problem]
[0007] In order to solve the above problems and achieve the objective, the control device of the present invention has a communication means for communicating with a plurality of imaging devices, and a control means for receiving media data from the plurality of imaging devices via the communication means, and the control means generates control information for switching a specific imaging device among the plurality of imaging devices to be used for distributing the media data based on status information received from the plurality of imaging devices, and transmits the control information to the plurality of imaging devices. Effect of the Invention
[0008] According to the present invention, it is possible to appropriately switch between a plurality of imaging devices to extend the operable time of the imaging devices, thereby reducing the number of situations in which media data for distribution is not transmitted from the imaging devices. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a system configuration diagram of the present embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating the configuration of a control device according to the embodiment. [Diagram 3] FIG. 1 is a block diagram illustrating the configuration of an imaging apparatus according to an embodiment of the present invention. [Figure 4] 5 is a diagram illustrating a processing sequence of media data by the control device of the embodiment; [Diagram 5] 5A and 5B are diagrams showing examples of how media data is displayed on the control device of the embodiment. [Figure 6] 4 is a flowchart showing a control process of the control device of the embodiment. [Figure 7] 6 is a diagram illustrating an example of a state information determination threshold value of the imaging device in the control process of the embodiment. [Figure 8]5A to 5C are diagrams for explaining a method for determining the severity of status information of the imaging device in the control process of the embodiment. [Figure 9] 5A to 5C are diagrams illustrating examples of switching patterns of the imaging apparatus according to the embodiment. [Figure 10] 2A to 2C are diagrams illustrating operation states and behaviors of the imaging apparatus according to the present embodiment. [Figure 11] 4A and 4B are diagrams illustrating state transitions of the imaging apparatus according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.
[0011] <System configuration> The system of this embodiment will be described with reference to FIG.
[0012] The system of this embodiment includes a distribution device 10, a control device 20, and multiple imaging devices 30. The multiple imaging devices 30 include a first imaging device 30A, a second imaging device 30B, and a third imaging device 30C, but in this embodiment, the multiple imaging devices 30A, 30B, and 30C may be collectively referred to as imaging devices 30. The multiple imaging devices 30A, 30B, and 30C capture images from different viewpoints for each imaging device, collect sounds around the imaging devices, generate media data in a predetermined format (for example, RTMP) including at least one of images and sounds, and transmit the media data to the control device 20. The control device 20 controls the shooting operation and the sound recording operation for each imaging device for the multiple imaging devices 30A, 30B, and 30C. In addition, the control device 20 receives media data in a predetermined format (for example, RTMP) including images captured by each imaging device and / or sounds collected by each imaging device from the imaging devices. Then, the control device 20 applies a predetermined data processing to the media data received from the imaging device to generate data for playback in the control device 20 or data for distribution by the distribution device 10. The distribution device 10 receives the media data for distribution from the control device 20 and distributes it to an external device such as a user device via an IP network such as the Internet or a LAN (Local Area Network).
[0013] The multiple imaging devices 30A, 30B, and 30C and the control device 20 are connected to each other so as to be able to communicate with each other through a network, using a wireless connection method such as wireless LAN or a wired connection method such as wired LAN. In the case of a wireless connection method, the multiple imaging devices 30A, 30B, and 30C and the control device 20 are directly connected by P2P or connected via an IP network. The media data transmitted from the imaging devices 30A, 30B, and 30C to the control device 20 or the media data transmitted from the control device 20 to the distribution device 10 includes at least one of an image and an audio. The images include moving images or still images, and the audio may include audio analysis data, subtitle data, and the like.
[0014] The imaging device 30 is a digital camera capable of taking images and collecting audio, but is not limited to a digital camera and may be a personal computer (notebook PC or tablet PC), a smartphone, a web camera such as a surveillance camera, a medical camera, or the like.
[0015] The control device 20 is a personal computer (notebook PC or tablet PC), a smartphone, etc. The control device 20 generates media data for playback without applying a data conversion process described later in FIG. 4 to the media data received from the imaging device 30, or generates media data in a data format suitable for distribution by the distribution device 10 by applying a data conversion process, and transmits the media data to the distribution device 10. The control device 20 also has a function of determining from which of the multiple imaging devices 30A, 30B, and 30C the media data received is to be transmitted to the distribution device 10.
[0016] The distribution device 10 is a server computer such as a service site that simultaneously distributes (for example, streaming) media data to multiple user terminals. The service site provides services for various purposes and uses, such as entertainment, promotion, and training.
[0017] <Configuration of the control device 20> Next, the configuration and functions of the control device 20 of this embodiment will be described with reference to FIG.
[0018] FIG. 2 is a block diagram illustrating the configuration of the control device 20 of the present embodiment.
[0019] The control unit 201 includes an arithmetic processor such as a CPU or MPU that controls the entire control device 20, and realizes the control processing of the flowchart described below by executing programs stored in the nonvolatile memory 202. The work memory 203 is a RAM or the like, and is also used as a work memory for expanding constants and variables for the operation of the control unit 201, programs read from the nonvolatile memory 202, and the like. The control unit 201 also controls the display of the display unit 206 described below.
[0020] The non-volatile memory 202 stores an OS (operating system), which is basic software executed by the control unit 201, and applications that cooperate with the OS to realize applied functions. In this embodiment, the non-volatile memory 202 also stores a camera application for communicating with the imaging device 30 and a distribution application for communicating with the distribution device 10.
[0021] The processing of the control device 20 in this embodiment is realized by reading software provided by an application. It is assumed that the application has software for utilizing basic functions of the OS installed in the control device 20. It is also possible for the OS of the control device 20 to have software for realizing the control processing in this embodiment. The application has a function of remotely controlling the imaging device 30 to receive media data, and a function of transmitting the media data received from the imaging device 30 to the distribution device 10.
[0022] The control unit 201 includes a control information generating unit 201a and a data processing unit 201b. The control information generating unit 201a generates a switching pattern for switching an imaging device (hereinafter, a distribution source imaging device) used by the distribution device 10 for distributing media data from among the multiple imaging devices 30A, 30B, and 30C, based on status information received from the imaging devices 30A, 30B, and 30c by the communication unit 207. The data processing unit 201b applies a data conversion process, which will be described later with reference to FIG. 4, to the media data received from the imaging device 30 to generate media data in a data format suitable for distribution by the distribution device 10.
[0023] A timer 204 is a timing unit that measures the time used for various controls and the time of a built-in clock.
[0024] The operation unit 205 is made up of operation members such as various switches and buttons that receive various operations from a user and notify the control unit 201. The operation unit 205 also includes a touch input function as a touch screen that is integrally configured with the display unit 206 described below. The operation unit 205 includes a power switch that turns the power of the control device 20 on or off and starts or shuts down the system.
[0025] The display unit 206 displays images, characters for interactive operations, etc. The display unit 206 is, for example, a display device such as a liquid crystal display or an organic EL display. The display unit 206 may be integrated with the control device 20, or may be an external device connected to the control device 20.
[0026] The communication unit 207 is communicatively connected to external devices such as the distribution device 10 and the imaging device 30 via a wireless antenna or a wired cable, and transmits and receives various information and data. The various information and data are, for example, control information to be transmitted to the imaging device 30, status information and media data received from the imaging device 30, and media data to be transmitted to the distribution device 10. The communication unit 207 includes an interface for communicating with the external device via wireless LAN. The control device 20 may be directly connected to the imaging device 30 via the communication unit 207, or may be connected via wireless LAN. The communication unit 207 is not limited to wireless LAN, and may use a wireless communication module such as infrared communication, Bluetooth (registered trademark), or Wireless USB, or a wired connection interface such as a USB cable, HDMI (registered trademark), or IEEE1394. The communication unit 207 may also be an interface for wireless communication via a wide area network (WAN) such as 4G / LTE or 5G.
[0027] The power supply control unit 208 includes a power supply unit, a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to be energized, and controls the DC-DC converter based on instructions from the control unit 201 to supply the required voltage for the required period to each unit of the control device 20. The power supply unit includes primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and Li-ion batteries, and an AC adapter.
[0028] <Configuration of imaging device 30> Next, the configuration and functions of the imaging device 30 of this embodiment will be described with reference to FIG.
[0029] The control unit 301 includes an arithmetic processor such as a CPU or MPU that controls the entire imaging device 30, and executes a program stored in the non-volatile memory 302 to realize communication processing and control processing with the control device 20. Note that instead of the control unit 301 controlling the entire device, the entire device may be controlled by multiple hardware devices sharing the processing.
[0030] The imaging unit 304 includes a group of lenses including a zoom lens and a focus lens, and a shutter with an aperture function. The imaging unit 304 also includes an image sensor configured with a CCD or CMOS that converts a subject image into an electric signal, and an A / D converter that converts an analog image signal output from the image sensor into a digital signal. Under the control of the control unit 301, the imaging unit 304 converts the subject image light formed by the lens included in the imaging unit 304 into an electric signal using the image sensor, performs noise reduction processing, and outputs image data consisting of a digital signal.
[0031] The audio input unit 308 is one or more microphones built into the imaging device 30 or connected via an audio terminal, and converts an analog audio signal generated by collecting audio around the imaging device 30 into a digital signal and outputs it to the control unit 301. The control unit 301 performs various types of audio processing on the digital audio signal generated by the audio input unit 308 to generate audio data.
[0032] The control unit 301 performs resizing such as pixel interpolation and reduction and color conversion processing on image data captured by the imaging unit 304. The control unit 301 also compresses and encodes the processed still image data using JPEG or the like, or encodes only moving image data or both moving image data and audio data using a moving image compression format such as H.264 / MPEG-4 / AVC to generate an image file, and records the image file on the recording medium 309. The control unit 301 also performs a predetermined calculation process using the captured image data, and controls the focus lens, aperture, and shutter of the imaging unit 304 based on the calculation result to perform AF (autofocus) processing and AE (autoexposure) processing. The control unit 301 also generates predetermined streaming data (for example, RTMP (Real-Time Messaging Protocol)) from media data including image data and / or audio data, and transmits the data to the control device 20.
[0033] The control unit 301 also includes an operation determination unit 301a and an imaging control unit 301b. The operation determination unit 301a determines the operation state of the imaging device 30 based on control information received from the control device 20, and transmits an operation instruction to the imaging control unit 301b. The imaging control unit 301b controls the operation of the imaging device 30 based on the operation instruction from the operation determination unit 301a.
[0034] Non-volatile memory 302 is an electrically erasable and recordable memory, and for example, an EEPROM is used. Constants, programs, etc. for the operation of control unit 301 are recorded in non-volatile memory 302. The program here refers to a program for executing a control process described later in FIG. 6.
[0035] The work memory 303 is used as a working area for expanding constants and variables for the operation of the control unit 301, programs read from the non-volatile memory 302, etc. The work memory 303 is also used as a buffer memory for temporarily storing image data captured by the imaging unit 304, and an image display memory for the display unit 306.
[0036] The operation unit 305 is composed of operation members such as various switches, buttons, and a touch screen that accept various operations from the user. The operation unit 305 includes a power switch that turns on or off the power of the imaging device 30 and starts or shuts down the system, a shooting switch that shoots an image, and a playback button that plays back the shot image. The operation unit 305 also includes a touch screen that is integrally formed with the display unit 306 described later. The shooting switches include a still image shooting switch that instructs the control unit 301 to shoot a still image, and an action shooting switch that instructs the control unit 301 to start or stop shooting (recording) a video. In response to an instruction from the still image shooting switch or the video shooting switch, the control unit 301 controls the imaging unit 304 to execute shooting preparation operations such as AF processing and AE processing, and also executes a series of shooting operations from reading a signal from the imaging unit 304 to writing image data to the recording medium 309.
[0037] The display unit 306 displays a live view image, a captured image, and text for interactive operations. The display unit 306 is, for example, a display device such as a liquid crystal display or an organic EL display. The display unit 306 may be integrated with the imaging device 30 or may be an external device connected to the imaging device 30. The imaging device 30 may be connected to the display unit 306 and may control the display of the display unit 306.
[0038] The communication unit 307 is communicably connected to an external device such as the control device 20 via a wireless antenna or a wired cable, and transmits and receives various information and data. The various information and data are, for example, control information received from the control device 20, and function information and media data transmitted to the control device 20. The communication unit 307 includes an interface for communicating with the external device via wireless LAN. The imaging device 30 may be directly connected to the control device 20 via the communication unit 307, or may be connected via wireless LAN. The communication unit 307 is not limited to wireless LAN, and may use a wireless communication module such as infrared communication, Bluetooth (registered trademark), or Wireless USB, or a wired connection interface such as a USB cable, HDMI (registered trademark), or IEEE1394. The communication unit 207 may also be an interface for wireless communication via a wide area network (WAN) such as 4G / LTE or 5G.
[0039] The communication unit 307 includes an AP mode in which the imaging device 30 operates as an access point (AP) in infrastructure mode, and a CL mode in which the imaging device 30 operates as a client (CL) in infrastructure mode. By operating the communication unit 307 in the CL mode, the imaging device 30 can operate as a CL in infrastructure mode. When the imaging device 30 operates as a CL, it can connect to a surrounding AP and participate in a network formed by the AP. By operating the communication unit 307 in the AP mode, the imaging device 30 can operate as a simple AP (hereinafter, a simple AP) which is a type of AP but has more limited functions. When the imaging device 30 operates as a simple AP, the imaging device 30 forms a network by itself. Peripheral devices of the imaging device 30 recognize the imaging device 30 as an AP and can participate in the network formed by the imaging device 30. A program for operating the imaging device 30 in the AP mode or the CL mode is stored in the non-volatile memory 302.
[0040] The audio input unit 308 is one or more microphones built into the imaging device 30 or connected via an audio terminal, and converts an analog audio signal generated by collecting audio around the imaging device 30 into a digital signal and outputs it to the control unit 301. The control unit 301 performs various types of audio processing on the digital audio signal generated by the audio input unit 308 to generate audio data.
[0041] Image files generated by the control unit 301 are recorded on the recording medium 309, and image files already recorded on the recording medium 309 are read out by the control unit 301. The recording medium 309 may be a memory card or a hard disk drive that is attached to the imaging device 30, or may be a flash memory or a hard disk drive built into the imaging device 30. It is sufficient that the imaging device 30 is at least capable of accessing the recording medium 309.
[0042] <Media data processing sequence> Next, a media data processing sequence performed by the control device 20 of this embodiment will be described with reference to FIG.
[0043] 4 is realized by the control unit 201 of the control device 20 functioning as the data processing unit 201b, reading data before and after processing from the work memory 203, and writing data to the work memory 203. Note that in this embodiment, an example will be described in which streaming data received from the multiple imaging devices 30A, 30B, and 30C includes video and audio, and media data for playback is generated without applying data conversion processing to the streaming data, or media data in a data format suitable for distribution by the distribution device 10 is generated by applying data conversion processing.
[0044] In the reception process 402, the control device 20 receives streaming data from the multiple imaging devices 30A, 30B, and 30C via the communication unit 207, and stores the data in the work memory 203. The format of the streaming data received in the reception process 402 is, for example, RTMP. Furthermore, in the reception process 402, media data including video and audio is generated from the streaming data stored in the work memory 203. The format of the media data generated from the streaming data is, for example, FLV.
[0045] In the DEMUX (demultiplexer) process 403, the media data input from the reception process 402 is separated into video data and audio data. The video data separated from the media data is in a format of, for example, H.264. The audio data separated from the media data is in a format of, for example, AAC.
[0046] In the decoding processes 404 and 405, the video data and audio data separated in the DEMUX process 403 are converted into a general-purpose format. Since the video data and audio data have been subjected to advanced encoding processing, the decoding processes 404 and 405 decode the data into a general-purpose data format, whereby the video data and audio data are generated as media data for playback that can be played back in the playback process 406.
[0047] In the encoding processes 407 and 408, the video data and audio data generated in the decoding processes 404 and 405 are subjected to a high-level encoding process again to generate encoded video data and audio data. The format of the encoded video data is, for example, H.264. The format of the encoded audio data is, for example, AAC. In the encoding processes 407 and 408, encoding processes in a data format that can be received by the distribution device 10 are applied.
[0048] A MUX (multiplexer) process 409 generates media data by synthesizing the video data and audio data encoded in the encoding processes 407 and 408. The format of the media data by synthesizing the video data and audio data is, for example, FLV.
[0049] In the transmission process 410, the media data generated in the MUX process 409 is converted into streaming data and transmitted to the distribution device 10. Note that the data format applied in the MUX process 409 and the transmission process 410 is a data format that can be received by the distribution device 10.
[0050] The data format applied in each process of the data processing flow in Fig. 4 is not limited to the above-mentioned example. For example, the formats of the video data and audio data may be determined according to the processing speed of decoding and encoding, and the quality characteristics of the video and audio, in addition to conforming to the data format output by the imaging device 30 and the data format that the distribution device 10 can receive.
[0051] FIG. 5 shows an example of a display of an image reproduced in the reproduction process 406 in FIG.
[0052] A display area 501 corresponds to the display screen of the display unit 206 of the control device 20. An image 502 is an image input from the first imaging device 30A. An image 503 is an image input from the second imaging device 30B. An image 504 is an image input from the imaging device 30C.
[0053] Image 502 is an example of an image with an angle of view that includes the entire subject. Images 503 and 504 are images from a different viewpoint from image 502, and are examples of images with an angle of view that enlarges a part of the subject compared to image 502.
[0054] A thick dashed attention frame is displayed around image 502. The attention frame indicates which of the multiple imaging devices the image was received from. In addition to the attention frame, information such as characters, figures, and coloring that can identify the imaging device that outputs the image to which the attention frame is added may be added and displayed. Which of the media data received from the multiple imaging devices is to be transmitted to the distribution device is determined by a control process described later with reference to Figs. 6 to 11. In addition, the user can select the media data that the distribution device 10 uses for distribution by touching image 502, image 503, or image 504.
[0055] <Control processing> Next, the control process performed by the control device 20 of this embodiment will be described with reference to FIGS.
[0056] Fig. 6 is a flowchart showing the control process of the control device 20 of this embodiment. The process of Fig. 6 is realized by the control unit 201 expanding a program stored in the non-volatile memory 202 into the work memory 203, executing the program, and controlling each component of the control device 20. The process of Fig. 6 is started when a camera application is started in the control device 20, the control device 20 and the multiple image capturing devices 30A, 30B, and 30C are connected to each other so as to be able to communicate with each other, and the multiple image capturing devices 30A, 30B, and 30C are in a shooting standby state.
[0057] The control device 20 of this embodiment switches the distribution source imaging device to be used by the distribution device 10 to distribute media data from among the multiple imaging devices 30A, 30B, 30C based on status information for each imaging device received from the multiple imaging devices 30A, 30B, 30C.
[0058] The control device 20 compares the status information received from the multiple imaging devices 30A, 30B, 30C with a threshold value, generates control information (switching pattern) for switching the source imaging device based on the comparison result, receives media data from the source imaging device that can be switched according to the switching pattern, and transmits it to the distribution device 10.
[0059] The control device 20 notifies the plurality of imaging devices 30A, 30B, and 30C of control information, and the plurality of imaging devices 30A, 30B, and 30C control their operating states based on the control information included in the notification.
[0060] In step S601, the control unit 201 registers a switching pattern for the image capture device 30. The switching pattern includes the order in which the multiple image capture devices 30A, 30B, and 30C are used as source image capture devices. The switching pattern is set before the start of the distribution process, for example, by the user of the control device 20 operating the operation unit 205 on the UI screen of the camera application displayed on the display unit 206. The control information generating unit 201a of the control unit 201 stores the switching pattern for the image capture device 30 in the non-volatile memory 202. The switching pattern for the image capture device 30 is control information for selecting a source image capture device for media data that the distribution device 10 automatically distributes and automatically records, and may be a pattern in which the source image capture device is switched from the multiple image capture devices 30A, 30B, and 30C at regular intervals, a pattern randomly set by the system, or a pattern in which the user sets the switching time and image capture device. The switching pattern may also set a minimum number of operable image capture devices. The minimum number of operable image capture devices is a parameter used in the determination process of step S610, which will be described later in FIG. 6. If the minimum number of operable imaging devices is not set, the initial value is set to 0.
[0061] In step S602, the control unit 201 starts the data processing unit 201b transmitting and receiving the media data from the multiple imaging devices 30A, 30B, and 30C. The transmitting and receiving process of the media data is a process from receiving the media data from the multiple imaging devices 30A, 30B, and 30C to transmitting the media data received from the source imaging device to the distribution device 10. When the control unit 201 receives a distribution start instruction by the user operating the operation unit 205 on the UI screen of the camera application displayed on the display unit 206, the data processing unit 201b starts the transmission and receiving process of the media data. Note that the distribution start instruction is not limited to the operation of the UI screen of the display unit 206 by the operation unit 205, but may be an operation by another operation member of the control device 20, a member other than the control device 20, or an operation by voice recognition. The data processing flow from receiving the media data from the imaging device to transmitting it to the distribution device 10 is as described in FIG. 4.
[0062] In step S603, the control unit 201 receives status information from the multiple imaging devices 30A, 30B, and 30C. The control information generating unit 201a of the control unit 201 receives the status information from the multiple imaging devices 30A, 30B, and 30C via the communication unit 207. The status information includes information about an unusable state in which the imaging device 30 cannot transmit media data, such as remaining battery power, heat generation amount, and remaining storage capacity. The status information also includes information about the operating state of the imaging device, such as the shooting mode.
[0063] In step S604, the control unit 201 determines whether or not there is an imaging device among the multiple imaging devices 30A, 30B, and 30C whose shooting mode is a mode for recording media data, based on the status information received from the multiple imaging devices 30A, 30B, and 30C in step S603. When the control information generating unit 201a of the control unit 201 determines from the status information received in step S603 that there is an imaging device whose shooting mode is a mode for recording media data, the control information generating unit 201a of the control unit 201 stores the storage remaining amount of the imaging device as a maximum value in the work memory 203. Note that, although the storage remaining amount is set to a maximum value in this embodiment, information indicating that the storage remaining amount is a maximum value may be received from the imaging devices 30A, 30B, and 30C, in which case the process of step S604 does not need to be executed.
[0064] In step S605, the control unit 201 compares the state information received from the multiple imaging devices 30A, 30B, and 30C in step S603 with a threshold value, and determines whether or not there is an imaging device whose state information is equal to or less than the threshold value. The control information generation unit 201a of the control unit 201 determines whether or not there is a parameter whose state information is equal to or less than the threshold value for each of the parameters of the state information illustrated in FIG. 7, that is, the remaining battery capacity, the amount of heat generated, and the remaining storage capacity, based on the state information received from the multiple imaging devices 30A, 30B, and 30C in step S603. In the example of FIG. 7, three threshold values A, B, and C are set for each parameter of the state information. If the control unit 201 determines that there is an imaging device whose state information is equal to or less than the threshold value, the control unit 201 advances the process to step S606, and if it determines that there is no imaging device whose state information is equal to or less than the threshold value, the control unit 201 advances the process to step S611.
[0065] In step S606, the control unit 201 determines the severity of the status information that is equal to or lower than the threshold. The control information generating unit 201a determines the severity of the status information based on the threshold range exemplified in FIG. 8. In the example of FIG. 8, the severity corresponding to the threshold range in FIG. 7 and the switching pattern of the imaging device for each severity are set. For example, when the remaining battery level in the status information received from the imaging device is 25%, it is within the range between threshold A and threshold B, so it corresponds to severity level 2. Also, for example, when the remaining battery level is 3%, it is within the range of threshold C or higher, so it corresponds to severity level 3. The higher the severity value, the higher the severity. In this embodiment, severity level 3 is the highest and is an unusable level.
[0066] In step S607, the control unit 201 corrects the switching patterns of the imaging devices 30A, 30B, and 30C. The control information generating unit 201b corrects the switching patterns of the imaging devices 30A, 30B, and 30C to which the severity has been assigned, with reference to FIG. 8, based on the severity of the status information below the threshold determined in step S606. In the example of FIG. 8, the number of times and the ratio are illustrated as the usage frequency based on the switching pattern. The switching pattern 1 is a pattern in which the usage frequency is changed by decreasing the number of times the source imaging device is used. The switching pattern 2 is a pattern in which the usage frequency is changed by decreasing the usage ratio (usage time) of the source imaging device. FIG. 8 is an example, and the numerical values may be changed as necessary. FIG. 9 illustrates a switching pattern corrected according to the severity of the status information in FIG. 8. FIG. 9(a) illustrates the switching pattern registered in step S601. In this case, the source imaging device is switched at regular intervals among the multiple imaging devices 30A, 30B, and 30C. FIG. 9(a) illustrates a normal switching pattern in which there is no status information below the threshold. Fig. 9(b) illustrates a switching operation in switching pattern 1 of Fig. 8 when the severity of the status information below the threshold is 1. Fig. 9(c) illustrates a switching operation in switching pattern 1 of Fig. 8 when the severity of the status information below the threshold is 2. Fig. 9(d) illustrates a switching operation in switching pattern 1 of Fig. 8 when the severity of the status information below the threshold is 3. Fig. 9(e) illustrates a switching pattern when there are multiple imaging devices with severity levels assigned to their status information. In this case, the pattern for imaging device 2 and the pattern for imaging device 3 are combined.
[0067] In step S608, the control unit 201 determines whether or not there is any status information that has reached the unusable level among the severity levels of the status information determined in step S606. If the control information generating unit 201a determines that any of the imaging devices 30A, 30B, and 30C whose status information has been assigned a severity level has reached the unusable level, the process proceeds to step S609, and if the control information generating unit 201a determines that none of the imaging devices has reached the unusable level, the process proceeds to step S611. The unusable level corresponds to severity level 3, in which the severity level of the status information of the imaging devices shown in FIG. 8 is such that transmission of media data to the distribution device 10 cannot be continued.
[0068] In step S609, the control unit 201 notifies the user that there is an imaging device whose status information has reached an unusable level in step S608. When the status information has reached an unusable level, the imaging device is in a state where it cannot continue transmitting media data to the distribution device 10, so the user needs to take some kind of measure even if the distribution device 10 is in the middle of distributing media data. Such measures include replacing the battery, changing the memory card, interrupting the distribution, or cooling with a fan. The control information generating unit 201a notifies the user that some kind of measure is required. The notification may be, for example, illuminating an LED, outputting a sound, or displaying a message in the imaging device 30 or the control device 20 whose status information has reached an unusable level, but any notification method may be used as long as it allows the user to recognize the notification.
[0069] In step S610, the control unit 201 determines whether the minimum number of operable imaging devices whose status information has not reached the unusable level is equal to or greater than a predetermined number. The control information generation unit 201a compares the minimum number set in the switching pattern registered in step S601 with the number of imaging devices used in the switching pattern corrected in step S607, and if it is determined that the minimum number of operable imaging devices is below the predetermined number, the process proceeds to step S614, whereas if it is determined that the minimum number of operable imaging devices is equal to or greater than the predetermined number, the process proceeds to step S611.
[0070] In step S611, the control unit 201 transmits control information based on the switching pattern set in step S601 or the switching pattern corrected in step S607 to the multiple imaging devices 30A, 30B, and 30C. The control information generating unit 201a transmits control information corresponding to the operation states of the imaging devices exemplified in FIG. 10 to the multiple imaging devices 30A, 30B, and 30C based on the corrected switching pattern. The control information includes the operation states of the imaging devices as in use, standby, and preparation for use, and notifies the imaging devices 30A, 30B, and 30C corresponding to each operation state of behavior (operation content) corresponding to each operation state. In use, the imaging device is transmitting media data for distribution to the distribution device 10. Standby is a state in which the imaging device stops operation or performs a minimum operation and waits until the next start of use. Preparation for use is a state in which the imaging device is prepared to immediately start transmitting media data for distribution to the distribution device 10 when the next start of use is started.
[0071] FIG. 11 illustrates the transition of the operation state of the imaging devices 30A, 30B, and 30C based on the switching pattern. In the example of FIG. 11, the imaging devices 30A, 30B, and 30C determine the operation state at each time interval t1 when the control device 20 notifies the imaging devices 30A, 30B, and 30C of the control information based on the switching pattern. The control information generating unit 201a may notify the imaging devices 30A, 30B, and 30C of the operation state of each imaging device as shown in FIG. 11 as control information at the timing when the switching pattern is updated, or may calculate the operation state of each imaging device from the switching pattern and notify the imaging devices 30A, 30B, and 30C each time the process of step S611 is executed. The notification time interval t1 of the control information may be, for example, 0.5 seconds, which is sufficient to notify the imaging device and start or execute the operation in the imaging device, as long as it is a time that allows a smooth transition between standby for use, preparation for use, and in use. When the control unit 301 of the imaging devices 30A, 30B, and 30C receives control information through the communication unit 307, the operation determination unit 301a of the imaging devices 30A, 30B, and 30C determines the operation state shown in FIG. 10 and transmits an operation instruction to the imaging control unit 301b. The imaging control unit 301b controls the operation of the imaging devices 30A, 30B, and 30C based on the operation instruction of the operation determination unit 301a. In the example of FIG. 10, four examples are shown as behaviors for each operation state of the imaging devices. In all examples, when in use, a video with a picture quality designated by the user is transmitted. In all examples, when ready for use, an operation is performed to return from a standby (sleep) state to an in-use state. In standby for use, behavior 1 is an operation to reduce battery power consumption by transmitting a video with the lowest picture quality to extend the operation time of the imaging device, and behavior 2 is an operation to reduce battery power consumption by going into sleep to extend the operation time of the imaging device. Behavior 3 is an operation for reducing battery power consumption and extending the operable time of the imaging device by maintaining the communication state but not going to sleep, but not transmitting media data. Behavior 4 is an operation for suppressing a decrease in the remaining storage capacity and extending the operable time of the imaging device by not recording media data. In addition to behaviors 1 to 4, examples include cooling with a fan during sleep to suppress an increase in heat generation, and thinning out the frame rate of video. Behaviors 1 to 4 illustrated in FIG. 11 may be combined as appropriate.Furthermore, any method other than those described above may be used as long as it suppresses a decrease in the remaining battery charge or storage charge or an increase in the amount of heat generated.
[0072] In step S612, the control unit 201 judges whether it is time for the control information generating unit 201a to receive status information from the imaging devices 30A, 30B, and 30C. If the control unit 201 judges that it is time to receive status information from the imaging devices 30A, 30B, and 30C, the control unit 201 advances the process to step S617. If the control unit 201 judges that it is not time to receive status information from the imaging devices 30A, 30B, and 30C, the control unit 201 returns the process to step S611. The timing to receive status information is determined based on the minimum processing time of the system, for example, 5 seconds. In this embodiment, the control device 20 notifies the imaging devices 30A, 30B, and 30C of control information based on the switching pattern at appropriate times, but the control device 20 may transmit the switching pattern corrected in step S607 to the imaging devices 30A, 30B, and 30C, and the operation judging unit 301a and the imaging control unit 301b of the control unit 301 may control the operation of the imaging devices 30A, 30B, and 30C according to the switching pattern.
[0073] In step S613, the control unit 201 determines whether or not the user has issued a distribution stop instruction by operating the operation unit 205 on the UI screen of the camera application displayed on the display unit 206. If the control unit 201 determines that a distribution stop instruction has been received, the process proceeds to step S614, and if the control unit 201 determines that a distribution stop instruction has not been received, the process returns to step S603 and the distribution continues.
[0074] In step S614, the control unit 201 performs distribution stop processing. The control unit 201 stops the transmission of the media data to the distribution device 10, instructs the distribution device 10 to stop the distribution of the media data, and further requests the multiple imaging devices 30A, 30B, and 30C to stop transmitting the media data.
[0075] According to the above-described embodiment, the control information (switching pattern) for switching the source imaging device used by the distributing device 10 to distribute media data is corrected based on the status information of the multiple imaging devices 30A, 30B, and 30C. In this manner, the source imaging device is appropriately switched among the multiple imaging devices 30A, 30B, and 30C to extend the operable time of the imaging device, thereby minimizing the state in which media data for distribution is not transmitted from the source imaging device.
[0076] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0077] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention.
[0078] The disclosure of this specification includes the following control device, imaging device, control method, and program. [Configuration 1] A communication means for communicating with a plurality of imaging devices; a control unit for receiving media data from the plurality of imaging devices via the communication unit; the control means generates control information for switching a predetermined imaging device to be used for delivering the media data among the plurality of imaging devices based on the status information received from the plurality of imaging devices; A control device that transmits the control information to the plurality of imaging devices. [Configuration 2] the control means generates a switching pattern for switching the predetermined imaging device among the plurality of imaging devices; 2. The control device according to configuration 1, wherein a predetermined switching pattern is corrected based on the state information. [Configuration 3] 3. The control device according to claim 2, wherein the state information includes a state in which the imaging device is unable to continue transmitting media data. [Configuration 4] the state information includes at least one of a remaining battery capacity, a heat generation amount, and a remaining storage capacity of the imaging device, 4. The control device according to configuration 2 or 3, wherein the control means corrects the predetermined switching pattern when it is determined that there is an imaging device whose status information is equal to or less than a threshold value. [Configuration 5] The control device according to any one of configurations 2 to 4, characterized in that the correction of the switching pattern includes reducing a usage frequency of an imaging device whose status information is equal to or less than a threshold based on a comparison between the status information and a threshold. [Configuration 6] The control device according to any one of configurations 2 to 5, characterized in that the control means stops distribution of the media data when the number of imaging devices used in the corrected switching pattern does not reach the number of operable imaging devices. [Configuration 7] 7. The control device according to any one of configurations 1 to 6, wherein the switching pattern includes, as the operating state of the imaging device, in use, standby for use, and preparation for use. [Configuration 8] 8. The control device according to configuration 7, wherein the operational state of the imaging device includes an operation for extending an operational time of the imaging device. [Configuration 9] The control device according to configuration 8, wherein the operation for extending the operable time of the imaging device includes an operation for reducing power consumption of the imaging device, an operation for reducing the amount of heat generated by the imaging device, or an operation for suppressing a decrease in remaining storage capacity of the imaging device. [Configuration 10] The control device according to any one of configurations 1 to 9, characterized in that, when it is determined that there is an imaging device whose status information is equal to or less than a threshold, the control means notifies a user that the imaging device cannot be used. [Configuration 11] 5. The control device according to configuration 4, wherein the control means determines that the remaining storage capacity of the imaging device in which no media data is being recorded is maximum. [Configuration 12] 12. The control device according to any one of configurations 1 to 11, wherein the control means converts the media data received from the specified imaging device into a data format suitable for distribution by a distribution device, and transmits the converted data to the distribution device. [Configuration 13] 13. The control device according to any one of configurations 1 to 12, wherein the media data includes video and audio generated by the imaging device. [Configuration 14] 1. An imaging device, comprising: A communication means for communicating with the control device; a control unit for transmitting media data to the control device by the communication unit; The control means transmits status information to the control device, An imaging device, characterized in that an operating state of the imaging device is controlled based on control information received from the control device. [Configuration 15] 15. The imaging device according to configuration 14, wherein the control information includes a switching pattern for switching an imaging device that transmits the media data to the control device. [Configuration 16] 16. The imaging device according to configuration 15, wherein the switching pattern includes, as the operating state of the imaging device, in use, standby for use, and preparation for use. [Configuration 17] 17. The imaging device according to any one of configurations 14 to 16, wherein the state information includes a state in which the imaging device is unable to continue transmitting media data. [Configuration 18] 18. The imaging device according to claim 17, wherein the status information includes at least one of a remaining battery charge, a heat generation amount, and a remaining storage capacity of the imaging device. [Configuration 19] 17. The imaging device according to configuration 16, wherein the operational state of the imaging device includes an operation for extending an operational time of the imaging device. [Configuration 20] The imaging device described in configuration 19, characterized in that the operation for extending the operational time of the imaging device includes an operation for reducing power consumption of the imaging device, an operation for reducing the amount of heat generated by the imaging device, or an operation for suppressing a decrease in remaining storage capacity of the imaging device. [Configuration 21] 21. The imaging device according to any one of configurations 14 to 20, wherein the media data includes video and audio generated by the imaging device. [Configuration 22] A control method for a control device, comprising: receiving media data from a plurality of imaging devices by a communication means; In the step, based on the status information received from the plurality of imaging devices, control information for switching a predetermined imaging device to be used for distributing the media data among the plurality of imaging devices is generated, A control method comprising: transmitting the control information to the plurality of image capture devices. [Configuration 23] A method for controlling an imaging device having a communication means for communicating with a control device, comprising: transmitting media data to the control device by the communication means; In the step, status information is transmitted to the control device; A control method comprising: controlling an operating state of the imaging device based on control information received from the control device. [Configuration 24] A program for causing a computer to function as a control device according to any one of configurations 1 to 13. [Configuration 25] A program for causing a computer to function as the imaging device according to any one of configurations 14 to 21. [Explanation of symbols]
[0079] 10... distribution device, 20... control device, 30, 30A, 30B, 30C... imaging device, 201, 301... control unit, 207, 307... communication unit
Claims
1. A communication means for communicating with a plurality of imaging devices; a control unit for receiving media data from the plurality of imaging devices via the communication unit; the control means generates control information for switching a predetermined imaging device to be used for delivering the media data among the plurality of imaging devices based on the status information received from the plurality of imaging devices; A control device that transmits the control information to the plurality of imaging devices.
2. the control means generates a switching pattern for switching the predetermined imaging device among the plurality of imaging devices; 2. The control device according to claim 1, further comprising: a control unit for controlling a switching pattern determined in advance based on the state information.
3. The control device according to claim 2 , wherein the state information includes a state in which the imaging device is unable to continue transmitting media data.
4. the state information includes at least one of a remaining battery capacity, a heat generation amount, and a remaining storage capacity of the imaging device, The control device according to claim 2 , wherein the control means corrects the predetermined switching pattern when it is determined that there is an imaging device whose status information is equal to or less than a threshold value.
5. The control device according to claim 2 , wherein the correction of the switching pattern includes reducing a frequency of use of an image capture device whose status information is equal to or less than a threshold based on a comparison between the status information and a threshold.
6. The control device according to claim 2 , wherein the control means stops the distribution of the media data when the number of imaging devices used in the corrected switching pattern does not reach the number of operable imaging devices.
7. The control device according to claim 1 , wherein the switching pattern includes, as the operating state of the imaging device, in use, standby for use, and preparation for use.
8. The control device according to claim 7 , wherein the operation state of the imaging device includes an operation for extending an operable time of the imaging device.
9. The control device according to claim 8, characterized in that the operation for extending the operational time of the imaging device includes an operation for reducing power consumption of the imaging device, an operation for reducing the amount of heat generated by the imaging device, or an operation for suppressing a decrease in remaining storage capacity of the imaging device.
10. The control device according to claim 1 , wherein, when it is determined that there is an image capturing device whose status information is equal to or less than a threshold value, the control means notifies a user that the image capturing device cannot be used.
11. The control device according to claim 4 , wherein the control means determines that the remaining storage capacity of the imaging device in which no media data is being recorded is maximum.
12. 2. The control device according to claim 1, wherein the control means converts the media data received from the predetermined imaging device into a data format suitable for distribution by a distribution device, and transmits the converted data to the distribution device.
13. The control device according to claim 1 , wherein the media data includes video and audio generated by the imaging device.
14. 1. An imaging device, comprising: A communication means for communicating with the control device; a control unit for transmitting media data to the control device by the communication unit; The control means transmits status information to the control device, An imaging device, characterized in that an operating state of the imaging device is controlled based on control information received from the control device.
15. The imaging device according to claim 14 , wherein the control information includes a switching pattern for switching an imaging device that transmits the media data to the control device.
16. 16. The imaging device according to claim 15, wherein the switching pattern includes an operating state of the imaging device being in use, being on standby, and being ready for use.
17. The imaging device of claim 14 , wherein the status information includes a condition that prevents the imaging device from continuing to transmit media data.
18. 18. The image capture device according to claim 17, wherein the state information includes at least one of a remaining battery charge, a heat generation amount, and a remaining storage capacity of the image capture device.
19. 17. The image capture device according to claim 16, wherein the operational state of the image capture device includes an operation for extending an operational time of the image capture device.
20. The imaging device according to claim 19, characterized in that the operation for extending the operational time of the imaging device includes an operation for reducing power consumption of the imaging device, an operation for reducing the amount of heat generated by the imaging device, or an operation for suppressing a decrease in remaining storage capacity of the imaging device.
21. The imaging device according to claim 14 , wherein the media data includes video and audio generated by the imaging device.
22. A control method for a control device, comprising: receiving media data from a plurality of imaging devices by a communication means; In the step, based on the status information received from the plurality of imaging devices, control information for switching a predetermined imaging device to be used for delivering the media data among the plurality of imaging devices is generated, A control method comprising: transmitting the control information to the plurality of image capture devices.
23. A method for controlling an imaging device having a communication means for communicating with a control device, comprising: transmitting media data to the control device by the communication means; In the step, status information is transmitted to the control device; A control method comprising: controlling an operating state of the imaging device based on control information received from the control device.
24. A program for causing a computer to function as the control device according to any one of claims 1 to 13.
25. A program for causing a computer to function as the imaging device according to any one of claims 14 to 21.
Citation Information
Patent Citations
Imaging control apparatus, imaging control method, and program
JP2019009651A
