Imaging system and imaging method

Abstract

To make it easier for a remote observer to obtain video which the observer desires.SOLUTION: The present invention relates to an imaging system 1 that images video using a wearable camera 2 attached to a worker. The imaging system 1 includes an imaging unit 3 provided in the wearable camera 2 that images video, a correction unit 4 provided in the wearable camera 2 that corrects shaking of the video imaged, a communication unit 5 provided in the wearable camera 2 that transmits the video corrected for shaking by the correction unit 4 to a remote location RM for monitoring at the remote location RM, and an adjustment unit that controls the wearable camera 2 from the remote location RM to adjust at least one of a zoom center C and the image quality of the video imaged by the wearable camera 2.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an imaging system and an imaging method for capturing moving images using an imaging camera provided on a moving object. [Background technology]

[0002] Patent Document 1 discloses a surveillance camera having a camera unit, a rotation mechanism for rotating the camera unit in a pan direction and a tilt direction, and a support part attached to a target facility and supporting an upper part of the rotation mechanism, the support part having at least one bending part that can be bent freely, and the bending part bends so that the pan axis of the camera unit of the surveillance camera attached to the target facility is perpendicular to the ground. Paragraph 0012 of the specification of Patent Document 1 describes that the surveillance camera has excellent freedom in terms of installation location. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-108568 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are cases where an imaging camera is attached to a moving object such as a person, a car, heavy machinery, or a work robot (hereinafter referred to as a "mobile object"), and the moving object captures the situation at the site while the captured video is monitored at a location away from the site (hereinafter referred to as a "remote location"). In such cases, since the imaging camera moves with the moving object, the type of video captured is largely determined by the location, orientation, and movement of the moving object. This presents a problem in that it is difficult for a monitor monitoring the video at a remote location to obtain the video that he or she desires.

[0005] The technology disclosed herein has been made in consideration of the above points, and its purpose is to make it easier for a monitor in a remote location to obtain video as desired. [Means for solving the problem]

[0006] A first aspect of the present disclosure relates to an imaging system for capturing a video using an imaging camera installed on a moving object, the imaging system including: an imaging unit installed in the imaging camera for capturing a video, a correction unit installed in the imaging camera for correcting a shake of the captured video, a communication unit installed in the imaging camera for transmitting the video corrected by the correction unit to a remote location for monitoring at the remote location, and an adjustment unit for controlling the imaging camera from the remote location of the imaging camera to adjust at least one of a zoom center and image quality of the video captured by the imaging camera.

[0007] According to the first aspect, the observer can control the imaging camera from a remote location to adjust at least one of the zoom center and image quality of the captured video. Furthermore, since the video is stabilized by the imaging camera, the observer can easily adjust at least one of the zoom center and image quality while watching the video. As a result, the observer can easily obtain the video he or she desires.

[0008] In a second aspect of the present disclosure, in the imaging system, the moving object is a person, and the imaging camera is a wearable camera.

[0009] According to the second aspect, for example, a worker working at a site can capture images of the site with the attached wearable camera while working, and a supervisor can monitor the video from a remote location, which is convenient. The wearable camera is provided with a correction unit, and the captured video is stabilized, so the worker does not need to keep the wearable camera still to capture the image, and can concentrate on the work at the site.

[0010] However, the color tone and brightness of a video are particularly susceptible to the influence of the surrounding environment of the subject. Here, in the third aspect of the present disclosure, in the imaging system, the adjustment unit can control the imaging camera so as to adjust at least one of the white balance and exposure of the imaging camera. Therefore, a monitor can adjust the color tone of the video by controlling the white balance of the imaging camera, or adjust the brightness of the video by controlling the exposure of the imaging camera. As a result, it becomes easier to obtain a video as the monitor desires.

[0011] A fourth aspect of the present disclosure relates to an imaging method for capturing a moving image by an imaging camera provided on a moving body and having a correction unit for performing shake correction, in which the imaging camera captures a moving image while performing shake correction, and the captured moving image is received and monitored at a remote location away from the moving body, and the imaging camera is controlled to adjust at least one of a zoom center and image quality of the moving image.

[0012] According to the fourth aspect, a monitor can adjust at least one of the zoom center and image quality of the video while monitoring the shake-compensated video from a remote location, making it easier to obtain the video that the monitor desires. Effect of the Invention

[0013] As described above, according to the present disclosure, a monitor in a remote location can easily obtain video as desired. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a conceptual diagram showing an imaging system and an imaging method according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of the imaging system. [Diagram 3] FIG. 3 is a schematic diagram for explaining adjustment of the zoom center. [Figure 4] FIG. 4 is a flowchart showing the procedure of the imaging method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. Note that the following description is merely an example.

[0016] (Embodiment) - Imaging system - First, an overview of an imaging system 1 according to an embodiment will be described. As shown in Fig. 1, the imaging system 1 is for capturing video using a wearable camera 2 (hereinafter referred to as "camera") attached to a worker (mobile body) performing work at a site. Note that the site referred to here is not particularly limited, but examples include civil engineering work sites, construction work sites, and factories.

[0017] As shown in FIG. 2, the imaging system 1 includes an imaging unit 3 for capturing video, a correction unit 4 for correcting shake of the video, a communication unit 5 for transmitting the video corrected by the correction unit 4 to a remote location RM for monitoring at the remote location RM, and a control unit 6 for controlling the operation of the camera 2.

[0018] The imaging system 1 also includes a web application (adjustment unit) running on a monitoring computer 7 installed at a remote location RM for controlling the camera 2 to adjust the zoom center C and image quality of the video captured by the camera 2 from the remote location RM of the camera 2.

[0019] In this embodiment, although there are a plurality of cameras 2 as shown in Fig. 1, it is sufficient to have at least one camera 2. Next, each component will be described in detail.

[0020] --Imaging section-- 2, the imaging unit 3 includes an imaging lens 30 that forms an image of a subject, an imaging element 31 that photoelectrically converts an optical image of the subject into an image signal, and a signal processing unit 32 that performs various processes on the image signal. The imaging lens 30 collects light from the subject and makes it incident on the imaging element 31. The imaging lens 30 is preferably, for example, a semi-spherical wide-angle lens. The imaging lens 30 may be a movable zoom lens that zooms in on the subject and forms an image.

[0021] The image sensor 31 photoelectrically converts an optical image of a subject using a plurality of pixels arranged on its light receiving surface to generate an electrical signal (image signal). Specifically, the image sensor 31 can be realized using a CCD image sensor, a CMOS image sensor, or the like.

[0022] The signal processing unit 32 is configured to perform processes such as A / D conversion, white balance adjustment, and color separation on the image signal generated by the imaging element 31. Specifically, the signal processing unit 32 can be realized using an IC, an LSI, an ASIC, or the like.

[0023] The image signal processed by the signal processing unit 32 is temporarily recorded on a frame-by-frame basis in a memory M provided in the camera 2. The image signal temporarily recorded in the memory M is an image signal of an imaging region R1 acquired through an imaging lens 30 as shown in Fig. 3, or an image signal of a first cropped region R2 or a second cropped region R3, which will be described later in detail.

[0024] The image signals recorded in the memory M are continuously recorded in a recording medium ST provided in the camera 2 as necessary. The memory M can be realized using a volatile memory such as a DRAM or SDRAM, and the recording medium ST can be realized using a non-volatile memory such as a flash memory. Note that the recording medium ST is not essential.

[0025] --Correction section-- The correction unit 4 is made up of a shake detection unit 4a that detects the shake of the camera 2, and a control unit 6. The shake detection unit 4a is realized by a sensor capable of detecting the shake of the camera 2, for example, an acceleration sensor and a gyro sensor.

[0026] A specific operation of the correction unit 4 is as follows, for example. The image signal of the imaging region R1 acquired by the imaging element 31 includes the positional deviation between frames caused by the shaking of the camera 2. The control unit 6 measures the positional deviation between frames caused by the shaking detected by the shaking detection unit 4a, and cuts out the first cut-out region R2 or the second cut-out region R3 for each frame from the imaging region R1 so as to eliminate the measured positional deviation, as shown in FIG. 3. Here, the region cut out by the correction unit 4 is the first cut-out region R2 before the setting of the zoom center C described in detail later, and the second cut-out region R3 after the setting of the zoom center C. In this way, an image signal of a moving image (frame) in which the positional deviation between frames caused by the shaking is eliminated is obtained, and the image signal is recorded in the memory M or the recording medium ST.

[0027] When the first cropped region R2 or the second cropped region R3 is cropped from the imaging region R1 for each frame, distortion due to aberration occurring in the periphery of the imaging region R1 may be corrected at the same time. Although the rectangular first cropped region R2 and second cropped region R3 are illustrated in Fig. 3, their shapes are not limited to rectangular.

[0028] --Communications Department-- The communication unit 5 is configured, for example, by a wireless router. The communication unit 5 connects the camera 2 and the monitoring computer 7 to be able to communicate with each other via a wireless LAN such as Wi-Fi (registered trademark). This enables data transmission and reception between the camera 2 and the monitoring computer 7. In other words, the communication unit 5 can transmit an image signal acquired by the camera 2 from the memory M or the recording medium ST to the monitoring computer 7.

[0029] -Control section- The control unit 6 is electrically connected to the imaging unit 3, the vibration detection unit 4a, the communication unit 5, the memory M, the recording medium ST, and other components of the camera 2 (not shown). The operations of the camera 2, such as imaging, are controlled by signals input from the control unit 6. The control unit 6 can be realized using an IC, an LSI, an ASIC, or the like, similar to the signal processing unit 32. The control unit 6 and the signal processing unit 32 may be configured integrally with each other.

[0030] -Monitoring computer- The monitoring computer 7 receives the video image signal transmitted from the communication unit 5 of the camera 2. Note that data transmission and reception between the monitoring computer 7 and the communication unit 5 of the camera 2 may be performed via a cloud server (not shown).

[0031] The video of the image signal received by the monitoring computer 7 is displayed on the display device D and monitored by the monitor. The video displayed on the display device D is the shake-compensated first cropped region R2 or second cropped region R3. The monitor can control the operation of the camera 2 using a web application running on the browser of the monitoring computer 7. As will be described in detail later, the monitor can transmit a signal for controlling the camera 2 from the monitoring computer 7 to the communication unit 5 of the camera 2 while monitoring the video so as to adjust the zoom center C and image quality of the video. Adjusting the zoom center C and image quality here means setting or resetting the zoom center C and image quality.

[0032] In addition to controlling the camera 2 to adjust the zoom center C and image quality of the video, the monitoring computer 7 can also adjust the zoom center C and image quality of the video sent to the monitoring computer 7 without controlling the camera 2.

[0033] --Zoom center adjustment-- When the observer controls the camera 2 from the monitoring computer 7 to adjust the zoom center C of the video, a video showing the entire first cropped region R2 is initially displayed on the display device D on the observer's side. The observer designates the center of the object T to be focused on in the first cropped region R2 using an input means such as a mouse. Then, as shown in Fig. 3, the camera 2 is controlled so that an image of the second cropped region R3, in which the center of the object T is set as the zoom center C, is displayed on the display device D. The control of the camera 2 here differs depending on whether the image zoom process is performed electronically or optically, and in detail, for example, is performed as follows for each case.

[0034] When the image zoom process is performed electronically, the image signal read from the image sensor 31 to the memory M of the camera 2 is limited to a second cropped area R3 that is narrower than the first cropped area R2. This reduces the data size of the image signal recorded in the memory M. Therefore, there is no need to reduce the resolution of the image recorded in the memory M, thin out the data, or increase the compression rate so that the video can be played back smoothly on the display device D by the observer. As a result, a clear video of the second cropped area R3 that the observer wants to focus on can be played back smoothly on the display device D with little delay even if the observer zooms in.

[0035] When the image zoom process is performed optically, the imaging lens 30 of the camera 2 must be a zoom lens, and the subject imaged in the imaging region R1 is zoomed in by moving this zoom lens. In this case, the camera 2 is provided with a pan-tilt head (not shown), and the zoom center C is adjusted by moving the pan-tilt head. This enlarges the subject imaged in the imaging region R1 of the imaging element 31, reducing the data size of the image signal recorded in the memory M. As a result, a clear moving image that does not cause any problems even when zoomed in is played back smoothly on the display device D with little delay, as described above.

[0036] When a camera 2 capable of both electronic image zoom processing and optical image zoom processing is used, it is preferable to prioritize optical zoom processing over electronic zoom processing from the viewpoint of obtaining clearer moving images, but the user may be allowed to select which one to prioritize, or electronic zoom processing and optical zoom processing may be combined. Furthermore, in addition to optical zoom processing and electronic zoom processing by controlling the camera 2, electronic zoom processing of moving images on the monitoring computer 7 without controlling the camera 2 may be combined. In that case, it is preferable in terms of image quality to perform optical zoom processing up to the telephoto end (maximum magnification) of the optical zoom, and then perform electronic zoom processing to achieve a further zoom magnification.

[0037] The observer can control the camera 2 to adjust not only the position of the zoom center C but also the size of the first cropped region R2 or the second cropped region R3.

[0038] - Picture quality adjustment - Examples of the image quality of the video that the monitor can adjust by controlling the camera 2 from the monitoring computer 7 include color tone, brightness, and dynamic range. When adjusting the color tone of the video, for example, the monitor can control the camera 2 to optimize the white balance in the first cut-out region R2 or the second cut-out region R3 according to various scenes such as fluorescent light, sunlight, shade, and cloudy weather, or can fine-tune the white balance in the first cut-out region R2 or the second cut-out region R3. The camera 2 may also have an auto white balance (AWB) function so that the white balance is automatically optimized in the first cut-out region R2 or the second cut-out region R3. Note that the region of the image for adjusting the image quality does not have to be the first cut-out region R2 or the second cut-out region R3. For example, the region for adjusting the image quality may be cut out independently from the imaging region R1.

[0039] When the brightness of the video is adjusted, for example, the observer can control the exposure of the camera 2 in the first clipped region R2 or the second clipped region R3. Specifically, the observer can control the aperture, shutter speed, ISO sensitivity, etc. of the camera 2 in the first clipped region R2 or the second clipped region R3.

[0040] -Transmitting and receiving audio signals- The camera 2 is equipped with a microphone, and audio signals from the scene are detected by the microphone and transmitted to the monitoring computer 7 .

[0041] 1, the observer wears a headset, and the voice emitted from the observer is detected by the microphone of the headset and transmitted to the camera 2. The observer can also give voice instructions to the worker on which subject to photograph, in which direction to point the camera 2, etc. The observer's voice signal is transmitted from the monitoring computer 7 to the communication unit 5 of the camera 2.

[0042] - Imaging method - As shown in FIG. 4, an imaging method S according to the embodiment includes an imaging step S1 of capturing a moving image, a transmission step S2 of transmitting the moving image to a monitoring computer 7, and an adjustment step S3 in which a monitor adjusts the image while watching the moving image.

[0043] In the imaging step S1, a video is captured with shake correction by the camera 2 attached to the worker. In the camera 2, when the zoom center C has not been set by the observer, a shake-corrected first cropped region R2 is recorded in the memory M from the imaging area R1 acquired by the imaging element 31. After the zoom center C has been set by the observer, a shake-corrected second cropped region R3 is recorded in the memory M from the imaging area R1 acquired by the imaging element 31.

[0044] In the transmission step S2, the image signal recorded in the memory M of the camera 2 or in the recording medium ST is transmitted by the communication unit 5 to the monitoring computer .

[0045] The observer monitors the video displayed on the display device D of the monitoring computer 7, and judges whether or not the zoom center C or image quality of the video of the camera 2 needs to be adjusted (set or reset), and if it is judged that adjustment is necessary, executes the adjustment step S3. In the adjustment step S3, the observer uses the input means to operate the camera 2 from the monitoring computer 7 so as to adjust the image quality or zoom center C of the video. By the input from the input means, a signal for controlling the camera 2 is received by the communication unit 5 of the camera 2, and the camera 2 is controlled. The control of the camera 2 in the adjustment step S3 is reflected in the imaging step S1 in the subsequent cycle. Finally, the observer judges whether or not to continue imaging, and if imaging is continued, the above procedure is repeated.

[0046] The imaging method S according to the embodiment has been described above with reference to FIG. 4 as being executed in sequence, but one cycle of imaging step S1 to adjustment step S3 is executed almost simultaneously, except in cases where the communication speed between the communication unit 5 and the monitoring computer 7 is slow.

[0047] --Action and Effects-- Incidentally, the video can be adjusted to a certain extent as desired by the observer by only adjusting the zoom center C or image quality in the video transmitted to the monitoring computer 7 without controlling the camera 2. However, in order to play the video smoothly on the display device D, it is necessary to limit the data size of the video transmitted from the camera 2 to the monitoring computer 7 to a certain extent, so that the image signal recorded in the memory M of the camera 2 tends to be limited to the minimum necessary information (image area, image quality). In that case, the freedom of image adjustment that the observer can perform on the monitoring computer 7 is also limited. Specifically, in order to play the video smoothly on the display device D, the image area of ​​the transmitted video needs to be small, like the second cut-out area R3 in FIG. 3. However, when considering the case of adjusting the zoom center C in the video transmitted to the monitoring computer 7 without controlling the camera 2, the range in which the observer can specify the zoom center C is limited to the narrow second cut-out area R3. Furthermore, if one were to adjust the image quality of the video transmitted to the monitoring computer 7, such as the color tone, brightness, and dynamic range, without controlling the camera 2, the information would already be limited at the time of capture, and as a result, for example, whiteout or blackout would occur in the image, making it impossible to adjust the image quality as desired by the monitor. For the above reasons, being able to control the camera 2 itself at the time of capture, rather than adjusting a video that has already been captured, would provide a higher degree of freedom in adjusting the video, and would make it easier for the monitor to obtain the video that he or she desires.

[0048] According to this embodiment, the observer can control the camera 2 from a remote position RM to adjust the zoom center C and image quality of the captured video. Furthermore, since the video is stabilized by the camera 2, the observer can easily adjust the zoom center C and image quality while watching the video. As a result, the observer can easily obtain the video he or she desires.

[0049] Moreover, the color tone and brightness of a video are particularly susceptible to the influence of the surrounding environment of the subject. Therefore, there is a strong demand for adjusting at least one of the color tone and brightness according to the surrounding environment of the subject. Here, in the imaging system 1 according to this embodiment, the camera 2 can be controlled so as to adjust the white balance and exposure of the camera 2 by an adjustment unit operating on the monitoring computer 7. Therefore, the monitor can adjust the color tone of the video by controlling the white balance of the camera 2, and can adjust the brightness of the video by controlling the exposure of the camera 2. As a result, it becomes easier to obtain a video as the monitor desires.

[0050] Furthermore, when the imaging lens 30 is a wide-angle lens, the video can be zoomed in to match the target T that the observer wants to focus on in the wide-range imaging area R1 acquired through the wide-angle lens, and the image quality can be optimized. Therefore, when the imaging lens 30 is a wide-angle lens, the imaging area R1 becomes wider, so that the observer has more freedom to zoom in and select the target T that he or she wants to focus on.

[0051] Furthermore, according to this embodiment, for example, a worker working on-site can capture images of the site with the attached camera 2 while working, and a monitor can monitor the video at a remote location RM, which is convenient. The camera 2 is provided with a correction unit 4, and the captured video is stabilized for shake, so the worker does not need to keep the camera 2 still to capture the image, and can concentrate on the work on-site.

[0052] (Other embodiments) In the above embodiment, the moving body is a worker, but instead, the moving body may be a vehicle, heavy machinery, work robot, or other object other than a person. In addition, the imaging camera may be any object that is provided on the moving body and capable of capturing images, and may be a wearable camera other than the wearable camera 2 attached to a helmet as shown in FIG. 1. The imaging camera does not have to be a wearable camera, and may be, for example, a camera that is held by a person or a camera that can be attached to a vehicle, heavy machinery, work robot, or the like. The imaging camera may be, for example, a smartphone or tablet terminal equipped with a camera function.

[0053] In the above embodiment, the imaging camera and the adjustment unit transmit and receive image signals and audio signals wirelessly to and from each other. However, the imaging camera and the adjustment unit may transmit and receive image signals and audio signals wirelessly to and from each other.

[0054] In the above embodiment, the observer can adjust the zoom center C and image quality (white balance, exposure) of the moving image using the adjustment unit, but it is sufficient if at least one of the zoom center C and image quality can be adjusted.

[0055] In the above embodiment, the camera 2 is controlled by the monitoring computer 7, but in addition to this, the camera 2 may be controlled by directly operating it.

[0056] In the above embodiment, the correction unit 4 performs shake correction on the image signal of the imaging region R1 recorded in the memory M, but instead, shake correction may be performed at a stage before the image signal is recorded from the imaging element 31 to the memory M.

[0057] The adjustment unit in the present invention is only required to be able to control the camera 2 to adjust at least one of the zoom center C and the image quality of the video, and therefore is not limited to a web app that runs on a desktop computer as shown in FIG. 1, but may be software other than a web app, or may be software that runs on various computers such as a laptop computer, a smartphone, a tablet terminal, etc. [Industrial Applicability]

[0058] INDUSTRIAL APPLICABILITY As described above, the present disclosure is useful for imaging systems. [Explanation of symbols]

[0059] 1. Imaging system 2. Wearable camera (imaging camera) 3. Imaging unit 4. Correction section 4a Shake detection unit (correction unit) 5. Communications Department 6 Control section (correction section) 7 Monitoring computer (control unit) 30 Imaging lens 31 Image sensor 32 Signal Processing Section M Memory ST Recording Media S Imaging Method S1 Imaging step S2 Send Step S3 Adjustment Step RM Remote Position C Zoom center R1 Imaging area R2 1st cutting area R3 2nd cutting area

Claims

1. An imaging system for capturing video using an imaging camera installed in a moving object, comprising: An imaging unit provided in the imaging camera for capturing a moving image; a correction unit provided in the imaging camera for correcting shake of the moving image to be captured; a communication unit provided in the imaging camera, the communication unit transmitting the video image corrected by the correction unit to a remote location for monitoring at the remote location; an adjustment unit for controlling the imaging camera from a remote position of the imaging camera so as to adjust at least one of a zoom center and image quality of a moving image captured by the imaging camera; An imaging system comprising:

2. 2. The imaging system according to claim 1, An imaging system, wherein the moving object is a person and the imaging camera is a wearable camera.

3. 2. The imaging system according to claim 1, An imaging system, wherein the adjustment unit is capable of controlling the imaging camera so as to adjust at least one of a white balance and an exposure of the imaging camera.

4. An imaging method for capturing a moving image using an imaging camera provided on a moving object and having a correction unit for performing shake correction, comprising the steps of: The imaging camera captures a video while performing shake correction; The imaging method includes receiving the captured video at a remote position of the moving object, monitoring the video, and adjusting at least one of a zoom center and image quality of the video.

Images