Imaging device and video data distribution method

Abstract

To provide an imaging device capable of redistributing video data on which a watermark image is superimposed and which has been distributed to a network once, while reproducing timing of superimposition of the watermark image.SOLUTION: A distribution photographic image processing unit 31 generates photographic image data for distribution. A watermark image superimposing unit 32 superimposes a watermark image on a frame of the photographic image data for distribution. An output unit 35 distributes the photographic image data for distribution on which the watermark image is superimposed. A data processing unit 41 generates watermark superimposition auxiliary information including at least information indicating timing at which the watermark image superimposing unit 32 superimposed the watermark image on the photographic image data for distribution. A recording photographic image processing unit 221 generates photographic image data for recording. A recording reproduction processing unit 23 adds additional data based on the watermark superimposition auxiliary information to the photographic image data for recording and records it on a recording medium 24.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging device and a method for distributing moving image data. [Background technology]

[0002] An imaging device may superimpose a watermark image onto video data captured by an imaging unit, and record the superimposed image onto a recording medium or distribute the superimposed image over a network (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-328747 A Summary of the Invention [Problem to be solved by the invention]

[0004] An imaging device may superimpose a watermark image onto video data at a predetermined timing operated by a user and distribute the superimposed video data over a network. There are cases where a user wishes to redistribute video data that has already been distributed over a network by reproducing the timing at which the watermark image was superimposed. In such cases, it is difficult for a user to instruct the superimposition of the watermark image at the same timing as the initial distribution. Thus, it is conceivable to record the video data for distribution, on which the watermark image has been superimposed at a predetermined timing, on a recording medium.

[0005] The imaging device records moving image data captured by the imaging unit on a recording medium. It is not preferable for the imaging device to record moving image data for distribution on a recording medium in addition to the moving image data captured by the imaging unit, since this requires more recording capacity than necessary. Moving image data for distribution is usually scaled to reduce the number of lines and pixels compared to the moving image data recorded on the recording medium. Therefore, it is not preferable to record only moving image data for distribution on a recording medium.

[0006] The present invention aims to provide an imaging device and a method for distributing video data that can re-distribute video data that has once been distributed over a network with a watermark image superimposed at a predetermined timing, while reproducing the timing of the watermark image superimposition. [Means for solving the problem]

[0007] The present invention includes an imaging unit that captures an image of a subject and generates captured image data, an image data storage unit that stores the captured image data, a captured image processing unit for distribution that generates captured image data for distribution based on the captured image data read from the image data storage unit, and a watermark image superimposing unit that superimposes a watermark image on a frame of the captured image data for distribution. The above an output unit that outputs the photographed image data for distribution on which the watermark image is superimposed to an external device so as to be distributed; The above The present invention provides an imaging device comprising: a data processing unit that generates watermark overlay auxiliary information including at least information indicating the timing at which a watermark image was overlaid; a recording image processing unit that generates recording image data based on the captured image data read from the image data holding unit; and a recording processing unit that adds additional data based on the watermark overlay auxiliary information to the recording image data and records the resulting image on a recording medium.

[0008] In the present invention, a captured image for distribution processing unit generates captured image data for distribution, a watermark image superimposing unit superimposes a watermark image on a frame of the captured image data for distribution, and an output unit The above The photographed image data for distribution on which the watermark image is superimposed is distributed, and a data processing unit performs the watermark image superimposition on the photographed image data for distribution. The abovea recording processing unit adds additional data based on the watermark superimposition auxiliary information to the photographed image data for recording and records the additional data on a recording medium; a reproduction processing unit reproduces the photographed image data for recording to which the additional data recorded on the recording medium has been added to generate reproduced image data; the photographed image processing unit for distribution generates reproduced image data for re-distribution based on the reproduced image data; and the watermark image superimposition unit extracts the additional data from the additional data. The above The watermark superimposition auxiliary information is added to the redistribution reproduction image data at a timing based on the watermark superimposition auxiliary information. The above A watermark image is superimposed, and the output unit The above The present invention provides a method for distributing moving image data, which redistributes the redistribution reproduction image data on which a watermark image is superimposed. Effect of the Invention

[0009] According to the imaging device and video data distribution method of the present invention, video data that has once been distributed over a network with a watermark image superimposed at a predetermined timing can be re-distributed while reproducing the timing of the watermark image superimposition. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an imaging apparatus according to an embodiment; [Figure 2A] FIG. 13 is a diagram conceptually showing an alpha channel set for the watermark image Wm1. [Figure 2B] FIG. 13 is a diagram conceptually showing an alpha channel set for a watermark image Wm2. [Diagram 3] 2 is a diagram showing one frame of captured image data for distribution D31 generated by the captured image for distribution processing unit 31 in FIG. 1. FIG. [Figure 4]2 is a diagram showing one frame of the captured image data for distribution D32 on which the watermark image Wm1 has been superimposed by the watermark image superimposing unit 32 in FIG. [Diagram 5] 13 is a diagram showing an example of the timing at which the watermark image superimposing unit 32 superimposes the watermark image on the captured image data D31 for distribution. FIG. [Figure 6] 2 is a diagram conceptually showing one frame of recordable photographed image data D221 generated by the recording and playback processing unit 23 in FIG. 1. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an imaging device and a method for distributing moving image data according to an embodiment will be described with reference to the accompanying drawings.

[0012] 1, the imaging unit 11 includes an imaging element and a plurality of lenses, and captures an image of a subject with an angle of view of, for example, an aspect ratio of 16:9 to generate captured image data. The imaging element may be a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image data storage unit 12 temporarily stores the captured image data output from the imaging unit 11. The image data storage unit 12 is a frame memory, and may be an SDRAM (Synchronous Dynamic Random Access Memory) or a DDR (Double-Data-Rate SDRAM).

[0013] The captured image for distribution processing unit 31 performs scaling processing to reduce the number of lines and pixels on the captured image data D12 read out from the image data storage unit 12, and performs noise reduction processing as necessary to generate captured image data for distribution D31. The captured image for distribution processing unit 31 may perform clipping processing to cut out a part of each frame of the captured image data D12.

[0014] The watermark image storage unit 33 stores watermark image data to be superimposed on the captured image data for distribution D31. The watermark image storage unit 33 is configured with a non-volatile memory. In the example shown in Fig. 1, the watermark image storage unit 33 stores watermark image data indicating two watermark images Wm1 and Wm2. The watermark image storage unit 33 may store watermark image data indicating one watermark image.

[0015] The watermark image data is composed of image data in a format having auxiliary data called an alpha channel that sets the opacity of the image. As an example, the watermark image data is in PNG format.

[0016] Fig. 2A conceptually shows the alpha channel set for the watermark image Wm1 indicated by the watermark image data. As an example, the background Wm00 in a rectangular area is set to 0% opacity (i.e., 100% transparency), and the mark Wm101 is set to 100% opacity (i.e., 0% transparency). In the example shown in Fig. 2A, the mark Wm101 is the characters "W.MK."

[0017] Fig. 2B conceptually shows the alpha channel set for the watermark image Wm2 indicated by the watermark image data. Similarly, the background Wm00 within the rectangular area is set to 0% opacity, and the mark Wm201 is set to 100% opacity. In the example shown in Fig. 2B, the mark Wm201 is the letters "ABC."

[0018] Operation data instructing the user to superimpose or stop the superimposition of a watermark image is input to the watermark image superimposing unit 32. When the watermark image storage unit 33 stores watermark image data of a plurality of watermark images, operation data including information for selecting a watermark image is input to the watermark image superimposing unit 32.

[0019] The watermark image superimposing unit 32 superimposes watermark image data indicating the selected watermark image on the photographed image data for distribution D31 at a predetermined timing in accordance with operation data input by a user operation, thereby generating photographed image data for distribution D32. The controller that supplies the operation data to the watermark image superimposing unit 32 may be a remote controller attached to the imaging device, or may be a mobile terminal such as a smartphone that communicates with the imaging device.

[0020] Fig. 3 shows one frame of the photographed image data for distribution D31 output from the photographed image for distribution processing unit 31. Fig. 4 shows one frame of the photographed image data for distribution D32 on which the watermark image Wm1 is superimposed by the watermark image superimposition unit 32. Because the background Wm00 is set to an opacity of 0% by the α channel, it appears that only the mark Wm101 with an opacity of 100% is superimposed on the photographed image data for distribution D32.

[0021] The watermark image superimposition unit 32 may automatically determine the position at which to superimpose the watermark image Wm1 or Wm2 on the distribution-purpose captured image data D31, or may set the position at which to superimpose the watermark image Wm1 or Wm2 based on operation data operated by the user.

[0022] The data processing unit 41 processes data to generate watermark superimposition auxiliary information indicating how the watermark image data has been superimposed on the captured image data for distribution D31 by the watermark image superimposition unit 32. Specifically, the data processing unit 41 generates the watermark superimposition auxiliary information including information indicating the timing at which the watermark image data was superimposed, information indicating the timing at which the superimposition was stopped, and the selected watermark image data.

[0023] 5, for example, suppose that a user performs an operation to superimpose a watermark image Wm1 at time t1, a predetermined time after time t0 which is the start timing of the distribution captured image data D31, and performs an operation to stop the superimposition at time t2. Also suppose that the user performs an operation to superimpose a watermark image Wm2 at time t3, and performs an operation to stop the superimposition at time t4.

[0024] In this case, the data processing unit 41 generates watermark superimposition auxiliary information including time t1 when the watermark image data was superimposed, time t2 when the superimposition was stopped, and watermark image data indicating the watermark image Wm1. The data processing unit 41 also generates watermark superimposition auxiliary information including time t3 when the watermark image data was superimposed, time t4 when the superimposition was stopped, and watermark image data indicating the watermark image Wm2. The data processing unit 41 supplies the generated watermark superimposition auxiliary information to the recording and playback processing unit 23. The recording and playback processing unit 23 includes a recording processing unit and a playback processing unit.

[0025] The timing when the watermark image data is superimposed and the timing when the superimposition is stopped may be represented by a time after the start time of the photographed image data for distribution D31, or may be represented by a frame number obtained by counting the frames after the start time of the photographed image data for distribution D31. The watermark superimposition auxiliary information may include a designation number for designating the watermark image Wm1 or Wm2 instead of including the watermark image data as it is. If the designation number is used, the data capacity of the watermark superimposition auxiliary information can be reduced.

[0026] 1, the output processing unit 34 encodes the captured image data for distribution D32, performs protocol packetization processing and SDI (Serial Digital Interface) serialization processing as necessary, and supplies the encoded captured image data for distribution D32 to the output unit 35. The output unit 35 outputs the encoded captured image data for distribution D32 supplied from the output processing unit 34 to a network external to the imaging device. The image data distributed via the network has a watermark image Wm1 or Wm2 superimposed thereon at the timing shown in FIG.

[0027] The recording captured image processing unit 221 performs at least one of scaling and noise reduction processing on the captured image data D12 read out from the image data storage unit 12 to generate recording captured image data D221. As an example of scaling processing, when the captured image data output from the imaging unit 11 has a horizontal pixel count x vertical pixel count of 3840 x 2160, called 4K, the recording captured image processing unit 221 reduces the pixel count to 1920 x 1080 for full HD. When the capacity of the recording medium 24 is sufficiently large, the recording captured image processing unit 221 does not need to reduce the captured image data.

[0028] The number of lines and the number of pixels of the captured image data for recording D221 are greater than the number of lines and the number of pixels of the captured image data for distribution D31.

[0029] The recording and playback processing unit 23 encodes and files the recording captured image data D221 output from the recording captured image processing unit 221 in order to record the recording captured image data D221 on the recording medium 24. At this time, the recording and playback processing unit 23 generates additional data (metadata) based on the watermark superimposition auxiliary information supplied from the data processing unit 41 and adds the data on a frame-by-frame basis.

[0030] Fig. 6 conceptually shows one frame of the filed recordable captured image data D221 generated by the recording and playback processing unit 23. If the frame shown in Fig. 6 is the frame at the timing when the watermark image data is superimposed, the additional data includes information indicating the start of the superimposition of the watermark image data. If the frame shown in Fig. 6 is the frame at the timing when the superimposition of the watermark image data is stopped, the additional data includes information indicating the stop of the superimposition of the watermark image data.

[0031] A file recorded on the recording medium 24 may be composed of a clip file made up of a header, a body, and a footer. The header stores basic information such as the length of the clip and the compression method used for encoding. The body stores image data and audio data in units of frames. Additional data may be added to each frame of the body. The footer stores additional data to be added to the entire clip.

[0032] For example, assume that the superimposition of watermark image data begins in frame number 100 and ends in frame number 800. Additional data indicating a value indicating the start of superimposition of watermark image data is added to the body frame with frame number 100. Additional data indicating a value indicating the end of superimposition of watermark image data is added to the body frame with frame number 800.

[0033] The additional data may include information indicating the position within the frame where the watermark image Wm1 or Wm2 is to be superimposed. For example, suppose that the position where the watermark image Wm1 or Wm2 is to be superimposed in the frame with frame number 100 is specified as pixel position 10 in the horizontal direction and line number 10 in the vertical direction. Suppose that the position where the watermark image Wm1 or Wm2 is to be superimposed in the frame with frame number 500 is changed to pixel position 100 in the horizontal direction and line number 100 in the vertical direction. In this case, a value indicating the change in the superimposition position of the watermark image data and additional data indicating the changed superimposition position are added to the frame with frame number 500 of the body.

[0034] The footer stores watermark image data indicating the watermark image Wm1 or Wm2. The watermark image data is not stored as additional data for each frame, but is stored in the footer of one clip file, so the data size of the additional data does not become very large, and the data size of the clip file does not become very large. When storing both the watermark image Wm1 and the watermark image Wm2 in the footer, a designation number that designates either the watermark image Wm1 or the watermark image Wm2 can be added to the additional data added to the frame of the body.

[0035] In this way, the recording and playback processing unit 23 records the filed recordable photographed image data D221 including the additional data on the recording medium 24. The recording medium 24 is any recording medium, and may be a memory card constituted by a flash memory or a hard disk drive.

[0036] The format of the additional data when the record-use photographed image data D221 is converted into a file is not limited to the above example. A pair of the frame number where the superimposition of the watermark image data started and a value indicating the start of the superimposition, a pair of the frame number where the superimposition of the watermark image data stopped and a value indicating the stop of the superimposition, and the watermark image data may be stored together in the footer. This makes it easier to search for the additional data.

[0037] Furthermore, additional data may be added such that if watermark image data is superimposed on each frame, the value is 1, and if no watermark image data is superimposed, the value is 0. In this case, as described above, the additional data may include a designation number that designates the watermark image Wm1 or Wm2.

[0038] Next, a description will be given of the re-distribution process performed after the for-distribution captured image processing unit 31 through the output unit 35 distribute the for-distribution captured image data D32 on which a watermark image is superimposed in response to a user operation.

[0039] 1, the recording and playback processing unit 23 reads out filed photographed image data for recording D221 including additional data recorded on the recording medium 24 from the recording medium 24. The recording and playback processing unit 23 decodes the read photographed image data for recording D221 and supplies the decoded playback image data to the playback image processing unit 222. The recording and playback processing unit 23 also extracts watermark superimposition auxiliary information from the additional data contained in the decoded photographed image data for recording D221 and supplies the watermark superimposition auxiliary information to the data processing unit 41. The data processing unit 41 holds the watermark superimposition auxiliary information.

[0040] The playback image processing unit 222 processes the playback image data supplied from the recording and playback processing unit 23 so as to become image data in the same format as the captured image data supplied from the imaging unit 11 to and stored in the image data storage unit 12. When the recording captured image processing unit 221 performs scaling processing to reduce the number of lines and the number of pixels on the captured image data D12, the playback image processing unit 222 performs scaling processing to increase (restore) the number of lines and the number of pixels. The image data storage unit 12 temporarily stores the playback image data D222 supplied from the playback image processing unit 222.

[0041] The captured image for distribution processing unit 31 performs scaling processing to reduce the number of lines and pixels on the playback image data D222 read out from the image data storage unit 12, and performs noise reduction processing as necessary to generate playback image data for re-distribution D31r. The captured image for distribution processing unit 31 may perform clipping processing to cut out a part of each frame of the playback image data D222.

[0042] 5 on the redistribution reproduction image data D31r, based on the watermark superposition auxiliary information held in the data processing unit 41, to generate redistribution reproduction image data D32r. If the watermark superposition auxiliary information held in the data processing unit 41 includes a designation number for designating the watermark image Wm1 or Wm2, the watermark image superposition unit 32 may read out the watermark image data from the watermark image holding unit 33 in accordance with the designation number and superimpose it on the redistribution reproduction image data D31r.

[0043] The output processing unit 34 encodes the redistribution reproduction image data D32r, performs protocol packetization processing and SDI serialization processing as necessary, and supplies the data to the output unit 35. The output unit 35 outputs the encoded redistribution reproduction image data D32r supplied from the output processing unit 34 to the network.

[0044] As described above, according to the imaging device and video data distribution method of this embodiment, video data that has been once distributed over a network with a watermark image superimposed at a predetermined timing can be re-distributed while reproducing the timing of the superimposition of the watermark image. According to the imaging device and video data distribution method of this embodiment, it is not necessary to record the video data for distribution (captured image data for distribution D32) on the recording medium 24.

[0045] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present invention. [Explanation of symbols]

[0046] 11 Imaging unit 12 Image data storage unit 23 Recording and playback processing section (recording processing section, playback processing section) 24 Recording media 31 Distribution image capture processing unit 32 Watermark image overlay section 33 Watermark image storage unit 34 Output Processing Section 35 Output section 41 Data Processing Unit 221 Recording image processing section 222 Reproduction image processing section

Claims

1. an imaging unit that captures an image of a subject and generates captured image data; an image data storage unit that stores the captured image data; a distribution-purpose captured image processing unit that generates distribution-purpose captured image data based on the captured image data read from the image data storage unit; a watermark image superimposing unit that superimposes a watermark image on a frame of the photographed image data for distribution; an output unit that outputs the photographed image data for distribution on which the watermark image is superimposed to an external device for distribution; a data processing unit that generates watermark superimposition auxiliary information including at least information indicating a timing at which the watermark image superimposition unit superimposed the watermark image on the captured image data for distribution; a record-use photographed image processing unit that generates record-use photographed image data based on the photographed image data read from the image data storage unit; a recording processing unit that adds additional data based on the watermark superimposition auxiliary information to the photographic image data for recording and records the result on a recording medium; An imaging device comprising:

2. a reproduction processing unit that reproduces the record-use photographed image data to which the additional data recorded on the recording medium has been added; a reproduction image processing unit that processes reproduction image data reproduced by the reproduction processing unit; Further equipped with the image data storage unit stores the reproduced image data processed by the reproduced image processing unit, the distribution captured image processing unit generates re-distribution reproduction image data based on the reproduction image data read from the image data storage unit; the watermark image superimposing unit superimposes the watermark image on the re-distribution reproduction image data at a timing based on the watermark superimposing auxiliary information extracted from the additional data; The output unit outputs the redistribution reproduction image data on which the watermark image is superimposed to an external device for redistribution. The imaging device according to claim 1 .

3. The imaging device of claim 2, wherein the watermark overlay auxiliary information includes data of the watermark image superimposed on the photographed image data for distribution and the playback image data for re-distribution, or a designation number for designating data of the watermark image superimposed on the photographed image data for distribution and the playback image data for re-distribution among a plurality of data of the watermark image.

4. A distribution-purpose captured image processing unit generates distribution-purpose captured image data, a watermark image superimposing unit superimposing a watermark image on a frame of the photographed image data for distribution; an output unit that distributes the photographed image data for distribution on which the watermark image is superimposed; a data processing unit generating watermark superimposition auxiliary information including at least information indicating a timing at which the watermark image superimposition unit superimposed the watermark image on the captured image data for distribution; a recording processing unit adding additional data based on the watermark superimposition auxiliary information to the photographic image data for recording and recording the resulting image data on a recording medium; a reproduction processing unit that reproduces the recording photographed image data to which the additional data recorded on the recording medium is added to generate reproduced image data; the distribution captured image processing unit generates re-distribution reproduction image data based on the reproduction image data; the watermark image superimposing unit superimposes the watermark image on the re-distribution reproduction image data at a timing based on the watermark superimposing auxiliary information extracted from the additional data; The output unit re-distributes the re-distribution reproduction image data on which the watermark image is superimposed. A method for distributing video data.

Images