Imaging device and information processing device, and control method thereof

Abstract

To provide an imaging device that, when a series of videos are divided and recorded into multiple files, can guarantee the authenticity of the combined video while reducing the processing load required for combining the divided videos. [Solution] This imaging device is capable of dividing and recording a series of video data into multiple files. The imaging device generates authenticity assurance data for the combined video data, which is obtained by combining the video data recorded in each of the multiple files in chronological order, and management information necessary for generating the file that records the combined video data. The imaging device then records the authenticity assurance data and management information along with the multiple files.

Description

[[Technical Field]]

[0001] The present invention relates to an imaging apparatus, an information processing apparatus, control methods therefor, and an imaging apparatus. [[Background Art]]

[0002] Improvements in the processing performance of smartphones, tablets and the like have enabled ordinary users to easily process images. In addition, processing technology has advanced through the use of AI and the like, making it difficult to determine whether an image has been processed or not. Against this background, technologies for guaranteeing the authenticity (that an image has not been altered) of images have been proposed.

[0003] In Patent Document 1, a digital camera generates a hash value of image data at the time of capturing and attaches the hash value to the image, thereby making it possible to verify whether or not the image has been altered since the time of capturing. [[Prior Art Document]] [[Patent Document]]

[0004] [[Patent Document 1]] Japanese Unexamined Patent Application Publication No. 2011-124663 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0005] When a series of moving images is recorded into a plurality of files in accordance with conditions, authenticity of each individual moving image can be verified by assigning a hash value of the recorded moving image to each file. However, when generating a file that stores a combined moving image obtained by combining moving images recorded in a plurality of files into one moving image, calculation of a hash value for the combined moving image and generation of management information required for the file are necessary, and thus the time and load required for file generation can become problematic.

[0006] In view of these problems, the present invention provides, in one embodiment, an imaging device and a control method therefor that, when a series of videos are divided and recorded into multiple files, can reduce the processing load required for combining the videos while ensuring the authenticity of the combined video. [Means for solving the problem]

[0007] In one embodiment, the present invention provides an imaging device comprising: an acquisition means for acquiring a series of video data; a recording means capable of dividing and recording the series of video data into multiple files; and a generation means for generating authenticity assurance data for combined video data obtained by combining the video data recorded in each of the multiple files in chronological order, and management information necessary for generating a file on which the combined video data is recorded, wherein the recording means records the authenticity assurance data and the management information together with the multiple files. [Effects of the Invention]

[0008] According to the present invention, when a series of videos are divided and recorded into multiple files, it is possible to provide an imaging device and a control method therefor that can reduce the processing load required for combining the videos while ensuring the authenticity of the combined video. [Brief explanation of the drawing]

[0009] [Figure 1] A schematic diagram showing an example configuration of an image processing system according to the embodiment. [Figure 2] Block diagram showing an example of the functional configuration of the imaging device 200. [Figure 3] Block diagram showing an example configuration of the information processing device 300. [Figure 4] This figure shows an example configuration of a split video file and a combined video file, both recorded using interleaved recording according to the first embodiment. [Figure 5] Flowchart showing an example of interleaved recording operation in the first embodiment [Figure 6] Flowchart relating to the file merging operation in the first embodiment [Figure 7] This figure shows an example of the configuration of a split video file recorded with file breaks according to the second embodiment and a combined video file. [Figure 8] Flowchart showing an example of file break recording operation in the second embodiment [Figure 9] A diagram showing an example of a method for splitting and recording video data. [Modes for carrying out the invention]

[0010] The present invention will be described in detail below with reference to the attached drawings, based on exemplary embodiments thereof. Note that the following embodiments do not limit the invention to the claims. Furthermore, while multiple features are described in the embodiments, not all of them are essential to the invention, and the multiple features may be combined arbitrarily. In addition, in the attached drawings, the same or similar configurations are given the same reference numeral, and redundant descriptions are omitted.

[0011] In the following description, the present invention will be implemented using a digital camera as an example of an imaging device. However, the present invention can be implemented with any electronic device capable of recording video. Such electronic devices include video cameras, computer equipment (personal computers, tablet computers, media players, PDAs, etc.), smartphones, game consoles, robots, drones, dashcams, and the like. These are examples, and the present invention can be implemented with other electronic devices as well.

[0012] ●(First Embodiment) <System Configuration> FIG. 1 is a schematic diagram showing a configuration example of an image processing system 100 according to the present invention. The image processing system 100 includes an imaging device 200 and an information processing device 300. The imaging device 200 generates a plurality of files in which moving image data is divided and recorded. The information processing device 300 combines the plurality of divided and recorded files. Here, it is assumed that a plurality of files are supplied from the imaging device 200 to the information processing device 300 via a recording medium 110 or a communication medium 120. In the former case, the imaging device 200 records files in the recording medium 110 such as a memory card, and the information processing device 300 reads the files from the recording medium 110. In the latter case, the imaging device 200 transmits the files to the information processing device 300 via the communication medium 120 such as a wireless LAN or a cable.

[0013] Further, although it is assumed here that the information processing device 300 combines the plurality of files, the imaging device 200 may perform the combining. Therefore, the information processing device 300 is not essential, and the communication medium 120 is also unnecessary when the information processing device 300 is not used.

[0014] <Internal Configuration of Imaging Device> FIG. 2 is a block diagram showing a hardware configuration example of the imaging device 200 as an example of the image processing apparatus according to the present invention. The MPU 201 is a microprocessor unit capable of executing programs. The functions of the imaging device 200 are realized by the MPU 201 reading a program stored in a ROM 210 into a RAM 209 and executing the program, and controlling the operations of other circuits as necessary.

[0015] A timing signal generation circuit 202 generates a timing signal necessary for the operation of an image sensor 203 and supplies the timing signal to the image sensor 203.

[0016] The image pickup device 203 may be, for example, a known CCD or CMOS color image sensor having a color filter of primary-color Bayer array. The image pickup device 203 includes a pixel array in which a plurality of pixels are two-dimensionally arranged, and a peripheral circuit for reading signals from each pixel. Each pixel has a photoelectric conversion region or a photoelectric conversion element, and accumulates electric charge according to the amount of incident light. By reading a signal having a voltage corresponding to the amount of electric charge accumulated during an exposure period from each pixel, a pixel signal group (analog image signal) representing a subject image formed on an imaging surface by an unillustrated optical system can be obtained.

[0017] The A / D converter 204 converts one frame of analog image signal read out from the image pickup device 203 into a digital image signal (image data). The image data may be still image data or moving image data, and in the following description, it is assumed to be moving image data. Note that although audio data is also attached to the moving image data, description of the handling of audio data is omitted.

[0018] The memory controller 205 controls reading and writing of moving image data, moving image files storing moving image data, and the like with respect to the buffer memory 206, and the refresh operation of the buffer memory 206, etc.

[0019] The buffer memory 206 is a memory for temporarily storing moving image data, moving image files, intermediate data, and the like.

[0020] The display unit 207 is provided on the surface of the housing of the image pickup apparatus 200, and displays live view video, reproduced images, user interfaces such as menu screens, settings of the image pickup apparatus 200, and the like. The display unit 207 may be, for example, a liquid crystal display or an organic EL display.

[0021] The recording medium I / F 208 is an interface for controlling reading and writing of data with respect to the recording medium 110. The recording medium 110 may be detachable like a memory card, for example. Further, a plurality of recording media 110 may be usable.

[0022] RAM209 is used as main memory for the MPU201 to load and execute programs, as well as video memory for the display unit 207, etc.

[0023] ROM210 is an electrically rewritable non-volatile memory. ROM210 stores programs executed by the MPU201, various settings for the imaging device 200, GUI data, and other similar information.

[0024] The hash generation unit 211 applies a known hash function such as SHA2 to the input data to generate a hash value for the input data. The hash value is an example of information used to guarantee the authenticity (that it has not been tampered with) of the source video data.

[0025] In this embodiment, the hash generation unit 211 is capable of splitting the input data for calculating the hash value. Splitting the input means inputting the data to be used to calculate the hash value in stages. The hash generation unit 211 performs hash value calculation on the input data up to the intermediate stage, but does not calculate the final hash value until a calculation instruction or notification of the end of the input data is received. Since the input data is no longer needed once the calculation process has reached the intermediate stage, it is not necessary to retain the entire video data from the start to the end of recording in order to calculate the hash value for video data being recorded, which has an undefined amount of data.

[0026] In this embodiment, hash values ​​are calculated for both the multiple video data recorded in segments and the video data obtained by combining the multiple video data recorded in segments. Therefore, if the hash generation unit 211 cannot perform hash value calculation processing for multiple independent input data in parallel, a separate hash generation unit 211 may be provided to calculate the individual hash values.

[0027] The communication unit 212 is a communication interface compliant with one or more wired and wireless communication standards (such as USB and wireless LAN). The imaging device 200 can communicate with external devices directly or via the communication medium 120 through the communication unit 212.

[0028] The video data splitting unit 213 splits a series of video data to record it into multiple video files. Figure 9 shows an example of a splitting method that the video data splitting unit 213 can perform. Video data 900 shows the first 12 frames of the video data before splitting. Here, each frame is assumed to be RAW data. RAW data is image data that has not undergone any development processing, and each pixel has luminance information for one of the RGB color components.

[0029] The development process is a series of image processing steps that the MPU201 applies to image data, specifically to convert RAW data into a general-purpose image data format that can be used for display, printing, and other purposes as photographic data. Generally, the development process includes white balance processing and color interpolation (demosaic processing), and may also include lens aberration correction, noise reduction (NR) processing, and gamma (tone conversion) processing.

[0030] The video data splitting unit 213 performs a splitting process according to the video recording method supported by the imaging device 200. Here, it is assumed that the imaging device 200 supports interleaved recording and file break recording as methods for recording video into multiple files.

[0031] Interleaved recording is a method of recording multiple frames that make up a video before splitting, by sequentially assigning them to multiple files to be recorded. For example, when interleaved recording is performed on two files, odd-numbered frames are recorded in one file and even-numbered frames are recorded in the other file. In the example shown in Figure 9, the original video data 900, which has 12 frames, is split and recorded into two separate video data 901: one recording the odd-numbered frames and the other recording the even-numbered frames.

[0032] File break recording is a method of recording video to a single file continuously until a predetermined condition is met, at which point the recording destination of the video is changed to a different file. The predetermined condition may be the duration, number of frames, or size of the recorded file, or the free space on the recording medium during recording. For example, if the predetermined condition is the recording time (30 minutes), a one-hour video will be split and recorded into a file recording the first 30 minutes and a file recording the next 30 minutes. In other words, file break recording can also be described as a method of recording video to multiple files in chronological order. In the example shown in Figure 9, the original video data 900, which has 12 frames, is split and recorded into two separate video data files: one recording the first 6 frames and another recording the last 6 frames.

[0033] Depending on the conditions and purpose, the individual files to be split and recorded may be recorded on separate recording media or on the same recording media. For example, if a file is split due to a decrease in the available space on the recording media, the individual files will be recorded on separate recording media. On the other hand, if a file is split at regular intervals, the individual files may be recorded on the same recording media. Also, in the case of interleaved recording for the purpose of high-speed recording, the individual files will be recorded on separate recording media. Multiple split and recorded files may share part of their filename, include information from other files related to metadata, and allow for easy identification of the necessary files and their order when combining them.

[0034] When compressing and encoding video data in GOP units, the frames in Figure 9 should be replaced with GOPs. In this case, the MPU201 applies development processing to each frame, and then compresses and encodes a predetermined number of consecutive frames as GOPs. Therefore, in interleaved recording, the video data is divided into two parts: one part containing video data corresponding to the odd-numbered GOPs, and another part containing video data corresponding to the even-numbered GOPs. Similarly, in file break recording, the file is split at the GOP boundaries.

[0035] The combined authenticity guarantee data generation unit 214 generates management information for the combined video data (or the video data before splitting) obtained by combining the video data split by the video data splitting unit 213 into a single video in chronological order. The combined authenticity guarantee data generation unit 214 also obtains a hash value as authenticity guarantee data for the combined video data (or the video data before splitting) using the hash generation unit 211. Note that, in addition to hash values, data conforming to standards for authenticity verification can be generated as authenticity guarantee data. The Coalition for Content Provenance and Authenticity (C2PA) is one example of such a standard. Therefore, the hash value in the following explanation can be replaced with other authenticity guarantee data.

[0036] The combined authenticity guarantee data addition unit 215 adds the management information and authenticity guarantee data generated by the combined authenticity guarantee data generation unit 214 to one or more of the video data divided by the video data division unit 213.

[0037] The authenticity guarantee data generation unit 216 generates management information for each video data that is divided and recorded in separate files by the video data division unit 213. The authenticity guarantee data generation unit 216 also obtains a hash value as authenticity guarantee data for each video data that is divided and recorded, using the hash generation unit 211.

[0038] The management information and authenticity guarantee data generated by the authenticity guarantee data generation unit 216 for each video data are recorded in the video file (split recording file) in which the video data is stored. On the other hand, for combined video data obtained by combining multiple split recordings of video data in chronological order, the management information and authenticity guarantee data generated by the combined authenticity guarantee data generation unit 214 are recorded in one or more of the split recording files.

[0039] Details of how the imaging device 200 operates when recording video in segments will be described later.

[0040] The operation unit 217 is a general term for input devices (buttons, switches, dials, etc.) provided for the user to input various instructions to the imaging device 200. The input devices constituting the operation unit 217 have names according to the function they are assigned to. For example, the operation unit 217 includes a release switch, a video recording switch, a shooting mode selection dial for selecting a shooting mode, a menu button, directional keys, and a select key. The release switch is a switch for still image recording, and the MPU 201 recognizes a half-pressed state of the release switch as a shooting preparation instruction and a fully pressed state as a shooting start instruction. In addition, the MPU 201 recognizes the video recording switch as a video recording start instruction when pressed in shooting standby mode, and as a recording stop instruction when pressed during video recording. The functions assigned to the same input device may be variable. Furthermore, the input devices may be software buttons or keys using a touch display. The MPU 201 may also include input devices that support non-contact input methods such as voice input and eye-tracking input.

[0041] Furthermore, the imaging device 200 performs video recording while in shooting standby mode, generates video data for display, and continuously displays it on the display unit 207, thereby enabling the display unit 207 to function as an electronic viewfinder (EVF). The video displayed to enable the display unit 207 to function as an EVF is called live view video.

[0042] In the shooting standby state, the MPU201 controls operations related to video recording and generates a live view video by applying development processing and resolution conversion processing to the video data stored in the buffer memory 206. The MPU201 also detects the subject area (e.g., the face area of ​​a person) from the live view video and generates information such as brightness information and focus accuracy to perform automatic exposure control (AE) and autofocus detection (AF).

[0043] Furthermore, when the MPU 201 detects a still image capture command via the operation unit 217, it generates still image data and a still image file containing the still image data, and records them on the recording medium 110. In addition, when the MPU 201 detects a video recording start command via the operation unit 217, it generates video data for recording and a video file containing the video data, and records them on the recording medium 110. The MPU 201 also performs encoding processing and additional information generation processing according to the recording format.

[0044] <Internal configuration of information processing device> Figure 3 is a block diagram showing an example of the hardware configuration of the information processing device 300. The information processing device 300 may be a general-purpose computer device such as a personal computer. However, any computer device capable of processing image data may be used as the information processing device 300. Therefore, for example, the imaging device 200 may be used as the information processing device 300. In this case, the imaging device 200 may have hardware corresponding to blocks 306 to 308 in Figure 3, or the MPU 201 may execute a program to perform the functions of blocks 306 to 308.

[0045] The MPU301 is a microprocessor unit capable of executing programs. The functions of the information processing device 300, described below, are realized by the MPU301 reading programs stored in the ROM303 into the RAM302, executing them, and controlling the operation of other circuits as needed.

[0046] RAM302 is used as main memory for the MPU301 to load and execute programs, and as video memory for a display unit (not shown). The display unit may be provided by the information processing device 300 or it may be an external display device.

[0047] ROM303 is an electrically rewritable non-volatile memory. ROM303 stores programs executed by the MPU301 (basic software and applications), various settings for the information processing unit 300, GUI data, etc. The information processing unit 300 may also have a large-capacity storage device such as a hard disk drive (HDD) or solid-state drive (SSD) in addition to ROM303. The basic software, applications, user data, etc., may be stored in the large-capacity storage device.

[0048] The recording medium I / F304 is an interface for controlling the reading and writing of data to the recording medium 110, which conforms to the same specifications as the recording medium used by the imaging device 200.

[0049] The communication unit 305 is a communication interface compliant with one or more wired and wireless communication standards (such as USB and wireless LAN). The information processing device 300 can communicate with external devices directly or via the communication medium 120 through the communication unit 305.

[0050] The video merging unit 306 combines multiple video data, which have been recorded separately and acquired through the recording medium I / F 304 or the communication medium 120, in chronological order to generate a single combined video data.

[0051] The additional data extraction unit 307 extracts management information and authenticity assurance data related to the combined video data that are attached to one or more video data combined by the video combining unit 306.

[0052] The combined video file generation unit 308 generates a combined video file using the management information and authenticity assurance data extracted by the supplemental data extraction unit 307 and the combined video data generated by the video merging unit 306.

[0053] The operation unit 309 consists of a group of input devices such as a keyboard, mouse, and touchpad. The MPU 301 detects operations performed by the operation unit 309 and executes processing corresponding to the detected operations.

[0054] <Interleaved recording operation> Next, using the flowchart shown in Figure 5, we will explain the operation of the imaging device 200 when it interleaves recording video data into two files at the GOP level, as an example of how the imaging device 200 divides and records video data. Here, we will focus particularly on the operation related to recording video files, and will only give a very brief explanation of known operations such as the generation of video data for recording.

[0055] Whether or not to split the video data for recording, and the type of split recording, may be determined according to user settings, or the MPU201 may decide based on factors such as frame rate and resolution.

[0056] The operation shown in the flowchart of Figure 5 is initiated when the MPU 201 detects a video recording start command via the operation unit 217 while in shooting standby mode.

[0057] In S501, the MPU201 initializes a counter for counting the number of GOPs. The counter may be a variable, for example, and its value can be stored in the RAM209. The initial value is 0. The MPU201 also creates two new video files (split video file 1 and split video file 2) for recording video data on the recording medium 110 and opens the files. In this embodiment, the video data corresponding to the odd-numbered GOPs of the generated recording video data is recorded in split video file 1, and the video data corresponding to the even-numbered GOPs is recorded in split video file 2. There are no particular restrictions on the number of frames that make up a GOP, but for example, it can be the reciprocal of the frame rate (fps) (number of frames per second) or half of that.

[0058] In S502, the MPU201 determines whether or not it has detected a command to end video recording via the operation unit 217. If it determines that it has detected the command, it executes S513; otherwise, it executes S503.

[0059] In S503, the MPU201 starts controlling the operation of the image sensor 203 to perform 1 GOP (Group of Pictures) worth of imaging.

[0060] In S504, the MPU201 generates encoded video data for 1 GOP. Specifically, for each frame constituting 1 GOP, the A / D converter 204 converts the analog image signal read from the image sensor 203 into image data, and the memory controller 205 stores the image data in the buffer memory 206. Each time that image data for one frame is stored in the buffer memory 206, the MPU201 applies image processing, including development processing, to the image data. Once the application of image processing to the image data for 1 GOP is complete, the MPU201 applies encoding processing using inter-frame prediction (for example, encoding processing compliant with the MPEG4 standard) to generate encoded video data for 1 GOP.

[0061] Furthermore, the combined authenticity assurance data generation unit 214 generates or updates management information for the combined video data, reflecting the generated encoded video data. The combined authenticity assurance data generation unit 214 stores and updates the generated management information in the RAM 209. The management information is metadata that indicates what data is stored in the video file and how. Here, the video data is recorded in a file compliant with a container format such as ISO / IEC 14496-12 or ISO / IEC 14496-14. In this case, the combined authenticity assurance data generation unit 214 generates management information based on the content to be recorded in the moov box. The combined authenticity assurance data generation unit 214 generates management information according to the file format in which the video data is recorded.

[0062] Furthermore, the MPU201 increments the counter value by one. The MPU201 then inputs the encoded video data for one GOP (Group of Processes) into the hash generation unit 211. The hash generation unit 211 processes the input encoded video data as split input data for calculating the hash value of the combined video data and as split input data for calculating the hash value of the image data recorded in split video file 1 or 2. The hash generation unit 211 determines, based on the counter value, whether to process the data as split input data for split video file 1 or 2. If a hash generation unit 211 is provided for each hash value calculated by the hash generation unit 211, the MPU201 selects the hash generation unit 211 supplying the encoded video data according to the counter value.

[0063] In S505, MPU201 determines whether the counter value is odd or not. If it is determined to be odd, it executes S506; otherwise, it executes S508.

[0064] In S506, the MPU201 records the encoded video data of the odd-numbered GOPs stored in buffer memory 206 into the segmented video file 1. The recorded encoded image data may be deleted from buffer memory 206.

[0065] In S507, the authenticity assurance data generation unit 216 generates management information that reflects the encoded video data recorded in S506. Then, the MPU 201 updates the management information recorded in the segmented video file 1 with the management information generated by the authenticity assurance data generation unit 216.

[0066] In S508, the MPU201 records the encoded video data of the even-numbered GOPs stored in buffer memory 206 into the segmented video file 2. The recorded encoded image data may be deleted from buffer memory 206.

[0067] In S509, the authenticity guarantee data generation unit 216 generates management information that reflects the encoded video data recorded in S508. Then, the MPU 201 updates the management information recorded in the segmented video file 2 with the management information generated by the authenticity guarantee data generation unit 216.

[0068] Note that management information does not need to be recorded in the segmented video files until recording is complete. In this case, the authenticity guarantee data generation unit 216 stores the management information in, for example, RAM 209, and updates the management information stored in RAM 209 in S507 and S508. The authenticity guarantee data generation unit 216 generates and updates management information in the same way as the combined authenticity guarantee data generation unit 214.

[0069] The MPU201 repeatedly executes the processes in S503-S509 on a GOP basis until it determines that it has detected an instruction to end recording in S502.

[0070] If it is determined in S502 that a recording termination instruction has been detected, in S510 the Authenticity Assurance Data Generation Unit 216 instructs the Hash Generation Unit 211 to calculate hash values ​​for the divided video file 1 and the divided video file 2. The Hash Generation Unit 211 calculates a hash value for the entire encoded image data that has been divided and input so far for each of the divided video file 1 and the divided video file 2, and outputs it to the Authenticity Assurance Data Generation Unit 216.

[0071] In S511, the MPU201 obtains hash values ​​from the authenticity assurance data generation unit 216 to be added to the split video file 1 and the split video file 2, and adds them to each split video file.

[0072] In S512, the authenticity guarantee data generation unit 216 instructs the hash generation unit 211 to calculate a hash value for the combined video data. The hash generation unit 211 calculates a hash value for the combined video data for the entirety of the encoded image data that has been divided and input so far, and outputs it to the authenticity guarantee data generation unit 216. Note that in S510, the hash value for the combined video data may also be obtained, in which case S512 is unnecessary.

[0073] In S513, the MPU201 adds a hash value and management information for the combined video data to at least one of the split video files 1 and 2. The MPU201 may also record the hash value and management information for the combined video data in a separate file from the split video files. In this case, the MPU201 may use a specific filename for the file containing the hash value and management information for the combined video data, or record information about split video files 1 and 2 in the metadata, so that the relationship with split video files 1 and 2 can be determined.

[0074] In S514, MPU201 closes split video file 1 and split video file 2.

[0075] Alternatively, instead of saving the segmented video files 1 and 2 to the recording medium 110, or in addition to saving them to the recording medium 110, the segmented video files 1 and 2 may be transmitted to an external device via the communication unit 212.

[0076] <Example of video file structure> Figure 4 shows the structure of interleaved segmented video files and a combined video file created by combining these video files. Here, odd-numbered GOPs are recorded in segmented video file 400, and even-numbered GOPs are recorded in segmented video file 410. The combined video file 420 is created by combining segmented video files 400 and 410 so that the GOPs are arranged in order. Note that if interleaved recording is performed on a frame-by-frame basis, the GOPs in the figure will be replaced with frames.

[0077] The segmented video file 400 contains authenticity guarantee data 401, video management information 402, video data 403, and metadata 404. Authenticity assurance data 401 is data that guarantees the authenticity (that it has not been tampered with) of video data 403, and in this case, it is the hash value of video data 403. Video management information 402 is information that indicates the format and storage location of the video data 403. Video data 403 contains encoded video data corresponding to odd-numbered GOPs. Metadata 404 is the metadata for the split video file 400. Here, it is assumed that, during split recording, management information and authenticity assurance data regarding the combined video data are recorded only in the split video file 400, out of the two split video files 400 and 410.

[0078] The combined authenticity guarantee data 405 is data that guarantees the authenticity of the combined video data 423, and in this case, it is the hash value of the combined video data 423. The combined video management information 406 is the management information for the combined video file 420. In this way, by recording management information for the composite video file containing the combined image data, as well as authenticity guarantee data, in one or more of the multiple video files generated during the split recording process, it becomes unnecessary to generate this information when combining the image data. Therefore, the authenticity of the combined video data can be guaranteed while generating the composite video file at high speed.

[0079] The split video file 410 contains authenticity assurance data 411, video management information 412, video data 413, and metadata 414. It is the same as the split video file 400, except that the video data 413 contains encoded video data corresponding to even-numbered GOPs, and the metadata 414 does not contain information about the combined video file.

[0080] The combined video file 420 contains combined authenticity assurance data 405, combined video management information 406, combined video data 423, and combined metadata 424. The combined video data 423 contains encoded video data obtained by combining the video data 403 and 413 stored in the split video files 400 and 410 so that the GOPs are arranged in order. The combined authenticity assurance data 405 and the combined video management information 406 are copied from the metadata 404 of the split video file 400. Note that the combined authenticity assurance data 405 and the combined video management information 406 may be copied from files other than the split video files.

[0081] <Combining interleaved video files> Next, using the flowchart shown in Figure 6, an example of the operation by which the information processing device 300 generates a combined video file 420 from the divided video files 400 and 410 will be explained. Here, it is assumed that the divided video files 400 and 410 are recorded on the same recording medium 110. However, the divided video files 400 and 410 may be recorded on different recording media, or they may be obtained through the communication unit 305.

[0082] In S601, MPU301 creates a new combined video file 420 on the recording medium 110 and opens the file.

[0083] In step S602, the MPU 301 opens the split video files 400 and 410 recorded on the recording medium 110 and begins acquiring the encoded video data. The MPU 301 can automatically detect, for example, that the split video files 400 and 410 are files to be combined based on their filenames. Alternatively, the user may instruct the MPU 301 via the operation unit 309 that the split video files 400 and 410 are files to be combined.

[0084] Based on the management information of the split video files 400 and 410, the MPU301 recognizes that each video file contains encoded image data for odd-numbered GOPs and encoded image data for even-numbered GOPs. The MPU301 also recognizes that the metadata 404 of the split video file 400 contains management information for the combined video file and authenticity assurance data for the combined video data, and that it is not necessary to generate this information during the merging process. In this way, the MPU301 determines how to combine the image data based on the management information of the multiple video files to be combined, and also determines whether it is necessary to generate the management information and authenticity assurance data to be used in the combined video file.

[0085] In S603, the additional data extraction unit 307 extracts the combined authenticity guarantee data 405 from the metadata 404 of the segmented video file 400 and adds it to the combined video file 420. In S604, the additional data extraction unit 307 extracts the combined video management information 406 from the metadata 404 of the segmented video file 400 and adds it to the combined video file 420. Alternatively, the combined authenticity guarantee data 405 and the combined video management information 406 may be extracted together in S603 and added to the combined video file 420; in that case, S604 is unnecessary.

[0086] In step S605, the MPU301 adds the encoded video data stored in the split video file 400 and the encoded video data stored in the split video file 410 to the combined video data 423 in chronological order.

[0087] In S606, MPU301 closes the combined video file 420.

[0088] As described above, according to this embodiment, when video data is divided and recorded, the necessary management information and authenticity assurance data are generated and recorded in the combined video file that stores the combined video data obtained by combining the divided video data. Therefore, it becomes possible to perform the combining process at high speed while guaranteeing the authenticity of the divided and recorded video data.

[0089] ●(Second Embodiment) Next, a second embodiment of the present invention will be described. This embodiment performs file break recording. Since the configuration of the imaging device 200 and the information processing device 300 can be the same as in the first embodiment, the differences from the first embodiment will be explained in detail.

[0090] <File break recording operation> Next, the operation of the imaging device 200 when recording video data via file break will be explained using the flowchart shown in Figure 8. Similar to the first embodiment, this explanation will focus particularly on the operation related to recording video files, and known operations such as the generation of video data for recording will be explained only very briefly. Furthermore, the file break conditions may be set by the user or determined by the MPU 201.

[0091] As described above, file breaks are possible under various conditions, but here, when the size of the encoded video data stored in one video file reaches a predetermined size (threshold), the encoded image data is stored in a new image file. Furthermore, all divided image files are recorded on the recording medium 110.

[0092] The operation shown in the flowchart of Figure 8 is initiated when the MPU 201 detects a video recording start command via the operation unit 217 while in shooting standby mode.

[0093] In S801, the MPU201 creates a new video file on the recording medium 110 and opens the file. At this point, it is not yet certain that a file break will occur, but for convenience, the image file created here will be called segmented video file 1. The MPU201 also initializes a counter that counts the cumulative size of the encoded video data. The counter may be a variable, for example, and its value can be stored in RAM209. The initial value is set to 0.

[0094] In S802, the MPU201 determines whether or not it has detected a command to end video recording via the operation unit 217. If it determines that it has detected the command, it executes S810; otherwise, it executes S803.

[0095] In S803, the MPU201 starts controlling the operation of the image sensor 203 to perform 1 GOP (Group of Pictures) worth of image capture.

[0096] In S804, the MPU201 generates encoded video data for 1 GOP in the same manner as in S504. The MPU201 then adds the size of the encoded video data to a counter. The combined authenticity guarantee data generation unit 214 generates or updates management information about the combined video data that reflects the generated encoded video data. Furthermore, the MPU201 inputs the generated 1 GOP of encoded video data to the hash generation unit 211. The hash generation unit 211 processes the input encoded video data as split input data for calculating the hash value of the combined video data and as split input data for calculating the hash value of the image data recorded in the split video files being recorded.

[0097] In S805, the MPU201 determines whether the cumulative data size of the currently recorded encoded video data has reached a predetermined value. If it determines that the value has been reached, it executes S806; otherwise, it executes S808.

[0098] In S806, the MPU201 adds the encoded video data generated in S804 to the segmented video file 1. The authenticity guarantee data generation unit 216 generates management information that reflects the added encoded video data. Then, the MPU201 updates the management information recorded in the segmented video file 1 with the management information generated by the authenticity guarantee data generation unit 216.

[0099] Furthermore, the authenticity guarantee data generation unit 216 instructs the hash generation unit 211 to calculate a hash value for the encoded video data of the divided video file. The hash generation unit 211 calculates a hash value for the entirety of the encoded image data that has been input for the divided video file and outputs it to the authenticity guarantee data generation unit 216. The MPU 201 adds the hash value obtained from the authenticity guarantee data generation unit 216 to the divided video file 1.

[0100] After that, MPU201 closes the split video file 1 and initializes the counter to 0.

[0101] In S807, MPU201 creates a new video file (split video file 2) on the recording medium 110 and opens the file.

[0102] In S808, MPU201 appends the encoded video data generated by S804 to the currently recording segmented video file.

[0103] In S809, the authenticity guarantee data generation unit 216 generates management information that reflects the encoded video data added in S808. Then, the MPU 201 updates the management information recorded in the segmented video file being recorded with the management information generated by the authenticity guarantee data generation unit 216.

[0104] The MPU201 repeatedly executes the processes in S803-S809 on a GOP basis until it determines that it has detected an instruction to end recording in S802. If the size of the encoded image data to be stored in the currently recorded segmented video file reaches a predetermined limit, a new segmented video file is generated and the encoded video data is added to the new segmented video file.

[0105] If it is determined in S802 that a recording termination instruction has been detected, in S810 the authenticity assurance data generation unit 216 instructs the hash generation unit 211 to calculate a hash value for the segmented video file being recorded. The hash generation unit 211 calculates a hash value for the entire encoded image data that has been input so far for the segmented video file being recorded, and outputs it to the authenticity assurance data generation unit 216.

[0106] In S811, the MPU201 obtains a hash value from the authenticity assurance data generation unit 216 to be added to the segmented video file being recorded, and adds it to the segmented video file.

[0107] In S812, the authenticity guarantee data generation unit 216 instructs the hash generation unit 211 to calculate a hash value for the combined video data. The hash generation unit 211 calculates a hash value for the combined video data for the entirety of the encoded image data that has been input so far and outputs it to the authenticity guarantee data generation unit 216. Note that the hash value for the combined video data may also be obtained in S810, in which case S812 is unnecessary.

[0108] In S813, the MPU201 adds the hash value and management information for the combined video data to the currently recorded segmented video file. Alternatively, the MPU201 may record the hash value and management information for the combined video data in a separate file from the segmented video file. In this case, the MPU201 can determine the relationship between the file containing the hash value and management information for the combined video data and the recorded segmented video file.

[0109] In S814, MPU201 closes the segmented video file that is being recorded.

[0110] Alternatively, instead of saving the segmented video files to the recording medium 110, or in addition to saving them to the recording medium 110, the segmented video files may be transmitted to an external device via the communication unit 212.

[0111] <Example of video file structure> Figure 7 shows the structure of split video files recorded using file breaks and the combined video file created by combining these video files. Here, it is assumed that the encoded video data is recorded in the order of split video file 700 and split video file 710. Furthermore, the combined video file 720 is created by combining split video file 700 and split video file 710 so that the GOPs are arranged in order. Note that if the data is not encoded at the GOP level, the GOPs in the figure will be replaced with frames.

[0112] The segmented video file 700 contains authenticity assurance data 701, video management information 702, video data 703, and metadata 704. Authenticity assurance data 701 is data that guarantees the authenticity (that it has not been tampered with) of video data 703, and in this case, it is the hash value of video data 703. Video management information 702 is information that indicates the format and storage location of the video data 703. Video data 703 contains encoded video data from the start of recording until the file break condition is met. Metadata 704 is the metadata for the split video file 700.

[0113] The split video file 710 contains authenticity assurance data 711, video management information 712, video data 713, and metadata 714. It is the same as the split video file 700, except that the video data 713 contains encoded video data corresponding to the GOP following the video data 703, and the metadata 714 contains information about the combined video file.

[0114] In this case, since split video file 710 is the last split video file recorded, the management information and authenticity assurance data regarding the combined video data are recorded only in split video file 710, out of split video files 700 and 710.

[0115] The combined authenticity guarantee data 715 is data that guarantees the authenticity of the combined video data 723, and in this case, it is the hash value of the combined video data 723. The combined video management information 716 is the management information for the combined video file 720.

[0116] Thus, according to this embodiment, when a file break is recorded, the management information of the composite video file containing the composited image data and authenticity assurance data are recorded in the last recorded video file. Therefore, the same effects as in the first embodiment can be achieved even when a file break is recorded.

[0117] The combined video file 720 contains combined authenticity guarantee data 715, combined video management information 716, combined video data 723, and combined metadata 724. The combined video data 723 contains encoded video data obtained by combining the video data 703 and 713 stored in the split video files 700 and 710 so that the GOPs are arranged in order. The combined authenticity guarantee data 715 and the combined video management information 716 are copied from the metadata 714 of the split video file 710. Note that the combined authenticity guarantee data 715 and the combined video management information 716 may be copied from files other than the split video files.

[0118] <Combining split video files recorded via file breaks> Furthermore, the operation by which the information processing device 300 generates a combined video file 720 from the divided video files 700 and 710 is substantially the same as the operation of combining interleaved recorded divided video files described with reference to Figure 6 in the first embodiment.

[0119] Specifically, by replacing the split video files 400 and 410 in Figure 6 with the split video files 700 and 710, a combined video file 720 can be generated.

[0120] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0121] The disclosure of this embodiment includes the following imaging device, information processing device, method for controlling the imaging device, method for controlling the information processing device, and program. (Item 1) A means for acquiring a series of video data, A recording means capable of dividing and recording the aforementioned series of video data into multiple files, The system includes a generation means for generating authenticity assurance data for combined video data obtained by combining video data recorded in each of the aforementioned multiple files in chronological order, and management information necessary for generating a file that records the combined video data. The recording means records the authenticity assurance data and the management information together with the plurality of files. An imaging device characterized by the following features. (Item 2) The imaging apparatus according to item 1, characterized in that the recording means interleaves the series of video data and records them in the plurality of files. (Item 3) The imaging device according to item 2, characterized in that the recording means records the authenticity assurance data and the management information in one or more of the plurality of files. (Item 4) The imaging device according to item 1, characterized in that the recording means records the series of video data into the plurality of files in a file break format. (Item 5) The imaging apparatus according to item 4, characterized in that the recording means records the authenticity assurance data and the management information in the last of the plurality of files to be recorded. (Item 6) The imaging apparatus according to item 1, characterized in that the recording means records the authenticity assurance data and the management information in a file separate from the plurality of files. (Item 7) The imaging device according to any one of items 1 to 6, characterized in that the aforementioned authenticity assurance data is a hash value. (Item 8) An acquisition means for acquiring a file recorded by an imaging device described in any one of items 1 to 7, A combining means for generating combined video data by combining the video data recorded in each of the multiple files, which are obtained by dividing and recording a series of video data, in chronological order, An extraction means for extracting the authenticity assurance data and the management information from any of the acquired files, A generation means that generates an image file containing the combined video data using the combined video data, the authenticity assurance data, and the management information, An information processing device characterized by having the following features. (Item 9) The information processing device according to item 8, characterized in that the extraction means extracts the authenticity assurance data and the management information from one of the plurality of files. (Item 10) The information processing device according to item 8, characterized in that the extraction means extracts the authenticity assurance data and the management information from a file separate from the plurality of files. (Item 11) A control method performed by an imaging device, The process of splitting a series of video data into multiple files and recording them together, The system includes generating authenticity assurance data for combined video data obtained by combining the video data recorded in each of the aforementioned multiple files in chronological order, and generating management information necessary for generating the file on which the combined video data is recorded. The aforementioned split recording includes recording the authenticity assurance data and the management information together with the plurality of files. A control method for an imaging device, characterized by the following: (Item 12) A control method executed by an information processing device, To obtain a file recorded by an imaging device described in any one of items 1 through 7, The process involves combining the video data recorded in each of the multiple files, which contain a series of video data split into separate files, in chronological order to generate combined video data. Extract the authenticity assurance data and the management information from one of the acquired files, Using the combined video data, the authenticity assurance data, and the management information, an image file containing the combined video data is generated. A control method for an information processing device, characterized by having the following features. (Item 13) A program to cause the computer of the imaging device to function as one of the means of the imaging device described in any one of items 1 to 7. (Item 14) A program that causes a computer to function as one of the means of the information processing device described in item 8.

[0122] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0123] 200...Imaging device, 201...MPU, 203...Image sensor, 211...Hash generation unit, 213...Video data splitting unit, 214...Combined authenticity guaranteed data generation unit, 215...Combined authenticity guaranteed data addition unit, 216...Authenticity guaranteed data generation unit

Claims

1. A means for acquiring a series of video data, A recording means capable of dividing and recording the aforementioned series of video data into multiple files, The system includes a generation means for generating authenticity assurance data for combined video data obtained by combining video data recorded in each of the aforementioned multiple files in chronological order, and management information necessary for generating a file that records the combined video data. The recording means records the authenticity assurance data and the management information together with the plurality of files. An imaging device characterized by the following features.

2. The imaging apparatus according to claim 1, characterized in that the recording means interleaves the series of video data and records them in the plurality of files.

3. The imaging apparatus according to claim 2, characterized in that the recording means records the authenticity assurance data and the management information in one or more of the plurality of files.

4. The imaging apparatus according to claim 1, characterized in that the recording means records the series of video data into the plurality of files in a file break format.

5. The imaging apparatus according to claim 4, characterized in that the recording means records the authenticity assurance data and the management information in the last of the plurality of files to be recorded.

6. The imaging apparatus according to claim 1, characterized in that the recording means records the authenticity assurance data and the management information in a file separate from the plurality of files.

7. The imaging apparatus according to claim 1, characterized in that the authenticity assurance data is a hash value.

8. An acquisition means for acquiring a file recorded by an imaging device described in any one of claims 1 to 7, A combining means for generating combined video data by combining the video data recorded in each of the multiple files, which are obtained by dividing and recording a series of video data, in chronological order, An extraction means for extracting the authenticity assurance data and the management information from any of the acquired files, A generation means that generates an image file containing the combined video data using the combined video data, the authenticity assurance data, and the management information, An information processing device characterized by having the following features.

9. The information processing apparatus according to claim 8, characterized in that the extraction means extracts the authenticity assurance data and the management information from one of the plurality of files.

10. The information processing apparatus according to claim 8, characterized in that the extraction means extracts the authenticity assurance data and the management information from a file separate from the plurality of files.

11. A control method performed by an imaging device, The process of splitting a series of video data into multiple files and recording them together, The system includes generating authenticity assurance data for combined video data obtained by combining the video data recorded in each of the aforementioned multiple files in chronological order, and generating management information necessary for generating the file on which the combined video data is recorded. The aforementioned split recording includes recording the authenticity assurance data and the management information together with the plurality of files. A control method for an imaging device, characterized by the following:

12. A control method executed by an information processing device, Acquiring a file recorded by an imaging device described in any one of claims 1 to 7, The process involves combining the video data recorded in each of the multiple files, which contain a series of video data split into separate files, in chronological order to generate combined video data. Extract the authenticity assurance data and the management information from one of the acquired files, Using the combined video data, the authenticity assurance data, and the management information, an image file containing the combined video data is generated. A control method for an information processing device, characterized by having the following features.

13. A program for causing the computer of an imaging device to function as each of the means of the imaging device described in any one of claims 1 to 7.

14. A program for causing a computer to function as one of the means of the information processing device described in claim 8.

Images