Imaging device, information processing method, and program

Abstract

To suppress an increase in processing load related to a calculation of a summary value by adaptively changing an object for the calculation of the summary value in ensuring the reliability of an image.SOLUTION: An information processing method includes the steps of: generating encoded data by encoding a captured image; generating a decoded image by decoding the encoded data; determining which of the data corresponding to a predetermined area in the decoded image and the encoded data is to be used as an object for the summary value calculation; generating a summary value by executing the summary value calculation on the data determined as the object for the summary value calculation; and generating distribution data including information based on the summary value and the encoded data.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing method. [Background technology]

[0002] Systems that can deliver images captured by an imaging device to a remote location are becoming widespread. One feature of captured image data is that it can be modified or altered using an information processing device such as a personal computer. Because of this feature, there are various technologies to ensure the reliability of digital images when they are used as evidence.

[0003] In Patent Document 1, a summary value is calculated from pixel data of a region of interest in a video image, and partial reliability is assigned to the video image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-027503 Summary of the Invention [Problem to be solved by the invention]

[0005] In general, the computational complexity of the digest value calculation depends on the size n of the input value that serves as the key, and can be set to O(n). When the input value is image data, the computational complexity of the digest value calculation increases in proportion to the size of the image data, i.e., the size n of the input value. Regarding this, Patent Document 1 proposes limiting the region in which the digest value calculation is performed to reduce the computational complexity, but the digest value calculation is performed uniformly on image data before encoding. Therefore, even if the computational complexity of the digest value calculation for image data after encoding (hereinafter, "encoded data") is actually smaller, in Patent Document 1, the digest value calculation is performed on image data before encoding, which can result in an increased processing load related to the digest value calculation.

[0006] Therefore, the present invention aims to suppress an increase in the processing load associated with the calculation of a digest value by adaptively changing the target of the calculation of the digest value in order to ensure the reliability of the image. [Means for solving the problem]

[0007] In order to solve the above problems, an imaging device of the present invention has the following configuration: encoding means for generating coded data by encoding a captured image, decoding means for generating a decoded image by decoding the coded data, determining means for determining which of data corresponding to a predetermined area in the decoded image and the coded data should be used for digest value calculation, calculating means for generating a digest value by performing the digest value calculation on the data determined by the determining means as the data to be used for digest value calculation, and generating means for generating distribution data including information based on the digest value and the coded data. The determining means compares the size of the encoded data with the size of data corresponding to the predetermined area in the decoded image, and determines the target of the digest value calculation according to the comparison result. do. [Effects of the Invention]

[0008] According to the present invention, in order to ensure the reliability of an image, the target of the summary value calculation is adaptively changed, thereby making it possible to suppress an increase in the processing load associated with the summary value calculation. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a system configuration. [Figure 2] FIG. 1 is a diagram illustrating a tampering determination system. [Figure 3] FIG. 2 is a diagram illustrating an example of functional blocks of the imaging device. [Figure 4] FIG. 2 is a diagram illustrating an example of functional blocks of the client device 101. [Figure 5] 10 is a flowchart showing the flow of information processing of the imaging device. [Figure 6] FIG. 1 is a diagram illustrating an example of a data structure in ARSEI. [Figure 7]10 is a flowchart showing the flow of metadata generation processing. [Figure 8] FIG. 2 is a diagram for explaining information displayed on a display. [Figure 9] FIG. 2 illustrates an example of a hardware configuration of each device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples and are not limited to the configurations shown in the drawings.

[0011] (Embodiment 1) 1 is a diagram showing the system configuration of this embodiment. The system of this embodiment includes an image capture device 100, a client device 101, a display 103, and a network .

[0012] The imaging device 100 and the client device 101 are connected to each other via a network 102. The network 102 is realized by a plurality of routers, switches, cables, etc. that comply with a communication standard such as ETHERNET (registered trademark).

[0013] The network 102 may be realized by the Internet, a wired local area network (LAN), a wireless LAN, a wide area network (WAN), or the like.

[0014] The imaging device 100 is a device that captures images. The imaging device 100 transmits distribution data including coded data obtained by encoding the captured images to an external device such as a client device 101 via a network 102. The client device 101 is, for example, an information processing device such as a personal computer installed with a program for realizing the processing functions described below.

[0015] The display 103 is configured with an LCD (Liquid Crystal Display) or the like, and displays a decoded image obtained by the client device 102 decoding encoded data included in the distribution data transmitted from the imaging device 100. The display 103 is connected to the client device 101 via a display cable that complies with a communication standard such as HDMI (High Definition Multimedia Interface), a registered trademark. The display 103 and the client device 101 may be provided in a single housing.

[0016] A conventional method for determining whether an image has been tampered with will now be described with reference to FIG. 2. FIG. 2 is a conceptual diagram of a system for determining whether an image has been tampered with. In the example shown in FIG. 2, the imaging device 100 first performs a digest value calculation on data of an image 200 captured by the imaging device 100 using a known hash function such as MD5 (Message Digest 5) or SHA-1 (Secure Hash Algorithm 1) to generate a digest value (hash value) 201. The imaging device 100 then encrypts the digest value 201 to generate an encrypted digest value 202, which is the encrypted digest value. Note that in this embodiment, a public key cryptosystem is used as the encryption method. Specifically, the client device 101 generates a public key paired with a private key stored in advance and transmits the public key to the imaging device 100. The imaging device 100 then encrypts the digest value 201 using the transmitted public key to generate the encrypted digest value 202. Thereafter, the image capturing device 100 generates distribution data 203 including the encrypted digest value 202 and the data of the image 200 , and transmits the distribution data 203 to the client device 101 .

[0017] The client device 101 receives the distribution data 203 transmitted from the imaging device 100 and separates it into image 200 data and an encrypted digest value 202. The client device 101 generates a digest value 201 by decrypting the encrypted digest value using a private key that is a pair with the public key used by the imaging device 100 for encryption. The digest value 201 generated by the client device 101 is used as the correct data for tampering detection. The client device 101 also performs digest value calculation on the image 200 data separated from the distribution data 203 using the same method as the imaging device 100. For example, if the imaging device 100 performed digest value calculation using the MD5 hash function, the client device 101 also performs digest value calculation using the MD5 hash function. The client device 101 then compares the digest value 201 corresponding to the correct data with the digest value 204 obtained by the client device 101 when calculating the digest value for the image 200. At this time, if the digest values ​​are the same, the client device 101 determines that the image 200 has not been tampered with during the transmission of the distribution data 203 from the imaging device 100 to the client device 101, and if the digest values ​​are different, it determines that tampering has occurred. Tampering with image data can be determined by the method described above with reference to Figure 2.

[0018] Now, with the increasing performance of image sensors and growing user needs, image capture devices used for monitoring, video streaming, etc. are increasingly being designed to have higher resolution and frame rates. As the resolution of captured images increases and the frame rate increases, the processing load associated with the above-described calculation of the summary value may increase.

[0019] Furthermore, when encoding images using encoding formats such as H264 or HEVC, frames may be composed of intraframes, which are compressed using spatial redundancy, and interframes, which are compressed using spatial and temporal redundancy. Consider a case where an image is encoded to obtain coded data, the coded data is decoded to obtain a decoded image, and a summary value is calculated for pixel value data of a partial region of the decoded image. In this case, if the coded data of the image to be processed is an interframe, the summary value calculation must be performed not only on the partial region of the image to be processed, but also on the partial region of the image of the reference frame, which may increase the processing load associated with the summary value calculation.

[0020] The process for suppressing an increase in the processing load associated with the above-described digest value calculation will be described with reference to the functional blocks shown in Fig. 3. Fig. 3 shows the functional blocks of the image capture device 100 according to this embodiment. Note that the functions of the functional blocks shown in Fig. 3 are realized by a CPU (Central Processing Unit) 900 of the image capture device 100 executing a computer program stored in a ROM (Read Only Memory) 920 of the image capture device 100, which will be described later with reference to Fig. 9, for example.

[0021] The imaging unit 101 captures an image of a subject using an imaging element such as a CCD (charge coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor, and generates an electrical signal by photoelectrically converting the captured image. The imaging unit 101 then generates an image by converting the photoelectrically converted electrical signal into a digital signal. The encoding unit 302 generates encoded data by performing a compression encoding process on the image captured by the imaging unit 101, such as H264 or HEVC, which combines intraframes and interframes.

[0022] The area detection unit 303 detects an important area (predetermined area) from the image input from the imaging unit 101, and outputs position information of the important area to the cropping unit 305. Here, the important area is an area that is important within the image, such as an area of ​​a person's face or an area of ​​a car's license plate in the image. The important area may be a partial area within the image, or the entire area of ​​the image.

[0023] The decoding unit 304 generates a decoded image by decoding the encoded data compressed and encoded by the encoding unit 302. The video format of the generated decoded image is assumed to be the same as the video format of the image captured by the imaging unit 301. Examples of such video formats include 4:4:4 format, 4:2:2 format, and 4:2:0 format. The cropping unit 305 extracts pixel value data in an important region based on the position information input from the region detection unit 303 from the decoded image input from the decoding unit 304, stores the extracted pixel value data as one-dimensional binary data, and outputs the one-dimensional binary data to the determination unit 306. The determination unit 306 determines which of the encoded data input from the encoding unit 302 and the one-dimensional binary data input from the cropping unit 305 should be used for digest calculation. For example, if the input encoded data to be processed corresponds to an interframe (P frame), the processing load associated with calculating the digest value for the decoded image data obtained by decoding the encoded data may increase. Therefore, when the coded data to be processed corresponds to an interframe, the determining unit 306 determines the one-dimensional binary data input by the clipping unit 105 as the target for digest value calculation. When the coded data to be processed corresponds to an intraframe (I frame), the determining unit 306 compares the size (amount of information) of the coded data with the size (amount of information) of one-dimensional binary data corresponding to an important region in the decoded image input by the clipping unit 105. The determining unit 306 then determines the coded data or the one-dimensional binary data corresponding to the important region, whichever has the smaller size (amount of information), as the target for digest value calculation. The determining unit 306 outputs the data determined as the target for digest value calculation to the digest value calculation unit 307. The encoding unit 302 may divide the image captured by the imaging unit 301 into tiles or slices according to the position of the important region, and perform compression encoding on the image, thereby enabling the important region to be decoded independently. In this case, the determination unit 306 may compare the size (amount of information) of the encoded data with the size (amount of information) of the data of a partial image (hereinafter referred to as a partial decoded image) in which the important area is independently decoded.

[0024] The digest value calculation unit 307 generates a digest value by performing digest value calculation on the data input from the determination unit 306 using a known hash function such as MD5 or SHA-1. The encryption unit 308 encrypts the digest value calculated by the digest value calculation unit 307 using a public key paired with a private key held by the client device 101 to generate an encrypted digest value, and outputs the encrypted digest value to the metadata generation unit 309. The metadata generation unit 309 generates metadata including the encrypted digest value generated by the encryption unit 308 and outputs the metadata to the distribution data generation unit 309. Note that if the data used for the digest value calculation is an important area in the decoded image, the metadata also includes position information of the important area. The distribution data generation unit 309 generates distribution data including the coded data input from the coding unit 302 and the metadata input from the metadata generation unit 108. The transmission unit 311 transmits the distribution data generated by the distribution data generation unit 310 to an external device such as the client device 101.

[0025] Next, processing of the client device 101 in this embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of functional blocks of the client device 101 in this embodiment. Note that the functions of the functional blocks shown in Fig. 4 are realized by a CPU (Central Processing Unit) 900 of the client device 101 executing a computer program stored in a ROM (Read Only Memory) 920 of the client device 101, which will be described later with reference to Fig. 9, for example.

[0026] First, the receiving unit 401 receives distribution data transmitted from the imaging device 100 and outputs it to the data separating unit 402. Next, the decoding unit 402 outputs metadata included in the distribution data to the metadata analyzing unit 403, and also generates a decoded image by decoding the coded data included in the distribution data, and outputs the generated decoded image to the digest value calculating unit 405 and the display control unit 408.

[0027] The metadata analysis unit 403 analyzes the output metadata and obtains at least an encrypted digest value. When a digest value is calculated for an important region in the decoded image, the metadata contains location information about the important region, so the metadata analysis unit 403 obtains the location information. Next, the calculation region determination unit 404 determines the data to be used for the digest value calculation. For example, if the metadata contains location information about the important region, the calculation region determination unit 404 determines that the data to be used for the digest value calculation is the data corresponding to the important region in the decoded image. On the other hand, if the metadata does not contain location information about the important region, the calculation region determination unit 404 determines the encoded data to be used for the digest value calculation.

[0028] When the calculation region determination unit 404 determines that data corresponding to an important region of the decoded image is to be subjected to digest value calculation, the digest value calculation unit 405 performs the following process. That is, it performs digest value calculation on one-dimensional binary data of pixel value data of an important region based on position information acquired in the image decoded by the decoding unit 402. On the other hand, when the calculation region determination unit 404 determines that coded data is to be subjected to digest value calculation, the digest value calculation unit 405 performs digest value calculation on the coded data. Note that the method of digest value calculation performed by the digest value calculation unit 405 is assumed to be the same as the method of digest value calculation performed by the digest value calculation unit 307 of the imaging device 100.

[0029] The encryption / decryption unit 407 generates a digest value corresponding to the correct data by decrypting the encrypted digest value acquired by the metadata analysis unit 403 using a private key that is paired with the public key used by the encryption unit 308 of the image capture device 100. The generated digest value is transmitted to the discrimination unit 406.

[0030] The determination unit 406 compares the digest value corresponding to the correct data output from the encryption / decryption unit 407 with the digest value output from the digest value calculation unit 405. If the digest values ​​are the same, the determination unit 406 determines that no tampering has occurred with the image during the process of transmitting the distribution data from the imaging device 100 to the client device 101. On the other hand, if the digest values ​​are different, the determination unit 406 determines that tampering has occurred.

[0031] The display control unit 408 causes the display 103 to display the decoded image decoded by the decoding unit 402. The display control unit 408 also causes the display 103 to display information (such as a message) indicating the determination result of the determination unit 406 as to whether or not tampering has occurred.

[0032] Next, processing of the imaging device 100 in this embodiment will be described with reference to the processing flow shown in Fig. 5. The processing flow shown in Fig. 5 is executed by the functional blocks of the imaging device 100 shown in Fig. 3, which are realized when the CPU 900 of the imaging device 100 executes a computer program stored in the ROM 920 of the imaging device 100, for example. The processing flow shown in Fig. 5 is a flowchart showing the processing flow from when the imaging device acquires (captures) an image to when distribution data related to the image is distributed. The processing shown in Fig. 5 is performed each time the imaging device 100 is powered on, appropriate settings are made, and an image is periodically generated at a certain timing.

[0033] First, in S501, an image is acquired by imaging using the imaging unit 301. Next, in S502, the encoding unit 302 generates encoded data by performing compression encoding processing on the image acquired by the imaging unit 301. The encoded data generated at this time is referred to as data A. Next, in S503, the decoding unit 304 generates a decoded image by performing decoding processing on the encoded data obtained by the compression encoding processing in S502. Next, in S504, the area detection unit 303 executes processing to detect an important area in the image acquired by the imaging unit 301. If an important area is detected (Yes in S504), the process proceeds to S505, and if an important area is not detected (No in S504), the process proceeds to S511. If an important area is detected, position information of the important area in the image is output to the cropping unit 305. If the user has set the entire image to be the important area in advance, the area detection unit 303 may not detect the important area in S504, but the entire image may be uniformly considered to have been detected as the important area, and the process may proceed to S505.

[0034] Next, in S505, the cropping unit 305 extracts pixel value data in the important region from the decoded image based on the position information input from the region detection unit 303, stores the extracted pixel value data as one-dimensional binary data, and outputs the one-dimensional binary data to the determination unit 306. The one-dimensional binary data generated at this time is referred to as data B.

[0035] Next, in S506, the determination unit 306 determines whether the encoded data currently being processed is an intraframe. If it is determined to be an intraframe (Yes in S506), the process proceeds to S507. On the other hand, if it is determined not to be an intraframe (it is an interframe) (No in S506), the process proceeds to S509. In S507, the determination unit 306 determines whether the size of the one-dimensional binary data of the important region in the decoded image corresponding to data B is smaller than the size of the encoded data corresponding to data A. If it is determined that the size of the one-dimensional binary data corresponding to data B is smaller than the size of the encoded data corresponding to data A (Yes in S507), the process proceeds to S508. On the other hand, if it is determined that the size of the one-dimensional binary data corresponding to data B is not smaller than the size of the encoded data corresponding to data A (No in S507), the process proceeds to S509. In S508, the digest value calculation unit 307 performs digest value calculation on the one-dimensional binary data of the important region in the decoded image currently being processed corresponding to data B, and generates a digest value. In S509, the digest value calculation unit 307 executes the digest value calculation process on the coded data corresponding to the data A currently being processed, and generates a digest value.

[0036] Next, in S510, the encryption unit 308 encrypts the digest value generated in S508 or S509 using a public key paired with the private key held by the client device 101, thereby generating an encrypted digest value.

[0037] Next, in S511, the metadata generation unit 309 generates metadata including the encrypted digest value generated in S510, and generates distribution data including the encoded data corresponding to the image currently being processed. Note that if the result in S504 is No, in S511 the metadata generation unit 309 generates distribution data including the encoded data.

[0038] Next, in S512, the transmission unit 311 transmits the distribution data generated in S511 to an external device such as the client device 101.

[0039] As described above, the imaging device 100 of this embodiment performs the following process when executing the digest value calculation to calculate the digest value used in tamper determination to ensure the reliability of the image. That is, the imaging device 100 adaptively changes the data on which the digest value calculation is performed depending on the size of the data, the prediction mode (intra or inter) of the encoded data, etc. In this way, it is possible to suppress an increase in the processing load related to the digest value calculation.

[0040] (Embodiment 2) In the second embodiment, the encoding unit 302 and the decoding unit 304 perform encoding and decoding processes in accordance with the HEVC standard, respectively. Another feature of the second embodiment is that the metadata generation unit 108 uses Annotated Region SEI (hereinafter referred to as ARSEI). ARSEI is an HEVC standard that makes it possible to include information such as data indicating the type and position of an object within the angle of view in a stream as metadata. Note that this embodiment will mainly describe the differences from the first embodiment, and components and processes that are the same as or equivalent to those in the first embodiment will be denoted by the same reference numerals, and redundant description will be omitted.

[0041] Fig. 6 shows the data structure of ARSEI used in this embodiment. Hereinafter, how to substitute metadata used in embodiment 2 into the data structure of ARSEI will be described with reference to Fig. 6.

[0042] The data structure 601 shown in Figure 6 is a pseudocode representation of the ARSEI data structure. In the data structure 601, the gray columns are responsible for controlling the data structure. The white columns are responsible for storing actual data, if present, according to the gray data structure control. ARSEI's unique features include the ability to update ar_label_idx[i], which identifies the ar_label (604) that can store up to 255 bytes of information (such as text), ar_object_idx[i], which identifies an object in an image, and the bounding box that indicates the object's location. Up to 256 ar_label_idx[i] and ar_object_idx[i] can be registered, and it is possible to determine which label to assign to an object when the object is updated.

[0043] If ar_num_label_updates (602), which indicates the number of ar_labels (604) to be updated, is non-zero, the index (0 to 255) associated with the ar_label (604) to be updated is entered into ar_label_idx (603), and the ar_label (604) is updated. The ar_label (604) can store 255 bytes of non-zero data.

[0044] Furthermore, when ar_num_object_updates (605), which indicates the number of objects to be updated, is non-zero, an index (any value between 0 and 255) associated with the object to be updated is entered in ar_object_idx (606), and ar_object_label_update_flag (607) is set to 1. Furthermore, if a specific ar_label (604) is to be associated with the object to be updated, the numerical value of ar_label_idx (603), which identifies the ar_label (604), is entered in ar_object_label_idx (608). Furthermore, when ar_bounding_box_update_flag is set to 1, the position information (610) of the object to be updated can be stored, and the object's position can be updated.

[0045] Also, by setting the ar_label_cancel_flag (611) for a certain numerical value of ar_label_idx to 1, it is possible to delete the information of the ar_label (604) linked to that ar_label_idx (603). Also, by setting the ar_object_cancel_flag (612) for a certain numerical value of ar_object_idx[i] to 1, it is possible to delete information (610, etc.) about the object of that ar_object_idx (606).

[0046] Here, processing of the imaging device 100 using the data structure of the ARSEI described above will be described with reference to the flow shown in Fig. 7. Note that the processing of the flow shown in Fig. 5 is executed by the functional blocks of the imaging device 100 shown in Fig. 3, which are realized by the CPU 900 of the imaging device 100 executing a computer program stored in the ROM 920 of the imaging device 100, for example. The processing of the flow shown in Fig. 7 is started every time an image is captured.

[0047] First, in S701, the metadata generation unit 309 determines whether updating of the ARSEI labels is necessary. Only when the number of important regions in the previously processed image is 0 and the number of important regions in the current frame is 0, it is determined that updating of the ARSEI labels is not necessary and the process proceeds to S709. In all other cases, it is determined that updating of the labels is necessary and the process proceeds to S702.

[0048] Next, in S702, the metadata generation unit 309 enters the number of labels (ar_label) that need to be updated in ar_num_label_updates, and proceeds to S703. The number of labels that need to be updated is the larger of the number of labels in the image that was previously processed and the number of important regions in the image that is currently being processed.

[0049] Next, in S703, if the metadata generation unit 309 determines that the summary value was calculated from the encoded data (Yes in S703), it transitions to S705. If the summary value was calculated from an important region in the decoded image (No in S703), it transitions to S704. In S704, the metadata generation unit 309 assigns a value between 0 and 254 to ar_label_idx[i] (603) for the important region in the image to be processed, and transitions to S706. Note that if a value for ar_label_idx[i] (603) has already been assigned to the important region in processing of the previous frame, the metadata generation unit 309 assigns the same value to ar_label_idx[i] (603) in the image to be processed, which is the current frame. In other words, the same value for ar_label_idx[i] (603) is assigned to the same important region. In S705, the metadata generation unit 309 assigns the value 255 to ar_label_idx[i] (603) and proceeds to S706. Note that although the value 255 assigned to the label ID indicates that a digest value has been calculated from the encoded data, it is sufficient that the client device 101 can distinguish this from a case where a digest value has been calculated from an important region in a decoded image. Therefore, for example, the values ​​from 0 to 255 defined in the ARSEI standard may be arbitrarily divided into two groups as follows: If the assigned value of ar_label_idx[i] (603) belongs to one group, it indicates that a digest value has been calculated from the encoded data, and if it belongs to the other group, it indicates that a digest value has been calculated from an important region in a decoded image.

[0050] Next, in S706, the metadata generation unit 309 determines whether it is necessary to cancel the labels. If there are fewer labels for the image currently being processed than for the previously processed image, the process proceeds to S708 to cancel the difference. If cancellation is not necessary, the process proceeds to S707. In S707, the metadata generation unit 309 stores the digest value calculated by the digest value calculation unit 307 in ar_label (604) associated with ar_label_idx[i] (603) assigned in S704 or S705, and then proceeds to S709. Here, since the ARSEI standard does not allow the data value of 0x00 to be entered, for example, 0xff is used as an escape sequence, and if 0x00 is found in the digest value, it is substituted as 0xff, 0x01, and if 0xff is found, it is substituted as 0xff, 0x02.

[0051] In S708, the metadata generation unit 309 sets the ar_label_cancel_flag (611) for the ar_label_idx to be cancelled to 1, and proceeds to S709. The processes from S703 to S708 must be repeated for each label that needs to be updated, but for the sake of simplicity, the case where the number of updates is 1 is described here.

[0052] Next, in S709, the metadata generation unit 309 determines whether the object needs to be updated. If there is no update to the label and no update to the position information of the important area, it is determined that there is no need to update the object (No in S709) and the processing of the flow shown in Fig. 7 ends; otherwise (Yes in S709), it is determined that there is a need to update the object and the processing proceeds to S710.

[0053] In S710, the metadata generation unit 309 assigns the number of objects that need to be updated to ar_num_object_updates (605), and the process proceeds to S711. In S711, the metadata generation unit 309 assigns different numerical values ​​between 0 and 255 to important areas in the image currently being processed, and assigns this to ar_object_idx (606). For example, if there is one important area, 0 is assigned as the ar_object_idx (606) for that important area. In this embodiment, the ar_object_idx (606) serves as an ID that identifies important areas in an image. Note that the numerical value of the ar_object_idx (606) assigned to a certain important area in the image of the previous frame that was previously processed is also assigned to the ar_object_idx (606) of that certain important area in the image being processed in the current frame. In other words, the same numerical value of the ar_object_idx (606) is assigned to the same important area.

[0054] Next, in S712, the metadata generation unit 309 determines whether cancellation of the object is necessary. If the number of important areas in the image of the current frame currently being processed is less than the number of ar_object_idx registered for the image of the previous frame, cancellation is necessary and the process proceeds to S713; otherwise, the process proceeds to S714.

[0055] In S713, the metadata generation unit 309 identifies an important area that needs to be canceled, sets ar_object_cancel_flag (612) in the ar_object_idx (606) associated with the important area to 1, and proceeds to S714. Here, for example, if a certain important area (e.g., the face area of ​​person A) that was detected by the area detection unit 303 in the image of the previous frame is no longer detected in the image of the current frame, the metadata generation unit 309 identifies the important area as one that needs to be canceled.

[0056] Next, in S714, the metadata generation unit 309 determines whether the position information of the important regions needs to be updated. At this time, if there is a change in the position of an important region to which a certain value (for example, 0) is assigned as ar_object_idx (606) between the image of the previous frame and the image of the current frame, the metadata generation unit 309 determines that the important region needs to be updated. At this time, if it is determined that at least one important region included in the image needs to be updated (Yes in S714), the process proceeds to S715; otherwise (No in S714), the process proceeds to S716.

[0057] In S715, the metadata generation unit 309 writes the position information in the image of the current frame for each of one or more important areas that need to be updated in ar_bounding_box_top[ar_object_idx[i]], ar_bounding_box_left[ar_object_idx[i]], ar_bounding_box_width[ar_object_idx[i]], and ar_bounding_box_height[ar_object_idx[i]], and transitions to S716.

[0058] In S716, the metadata generation unit 309 generates a NAL (Network Abstraction Layer) unit from the ARSEI information stored in the data structure in S701 to S715, and ends the processing flow shown in Fig. 7. Note that NAL stands for Network Abstraction Layer, and is defined in the HEVC standard.

[0059] 7, and the encoded data of the image obtained by the encoding unit 302. The transmission unit 311 transmits the distribution data to an external device such as the client device 101.

[0060] As described above, the imaging device 100 in this embodiment is capable of generating distribution data including ar_label_idx[i] that identifies the ar_label that stores information about the encrypted digest value obtained by encrypting the digest value, and transmitting the distribution data to the client device 101.

[0061] 4 and 8, the processing in the client device 101 in this embodiment will be described. The receiving unit 401 of the client device 101 receives distribution data transmitted from the imaging device 100, and outputs the HEVC format data included in the distribution data to the decoding unit 402 on a frame-by-frame basis.

[0062] The decoding unit 402 decodes the HEVC format data input from the receiving unit 601, generates a decoded image, outputs it to the display control unit 408, and outputs the ARSEI information to the metadata analysis unit 403.

[0063] The metadata analysis unit 403 analyzes the input ARSEI information, deletes the object information linked to the ar_object_idx (606) for which the ar_object_cancel_flag (612) was set to 1, and deletes from the record the ar_label (604) linked to the ar_label_idx (603) for which the ar_label_cancel_flag (611) was set to 1. It also stores the ar_label (604) linked to the newly transmitted ar_label_idx (603), and similarly stores the ar_label_idx (603) and location information associated with the newly registered ar_object_idx (606).

[0064] The metadata analysis unit 403 also refers to the ar_object_label_idx (608) to identify the ar_label_idx (603) associated with the ar_object_idx (606) of a certain important region. The metadata analysis unit 403 then reads out information about the encrypted digest value stored in the ar_label (604) corresponding to the identified ar_label_idx (603), and outputs the information to the encryption / decryption unit 407. The encryption / decryption unit 407 uses a private key to decrypt the encrypted digest value to generate a digest value, and outputs the digest value to the determination unit 406 as correct answer data.

[0065] The metadata analysis unit 403 also outputs to the calculation region determination unit 404 the ar_label_idx (603) linked to the ar_object_idx (606), the ar_label (604) corresponding to the ar_label_idx (603), and the position information (610) linked to the ar_object_idx (606).

[0066] The calculation area determination unit 604 determines whether the digest value obtained after decrypting the encrypted digest value stored in the ar_label(604) corresponding to the input ar_label_idx(603) is obtained by calculating the digest value for the coded data, or whether the digest value obtained by calculating the digest value for data corresponding to an important region in the decoded image. For example, if ar_label_idx(603) is 255, the calculation area determination unit 604 determines that the digest value obtained after decrypting the encrypted digest value stored in the ar_label(604) associated with the ar_label_idx(603) is obtained by calculating the digest value for the coded data. On the other hand, if ar_label_idx(603) is not 255, the calculation area determination unit 604 determines that the digest value obtained after decrypting the encrypted digest value stored in the ar_label(604) associated with the ar_label_idx(603) is obtained by calculating the digest value for data corresponding to an important region in the decoded image. The result of the determination as to whether the digest value corresponds to the coded data or to the data of the important region in the decoded image is input to the digest value calculation unit 406 .

[0067] If the source for which the digest value is to be calculated is coded data, the digest value calculation unit 406 performs digest value calculation on the coded data included in the distribution data 401. On the other hand, if the source for which the digest value is to be calculated is an important region in a decoded image, the digest value calculation unit 406 performs digest value calculation on one-dimensional binary data of the important region in the decoded image generated by the decoding unit 402.

[0068] The discrimination unit 406 compares the digest value calculated by the digest value calculation unit 405 with the digest value corresponding to the correct data output by the encryption / decryption unit 407, and determines that there is no tampering if the values ​​are the same, and that there is tampering if the values ​​are different. The discrimination unit 406 outputs tampering information, which is information on the result of the determination, and information on the source (encoded data or data in an important region of the decoded image) that is the target of the digest value calculation, to the display control unit 408. If the source that is the target of the digest value calculation is data in an important region of the decoded image, the discrimination unit 406 outputs position information of the important region to the display control unit 408.

[0069] The display control unit 408 displays the decoded image obtained by the decoding unit 402 on the display 103. FIGS. 8(a) and 8(b) show examples of information displayed by the display control unit 408. FIG. 8(a) shows a case where the source for which the digest value is calculated is data in an important region of the decoded image. In the example shown in FIG. 8(a), the display control unit 408 displays a decoded image 800, tampering information 802 indicating whether or not tampering has occurred, and a frame 801 indicating the position of the important region that was the target of the determination of whether or not the tampering has occurred. In the example shown in FIG. 8(b), the source for which the digest value is calculated is coded data. As shown in FIG. 8(b), the display control unit 408 may superimpose a frame 801 indicating the position of the important region detected by the region detection unit 303 on the decoded image. Also, as shown in FIG. 8(b), the display control unit 408 superimposes tampering information 804 on the decoded image 800. Note that a bounding box that includes the entire decoded image 800 may be superimposed on the decoded image 800 so that it is clear that the target of the digest value calculation is coded data.

[0070] As described above, in this embodiment, it is possible to adaptively change the data to be used for calculating the digest value to reduce the processing load associated with calculating the digest value, while transmitting the encrypted digest value using ARSEI so that tampering can be determined on the client device 101 side.

[0071] (Other embodiments) Next, the hardware configuration of the imaging device 100 for realizing each function of the above-described embodiment will be described with reference to Fig. 9. Note that although the hardware configuration of the imaging device 100 will be described in the following explanation, it is assumed that the client device 101 is also realized by a similar hardware configuration.

[0072] The imaging device 100 of this embodiment has a CPU 900, a RAM 910, a ROM 920, a HDD 930, and an I / F 940. The CPU 900 is a central processing unit that controls the imaging device 100. The RAM 910 temporarily stores computer programs executed by the CPU 900. The RAM 910 also provides a work area used by the CPU 900 when executing processing. The RAM 910 also functions as, for example, a frame memory or a buffer memory.

[0073] The ROM 920 stores programs and the like used by the CPU 900 to control the imaging device 100. The HDD 930 is a storage device that records image data and the like. The I / F 910 communicates with external devices via the network 300 in accordance with TCP / IP, HTTP, or the like.

[0074] Although the above-described embodiments have been described with reference to examples in which the CPU 900 executes the processing, at least a portion of the processing by the CPU 900 may be performed by dedicated hardware. For example, the processing of reading program code from the ROM 920 and loading it into the RAM 910 may be performed by a DMA (Direct Memory Access) that functions as a transfer device.

[0075] The present invention can also be realized by a process in which one or more processors read and execute a program that realizes one or more functions of the above-described embodiments. The program may be supplied to a system or device having a processor via a network or a storage medium. The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions of the above-described embodiments. Each unit of the imaging device 100 may be realized by hardware as shown in FIG. 9, or by software. One or more functions of the imaging device 100 according to the above-described embodiment may be implemented by another device.

[0076] Although the present invention has been described above with reference to the embodiments, the above embodiments merely illustrate specific examples of how the present invention can be implemented, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from the technical concept or main features of the present invention. For example, combinations of the embodiments are also included in the disclosure of this specification. [Explanation of symbols]

[0077] 100 Imaging device 101 Client device 102 Display

Claims

1. an encoding means for generating encoded data by encoding the captured image; a decoding means for generating a decoded image by decoding the encoded data; a determining means for determining which of the data corresponding to a predetermined area in the decoded image and the coded data is to be used for calculating a digest value; a calculation unit that generates a summary value by performing the summary value calculation on the data determined by the determination unit as the target of the summary value calculation; generating means for generating distribution data including information based on the digest value and the encoded data; an imaging device characterized in that the determination means compares the size of the encoded data with the size of data corresponding to the specified area in the decoded image, and determines the target of the summary value calculation according to the comparison result.

2. 2. The imaging device according to claim 1, wherein the determining means performs the summary value calculation on the data corresponding to the predetermined area when the size of the data corresponding to the predetermined area in the decoded image is smaller than the size of the encoded data.

3. 3. The imaging device according to claim 1, wherein the distribution data includes, in accordance with ARSEI, an ar_label that stores information based on the digest value, and an ar_label_idx that identifies the ar_label.

4. 4. The imaging device according to claim 3, wherein the distribution data includes an ar_object_idx that identifies the specified area from which the summary value is obtained, and the ar_label_idx that identifies the ar_label in which information based on the summary value is stored is associated with the ar_object_idx.

5. 5. The imaging device according to claim 3, wherein the ar_label_idx value, which specifies the ar_label that stores information based on the summary value, can be used to identify whether the summary value was obtained by calculating the summary value for the encoded data or by calculating the summary value for data corresponding to the specified area in the decoded image.

6. further comprising an encryption means for encrypting the digest value to generate an encrypted digest value; 6. The imaging device according to claim 1, wherein the information based on the digest value corresponds to the encrypted digest value.

7. an encoding step of generating encoded data by encoding the captured image; a decoding step of generating a decoded image by decoding the encoded data; a determining step of determining which of the data corresponding to a predetermined area in the decoded image and the coded data is to be used for calculating a digest value; a calculation step of generating a summary value by performing the summary value calculation on the data determined in the determination step as the target of the summary value calculation; a generating step of generating distribution data including information based on the digest value and the encoded data, an information processing method characterized in that, in the determination step, a size of the encoded data is compared with a size of data corresponding to the specified area in the decoded image, and a target for the summary value calculation is determined according to the comparison result.

8. A computer program for causing a computer to function as each of the means of the imaging device according to any one of claims 1 to 6.

Images