Video storage system, video compression apparatus, video storage method, and computer program

Abstract

To provide a video storage system for storing videos which can be verified suitably, without damage of user's operability.SOLUTION: The video storage system comprises: a scene recognition unit which based on a video photographed by a camera, recognizes a scene shown by the video; a related information determination unit which based on the scene, determines related information related with a region in the video or the compressibility of an image; an attention region extraction unit which extracts an attention region from the video; an important region determination unit which on the basis of the attention region and the related information, determines an important region and an unimportant region not including the important region, in the video; a video compression unit which compresses the video in such a manner where the important region is to be compressed with lower compressibility than the unimportant region; and a video storage unit which stores the video compressed by the video compression unit.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a video storage system, a video compression device, a video storage method, and a computer program.

Background Art

[0002] Cameras are used for work monitoring applications, security applications, in-vehicle applications, etc. in factories. However, in order to transmit high-resolution video captured by a camera to a video receiving device and store it in a storage unit, a large amount of transmission cost and storage cost are required, which is a problem. On the other hand, from the perspective of quality control of products produced in a factory, it may be obligatory to store work monitoring videos for a certain period of time.

[0003] Therefore, it is conceivable to reduce the data amount of the compressed video by compressing important areas necessary for verification when a problem occurs after product shipment at a low compression ratio and compressing other areas at a high compression ratio. For example, Patent Document 1 discloses an imaging device that performs compression encoding at different image quality levels for important areas and other areas.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the imaging device described in Patent Document 1, the user has to operate a region designation lever to designate an important region in the image, and the operability is poor. In particular, when the scene of the video changes, the important region has to be designated again, leading to deterioration of operability. Also, when designating an important region manually, setting mistakes of the important region are likely to occur. For this reason, there is a possibility that videos for which appropriate verification is difficult are stored.

[0006] The present disclosure has been made in view of such circumstances, and an object thereof is to provide a video storage system, a video compression device, a video storage method, and a computer program capable of storing a video that can be appropriately verified without impairing the operability of a user.

Means for Solving the Problems

[0007] A video storage system according to an aspect of the present disclosure includes a scene recognition unit that recognizes a scene shown in the video based on the video captured by a camera, a related information determination unit that determines related information related to a compression rate of a region or an image in the video based on the scene, a region of interest extraction unit that extracts a region of interest from the video, a significant region determination unit that determines a significant region and a non-significant region that does not include the significant region in the video based on the region of interest and the related information, a video compression unit that compresses the video so that the significant region has a lower compression rate than the non-significant region, and a video storage unit that stores the compressed video by the video compression unit.

[0008] The present invention can be realized not only as a video storage system including such a characteristic processing unit, but also as a video storage method including such characteristic processing as steps. Further, the present invention can be realized as a video compression device constituting the video storage system, or as a computer program for causing a computer to function as a characteristic processing unit included in the video compression device. Furthermore, the present invention can be realized as a semiconductor integrated circuit that realizes part or all of the video compression device.

Effects of the Invention

[0009] According to the present disclosure, it is possible to store a video that can be appropriately verified without impairing the operability of a user.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0011] [Summary of Embodiments of the Present Disclosure] First, the summary of the embodiments of the present disclosure will be listed and described. (1) A video storage system according to an embodiment of the present disclosure includes a scene recognition unit that recognizes a scene shown in the video based on the video captured by a camera, a related information determination unit that determines related information related to the compression rate of a region or an image in the video based on the scene, a region of interest extraction unit that extracts a region of interest from the video, an important region determination unit that determines an important region and a non-important region that does not include the important region in the video based on the region of interest and the related information, a video compression unit that compresses the video so that the important region has a lower compression rate than the non-important region, and a video storage unit that stores the compressed video by the video compression unit.

[0012] According to this configuration, a region of interest is extracted from the video, and related information is determined based on the scene recognized from the video. Therefore, even when the scene of the video changes without any operation by the user, the important region can be appropriately determined. Thus, it is possible to accumulate videos that can be appropriately verified without impairing the operability of the user.

[0013] (2) In the above (1), the related information includes information on a prohibited region that prohibits compression at a low compression rate in the video, and the important region determination unit may determine the prohibited region as the non-important region.

[0014] According to this configuration, a region unnecessary for verification can be determined as a prohibited region according to the scene. Therefore, when the scene of the video changes, a region unnecessary for verification can be compressed at a high compression rate according to the scene. As a result, the data amount of the compressed video can be reduced.

[0015] (3) In the above (2), the region of interest extraction unit may extract the region of interest from a region excluding the prohibited region in the video.

[0016] Since the prohibited region is regarded as a non-important region, it is not necessary to extract the region of interest from the prohibited region. Therefore, according to this configuration, unnecessary extraction processing of the region of interest can be reduced.

[0017] (4) In any of the above (1) to (3), the related information includes information on a forced region that forces compression at a low compression rate in the video, and the important region determination unit may determine the forced region as the important region.

[0018] According to this configuration, a region necessary for verification can be determined as a forced region according to the scene. Therefore, even when the scene of the video changes, a region necessary for verification can be compressed at a low compression rate according to the scene. As a result, it is possible to accumulate videos that can be appropriately verified.

[0019] (5) In the above (4), the attention area extraction unit may extract the attention area from an area excluding the forced area in the video.

[0020] Since the forced area is regarded as an important area, there is no need to extract the attention area from the forced area. Therefore, according to this configuration, unnecessary extraction processing of the attention area can be reduced.

[0021] (6) In any one of the above (1) to (5), the related information includes information on a predetermined offset amount from the attention area, and the important area determination unit may determine an offset area obtained by moving the attention area by the predetermined offset amount as the important area.

[0022] According to this configuration, for example, when the attention area is a human face area, based on the offset amount indicating the relative positional relationship between the face and the hand, the area of the fingertip can be set as the offset area. Therefore, even when the attention area extraction unit cannot extract the area of the fingertip, the area of the fingertip can be included in the important area. In this way, an area having a predetermined relevance to the attention area can be included in the important area.

[0023] (7) In any one of the above (1) to (6), the video storage system may further include an area information output unit that outputs area information indicating at least one of the important area and the non-important area to the video storage unit or externally.

[0024] According to this configuration, based on the area information, the compressed video can be decompressed by an appropriate decompression method for each area. Also, based on the area information, since the important areas that should be emphasized in the verification of the video can be discriminated, the verification work can be performed efficiently.

[0025] (8) A video compression device according to another embodiment of the present disclosure includes a scene recognition unit that recognizes a scene shown in the video based on the video captured by a camera, a related information determination unit that determines related information related to the compression ratio of an area or an image in the video based on the scene, a region of interest extraction unit that extracts a region of interest from the video, a significant region determination unit that determines a significant region and a non-significant region that does not include the significant region in the video based on the region of interest and the related information, and a video compression unit that compresses the video such that the significant region has a lower compression ratio than the non-significant region.

[0026] This configuration includes the same processing units as those of the video storage system described above. Therefore, it exhibits the same operations and effects as the video storage system.

[0027] (9) A video storage method according to another embodiment of the present disclosure includes steps of recognizing a scene shown in the video based on the video captured by a camera, determining related information related to the compression ratio of an area or an image in the video based on the scene, extracting a region of interest from the video, determining a significant region and a non-significant region that does not include the significant region in the video based on the region of interest and the related information, compressing the video such that the significant region has a lower compression ratio than the non-significant region, and storing the compressed video.

[0028] This configuration includes steps corresponding to each processing unit included in the video storage system described above. Therefore, the video storage method exhibits the same operations and effects as the video storage system.

[0029] (10) A computer program according to another embodiment of the present disclosure causes a computer to function as a scene recognition unit that recognizes a scene shown in the video based on the video captured by a camera, a related information determination unit that determines related information related to the compression rate of a region or an image in the video based on the scene, a region of interest extraction unit that extracts a region of interest from the video, a significant region determination unit that determines a significant region and a non-significant region that does not include the significant region in the video based on the region of interest and the related information, and a video compression unit that compresses the video so that the significant region has a lower compression rate than the non-significant region.

[0030] According to this configuration, the computer can function as the above-described video compression device. Therefore, the computer program has the same operations and effects as the above-described video compression device.

[0031] [Details of Embodiments of the Present Disclosure] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present disclosure. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are examples and do not limit the present disclosure. In addition, among the components in the following embodiments, components not described in the independent claims are components that can be arbitrarily added. Each drawing is a schematic diagram and is not necessarily drawn precisely.

[0032] Also, the same reference numerals are given to the same components. Since their functions and names are the same, their descriptions will be omitted as appropriate.

[0033] [Overall Configuration of Video Storage System] FIG. 1 is a block diagram showing the configuration of a video storage system according to an embodiment of the present disclosure. The video storage system 100 is a system for compressing and storing video data (hereinafter simply referred to as "video"), and includes a video compression device 1 and a video decompression device 2.

[0034] The video compression device 1 compresses a video and transmits the compressed video to the video decompression device 2 via the network 3.

[0035] The video decompression device 2 receives the compressed video from the video compression device 2 via the network 3 and restores the video by decompressing the compressed video.

[0036] The network 3 is, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), or the like.

[0037] 〔Configuration of the video compression device 1〕 FIG. 2 is a block diagram showing the configuration of the video compression device 1. The video compression device 1 includes a video input unit 11, a scene recognition unit 12, a related information determination unit 13, a region of interest extraction unit 14, a significant region determination unit 15, a video compression unit 16, a multiplexing unit 17, and a data transmission unit 18.

[0038] The video input unit 11 receives, as an input, the video captured by the camera 4. The video is composed of image data (hereinafter simply referred to as "image") in time series. The video input unit 11 may receive the video by receiving one image at a time from the camera 4, or may receive the video composed of a plurality of images from the camera 4.

[0039] The scene recognition unit 12 recognizes the scene shown in the video based on the video received by the video input unit 11. A scene refers to the scenario shown in the video. For example, when the camera 4 is installed at a position where it can capture the working area inside the factory, the scene of the video indicates the working area in the factory. Also, when the camera 4 is installed on a moving object such as an automobile and is capturing the front of the moving object, and the moving object stops in front of an intersection, the scene of the video indicates the intersection. As an example, the scene recognition unit 12 inputs the images constituting the video into a learning model that has previously machine-learned the correspondence between images and scenes, thereby recognizing the scene of the image. The learning model is, for example, a CNN (Convolution Neural Network), an RNN (Recurrent Neural Network), an AutoEncoder, etc. However, the method for recognizing the scene is not limited to this.

[0040] The related information determination unit 13 determines related information related to the compression rate of the regions included in the images constituting the video based on the scene of the video recognized by the scene recognition unit 12. The related information includes prohibited region information, mandatory region information, and offset specification region information, which will be described below.

[0041] The prohibited region information indicates a prohibited region where compression at a low compression rate is prohibited in the image. That is, the prohibited region indicates a region that is compressed at a high compression rate and is not important for the verification work in case any problems occur. For example, the prohibited region for the scene of the working area in the factory is a region where no workers are present, such as the ceiling or the wall near the ceiling. When the prohibited region is a rectangular region, the prohibited region information includes, as an example, the upper left corner coordinates and the lower right corner coordinates of the prohibited region.

[0042] The forced area information indicates a forced area that forces compression at a low compression rate in the image. That is, the forced area indicates an area to be compressed at a low compression rate and is an area important for the verification work. For example, in the scene of an intersection, the forced area is an area that a moving object such as a pedestrian must pay attention to, such as a crosswalk. In addition, when the forced area is a rectangular area, the forced area information includes, as an example, the upper left corner coordinates and the lower right corner coordinates of the forced area.

[0043] The offset specified area information is an area for specifying whether to compress an offset area specified by adding a predetermined offset amount to the attention area extracted by the attention area extraction unit 14 described later at a low compression rate. Specifically, if the offset area is within the offset specified area, the offset area is compressed at a low compression rate, and if the offset area is not within the offset specified area, the offset area is not compressed at a low compression rate. In addition, when the offset specified area is a rectangular area, the offset specified area information includes, as an example, the upper left corner coordinates and the lower right corner coordinates of the offset specified area.

[0044] The related information further includes an object (hereinafter referred to as "attention area extraction target") to be extracted as an attention area by the attention area extraction unit 14 described later. The attention area extraction target includes, for example, a person's face in the case of a scene of a working place in a factory, and includes pedestrians, automobiles, and signs in the case of a scene of an intersection. In addition, in the case of a highway scene, since there are no pedestrians on the highway, the attention area extraction target does not include pedestrians and includes automobiles and signs. The related information further includes information on a predetermined offset amount from the attention area.

[0045] It is assumed that the correspondence relationship between the scene and the related information is determined in advance. For example, the related information determination unit 13 may refer to the table information indicating the correspondence relationship between the scene and the related information, and determine the related information of the scene from the scene. Note that the method for determining the related information is not limited to the above. The related information determination unit 13 may determine the related information using the video. For example, the related information determination unit 13 may input the scene and the images constituting the video into a learning model that has previously learned the correspondence relationship between the scene, the image, and the related information by machine learning, and determine the related information of the image. The learning model is, for example, CNN, RNN, AutoEncoder, etc.

[0046] FIG. 3 is a diagram for explaining an example of related information. FIG. 3 is an image of a scene at an intersection. For a moving body traveling at an intersection, the empty area is not very relevant to driving. Therefore, the empty area is set as the prohibited area 40E. On the other hand, the area of the road that the moving body will travel immediately after is an area important for driving, so it is set as the mandatory area 50C. Furthermore, the area including the crosswalk ahead is an area that requires special attention, so it is set as the mandatory area 50D.

[0047] Referring to FIG. 2 again, the attention area extraction unit 14 extracts the attention area included in the video based on the video received by the video input unit 11 and the related information determined by the related information determination unit 13. That is, the attention area extraction unit 14 extracts the attention area by extracting the image of the attention area extraction target shown in the related information from the video. For example, the attention area extraction unit 14 divides the images constituting the video into a plurality of blocks of a predetermined size. The attention area extraction unit 14 determines whether or not the attention area extraction target is included in each block image using a learning model. The learning model is, for example, CNN, RNN, AutoEncoder, etc. For example, when the attention area extraction target is a human face, images including various human faces are used as teacher data, and machine learning techniques such as deep learning are used to advance the learning of human faces, and the parameters of the learning model are determined. The attention area extraction unit 14 extracts the block including the attention area extraction target as the attention area.

[0048] Based on the attention area extracted by the attention area extraction unit 14 and the related information determined by the related information determination unit 13, for each image constituting the video, the important area determination unit 15 determines an important area and an unimportant area that does not include the important area. Here, the important area is an area compressed at a lower compression rate compared to the unimportant area. That is, the important area is an area important for the above-described verification work, and the unimportant area is an area of lower importance for the verification work compared to the important area.

[0049] Specifically, the important area determination unit 15 performs the following processing for each image constituting the video. That is, when the attention area is included in the prohibited area, the important area determination unit 15 deletes the attention area. Thereby, even if there is an attention area within the prohibited area, the prohibited area is regarded as an unimportant area. The important area determination unit 15 adds the other attention areas to the important area. Also, when the offset area calculated by adding a predetermined offset amount to the position coordinates of the attention area is within the offset designated area, the important area determination unit 15 adds the offset area to the important area. Furthermore, the important area determination unit 15 adds the forced area to the important area. The important area determination unit 15 determines all areas other than the important area as unimportant areas.

[0050] FIG. 4 and FIG. 5 are diagrams for explaining the determination process of the important area by the important area determination unit 15. FIG. 4 and FIG. 5 show images. For example, referring to FIG. 4, the related information is assumed to show prohibited areas 40A, 40B, 40C, forced areas 50A, 50B, and an offset designated area 60A. Also, it is assumed that the attention area extraction unit 14 extracts attention areas 30A to 30F from the attention area 30A.

[0051] Since the attention area 30A is included in the prohibited area 40A, the important area determination unit 15 deletes the attention area 30A. Also, the important area determination unit 15 adds the other attention areas 30B, 30C, 30D, 30E, 30F to the important area.

[0052] In addition, since the offset region 31B calculated by adding a predetermined offset amount to the position coordinates of the region of interest 30B is within the offset specified region 60A, the important region determination unit 15 adds the offset region 31B to the important region. For example, assume that the offset amount is defined as (xo1, yo1, xo2, yo2). When the upper left corner coordinates and the lower right corner coordinates of the region of interest 30B are (x1, y1) and (x2, y2), respectively, the important region determination unit 15 calculates the upper left corner coordinates and the lower right corner coordinates of the offset region 31B as (x1 + xo1, y1 + yo1) and (x2 + xo2, y2 + yo2) by adding the corresponding offset amounts to each coordinate.

[0053] In addition, since the offset region 31C calculated by adding a predetermined offset amount to the position coordinates of the region of interest 30C is within the offset specified region 60A, the important region determination unit 15 adds the offset region 31C to the important region. The calculation method of the offset region 31C is the same as that of the offset region 31B. In addition, the important region determination unit 15C adds the forced regions 50A and 50B to the important region.

[0054] FIG. 5 is a diagram showing the important region determined as described above. That is, the important region determination unit 15 determines the important regions 70A to 70G. The important region 70A corresponds to the region of interest 30C. The important region 70B corresponds to the offset region 31C. The important region 70C corresponds to the region of interest 30B. The important region 70D corresponds to the offset region 31B. The important region 70E corresponds to the region of interest 30D. The important region 70F corresponds to the forced region 50A. The important region 70G corresponds to the forced region 50B.

[0055] FIG. 6 is a diagram for explaining the determination process of the important area by the important area determination unit 15. FIG. 6 is an image of a scene at a workplace in a factory, showing a state where workers 82A and 82B are performing predetermined operations on a product 81 flowing on a belt conveyor 80. The image includes a prohibited area 40D and an offset designation area 60B. The prohibited area 40D includes the ceiling or a wall near the ceiling, etc. Since the workers 82A and 82B do not perform operations in this area, it is an area that is not important for the verification operation. The offset designation area 60B is an area where the workers 82A and 82B perform operations. Assume that the attention area extraction unit 14 extracts the face areas of the workers 82A and 82B as attention areas 30G and 30H, respectively. Since the attention areas 30G and 30H exist outside the prohibited area 40D, the important area determination unit 15 adds the attention areas 30G and 30H to the important areas. Further, the important area determination unit 15 calculates an offset area 31G by adding a predetermined offset amount to the position coordinates of the attention area 30G, and determines whether the offset area 31G is included in the offset designation area 60B. Since the offset area 31G is included in the offset designation area 60B, the important area determination unit 15 adds the offset area 31G to the important areas. The offset amount is the movement amount to the area including the hand when the face area of the worker 82A is used as a reference. Thereby, the hand area of the worker 82A can be calculated as the offset area 31G and added to the important areas. Similarly, the important area determination unit 15 calculates an offset area 31H by adding the offset amount to the position coordinates of the attention area 30H, and since the offset area 31H is included in the offset designation area 60B, the offset area 31H is added to the important areas.

[0056] Referring again to FIG. 2, the video compression unit 16 compresses the video after setting the compression ratio of the important region determined by the important region determination unit 15 to be lower than that of the non-important region. However, the important region and the non-important region determined by the important region determination unit 15 differ for each image. Therefore, the video compression unit 16 compresses the video while changing the important region and the non-important region for each image constituting the video. The video compression method is, for example, H.264 / MPEG-4 AVC, or H.265 / MPEG-H HEVC, etc. The greater the compression ratio, the lower the image quality of the decompressed image (region).

[0057] The multiplexing unit 17 generates multiplexed data by multiplexing region information indicating at least one of the important region and the non-important region determined by the important region determination unit 15 onto the compressed video obtained by compressing the video by the video compression unit 16. The region information may include, for example, information on the position and size of the important region for each image, information on the position and size of the non-important region for each image, or both pieces of information. The data transmission unit 18 transmits the multiplexed data generated by the multiplexing unit 17 to the video decompression device 2.

[0058] 〔Configuration of Video Decompression Device 2〕 FIG. 7 is a block diagram showing the configuration of the video decompression device 2. The video decompression device 2 includes a data reception unit 21, a separation unit 22, a video storage unit 23, a video decompression unit 24, and a video recognition unit 25.

[0059] The data reception unit 21 receives multiplexed data from the video compression device 1. The separation unit 22 separates the multiplexed data received by the data reception unit 21 into the compressed video and the region information. The separation unit 22 writes the separated compressed video and region information to the video storage unit 23. Also, the separation unit 22 outputs the compressed video and the region information to the video decompression unit 24.

[0060] The video storage unit 23 is composed of a non-volatile memory element such as a flash memory or an EEPROM (Electrically Erasable Programmable Read Only Memory), or a magnetic storage device such as a hard disk. The video storage unit 23 stores the compressed video and the region information for a certain period for subsequent verification work.

[0061] The video decompression unit 24 receives the compressed video and the region information from the separation unit 22, and restores the video by decompressing the compressed video based on the region information.

[0062] Specifically, the video decompression unit 24 determines the important regions and non-important regions of each image based on the region information. For example, when the region information indicates an important region and does not indicate a non-important region, the video decompression unit 24 determines the regions other than the important regions as non-important regions. Also, when the region information indicates a non-important region information and does not indicate an important region, the video decompression unit 24 determines the regions other than the non-important regions as important regions.

[0063] The video decompression unit 24 decompresses each image constituting the compressed video based on the important regions and non-important regions determined for each image. That is, the video decompression unit 24 decompresses each region by a decompression method corresponding to the compression method of each of the important regions and non-important regions.

[0064] The video recognition unit 25 recognizes the video restored by the video decompression unit 24 decompressing the compressed video. For example, the video recognition unit 25 recognizes the attention regions included in the video. The recognition process of the attention regions may be the same as the extraction process of the attention regions by the attention region extraction unit 14. The recognition process executed by the video recognition unit 25 is not limited to the recognition process of the attention regions.

[0065] 〔Processing Procedure of Video Compression Device 1〕 FIG. 8 is a flowchart showing an example of the processing procedure executed by the video compression device 1.

[0066] The video compression device 1 inputs the video captured by the camera 4 (step S11). For each image constituting the video input in step S11, the video compression device 1 determines an important region (step S12, loop A). Details of the important region determination process (step S12) will be described later.

[0067] Based on the result of the important region for each image determined in step S12, the video compression device 1 compresses the video after setting the compression rate of the important region to be lower than that of the non-important region (step S13). The video compression device 1 compresses the video, for example, in units of GOP (Group of Pictures).

[0068] The video compression device 1 multiplexes region information indicating the important region determined in step S12 onto the compressed video to generate multiplexed data (step S14).

[0069] The video compression device 1 transmits the multiplexed data generated in step S14 to the video decompression device 2 (step S15). FIG. 9 is a flowchart showing details of the important region determination process (step S12 in FIG. 8). Based on the image to be processed, the video compression device 1 recognizes the scene shown in the image (step S21).

[0070] Based on the scene recognized in step S21, the video compression device 1 determines relevant information on the compression rate of the regions of the image (step S22). The video compression device 1 extracts a region of interest from the image (step S23).

[0071] For each region of interest extracted in step S23, the video compression device 1 executes the processes of steps S24 to S29 described below (loop B).

[0072] That is, the video compression device 1 determines whether the target area is included in the prohibited area indicated by the related information determined in step S22 (step S24). Here, when the entire area of the target area is included in the prohibited area, the video compression device 1 determines that the target area is included in the prohibited area. However, even if a part of the target area is included in the prohibited area, the video compression device 1 may determine that the target area is included in the prohibited area.

[0073] When the video compression device 1 determines that the target area is included in the prohibited area (YES in step S24), it deletes the target area (step S25).

[0074] When the video compression device 1 determines that the target area is not included in the prohibited area (NO in step S24), it calculates an offset area by adding a predetermined offset amount to the position coordinates of the target area (step S26). The video compression device 1 determines whether the offset area is included in the offset specified area (step S27). Here, when the entire area of the offset area is included in the offset specified area, the video compression device 1 determines that the offset area is included in the offset specified area. However, even if a part of the offset area is included in the offset specified area, the video compression device 1 may determine that the offset area is included in the offset specified area.

[0075] When the video compression device 1 determines that the offset area is included in the offset specified area (YES in step S27), it adds the offset area to the important area (step S28).

[0076] When the video compression device 1 determines that the target area is not included in the offset specified area (NO in step S27), or after the process of step S28, it adds the target area to the important area (step S29). After the process of loop B, the video compression device 1 adds the forced area to the important area (step S30). The video compression device 1 determines an area other than the important area in the image as an unimportant area (step S31).

[0077] As described above, the video compression device 1 extracts the attention area from the video and determines the relevant information based on the scene recognized from the video. Therefore, even when the scene of the video changes without any operation by the user, the important area can be appropriately determined. Thus, it is possible to accumulate videos that can be appropriately verified without impairing the operability of the user.

[0078] In addition, the video compression device 1 can determine, as a prohibited area, an area unnecessary for verification according to the scene. For this reason, when the scene of the video changes, the video compression device 1 can compress the area unnecessary for verification at a high compression rate according to the scene. As a result, the data amount of the compressed video can be reduced.

[0079] In addition, the video compression device 1 can determine, as a mandatory area, an area necessary for verification according to the scene. For this reason, even when the scene of the video changes, the video compression device 1 can compress the area necessary for verification at a low compression rate according to the scene. Thus, it is possible to accumulate videos that can be appropriately verified.

[0080] In addition, for example, when the attention area is a human face area, the video compression device 1 can extract the area of the fingertip as an offset area based on the offset amount indicating the relative positional relationship between the face and the hand. For this reason, even when the area of the fingertip cannot be extracted by the attention area extraction unit 14, the video compression device 1 can include the area of the fingertip in the important area. In this way, the video compression device 1 can include an area having a predetermined relevance to the attention area in the important area.

[0081] In addition, the video compression device 1 can transmit area information indicating an important area or an unimportant area to the video decompression device 2. For this reason, the video decompression device 2 can decompress the compressed video by an appropriate decompression method for each area. In addition, since the video decompression device 2 can determine the important area that should be emphasized for verification in the video based on the area information, the verification work can be efficiently performed.

[0082] <Modification Example 1> In Embodiment 1, the video compression device 1 extracted the target area from all areas in the image. However, the target area included in the prohibited area is deleted (step S25 in FIG. 9). Also, the mandatory area is added to the important area regardless of whether the target area is included (step S30 in FIG. 9).

[0083] Therefore, the target area extraction unit 14 of the video compression device 1 extracts the target area from the area excluding both the prohibited area and the mandatory area in the image. Thereby, the processing time related to the extraction of the target area can be reduced.

[0084] Note that the target area extraction unit 14 may extract the target area from the area excluding one of the prohibited area and the mandatory area in the image. Thereby, the processing time related to the extraction of the target area can be reduced.

[0085] <Modification Example 2> In Embodiment 1, the video compression device 1 determined the related information related to the compression rate of the area included in the image constituting the video based on the scene of the video. However, the compression rate may be determined for each area or for each image.

[0086] In Modification Example 2, the video compression device 1 determines the related information related to the image constituting the video based on the scene of the video. For example, when the scene of the video shows a working place in a factory when the lighting is turned off, the important area determination unit 15 of the video compression device 1 may set all areas of the image showing the scene as the mandatory area and compress them at a low compression rate. Thereby, it is possible to efficiently monitor a suspicious person who intrudes into the working place in a situation where there is no operator.

[0087] <Modification Example 3> The important area determination unit 15 may determine related areas according to the time zone. For example, when the scene of the video shows a work area in a factory and it is the lunch break or night time zone, all areas of the image in that time zone may be set as forced areas and compressed at a low compression rate. Thereby, it is possible to efficiently monitor suspicious persons who enter the work area during the time when no workers are present.

[0088] Also, in the scene of a store such as a department store, the important area determination unit 15 may set the area near the showcase as a forced area during the daytime time zone, and all areas of the image in that time zone as forced areas during the night time zone. Thereby, during the daytime time zone, it is possible to intensively monitor the flow of people near the showcase, and during the night time zone, it is possible to monitor not only suspicious persons near the showcase but also those who enter the store. That is, even when the monitoring target or monitoring area differs depending on the time zone, efficient monitoring can be performed.

[0089] <Modification Example 4> In Embodiment 1, the video compression device 1 determined related information based on the scene of the video. In Modification Example 4, camera parameters of the camera 4 are used when determining the related information.

[0090] Assume that the camera 4 is, for example, a PTZ camera capable of panning, tilting, and zooming. The related information determination unit 13 of the video compression device 1 acquires camera parameters including the pan angle, tilt angle, and zoom magnification from the camera 4. The related information determination unit 13 corrects the related information determined based on the scene of the video based on the camera parameters. For example, when the camera 4 pans left or right, the related information determination unit 13 corrects the related information by moving the prohibited area, forced area, and offset specified area according to the pan angle. Also, when the zoom magnification of the camera 4 changes, the related information determination unit 13 corrects the related information by expanding or shrinking the prohibited area, forced area, and offset specified area according to the zoom magnification.

[0091] [Appendix] The non-important region described above may be all regions excluding the important region in the image. Also, the non-important region may be a part of the regions excluding the important region in the image. That is, the regions excluding the important region in the image may include regions other than the non-important region. For example, the regions excluding the important region in the image may include a non-important region and a semi-important region. Here, the semi-important region is a region whose importance for the verification work is second only to that of the important region and higher than that of the non-important region.

[0092] Each process (each function) of the above-described embodiment is realized by a processing circuit including one or more processors. The processing circuit may be configured by, in addition to the one or more processors, an integrated circuit in which one or more memories, various analog circuits, and various digital circuits are combined. The one or more memories store a program (instructions) for causing the one or more processors to execute each of the above processes. The one or more processors may execute each of the above processes according to the program read from the one or more memories, or may execute each of the above processes according to a logic circuit designed in advance to execute each of the above processes. The processor may be various processors suitable for computer control, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit). Note that the plurality of physically separated processors may cooperate with each other to execute each of the above processes. For example, the processors mounted on each of a plurality of physically separated computers may cooperate with each other via a network such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet to execute each of the above processes. The program may be installed in the memory via the network from an external server device or the like, or may be distributed in a state stored in a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a semiconductor memory, and may be installed in the memory from the recording medium.

[0093] In addition, at least a part of the above embodiment and the above modification example may be arbitrarily combined.

[0094] The embodiments disclosed this time should be considered as illustrative in all respects and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and it is intended that all modifications within the meaning and scope equivalent to the claims are included.

Explanation of Signs

[0095] 1 Video compression device 2 Video decompression device 3 Network 4 Camera 11 Video input section 12 Scene recognition section 13 Related information determination section 14 Region of interest extraction section 15 Important region determination section 15C Important region determination section 16 Video compression section 17 Multiplexing section 18 Data transmission section (region information output section) 21 Data reception section 22 Separation section (region information output section) 23 Video storage section 24 Video decompression section 25 Video recognition section 30A Region of interest 30B Region of interest 30C Region of interest 30D Region of interest 30E Region of interest 30F Region of interest 30G Region of interest 30H Region of interest 31B Offset region 31C Offset region 31G Offset region 31H Offset region 40A Prohibited region 40B Prohibited region 40C Prohibited region 40D Prohibited region 40E Prohibited region 50A Compulsory region 50B Compulsory region 50C Forced Area 50D Forced Area 60A Offset Specified Area 60B Offset Specified Area 70A Important Area 70B Important Area 70C Important Area 70D Important Area 70E Important Area 70F Important Area 70G Important Area 80 Belt Conveyor 81 Product 82A Operator 82B Operator 100 Video Storage System

Claims

1. A scene recognition unit that recognizes a scene shown in the video based on the video captured by a camera; A related information determination unit that determines related information related to the compression rate of a region or an image in the video based on the scene; A region of interest extraction unit that extracts a region of interest from the video; A significant region determination unit that determines a significant region and a non-significant region that does not include the significant region in the video based on the region of interest and the related information; A video compression unit that compresses the video so that the significant region has a lower compression rate than the non-significant region; A video storage system comprising a video storage unit that stores the compressed video in the video compression unit.

2. The related information includes information on a prohibited region that prohibits compression at a low compression rate in the video, The significant region determination unit determines the prohibited region as the non-significant region, and the video storage system according to claim 1.

3. The region of interest extraction unit extracts the region of interest from a region excluding the prohibited region in the video, and the video storage system according to claim 2.

4. The related information includes information on a forced region that forces compression at a low compression rate in the video, The significant region determination unit determines the forced region as the significant region, and the video storage system according to any one of claims 1 to 3.

5. The region of interest extraction unit extracts the region of interest from a region excluding the forced region in the video, and the video storage system according to claim 4.

6. The related information includes information on a predetermined offset amount from the region of interest, The significant region determination unit determines an offset region obtained by moving the region of interest by the predetermined offset amount as the significant region, and the video storage system according to any one of claims 1 to 3.

7. The video storage system according to any one of claims 1 to 3, further comprising a region information output unit that outputs region information indicating at least one of the important region and the non-important region to the video storage unit or externally.

8. A scene recognition unit that recognizes a scene shown in the video based on the video captured by a camera; A related information determination unit that determines related information related to the compression rate of a region or an image in the video based on the scene; A region of interest extraction unit that extracts a region of interest from the video; A significant region determination unit that determines a significant region and a non-significant region that does not include the significant region in the video based on the region of interest and the related information; A video compression device comprising a video compression unit that compresses the video so that the significant region has a lower compression rate than the non-significant region.

9. A step of recognizing a scene shown in the video based on the video captured by a camera; A step of determining related information related to the compression rate of a region or an image in the video based on the scene; A step of extracting a region of interest from the video; A step of determining a significant region and a non-significant region that does not include the significant region in the video based on the region of interest and the related information; A step of compressing the video so that the significant region has a lower compression rate than the non-significant region; A video storage method including a step of storing the compressed video.

10. A computer, A scene recognition unit that recognizes a scene shown in the video based on the video captured by a camera; A related information determination unit that determines related information related to the compression rate of a region or an image in the video based on the scene; A region of interest extraction unit that extracts a region of interest from the video; An important area determination unit that determines an important area and an unimportant area that does not include the important area in the video based on the area of interest and the related information; A computer program for causing the video compression unit to function as a video compression unit that compresses the video so that the important area has a lower compression ratio than the unimportant area.

Images