Computer system, video data compression method, and program

Abstract

The process of determining whether or not an object is included in a frame is adjusted to compress the data in a shorter processing time. [Solution] The computer system comprises at least one arithmetic unit and a memory device, the memory device storing video data consisting of multiple frames, and each frame contains at least one object. The arithmetic unit analyzes the motion characteristics of objects in the video data and controls the execution of a process that generates annotation data indicating the location of important objects contained in the frame based on the results of the analysis. Using the annotation data, the system compresses the frame so that the amount of data in the areas where important objects exist is large and the amount of data in the areas where important objects do not exist is small.

Description

Technical Field

[0001] The present invention relates to a compression technique for reducing the capacity of video data.

Background Art

[0002] From the viewpoint of reducing the cost required for storage and transfer of video data, an irreversible compression technique with a high compression ratio is required. Further, these irreversible compression techniques are required to be highly efficient from the viewpoint of suppressing the computational cost required for compression in addition to having a high compression ratio.

[0003] A technique is known in which by using a Deep Neural Network (DNN) such as an autoencoder, the bit allocation amount is controlled for each region based on the importance of each region of multi-dimensional data, and compressed data is generated (from paragraph 0169 to paragraph 0178 of Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The process of determining the presence or absence of important objects for all frames has a high processing cost, which is a bottleneck in the compression process. An object of the present invention is to provide a technique for adjusting the execution of the process of determining the presence or absence of important objects and compressing data at high speed in a shorter time than determining the presence or absence of important objects for all frames.

Means for Solving the Problems

[0006] A typical example of the invention disclosed in this application is as follows: a computer system comprising at least one arithmetic unit and a storage device connected to the at least one arithmetic unit, wherein the storage device stores video data consisting of a plurality of frames, each frame contains at least one object, the at least one arithmetic unit analyzes the motion characteristics of the object in the video data, controls the execution of a process that generates annotation data indicating the position of important objects included in the frame based on the results of the analysis, and uses the annotation data to compress the frame such that the amount of data in the area where the important object exists is large and the amount of data in the area where the important object does not exist is small. [Effects of the Invention]

[0007] According to the present invention, by adjusting the execution of the process for determining the presence or absence of important objects, data can be compressed more quickly and in less time than determining the presence or absence of important objects for all frames. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0008] [Figure 1] This is a flowchart illustrating the outline of the present invention. [Figure 2] This figure shows an example of the system configuration of Example 1. [Figure 3] This figure shows an example of the configuration of the compression system in Example 1. [Figure 4] This figure shows an example of the functional configuration of the compression service unit in Example 1. [Figure 5A] This figure shows an example of the compression settings information stored in the compression settings database of Example 1. [Figure 5B] This figure shows an example of the compression settings information stored in the compression settings database of Example 1. [Figure 6] This figure shows an example of the interface presented by the setting unit in Example 1. [Figure 7] This diagram shows the video data compression process flow of the compression system in Example 1. [Figure 8] This flowchart illustrates an example of the processing performed by the annotation control unit in Example 1. [Figure 9] This flowchart illustrates an example of the processing performed by the annotation data generation unit in Example 1. [Figure 10] This figure shows an example of the configuration of the compression system in Example 2. [Figure 11] This figure shows the video data compression process flow of the compression system in Example 2. [Figure 12] This flowchart illustrates an example of the processing performed by the annotation control unit in Example 3. [Figure 13] This flowchart illustrates an example of the processing performed by the annotation data generation unit in Example 3. [Figure 14] This figure shows an example of the data structure of frequency adjustment information in Example 4. [Figure 15] This flowchart illustrates an example of the processing performed by the annotation control unit in Example 4. [Figure 16] This flowchart illustrates an example of the processing performed by the annotation data generation unit in Example 4. [Figure 17] This figure shows an example of the data structure of frequency adjustment information in Example 5. [Figure 18] This flowchart illustrates an example of the processing performed by the annotation control unit in Example 5. [Modes for carrying out the invention]

[0009] The embodiments of the present invention will be described below with reference to the drawings. However, the present invention is not to be construed as being limited to the embodiments described below. It will be readily apparent to those skilled in the art that the specific configuration can be modified without departing from the spirit or intent of the present invention.

[0010] In the configuration of the invention described below, the same or similar components or functions are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] First, the outline of the present invention will be described.

[0012] The compression system of the present invention is a system for compressing video data, and includes, as functional components, an annotation data generation unit, an annotation control unit, and an encoder. The video data is composed of a plurality of frames. The frames include, as objects, artifacts such as people, vehicles, luggage, and signs, and natural objects such as trees and mountains.

[0013] The annotation data generation unit generates annotation data indicating the positions of important objects included in the frames constituting the video data. The important objects are objects specified by the user. The annotation control unit controls the execution of the annotation data generation process. The encoder compresses the frames using the annotation data.

[0014] FIG. 1 is a flowchart for explaining the outline of the present invention.

[0015] The annotation control unit analyzes the movement characteristics of the objects in the video data (step S101). Here, the movement characteristics of the objects represent the degree of movement of the objects included in the frames.

[0016] The annotation control unit generates a control instruction based on the analysis result (step S102). The annotation control unit transmits the control instruction to the annotation data generation unit.

[0017] Here, control instructions are information used to control the execution of the annotation data generation process. When the degree of object movement between frames is small, the annotation data can be reused. Therefore, the annotation control unit generates control instructions to execute the annotation data generation process at intervals of a number of frames corresponding to the degree of object movement.

[0018] The annotation data generation unit controls the execution of the annotation data generation process based on the control instructions (step S103).

[0019] The encoder uses annotation data to compress each frame so that the amount of data is large in areas where important objects exist and small in areas where important objects do not exist (step S104).

[0020] In this way, by controlling the execution of annotation data generation processing based on the motion characteristics of objects in video data, the frequency of execution of annotation data generation processing during compression can be reduced. This allows for efficient and high-speed compression processing. By compressing the data, storage capacity can be reduced, and the cost of compression processing can also be reduced. [Examples]

[0021] In Example 1, the annotation control unit analyzes the motion characteristics of objects in the video data based on the motion vectors contained in the frames that make up the video data, and controls the annotation data generation process.

[0022] Figure 2 shows an example of the system configuration of Example 1.

[0023] The system includes a compression system 100, a terminal 101, a camera 102, and a user data database 103. The compression system 100 connects to the terminal 101, camera 102, and user data database 103 via a network 110 such as a LAN (Local Area Network).

[0024] Camera 102 generates video data. The frames that make up the video data are assumed to contain motion vectors. Terminal 101 may also generate the video data. In this embodiment, the following video data storage case is assumed. Case 1: Terminal 101 generates video data and stores it in user data database 103. Case 2: Terminal 101 acquires video data captured by camera 102 and stores it in user data database 103. Case 3: Camera 102 stores video data in user data database 103.

[0025] The user data database 103 is a database that stores user data such as video data. The user data database 103 may be implemented using either an on-premise system or a cloud-based system used by the user.

[0026] The compression system 100 compresses video data stored in the user data database 103 using a lossy compression method. The compression system 100 may be implemented using either an on-premise system or a cloud-based system used by the user. Alternatively, the compression system 100 may be implemented using a cloud-based system used by a service provider different from the user who owns the video data, and provided as a service.

[0027] The compression system 100 may also acquire video data directly from the terminal 101 and the camera 102.

[0028] Figure 3 shows an example of the configuration of the compression system 100 of Example 1.

[0029] The compression system 100 has a hardware configuration consisting of a CPU 200, a GPU 201, memory 202, a DMA 203, a storage device 204, and a network interface 205. Each hardware element is connected to the others via a bus 206. There may be two or more of each hardware element.

[0030] The CPU200 and GPU201 are arithmetic units that perform various calculations. The functional units described later are realized when the CPU200 or GPU201 executes a program.

[0031] Memory 202 stores the program executed by the CPU 200 and the data used by that program. DMA 203 is a device that performs DMA transfers. Network interface 205 is a device that communicates over the network.

[0032] The storage device 204 is an HDD (Hard Disk Drive) or SSD (Solid State Drive), etc. The storage device 204 stores the program that implements the compression service unit 210 and the motion vector extraction unit 211, as well as the video data database 212 and the compression setting database 213.

[0033] The compression service unit 210 performs various settings related to the compression process and compresses the video data. Figure 4 shows an example of the functional configuration of the compression service unit 210 in Embodiment 1.

[0034] The compression service unit 210 includes a setting unit 300 and a compression unit 301. The setting unit 300 presents an interface for defining important objects and setting the compression method, and performs various settings. The compression unit 301 compresses the video data. The compression unit 301 includes a decoder 401, an annotation control unit 402, an annotation data generation unit 403, and an encoder 404, which will be described later.

[0035] The motion vector extraction unit 211 extracts motion vectors contained in the frames that make up the image data.

[0036] The video data database 212 stores video data obtained from the user data database 103. The compression settings database 213 stores various settings for compressing video data.

[0037] The compression settings database 213 stores important object data, including, for example, object type, frame, and annotation data. The annotation data is, for example, a monochrome image indicating the region within the frame where the important object is located.

[0038] Furthermore, the compression settings database 213 stores compression setting information related to the video data compression method. Figures 5A and 5B show examples of compression setting information stored in the compression settings database 213 of Embodiment 1.

[0039] The compression settings information shown in Figure 5A includes the GOP (Group of Pictures), thresholds, and the maximum number of skips. The compression settings information shown in Figure 5B includes the GOP, thresholds for each frame type, and the maximum number of skips. Users can set various compression settings depending on the type of video data.

[0040] Figure 6 shows an example of the interface provided by the setting unit 300 of Embodiment 1.

[0041] The configuration unit 300 displays the management screen 600. The user operates the management screen 600 to configure the compression settings. The user can configure multiple compression settings.

[0042] Figure 7 shows the flow of video data compression processing in the compression system 100 of Example 1.

[0043] The video data stored in the video data database 212 of the storage device 204 is sent to the CPU 200 and also to the GPU 201 via the DMA 203.

[0044] The CPU 200 executes a program that implements the motion vector extraction unit 211, and extracts motion vectors contained in each frame that makes up the video data. The CPU 200 sends the motion vectors, which have frame identification information attached, to the GPU 201.

[0045] GPU201 stores video data and motion vectors in GPU memory 400.

[0046] The decoder 401 converts the video data into frames and transmits them to the annotation data generation unit 403 and the encoder 404.

[0047] The annotation control unit 402 analyzes the motion characteristics of objects in the video data using motion vectors and determines whether or not to execute the annotation data generation process. Based on the determination result, the annotation control unit 402 transmits a control instruction generated by the annotation control unit 403.

[0048] The annotation data generation unit 403 determines, based on the control instruction, whether or not to perform annotation data generation processing on the received frame. If the annotation data generation processing is to be performed, the annotation data generation unit 403 performs annotation data generation processing on the received frame and transmits the generated annotation data to the encoder 404. If the annotation data generation processing is not to be performed, the annotation data generation unit 403 transmits the annotation data generated in the previous annotation data generation processing to the encoder 404.

[0049] The annotation data generation unit 403 generates annotation data using a machine learning model generated, for example, by few-shot learning or zero-shot learning. The machine learning model takes important object data and frames as input and calculates the probability that each pixel in the frame is an important object. The annotation data generation unit 403 generates annotation data based on the output of the machine learning model.

[0050] Alternatively, a separate machine learning model can be prepared for each type of object, and the machine learning model can be switched depending on the specified object type. The machine learning models used in this method do not require input of important object data.

[0051] The encoder 404 compresses the frames into a predetermined format based on the annotation data and compression setting information, and outputs compressed data consisting of the compressed frames. The encoder 404 is, for example, an encoder for a standardized video codec such as AVC.

[0052] Figure 8 is a flowchart illustrating an example of the processing performed by the annotation control unit 402 in Embodiment 1. The following explanation will use the case where compression setting information as shown in Figure 5A is configured as an example.

[0053] The annotation control unit 402 manages the number of times it has determined that annotation data generation is not necessary as the skip count. The initial value of the skip count is 0.

[0054] The annotation control unit 402 calculates an evaluation index using the motion vectors included in the frame to be processed (step S201).

[0055] For example, the annotation control unit 402 calculates the absolute difference between the motion vector contained in the frame to be processed and the motion vector contained in the frame immediately preceding the frame to be processed as an evaluation index. The evaluation index described above is just an example and is not limited to it. Any index that can evaluate the magnitude of the motion of the objects contained in the frame is acceptable.

[0056] When considering frame types, evaluation metrics should be calculated using frames of the same type.

[0057] The annotation control unit 402 determines whether or not it is necessary to generate annotation data using evaluation metrics (step S202).

[0058] Specifically, the annotation control unit 402 determines whether the evaluation metric is greater than the threshold included in the compression setting information. If the evaluation metric is greater than the threshold, it determines that annotation data needs to be generated.

[0059] When considering the frame type, the determination can be made using a threshold corresponding to the frame type.

[0060] If it is determined that annotation data needs to be generated, the annotation control unit 402 initializes the skip count to 0 (step S203), generates an execution instruction to instruct the annotation data generation process to be executed, and sends it to the annotation data generation unit 403 (step S204). After that, the annotation control unit 402 terminates the process.

[0061] If it is determined that annotation data does not need to be generated, the annotation control unit 402 determines whether the current number of skips is greater than the upper limit of skips included in the compression setting information (step S205).

[0062] If the current number of skips is less than or equal to the maximum number of skips, the annotation control unit 402 proceeds to step S203.

[0063] If the current number of skips is greater than the maximum number of skips, the annotation control unit 402 adds 1 to the current number of skips (step S206), generates a skip instruction to skip the annotation data generation process, and sends it to the annotation data generation unit 403 (step S207). After that, the annotation control unit 402 terminates the process.

[0064] The annotation control unit 402 determines whether the movement of objects in a frame is large or small based on the evaluation index. If the movement of objects in a frame is large, it is highly likely that the position of important objects has moved from the previous frame, so the annotation control unit 402 sends an execution instruction. If the movement of objects in a frame is small, it is highly likely that the position of important objects has not moved from the previous frame, so the annotation control unit 402 sends a skip instruction. By controlling the execution of the annotation data generation process based on the motion vector in this way, the frequency of execution of the annotation data generation process in the compression process can be reduced.

[0065] If the annotation data generation process is skipped a certain number of times in a row, the probability of misalignment of important objects increases. Therefore, if the number of skips exceeds the maximum number of skips, the system is controlled to execute the annotation data generation process. Note that steps S205-S207 may be omitted.

[0066] Figure 9 is a flowchart illustrating an example of the processing performed by the annotation data generation unit 403 in Example 1.

[0067] The annotation data generation unit 403 acquires a frame (step S301) and receives a control instruction from the annotation control unit 402 (step S302).

[0068] The annotation data generation unit 403 determines whether the received control instruction is an execution instruction (step S303).

[0069] If the received control instruction is an execution instruction, the annotation data generation unit 403 generates annotation data by executing the annotation data generation process (step S304). After that, the annotation data generation unit 403 proceeds to step S306.

[0070] In step S304, the annotation data generation unit 403 stores the annotation data in the GPU memory 400. If annotation data is already stored, the annotation data generation unit 403 overwrites the existing annotation data.

[0071] If the received control instruction is a skip instruction, the annotation data generation unit 403 retrieves the previous annotation data from the GPU memory 400 (step S305). After that, the annotation data generation unit 403 proceeds to step S306.

[0072] The annotation data generation unit 403 transmits annotation data to the encoder 404 (step S306). After that, the annotation data generation unit 403 terminates processing.

[0073] As described above, the compression system 100 of Example 1 can analyze the motion characteristics of objects in video data based on motion vectors included in the frames and control the execution of annotation data generation processing based on the analysis results. [Examples]

[0074] Example 2 differs from Example 1 in its method of extracting motion vectors. The following description focuses on the differences between Example 2 and Example 1.

[0075] The system configuration of Example 2 is the same as that of Example 1. Figure 10 shows an example of the configuration of the compression system 100 of Example 2.

[0076] The hardware configuration of the compression system 100 in Example 2 is the same as that of Example 1. The software configuration of the compression system 100 in Example 2 is slightly different. In Example 2, the decoder 401 includes the functionality of the motion vector extraction unit 211.

[0077] Figure 11 is a diagram showing the video data compression process of the compression system 100 in Example 2.

[0078] The video data stored in the video data database 212 of the storage device 204 is transmitted to the GPU 201 via DMA 203.

[0079] GPU201 stores video data in GPU memory 400.

[0080] The decoder 401 converts the video data into frames and extracts motion vectors from the frames. The decoder 401 stores the motion vectors, which have frame identification information attached, in the GPU memory 400. The decoder 401 also transmits the frames to the annotation data generation unit 403 and the encoder 404.

[0081] The annotation control unit 402 analyzes the motion characteristics of objects in the video data using motion vectors and determines whether or not to execute the annotation data generation process. Based on the determination result, the annotation control unit 402 transmits a control instruction generated by the annotation control unit 403.

[0082] The annotation data generation unit 403 determines, based on the control instruction, whether or not it is necessary to perform annotation data generation processing for the received frame. If it is necessary to perform annotation data generation processing, the annotation data generation unit 403 performs the annotation data generation processing and transmits the generated annotation data to the encoder 404. If it is not necessary to perform annotation data generation processing, the annotation data generation unit 403 transmits the annotation data generated in the previous annotation data generation processing to the encoder 404.

[0083] The processing performed by the annotation control unit 402 and the annotation data generation unit 403 in Example 2 is the same as in Example 1.

[0084] According to Example 2, since the GPU201 performs all processing, the processing load on the CPU200 can be reduced. [Examples]

[0085] In Example 3, the processing performed by the annotation control unit 402 and the annotation data generation unit 403 differs in some respects. Below, Example 3 will be described focusing on the differences from Example 1.

[0086] The system configuration of Example 3 is the same as that of Example 1. The configuration of the compression system 100 in Example 3 is the same as that of Example 1. Alternatively, the configuration of Example 2 may be adopted.

[0087] The compression process in Example 3 is the same as in Example 1. Alternatively, the processing method in Example 2 may be used.

[0088] Figure 12 is a flowchart illustrating an example of the processing performed by the annotation control unit 402 in Example 3.

[0089] The annotation control unit 402 calculates an evaluation index for the right-side region using motion vectors contained in the right-side region of the frame to be processed, and calculates an evaluation index for the left-side region using motion vectors contained in the left-side region of the frame to be processed (step S401).

[0090] For example, the annotation control unit 402 calculates the absolute difference between the motion vector contained in the right-hand region of the frame being processed and the motion vector contained in the right-hand region of the frame immediately preceding the frame being processed, as an evaluation index for the right-hand region. The evaluation index described above is just an example and is not limited to it. Any index that can evaluate the magnitude of the movement of an object contained in the right-hand region of a frame is acceptable.

[0091] The annotation control unit 402 determines whether or not it is necessary to generate annotation data for the right-side region using the evaluation index for the right-side region (step S402).

[0092] Specifically, the annotation control unit 402 determines whether the evaluation index of the right-hand region is greater than the threshold included in the compression setting information. If the evaluation index of the right-hand region is greater than the threshold, it determines that annotation data for the right-hand region needs to be generated.

[0093] If it is determined that annotation data needs to be generated for the right-side region, the annotation control unit 402 generates a first execution instruction to instruct the annotation data generation process for the right-side region to be executed and sends it to the annotation data generation unit 403 (step S403). After that, the annotation control unit 402 proceeds to step S405.

[0094] If it is not necessary to generate annotation data for the right-hand region, the annotation control unit 402 generates a first skip instruction and sends it to the annotation data generation unit 403 (step S404). The annotation control unit 402 then proceeds to step S405.

[0095] In step S405, the annotation control unit 402 determines whether or not it is necessary to generate annotation data for the left region using the evaluation index for the left region (step S405).

[0096] Specifically, the annotation control unit 402 determines whether the evaluation index of the left-hand region is greater than the threshold included in the compression setting information. If the evaluation index of the left-hand region is greater than the threshold, it determines that annotation data for the left-hand region needs to be generated.

[0097] If it is determined that annotation data needs to be generated for the left-side region, the annotation control unit 402 generates a second execution instruction to instruct the annotation data generation process for the left-side region to be executed and sends it to the annotation data generation unit 403 (step S406). After that, the annotation control unit 402 terminates the process.

[0098] If it is not necessary to generate annotation data for the left-hand region, the annotation control unit 402 generates a second skip instruction and sends it to the annotation data generation unit 403 (step S407). After that, the annotation control unit 402 terminates processing.

[0099] Figure 13 is a flowchart illustrating an example of the processing performed by the annotation data generation unit 403 in Example 3.

[0100] The annotation data generation unit 403 acquires a frame (step S301) and receives a control instruction from the annotation control unit 402 (step S302). In Embodiment 3, two control instructions are received.

[0101] The annotation data generation unit 403 determines whether the received control instruction is a first execution instruction (step S503).

[0102] If the received control instruction is a first execution instruction, the annotation data generation unit 403 generates annotation data for the right-side region by executing an annotation data generation process using the right-side region of the frame (step S504). At this time, the annotation data generation unit 403 stores the annotation data for the right-side region in the GPU memory 400. If annotation data for the right-side region is already stored, the annotation data generation unit 403 overwrites the annotation data for the right-side region.

[0103] If the received control instruction is a first skip instruction, the annotation data generation unit 403 retrieves the annotation data for the previous right-hand region from the GPU memory 400 (step S505).

[0104] The annotation data generation unit 403 determines whether the received control instruction is a second execution instruction (step S506).

[0105] If the received control instruction is a second execution instruction, the annotation data generation unit 403 generates annotation data for the left region by executing annotation data generation processing using the left region of the frame (step S507). At this time, the annotation data generation unit 403 stores the annotation data for the left region in the GPU memory 400. If annotation data for the left region is already stored, the annotation data generation unit 403 overwrites the annotation data for the left region.

[0106] If the received control instruction is a second skip instruction, the annotation data generation unit 403 retrieves the annotation data for the previous left region from the GPU memory 400 (step S508).

[0107] The annotation data generation unit 403 integrates the annotation data for the right region and the annotation data for the left region to generate annotation data for the entire frame (step S509).

[0108] The annotation data generation unit 403 transmits annotation data to the encoder 404 (step S510). After that, the annotation data generation unit 403 terminates processing.

[0109] In Example 3, the annotation data generation process is performed using an image smaller than the entire frame, thus reducing the processing load.

[0110] Although the explanation described the process of dividing the frame into two regions, the right and left regions, it is not limited to this. The frame may be divided into any number of regions, and the same process may be performed. [Examples]

[0111] In Example 4, the annotation control unit 402 analyzes the motion characteristics of objects in the video data based on the type of video data and controls the annotation data generation process. The following describes Example 4, focusing on the differences from Example 1.

[0112] The system configuration of Example 4 is the same as that of Example 1. The configuration of the compression system 100 in Example 4 is the same as that of Example 1. Alternatively, the configuration of Example 2 may be adopted.

[0113] The compression settings database 213 in Example 4 stores frequency adjustment information. Figure 14 shows an example of the data structure of the frequency adjustment information in Example 4.

[0114] The frequency adjustment information 1400 stores entries including type 1401 and frequency 1402. Type 1401 is a field that stores the type of video data. In this embodiment, the type of video data is represented by the equipment and shooting environment used to capture the video data. Frequency 1402 is a field that stores the execution frequency of the annotation data generation process. Frequency 1402 stores the number of frames.

[0115] In video data recorded by a dashcam, significant object movement is expected. Therefore, it is advisable to set a high frequency for the annotation data generation process. On the other hand, in video data recorded by a camera fixed in a warehouse, little object movement is expected. Therefore, a low frequency for the annotation data generation process can be set.

[0116] In Example 4, metadata including equipment and shooting environment is attached to the video data.

[0117] Figure 15 is a flowchart illustrating an example of the processing performed by the annotation control unit 402 in Embodiment 4. In Embodiment 4, the annotation control unit 402 performs the processing described below when video data compression begins.

[0118] The annotation control unit 402 identifies the type of video data based on the metadata attached to the video data (step S601).

[0119] The annotation control unit 402 refers to the frequency adjustment information 1400 to search for an entry corresponding to the type of video data, and determines the frequency based on the searched entry (step S602).

[0120] The annotation control unit 402 generates a control instruction including the determined frequency (step S603) and transmits the control instruction to the encoder 404 (step S604). After that, the annotation control unit 402 terminates processing.

[0121] Figure 16 is a flowchart illustrating an example of the processing performed by the annotation data generation unit 403 in Example 4.

[0122] In Example 4, the annotation data generation unit 403 initializes the variable k to 0 when it receives a control instruction. The annotation data generation unit 403 then executes the process described below.

[0123] The annotation control unit 402 acquires the frame (step S701) and increments the variable k by 1 (step S702).

[0124] The annotation data generation unit 403 determines whether the variable k matches the frequency included in the control instruction (step S703).

[0125] If the frequency of variable k matches that included in the control instruction, the annotation data generation unit 403 generates annotation data by executing the annotation data generation process (step S704). The annotation data generation unit 403 initializes variable k to 0 (step S705), and then proceeds to step S707.

[0126] At this time, the annotation data generation unit 403 stores the annotation data in the GPU memory 400. If annotation data is already stored, the annotation data generation unit 403 overwrites the existing annotation data.

[0127] If the frequency of variable k does not match that included in the control instruction, the annotation data generation unit 403 retrieves the previous annotation data from the GPU memory 400 (step S706). Then, the annotation data generation unit 403 proceeds to step S707.

[0128] The annotation data generation unit 403 transmits annotation data to the encoder 404 (step S707). After that, the annotation data generation unit 403 terminates processing.

[0129] The compression system 100 of Example 4 can analyze the motion characteristics of objects in video data based on the type of video data, and control the execution of annotation data generation processing based on the analysis results. [Examples]

[0130] In Example 5, the annotation control unit analyzes the motion characteristics of objects in the video data based on the frame rate of the video data and controls the annotation data generation process. The following describes Example 5, focusing on the differences from Example 1.

[0131] The system configuration of Example 5 is the same as that of Example 1. The configuration of the compression system 100 in Example 5 is the same as that of Example 1. Alternatively, the configuration of Example 2 may be adopted.

[0132] The compression settings database 213 in Example 5 stores frequency adjustment information. Figure 17 shows an example of the data structure of the frequency adjustment information in Example 5.

[0133] The frequency adjustment information 1700 stores entries including frame rate 1701 and frequency 1702. Frame rate 1701 is a field that stores the frame rate of the video data. Frequency 1702 is a field that stores the execution frequency of the annotation data generation process. Frequency 1702 stores the number of frames.

[0134] When the frame rate is high, the movement of objects between frames is expected to be small, so the frequency of annotation data generation processing can be set low. When the frame rate is low, the movement of objects between frames is expected to be large, so it is better to set the frequency of annotation data generation processing high.

[0135] Figure 18 is a flowchart illustrating an example of the processing performed by the annotation control unit 402 in Example 5. In Example 5, the annotation control unit 402 performs the processing described below when video data compression begins.

[0136] The annotation control unit 402 determines the frame rate of the video data (step S801). The method for determining the frame rate can be any known technique. For example, one method is to pre-add metadata, including the frame rate, to the video data.

[0137] The annotation control unit 402 refers to the frequency adjustment information 1700 to search for entries corresponding to the frame rate of the video data and determines the frequency based on the searched entries (step S802).

[0138] The annotation control unit 402 generates a control instruction including the specified frequency (step S803) and transmits the control instruction to the encoder 404 (step S804). After that, the annotation control unit 402 terminates processing.

[0139] The processing performed by the annotation data generation unit 403 in Example 5 is the same as in Example 4, so its explanation will be omitted.

[0140] The compression system 100 of Example 5 can analyze the motion characteristics of objects in video data based on the frame rate of the video data and the type of video data, and can control the execution of annotation data generation processing based on the analysis results.

[0141] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. Furthermore, for example, the embodiments described above are detailed explanations of the configuration in order to clearly illustrate the present invention, and are not necessarily limited to those having all the configurations described. In addition, some of the configurations in each embodiment can be added to, deleted from, or replaced with other configurations.

[0142] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, in whole or in part, for example, by designing them as integrated circuits. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0143] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, Python, and Java (registered trademark).

[0144] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0145] In the above-described embodiment, the control lines and information lines shown are those deemed necessary for explanation and do not necessarily represent all control lines and information lines in the actual product. All components may be interconnected. [Explanation of Symbols]

[0146] 100 Compression Systems 101 terminals 102 Cameras 103 User Data Database 110 Network 200 CPU 201 GPU 202 memory 203 DMA 204 Storage device 205 Network Interfaces 206 Bus 210 Compression Service Department 211 Motion vector extraction unit 212 Video Data Database 213 Compression Settings Database 300 Setting section 301 Compression section 400 GPU memory 401 Decoder 402 Annotation Control Unit 403 Annotation Data Generation Unit 404 IOS 1400 Frequency adjustment information 1700 Frequency adjustment information

Claims

1. A computer system, The system comprises at least one computing device and a storage device connected to the at least one computing device, The aforementioned storage device stores video data consisting of multiple frames. The frame contains at least one object, The at least one arithmetic unit is, The motion characteristics of the object in the aforementioned video data are analyzed, Based on the results of the above analysis, control the execution of the process that generates annotation data indicating the location of important objects included in the frame. A computer system characterized by compressing the frame using the annotation data such that the amount of data in the areas where the important objects exist is large, and the amount of data in the areas where the important objects do not exist is small.

2. A computer system according to claim 1, The at least one arithmetic unit is, Analyze the magnitude of the movement of the object included in the aforementioned frame, A computer system characterized by determining whether to perform the processing on the frame based on the results of the analysis.

3. A computer system according to claim 2, The at least one arithmetic unit is, Using the motion vectors contained in the frame, an evaluation index is calculated to evaluate the magnitude of the motion of the objects contained in the frame. Based on the comparison result between the evaluation index and the first threshold, it is determined whether or not to perform the processing on the frame. A computer system characterized in that, if the processing is not performed on the frame, the frame is compressed using the annotation data generated by the processing performed previously.

4. A computer system according to claim 3, The at least one arithmetic unit is, The number of times it is determined that the above process should not be performed for the above frame is counted. A computer system characterized in that, if the number of times is greater than a second threshold, it is determined to perform the processing on the frame.

5. A computer system according to claim 1, The at least one arithmetic unit is, The aforementioned video data was analyzed by type, A computer system characterized by determining the frequency of execution of the process based on the results of the analysis.

6. A computer system according to claim 1, The at least one arithmetic unit is, Analyze the frame rate of the aforementioned video data. A computer system characterized by determining the frequency of execution of the process based on the results of the analysis.

7. A method for compressing video data performed by a computer system, The computer system has at least one arithmetic unit and a storage device connected to the at least one arithmetic unit. The aforementioned storage device stores video data consisting of multiple frames. The frame contains at least one object, The aforementioned video data compression method is: The first step involves at least one computing device analyzing the motion characteristics of an object in the video data, A second step involves the execution of a process by which at least one computing device generates annotation data indicating the locations of important objects included in the frame based on the results of the analysis. A third step in which at least one computing device compresses the frame using the annotation data such that the amount of data in the areas where the important objects exist is large and the amount of data in the areas where the important objects do not exist is small. A method for compressing video data, characterized by including the following:

8. A method for compressing video data according to claim 7, The first step includes the step of at least one computing unit analyzing the magnitude of motion of an object included in the frame, The second step is a method for compressing video data, characterized in that the at least one computing device determines whether to perform the processing on the frame based on the results of the analysis.

9. A method for compressing video data according to claim 8, The first step includes the step of the at least one computing device calculating an evaluation index for evaluating the magnitude of motion of an object included in the frame using the motion vectors included in the frame, The second step includes the step of determining whether or not to perform the processing on the frame based on the result of comparing the evaluation index with the first threshold, A method for compressing video data, characterized in that, if the processing is not performed on the frame, at least one computing unit compresses the frame using the annotation data generated by the processing performed previously.

10. A method for compressing video data according to claim 7, The first step includes the step of at least one computing device analyzing the type of video data, The second step is a method for compressing video data, characterized in that the at least one computing device determines the frequency of execution of the process based on the results of the analysis.

11. A method for compressing video data according to claim 7, The first step includes the step of at least one computing device analyzing the frame rate of the video data, The second step is a method for compressing video data, characterized in that the at least one computing device determines the frequency of execution of the process based on the results of the analysis.

12. This is a program to be executed by a computer that compresses video data. The computer has at least one arithmetic unit and a storage device connected to the at least one arithmetic unit. The aforementioned storage device stores video data consisting of multiple frames. The frame contains at least one object, The aforementioned program, A first step in analyzing the motion characteristics of an object in the aforementioned video data, A second step controls the execution of a process that generates annotation data indicating the location of important objects included in the frame, based on the results of the analysis described above. A third step involves compressing the frame using the annotation data such that the amount of data in the areas where the important objects exist is large, and the amount of data in the areas where the important objects do not exist is small. A program characterized by causing the computer to execute the above.

13. The program according to claim 12, The first step described above includes a step of analyzing the magnitude of motion of an object included in the frame, The program is characterized in that the second step includes a step of determining whether to perform the processing on the frame based on the results of the analysis.

14. The program according to claim 13, The first step described above includes a step of calculating an evaluation index for evaluating the magnitude of motion of an object included in the frame using the motion vector included in the frame, The second step includes a step of determining whether or not to perform the process on the frame based on the result of comparing the evaluation index with the first threshold, The program is characterized in that, if the processing is not performed on the frame, it includes a step of compressing the frame using the annotation data generated by the processing performed previously.

15. The program according to claim 12, The first step described above includes a step of analyzing the type of video data, The program is characterized in that the second step includes a step of determining the frequency of execution of the process based on the results of the analysis.

16. The program according to claim 12, The first step described above includes a step of analyzing the frame rate of the video data, The program is characterized in that the second step includes a step of determining the frequency of execution of the process based on the results of the analysis.

Images