Computer system, program and data compression method

Abstract

To efficiently generate compressed data that is compatible with a transmission destination.SOLUTION: A computer system calculates a first feature quantity of an object type using object type definition data that defines the object type to be detected in a space defined by the dimension of the processed data, which is extracted from the multidimensional data to be compressed on a plane of any dimension, acquires the processed data and calculates a second feature quantity of the processed data, uses the first feature quantity and the second feature quantity to estimate an important area where the object type exists in the space of the processed data, generates compression level information containing parameters for determining the data volume of the important area and the area outside the important area, and performs non-reversible compression using the compression level information to generate compressed data in a data format compatible with a transmission destination.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a compression technique for reducing the volume of data. [Background technology]

[0002] Lossy compression technologies with high compression ratios are required to reduce the costs required for data storage and transmission. Furthermore, these lossy compression technologies must be highly efficient in order to reduce the computational costs required for compression, in addition to having a high compression ratio. Furthermore, it is desirable for the compressed data generated by lossy compression technologies to conform to commonly used data formats in terms of compatibility.

[0003] Known examples of lossy compression techniques for video data include standardized compression techniques such as Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC).

[0004] In addition, a technology is known that uses a Deep Neural Network (DNN) such as an autoencoder to control the bit allocation amount of multidimensional data for each region based on user specifications and generate compressed data (paragraphs 0169 to 0178 of Patent Document 1).

[0005] In industrial data, it is not always necessary to reproduce all information contained in the data with high fidelity after compression and decompression. For example, when inspecting a power transmission tower using video data captured by a drone, high image quality is required in the area where the power transmission tower is captured, while degradation in image quality can be tolerated in areas such as vegetation in the background. According to Patent Document 1, by controlling the bit allocation amount so that areas containing object types such as power transmission towers have high image quality and other areas are highly compressed, it is possible to generate data that is suited to the application and has a high compression rate. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2020-155071 Summary of the Invention [Problem to be solved by the invention]

[0007] The technology disclosed in Patent Document 1 is expected to achieve a high compression rate. However, because the type of bit string that a DNN generates as compressed data is determined through learning, there is a problem (Problem 1) in that the compressed data generated by the lossy compression technology disclosed in Patent Document 1 is not compatible with commonly used data formats such as AVC. [Means for solving the problem]

[0008] A representative example of the invention disclosed in the present application is as follows: That is, a computer system includes a processor, a storage device connected to the processor, and a connection device connected to the processor for connecting to an external device, wherein the processor executes the following steps: a first process of calculating a first feature amount of an object type to be detected using object type definition data that defines the object type in a space defined by the dimensions of processed data obtained by cutting out multidimensional data to be compressed on a plane of any dimension; a second process of acquiring the processed data and calculating a second feature amount of the processed data; a third process of estimating an important region in the space of the processed data in which the object type exists using the first feature amount and the second feature amount; a fourth process of generating compression level information including parameters for determining the data amount of each of the important region and a region other than the important region; and a fifth process of generating the compressed data by converting the processed data into a data format that ensures compatibility with a destination of the compressed data through lossy compression using the compression level information. [Effects of the Invention]

[0009] According to the present invention, it is possible to efficiently generate compressed data that is compatible with the destination. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiment. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an overview of a system according to a first embodiment. [Figure 2] An example of a machine learning model for semantic segmentation using conventional few-shot learning techniques is shown below. [Figure 3] FIG. 1 illustrates an example of a system configuration according to a first embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of a data structure of pre-processing parameter management information according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a data structure of object type information according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a data structure of object type definition information according to the first embodiment. [Figure 7] 10 is a flowchart illustrating an example of a registration process of object type information executed by a compression unit according to the first embodiment. [Figure 8] FIG. 10 illustrates an example of an object type setting interface provided by the compression unit according to the first embodiment. [Figure 9] 10 is a flowchart illustrating an example of a registration process of object type definition information executed by a compression unit according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of an object type definition interface provided by the compression unit according to the first embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of an object type definition interface provided by the compression unit according to the first embodiment. [Figure 12] 10 is a flowchart illustrating an example of a compression process executed by a compression unit according to the first embodiment. [Figure 13] 10 is a flowchart illustrating an example of pre-processing executed by a compression unit according to the first embodiment. [Figure 14] 10 is a flowchart illustrating an example of a compression level information generation process executed by a compression unit according to the first embodiment. [Figure 15] FIG. 10 is a diagram illustrating an outline of a system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] In addition to the above-mentioned problems, the technology described in Patent Document 1 also has the following problems. (Problem 2) The definition of object types (object types) and the bit allocation amount for each region are hard-coded as learning parameters for the DNN. Therefore, when changing the definition of an object type, a large amount of learning data including training data indicating the object type is required, and re-learning takes time. (Problem 3) The DNN accepts high-resolution original data as input, determines the bit allocation amount, and generates compressed data, resulting in slow compression speed. For example, when a convolutional neural network is used as the DNN, the amount of calculation generally increases in proportion to the input resolution. Therefore, it takes a long time to process high-resolution data such as Full-HD and 4K.

[0012] Hereinafter, embodiments of the present invention that solve the three problems will be described with reference to the drawings. However, the present invention should not be construed as being limited to the description of the embodiments shown below. Those skilled in the art will readily understand that the specific configuration can be changed within the scope of the idea or purpose of the present invention.

[0013] In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and duplicated explanations will be omitted. [Example]

[0014] First, an overview of the system of the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating the overview of the system of the first embodiment.

[0015] The system of the first embodiment comprises a data generation source 100 , an object type setting interface 101 , an object type definition interface 102 , and a compression unit 103 .

[0016] The data generation source 100 is an entity that generates multidimensional data to be compressed, such as an image sensor that generates video data. Video data has spatial, temporal, and channel dimensions. A frame (image) is data obtained by cutting out video data on a time plane. In this embodiment, an example will be described in which the data generation source 100 is an image sensor that generates video data.

[0017] The data generation source 100 and the generated data are not limited to these, and may be, for example, an image sensor that generates still image data, a vibration sensor that generates one-dimensional time-series data, or the like. The data generation source 100 is not limited to a sensor, but may be software such as computer graphics that generates video data and still image data. The data from the data generation source 100 may be data obtained by processing data generated by a sensor and software, such as a segmentation map obtained by applying a semantic segmentation machine learning model to each frame of video data. The data from the data generation source 100 may be a video file or the like stored in a recording device. There may be multiple data generation sources 100.

[0018] The object type setting interface 101 is an interface that allows the user to specify the type of object (object type) to be detected. Here, object types include, for example, people, cars, roads, steel towers, and buildings. Information on the specified object type is managed as object type information 332. In this embodiment, the compression unit 103 compresses each frame of video data generated by the data generation source 100 so that areas where the specified object type exists have high image quality and areas where the object type does not exist are highly compressed.

[0019] The object type information 332 stores entries including a data source ID 501 and an object type ID 502. The data source ID 501 stores an identifier representing the data source 100, and the object type ID 502 stores an identifier representing the object type.

[0020] 1, a plurality of object types may be specified for one data generation source 100. The number of object types specified for one data generation source 100 may be one.

[0021] The object type definition interface 102 is an interface for inputting object type definition data 221 (see FIG. 2) that defines the characteristics of an object type.

[0022] The compression unit 103 is a module that compresses the multidimensional data generated by the data generation source 100. The compression unit 103 may generate compressed data 104 for each frame of a video, for every predetermined number of frames, or by compressing the entire video file to generate compressed data 104. The compression unit 103 includes an object type definition data conversion unit 120, a preprocessing unit 121, an image conversion unit 122, a similarity calculation unit 123, a compression level information generation unit 124, and an encoder 125.

[0023] The object type definition data conversion unit 120 is a unit that converts the object type definition data input by the user via the object type definition interface 102 into an object type vector (for example, vectors 141 and 142 in FIG. 1), which is a feature that expresses the object type. Note that the object type vector is typically a one-dimensional vector, but is not limited to this and may be data of any data structure, such as a tensor or associative array having two or more dimensions.

[0024] The object type vector calculated by the object type definition data conversion unit 120 is stored in the object type definition information 333 .

[0025] The object type definition information 333 stores entries including an object type ID 601 and an object type feature amount 602. The object type ID 601 stores an identifier representing the object type, and the object type feature amount 602 stores an object type vector, which is the feature amount of the object type. Note that the entry may also include a field for managing parameters for setting the image quality of an area where an object corresponding to the object type exists.

[0026] By combining the object type information 332 and the object type definition information 333, it is possible to manage the object type feature amount 602 corresponding to each data generation source ID 501. For example, the object type information 332 and the object type definition information 333 shown in Fig. 1 indicate that the object type vectors 141 and 142 correspond to the data generation source ID "A."

[0027] When the compression unit 103 acquires video data to be compressed, it inputs a frame of the data (hereinafter referred to as an original frame) to the pre-processing unit 121. The pre-processing unit 121 performs pre-processing such as downscaling on the input original frame, and generates a processed frame with a changed resolution, etc.

[0028] The image conversion unit 122 calculates the image feature 143 of the processed frame from the processed frame. The image feature is, for example, a tensor.

[0029] The similarity calculation unit 123 calculates the similarity between the image feature 143 and the object type vector. The output is, for example, a two-dimensional array that represents the detection result of the object type represented by the object type vector.

[0030] The compression level information generation unit 124 calculates compression level information for each compression unit of the encoder 125 based on the output of the similarity calculation unit 123 .

[0031] The encoder 125 compresses the original frames based on the compression level information generated by the compression level information generation unit 124, and generates compressed data 104. The encoder 125 is, for example, an encoder for a standardized video codec such as AVC. Note that the encoder 125 is not limited to the software encoder described above, and may be an HEVC encoder or a hardware encoder.

[0032] Here, the compression level information is a parameter of the encoder 125 that controls the bit allocation amount for each region. If the encoder 125 is an AVC-compliant encoder, the compression unit of the encoder 125 is a macroblock, and the compression level information is, for example, a quantization parameter value (QP value) for each macroblock, QP value difference information for each macroblock, information specifying the degree of image quality enhancement for each macroblock, etc. In this case, the compression level information generation unit 124, for example, calculates the maximum probability for each macroblock with respect to the output of the similarity calculation unit 123, assigns a predetermined QP value to macroblocks whose maximum value is greater than a predetermined threshold, and generates, as compression level information, spatial distribution information of QP values ​​in which relatively large predetermined QP values ​​are assigned to other macroblocks. Note that the above-mentioned compression level information is merely an example and is not limited to this.

[0033] The object type definition data conversion unit 120, the image conversion unit 122, and the similarity calculation unit 123 are units included in a machine learning model for semantic segmentation that uses, for example, few-shot learning or zero-shot learning techniques.

[0034] FIG. 2 shows an example of a machine learning model 200 for semantic segmentation using conventional few-shot learning techniques.

[0035] The machine learning model 200 receives an image 211 and object type definition data 221 as input, and outputs a detection result 240 of an area in the image 211 where an object corresponding to the object type specified by the object type definition data 221 exists.

[0036] 2 shows object type definition data 221 for setting a power transmission tower as an object type, and is composed of an image 222 showing the power transmission tower and a mask image 223 showing the area in the image where the power transmission tower is located. At least one piece of object type definition data 221 is required for one object type. Note that there may be multiple pieces of object type definition data 221 for one object type.

[0037] The image 211 is converted into image features 231 by the image conversion unit 122. The image conversion unit 122 is, for example, a convolutional neural network such as a Residual Network (ResNet), and converts the image 211 into a three-dimensional tensor consisting of spatial (length, width) and channel dimensions.

[0038] Note that the image conversion unit 122 is not limited to this, and may be, for example, a vision encoder using a Transformer such as Contrastive Language-Image Pre-Training (CLIP), a neural network with other structures, or any other processing module.

[0039] The object type definition data conversion unit 120 calculates an object type vector 232 representing an object type using the image 222 and mask image 223 included in the object type definition data 221. The object type definition data conversion unit 120 is, for example, a neural network such as ResNet, and converts the image 222 into a three-dimensional tensor consisting of spatial (vertical and horizontal) and channel dimensions, and applies average pooling in the spatial direction to a region of the tensor that is marked as an object type region in the mask image 223, to calculate the object type vector 232 representing the object type.

[0040] In addition, the object type definition data conversion unit 120 may calculate a vector representing the background by applying Average Pooling in the spatial direction to the area (background) of the tensor that is not marked as an object type area in the mask image 223, and may set the vector 232 representing the object type and the vector representing the background as a set to be the object type vector 232.

[0041] Note that the object type definition data conversion unit 120 is not limited to this, and may include, for example, a vision encoder using a Transformer such as CLIP.

[0042] The similarity calculation unit 123 calculates the similarity between each spatial position of the image feature 231 and the object type vector 232, and outputs the object type detection result 240. The similarity can be calculated using, for example, cosine similarity, but is not limited to this.

[0043] In general, the machine learning model 200 for semantic segmentation is treated as one module composed of an image conversion unit 122, an object type definition data conversion unit 120, and a similarity calculation unit 123.

[0044] Therefore, in conventional implementations, it is necessary to provide a machine learning model 200 for each object type, and as the number of object types increases, the calculation cost increases and it becomes difficult to speed up processing.

[0045] In the present invention, by using the functional configuration shown in FIG. 1, an increase in calculation cost is suppressed and an improvement in compression speed is realized.

[0046] 2, the object type definition data conversion unit 120 receives object type definition data 221 as input and does not depend on the image 211. Therefore, in this embodiment, the object type definition data conversion unit 120 executes processing in response to the setting of object type definition data via the object type definition interface 102, and stores the results in object type definition information 333. In other words, it is arranged independently from the unit that performs frame compression processing.

[0047] 2, the image conversion unit 122 receives an image 211 as input and does not depend on the object type definition data 221. Therefore, in this embodiment, the image conversion unit 122 executes processing when each processing frame is input, and stores the results in a cache. By storing the results in the cache, they can be reused in the detection process for each object type.

[0048] The similarity calculation unit 123 calculates the similarity for each object type for one frame, which is a process for estimating the area in which the object type exists within the frame.

[0049] 1, the number of calculations can be reduced compared to a naive implementation that runs the machine learning model 200 for each object type in each frame, thereby improving the compression speed. In conventional implementations, the machine learning model 200 is executed in proportion to the number of object types, so the effect of improving speed is particularly noticeable when there are multiple object types.

[0050] The machine learning model 200 is not limited to the one described above, and may be, for example, a deep learning model for object detection that utilizes few-shot learning technology.

[0051] Furthermore, the machine learning model 200 may input a natural language (for example, a character string such as "power transmission tower") as the object type definition data 221. In this case, the object type definition data conversion unit 120 may be, for example, a text encoder that converts the character string into a tensor.

[0052] The configuration of the system according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram illustrating an example of the configuration of the system according to the first embodiment.

[0053] The computer 300 is hardware that realizes the compression unit 103, and includes, for example, an arithmetic unit 310, a switch 311, a memory 312, a front-end interface 313, and a back-end interface 314.

[0054] The front-end interface 313 is an interface for connecting to the data generation source 100 and the management terminal 301. The back-end interface 314 is an interface for connecting to the storage device 302 and the network 303.

[0055] The arithmetic device 310 is a device that controls the entire computer 300, and may be, for example, a general-purpose arithmetic device such as a CPU (Central Processing Unit), an accelerator such as a GPU (Graphical Processing Unit) and an FPGA (Field Programmable Gate Array), or a hardware encoder / decoder for a standard codec such as HEVC, or a combination of the above. The arithmetic device 310 is connected to a memory 312 and the like via a switch 311.

[0056] The memory 312 stores programs executed by the arithmetic device 310 and information used by the programs. The memory 312 is also used as a work area. The memory 312 of the first embodiment stores a compression program 330 that implements the compression unit 103, preprocessing parameter management information 331, object type information 332, and object type definition information 333. The memory 312 also includes an image feature cache 334. The memory 312 may also store programs and information such as an OS (Operating System).

[0057] The program may be pre-installed in the computer 300, or may be stored in a non-transitory recording medium and installed.

[0058] The storage device 302 may be a block device configured with a hard disk drive (HDD) and a solid state drive (SSD), a file storage, a content storage, a volume constructed on a storage system, or any other method for accumulating data. Furthermore, if there is no need to store compressed data, the storage device 302 may be omitted.

[0059] The network 303 is a communication network such as a local area network (LAN) or the Internet. The compression unit 103 can transmit the compressed data 104 to other devices via the network 303. Note that if there is no need to transmit the compressed data 104 to other devices, the network 303 may be omitted.

[0060] The compression unit 103 may be realized using hardware in which hardware such as ICs (Integrated Circuits) are interconnected, or some of the functions of the compression unit 103 may be realized using a single semiconductor element such as an ASIC (Application Specific Integrated Circuit) and an FPGA. The compression unit 103 may also be realized using a VM (Virtual Machine) realized by virtualization technology. Components other than those shown here may also be added.

[0061] The data generation source 100, management terminal 301, computer 300, and storage device 302 may be different hardware devices, VMs running on the same computer, different containers running on the same operating system (OS), or applications running on the same OS. Also, a combination of multiple implementations may be used. For example, the data generation source 100 is an image sensor, the compression unit 103 is an edge device connected to the image sensor and including a computing device 310, the management terminal 301 is a terminal that can be operated by a user, and the storage device 302 is an HDD.

[0062] FIG. 4 is a diagram illustrating an example of the data structure of the pre-processing parameter management information 331 according to the first embodiment.

[0063] The preprocessing parameter management information 331 is, for example, data in a table format, and stores entries including a data generation source ID 401, a downscaling coefficient 402, and a downscaling algorithm 403. Note that the fields included in the entries are merely examples and are not limited to these.

[0064] The data source ID 401 is a field that stores the identifier of the data source 100. The identifier of the data source is, for example, a character string named by a user, a Media Access Control (MAC) address or an Internet Protocol (IP) address assigned to the data source 100, or any code that can identify the data source 100. Note that if the data source 100 is self-evident, the entry does not need to include the data source ID 401.

[0065] The downscale coefficient 402 and the downscale algorithm 403 are fields that store parameters for controlling the conversion of the original frame.

[0066] In the pre-processing parameter management information 331 shown in FIG. 4, an entry is set that defines pre-processing for reducing the length and width of the original frame of data generation source A to 1 / 16 using a bilinear algorithm.

[0067] The preprocessing parameter management information 331 may be set by a user via the management terminal 301, or may be automatically set by the arithmetic device 310 when the compression unit 103 is started up or a new data generation source 100 is added, or may be set by some other method. For example, when a data generation source 100 is added, the arithmetic device 310 may check the codec of the encoder 125 and determine the downscale coefficient based on the check result. For example, if the codec of the encoder 125 is AVC and compression level information can be specified by a QP value in units of 16 pixel x 16 pixel macroblocks, the arithmetic device 310 may set the downscale coefficient 402 to 1 / 16 based on the information that the encoder 125 is an AVC encoder.

[0068] The method for setting the various fields of the preprocessing parameter management information 331 is not limited to this.

[0069] The preprocessing parameter management information 331 may be any data structure that can manage parameters related to frame conversion, and may be a data structure other than a table, such as XML (Extensible Markup Language), YAML (YAML Ain't Markup Language), a hash table, or a tree structure.

[0070] FIG. 5 is a diagram illustrating an example of the data structure of the object type information 332 according to the first embodiment.

[0071] The object type information 332 is, for example, data in a table format, and stores entries including a data generation source ID 501 and an object type ID 502. Note that the fields included in the entries are merely examples and are not limited to these.

[0072] The data source ID 501 is a field for storing the identifier of the data source 100 and is the same as the data source ID 401 .

[0073] The object type ID 502 is a field for storing an identifier that indicates the type of object. The identifier is, for example, a character string named by the user, but is not limited to this.

[0074] The object type information 332 manages the object type ID 502 of the detection target in the video data generated by the data generation source 100 corresponding to the data generation source ID 501. For example, in the object type information 332 shown in Fig. 5, the object type IDs "obj1" and "obj2" are set for the data generation source ID "A."

[0075] Note that the information managed by the object type information 332 is not limited to the data generation source ID 501 and the object type ID 502. For example, parameters (e.g., Constant Rate Factor, Quantization Parameter, etc.) that control the compression rate and image quality when the video data generated by the data generation source 100 corresponding to the data generation source ID 501 is compressed by the encoder 125 may also be managed.

[0076] The data structure of the object type information 332 is not limited to table format data, and may be a data structure other than a table, such as XML, YAML, a hash table, or a tree structure.

[0077] FIG. 6 is a diagram illustrating an example of the data structure of the object type definition information 333 according to the first embodiment.

[0078] The object type definition information 333 is, for example, data in a table format, and stores entries including an object type ID 601, an object type feature amount 602, an image quality parameter 603, and a peripheral parameter 604. Note that the fields included in the entries are merely examples and are not limited to these.

[0079] The object type ID 601 is a field for storing an identifier of an object type, and is the same as the object type ID 502 .

[0080] The object type feature amount 602 is a field for storing an object type vector representing the object type corresponding to the object type ID 601 .

[0081] The image quality parameter 603 is a field for storing a parameter that specifies the image quality at the time of compression of an area in which an object type corresponding to the object type ID 601 exists.

[0082] For example, FIG. 6 shows an example in which image quality is specified in three levels: High, Mid, and Low. However, the image quality parameter 603 is not limited to this and may be, for example, an offset value to be added to the AVC QP value in an area where an object type exists. In this case, for example, if −10 is specified as the image quality parameter for the object type ID “obj1,” the compression level information generation unit 124 generates −10 as the QP value offset (compression level information) for the macroblock in the area where “obj1” exists, and the encoder 125 compresses the video by subtracting 10 from the QP value of that area. However, the definition of the image quality parameter 603 and the method of generating compression level information based on the image quality parameter are not limited to this. For example, if the encoder 125 receives information specifying the degree of enhancement of image quality for each macroblock as compression level information, the information may be managed as the image quality parameter.

[0083] The peripheral parameters 604 are fields for storing parameters for correcting an area in which the image quality is controlled for an area in which the object type corresponding to the object type ID 601 exists.

[0084] For example, a correction method may involve expanding the area in which image quality is controlled relative to the detected area. For example, in the maintenance of a power transmission tower, it may be necessary to retain not only the area where the tower is located but also its surrounding area in high image quality in order to inspect for overhanging tree branches. In this case, the surrounding parameter 604 is, for example, a parameter that represents the size of the surrounding area of ​​the object type to be retained in high image quality, and is, for example, a parameter that specifies how many pixels surrounding the area containing the object type should be compressed to high image quality. However, the definition of the surrounding parameter 604 is not limited to this and may be any parameter related to the process of correcting the area in which image quality is controlled relative to the area of ​​the detection result.

[0085] Note that the information managed by the object type definition information 333 is not limited to the object type ID 601, the object type feature amount 602, the image quality parameter 603, and the peripheral parameter 604. For example, trained parameters of the neural network of the image conversion unit 122 may be stored.

[0086] The data structure of the object type definition information 333 is not limited to table format data, and may be a data structure other than a table, such as XML, YAML, a hash table, or a tree structure.

[0087] Here, specific examples of information stored in the object type information 332 and the object type definition information 333 will be described with reference to FIGS.

[0088] The object type information 332 shown in FIG. 5 indicates that “obj1” and “obj2” are set as object type IDs 502 for video data generated by the data generation source 100 to which “A” is assigned as the data generation source ID 501.

[0089] 6 indicates that an object type vector 521 is set for the object type "obj1," that the image quality parameter 603 for the area where the object type "obj1" exists is set to "High," and that the peripheral parameter 604 is set to "10." The same is true for the object type "obj2."

[0090] For example, if the object types "obj1" and "obj2" represent a power transmission tower and a wind turbine, respectively, entry 611 is set with information for compressing the video data generated by data generation source A so that the area where the power transmission tower is located and the surrounding 10 pixels have "High" image quality. Entry 612 is set with information for compressing the video data generated by data generation source A so that the area where the wind turbine is located and the surrounding 20 pixels have "Mid" image quality.

[0091] Next, the processing executed by the compression unit 103 will be described with reference to FIGS.

[0092] Fig. 7 is a flowchart illustrating an example of a registration process of the object type information 332 executed by the compression unit 103 according to the first embodiment. Fig. 8 is a diagram illustrating an example of the object type setting interface 101 provided by the compression unit 103 according to the first embodiment.

[0093] The computing device 310 functioning as the compression unit 103 starts the processing described below when it receives a request from the management terminal 301 via the front-end interface 313. The trigger for executing the processing is not limited to the above-mentioned, and may also be the startup of the computer 300, etc.

[0094] The arithmetic device 310 provides the management terminal 301 with an object type setting interface 101 for setting an object type via the front-end interface 313 (step S701). The object type setting interface 101 will now be described with reference to FIG.

[0095] The object type setting interface 101 is displayed on a display device (not shown) of the management terminal 301. The user operates the object type setting interface 101 using an input device (not shown) of the management terminal 301.

[0096] The object type setting interface 101 displays a table 801. The table 801 is a table for confirming and registering entries including a data source ID 811 and an object type ID 812. The data source ID 811 and the object type ID 812 are the same fields as the data source ID 501 and the object type ID 502. Note that the table 801 may also display entries registered in the object type information 332.

[0097] The delete button 802 is an operation button for deleting an entry from the table 801. When the user operates the delete button 802, the entry is deleted from the object type information 332. Note that an entry in the object type information 332 may be deleted by an operation other than the operation of the delete button 802.

[0098] The add button 803 is an operation button for adding an entry to the table 801. Note that an entry may be automatically added after a value is set in the last entry. In this case, the add button 803 is not necessary.

[0099] Entry 821 is an entry added by operating the add button 803. For example, the user can input the data source ID as text into the data source ID 811. The user can directly input text into the object type ID 812, or a drop-down list can be displayed so that the user can make a selection. The drop-down list includes existing object type IDs and "New." If "New" is selected, the computing device 310 automatically generates and assigns an identifier.

[0100] The set button 804 is an operation button for registering the contents of the table 801 in the object type information 332. When the user operates the set button 804, the management terminal 301 sends a registration request including the table 801 to the compression unit 103. The compression unit 103 receives the registration request via the front-end interface 313 and updates the object type information 332 in accordance with the contents of the table 801.

[0101] 8 may be displayed in the table 801 of the object type setting interface 101. For example, a field may be displayed for inputting parameters (e.g., Constant Rate Factor, Quantization Parameter, etc.) that control the compression rate and image quality when the data generated by the data generation source 100 to which the data generation source ID 501 is assigned is compressed by the encoder 125.

[0102] This concludes the description of the object type setting interface 101. Note that the object type setting interface 101 is not limited to that shown in Fig. 8, and other information not shown may be displayed, a different operation method may be used, or the design may be different. Returning to the description of Fig. 7.

[0103] The arithmetic device 310 acquires the information input via the object type setting interface 101 (step S702), and updates the object type information 332 based on the information (step S703). After that, the arithmetic device 310 ends the object type information registration process.

[0104] Fig. 9 is a flowchart illustrating an example of a registration process of the object type definition information 333 executed by the compression unit 103 according to the first embodiment. Fig. 10 and Fig. 11 are diagrams illustrating an example of the object type definition interface 102 provided by the compression unit 103 according to the first embodiment.

[0105] The computing device 310 functioning as the compression unit 103 starts the processing described below when it receives a request from the management terminal 301 via the front-end interface 313. The trigger for executing the processing is not limited to the above-mentioned, and may also be the startup of the computer 300, etc.

[0106] The arithmetic device 310 provides the management terminal 301 with an object type definition interface 102 for setting the object type definition information 333 via the front-end interface 313 (step S901). The object type definition interface 102 will now be described with reference to Figs. 10 and 11.

[0107] The object type definition interface 102 is displayed on a display device (not shown) of the management terminal 301. The user operates the object type definition interface 102 using an input device (not shown) of the management terminal 301.

[0108] 10 displays a table 1031. Table 1031 is a table for checking and registering entries including an object type ID 1041, an image 1042, a mask image 1043, an image quality parameter 1044, and a peripheral parameter 1045. The object type ID 1041, the image quality parameter 1044, and the peripheral parameter 1045 are the same fields as the object type ID 601, the image quality parameter 603, and the peripheral parameter 604. Note that table 1031 may also display entries registered in object type definition information 333.

[0109] The delete button 1032 is an operation button for deleting an entry from the table 1031. When the user operates the delete button 1032, the entry is deleted from the object type definition information 333. Note that an entry in the object type definition information 333 may be deleted by an operation other than the operation of the delete button 1032.

[0110] The add button 1033 is an operation button for adding an entry to the table 1031. Note that an entry may be automatically added after a value is set in the last entry. In this case, the add button 1033 is not necessary.

[0111] The user can input the object type ID, for example, as text, into the object type ID 1041. Alternatively, the object type IDs that have already been set in the object type information 332 or the object type definition information 333 may be displayed as a drop-down list, allowing the user to select one.

[0112] An image and a mask image constituting object type definition data that define the characteristics of an object type are input into image 1042 and mask image 1043. The user may directly input a file path for image 1042, or may select the image by operating an operation button such as a Browse button displayed in the field. The user may directly input a mask image created in advance for mask image 1043, or may select the image by operating an operation button such as a Browse button displayed in the field. Furthermore, when mask image 1043 is clicked, management terminal 301 may display a mask image drawing screen, and may have the user draw a mask image on the screen that represents the area where the object type exists.

[0113] Image quality parameters for each object type are input into image quality parameters 1044. The user may be prompted to input the image quality parameters in text, or a list of compression parameters may be displayed in a drop-down list or the like, allowing the user to select from them. Peripheral parameters for each object type are input into peripheral parameters 1045.

[0114] Note that the image quality parameters for each object type may not only be directly specified, but other information for determining the image quality parameters may also be input into image quality parameters 1044. For example, the user may specify a target image quality for the area of ​​that object type, and the management terminal 301 or compression unit 103 may convert the specified value into an image quality parameter. Alternatively, the user may specify a target image quality for each object type and a target bit rate for the entire video, and based on these, the management terminal 301 or compression unit 103 may determine the image quality parameters for each object type and the setting values ​​of parameters (e.g., Constant Rate Factor) to be set in the encoder 125.

[0115] The verification button 1035 is an operation button for verifying the contents of the table 1031. When the user operates the verification button 1035, the management terminal 301 sends a verification request for the table 1031 to the compression unit 103. The compression unit 103 receives the verification request via the front-end interface 313, verifies the contents of the table 1031, and returns the results to the management terminal 301. The management terminal 301 displays the results. For example, the compression unit 103 performs semantic segmentation processing on the image 1042 and returns an image that visualizes the results as the verification result. However, the specific contents of the verification processing are not limited to this. The verification processing may also be executed by the management terminal 301.

[0116] The set button 1034 is an operation button for registering the contents of the table 1031 in the object type definition information 333. When the user operates the set button 1034, the management terminal 301 sends a registration request including the table 1031 to the compression unit 103. The compression unit 103 receives the registration request via the front-end interface 313 and updates the object type definition information 333 in accordance with the contents of the table 1031.

[0117] This concludes the description of the object type definition interface 102. Note that the object type definition interface 102 is not limited to the one shown in Fig. 10, and other information not shown may be displayed, a different operation method may be used, or the design may be different.

[0118] FIG. 11 shows another example of the object type definition interface 102.

[0119] First, the user operates the file selection button 1111 to select a video file that contains the object type they wish to specify. The selected video file is displayed in the video player field 1112. In the video player field 1112, the user can play and pause the video file. In addition, the user can use the playback time bar 1113 to adjust the display time of the video file.

[0120] Next, the user specifies the object type in the object type definition data input field 1120. In the object type ID field 1121, the user inputs the object type ID, for example, as text.

[0121] Next, the user operates the playback time bar 1113 to display a frame showing a desired object type in the video player field 1112. In this state, when the user operates the frame selection button 1123, the image of that frame is displayed in the drawing field 1124.

[0122] The user draws a bounding box 1125 on the drawing field 1124 using a cursor 1126. The user can draw a mask image by drawing the bounding box 1125 so that it surrounds a desired object type. For example, the mask image can be an image in which the inside of the bounding box drawn by the user is filled in.

[0123] However, the method of converting a bounding box into a mask image is not limited to this. The largest object type contained in the bounding box may be detected by deep learning or the like, and the detection result may be used as the mask image.

[0124] Note that the method of allowing the user to specify an object type in the drawing field 1124 is not limited to drawing a bounding box. For example, by clicking a part of the object type with the cursor 1126, the area of ​​the object type may be detected by deep learning or the like, and the detection result may be used as a mask image.

[0125] By operating the line addition button 1127, the user can specify object type definition data consisting of a pair of an image of the object type and a mask image.

[0126] The user can specify image quality parameters and peripheral parameters by operating the image quality parameter specification slide bar 1128 and the peripheral parameter slide bar 1129. Note that the method of specifying the image quality in the area of ​​the object type is not limited to this. For example, the user may specify the target image quality in the area of ​​the object type using the slide bar, and the management terminal 301 or compression unit 103 may convert the specified value into image quality parameters. Alternatively, the user may specify the target image quality for each object type and the target bit rate for the entire video, and based on these, the management terminal 301 or compression unit 103 may determine the image quality parameters for each object type and the setting values ​​of the parameters (e.g., Constant Rate Factor) to be set in the encoder 125.

[0127] By operating the object type addition button 1130, the user can add an object type definition data input field 1120 and define the characteristics of multiple object types.

[0128] By operating the frame selection button 1141 of the verification function, the frame displayed in the video player field 1112 can be displayed in field 1142. By operating the verification start button 1143 in this state, the management terminal 301 sends a verification request for the object type definition data input field 1120 to the compression unit 103. The compression unit 103 receives the verification request via the front-end interface 313, executes semantic segmentation processing using the object type definition data specified in the object type definition data input field 1120 and the image displayed in field 1142 as input, visualizes the results, and responds. The returned verification results are displayed in visualized image 1144. By looking at visualized image 1144, the user can confirm whether the object type has been specified as expected.

[0129] The set button 1150 is an operation button for registering the contents of the object type definition data input field 1120 in the object type definition information 333. When the user operates the set button 1150, the management terminal 301 sends a registration request including the object type ID, image, mask image, image quality parameters, and peripheral parameters to the compression unit 103. The compression unit 103 receives the registration request via the front-end interface 313 and updates the object type definition information 333 in accordance with the contents of the object type definition data input field 1120.

[0130] 11 may be divided into multiple screens. For example, the object type definition data input field 1120 and the verification field may be displayed as separate screens.

[0131] Returning to the explanation of Fig. 9, the arithmetic device 310 acquires information input via the object type definition interface 102 (step S902).

[0132] The arithmetic device 310 starts loop processing of the object type (step S903). Specifically, the arithmetic device 310 selects one of the object type IDs 1041 included in the table 1031 acquired in step S902. For example, if the table 1031 includes the object type IDs "obj1" and "obj2", the arithmetic device 310 selects either "obj1" or "obj2". In this case, the loop processing is executed twice.

[0133] The arithmetic device 310 acquires object type definition data for the selected object type (step S904). Specifically, the arithmetic device 310 acquires from the table 1031 an entry whose object type ID 1041 matches the identifier of the selected object type. For example, if the selected object type ID is "obj1", then entries 1051 and 1052 are acquired.

[0134] The arithmetic unit 310 converts the object type definition data into an object type vector (step S905). Specifically, the arithmetic unit 310 calculates the object type vector using the image 1042 and mask image 1043 of the entry acquired in step S904.

[0135] If there are multiple entries acquired in step S904, the arithmetic device 310 calculates one object type vector from each entry. In this case, the number of entries corresponds to the number of shots in the machine learning model 200 of semantic segmentation using few-shot learning. In general, the greater the number of shots, the higher the accuracy of object type detection.

[0136] The arithmetic device 310 updates the object type definition information 333 (step S906). Specifically, the arithmetic device 310 adds an entry to the object type definition information 333, sets the identifier of the object type selected in step S903 in the object type ID 601 of the added entry, and sets the parameters included in the object type definition data acquired in step S904 in the image quality parameters 603 and peripheral parameters 604. The arithmetic device 310 also sets the calculated object type vector in the object type feature amount 602 of the added entry.

[0137] In step S907, the arithmetic device 310 determines whether processing has been completed for all object types. If processing has not been completed for all object types, the arithmetic device 310 returns to step S903 and executes the same processing. If processing has been completed for all object type IDs, the arithmetic device 310 ends the registration processing of the object type definition information 333.

[0138] Note that a re-learning process such as fine-tuning may be performed on all or part of the image conversion unit 122, or all or part of the object type definition data conversion unit 120, or both. For example, in a step between step S902 and step S903, re-learning may be performed using the image 1042 and the mask image 1043 as training data. The re-learned parameters may be managed as object type definition data. Furthermore, the image conversion unit 122, the object type definition data conversion unit 120, or both may be re-learned for multiple object types using the images 1042 and mask images 1043 of multiple object type IDs 1041 corresponding to a certain data generation source ID as training data.

[0139] FIG. 12 is a flowchart illustrating an example of the compression process executed by the compression unit 103 according to the first embodiment.

[0140] When the arithmetic device 310 functioning as the compression unit 103 receives a new original frame via, for example, the front-end interface 313, it starts the compression process described below.

[0141] The arithmetic unit 310 performs pre-processing on the acquired original frame (step S1201). Details of the pre-processing will be described with reference to Fig. 13. In the pre-processing, for example, a processed frame is generated by performing pre-processing such as downscaling.

[0142] The arithmetic unit 310 calculates the image feature 143 from the processed frame and stores it in the image feature cache 334 (step S1202).

[0143] The arithmetic unit 310 refers to the object type information 332 to identify the object type associated with the data generation source 100 of the original frame (step S1203).

[0144] 5, the calculation device 310 searches for an entry in which the identifier of the data generation source 100 of the original frame is stored in the data generation source ID 501, and extracts the object type ID 502 of that entry. If the identifier of the data generation source 100 is "A", "obj1" and "obj2" are extracted.

[0145] The arithmetic device 310 starts loop processing of the object types (step S1204). Specifically, the arithmetic device 310 selects one of the object types identified in step S1203. For example, if the object types identified in step S1203 are "obj1" and "obj2," the arithmetic device 310 selects either "obj1" or "obj2." In this case, the loop processing is executed twice.

[0146] The arithmetic device 310 acquires an entry related to the selected object type from the object type definition information 333 (step S1205). For example, if the selected object type is "obj1", the arithmetic device 310 acquires entry 611 from the object type definition information 333 in FIG. 6.

[0147] The arithmetic unit 310 calculates the similarity between the object type vector and the image feature 143 stored in the image feature cache 334 (step S1206).

[0148] The calculation result of the similarity is the detection result of the object type in the processed frame, and is, for example, a two-dimensional tensor that indicates the probability that an object of the object type exists in each pixel of the processed frame.

[0149] In step S1207, the arithmetic device 310 determines whether or not processing has been completed for all identified object types. If processing has not been completed for all identified object types, the arithmetic device 310 returns to step S1204 and executes the same processing.

[0150] When the process has been completed for all identified object types, the arithmetic device 310 executes a compression level information generation process using the detection results for each object type (step S1208). Details of the compression level information generation process will be described with reference to FIG. 14.

[0151] The arithmetic device 310 generates compressed data 104 based on the original frame and compression level information (step S1209). Thereafter, the arithmetic device 310 transmits the compressed data 104 via the backend interface 314 to the storage device 302 or a device connected via the network 303, and ends the process.

[0152] Alternatively, steps S1201 to S1208 may be performed only once for a predetermined number of original frames, and the resulting compression level information may be reused for compressing those original frames. For example, steps S1201 to S1208 may be performed only for original frames that are multiples of four, counting from the beginning of the video data, and the three subsequent frames may be compressed using the most recently calculated compression level information. This reduces the amount of calculation per original frame, thereby improving compression speed.

[0153] FIG. 13 is a flowchart illustrating an example of pre-processing executed by the compression unit 103 according to the first embodiment.

[0154] The arithmetic device 310 acquires preprocessing parameters from the preprocessing parameter management information 331 (step S1301). Specifically, the arithmetic device 310 refers to the preprocessing parameter management information 331 and searches for an entry in which the identifier of the data source 100 that transmitted the original frame is stored in the data source ID 401. The arithmetic device 310 acquires the parameters set in the downscaling coefficient 402 and downscaling algorithm 403 of the entry.

[0155] The arithmetic unit 310 converts the original frame into a processed frame based on the acquired parameters (step S1302), and ends the pre-processing.

[0156] For example, in the case of the pre-processing parameter management information 331 shown in FIG. 4, the length and width of the frame received from the data generation source A are reduced to 1 / 16 based on the bilinear algorithm.

[0157] FIG. 14 is a flowchart illustrating an example of a compression level information generation process executed by the compression unit 103 according to the first embodiment.

[0158] The arithmetic device 310 starts loop processing of object types (step S1401). Specifically, the arithmetic device 310 selects one of the object types identified in step S1203. For example, if the object types identified in step S1203 are "obj1" and "obj2," the arithmetic device 310 selects either "obj1" or "obj2." In this case, the loop processing is executed twice.

[0159] The arithmetic unit 310 calculates the maximum value for each compression unit of the encoder 125 for the detection result of the selected object type, and calculates a new two-dimensional tensor (step S1402).

[0160] For example, in the case of the pre-processing parameter management information 331 shown in FIG. 4, the object type definition information 333 shown in FIG. 6, and the encoder 125 being an AVC encoder with a macroblock size of 16×16, the two-dimensional tensor (detection result) calculated in step S1206 has a resolution that corresponds one-to-one to the number of AVC macroblocks in the original frame, and therefore, identity conversion is performed in step S1402. For example, if the downscale coefficient 402 in the pre-processing parameter management information 331 is 1 / 4, the arithmetic unit 310 applies max pooling to the two-dimensional tensor calculated in step S1206 for each 4×4 tile, thereby converting the two-dimensional tensor so that each element corresponds one-to-one to the macroblock, which is the compression unit. Note that the process performed for each compression unit in step S1402 is not limited to calculation of a maximum value, and may also be calculation of an average value, etc.

[0161] The arithmetic device 310 acquires compression parameters from the object type definition information 333 (step S1403). Specifically, the arithmetic device 310 references the object type definition information 333 and searches for an entry in which the identifier of the selected object type is stored in the object type ID 601. The arithmetic device 310 acquires the parameters set in the image quality parameters 603 and peripheral parameters 604 of the searched entry.

[0162] The arithmetic unit 310 corrects the two-dimensional tensor calculated in step S1402 based on the peripheral parameters (step S1404).

[0163] For example, if the value of the peripheral parameter 604 is 32 and the encoder 125 is an AVC encoder with a macroblock size of 16×16, a correction is performed to expand the area for controlling image quality by two peripheral macroblocks beyond the area detected in step S1206. Specifically, this is processing to apply Max Pooling with a kernel size of 2×2 and a stride of 1 to the two-dimensional tensor calculated in step S1402. However, the processing in step S1404 is not limited to Max Pooling.

[0164] If the value of the peripheral parameter 604 is not a multiple of the macroblock size (for example, if the peripheral parameter 604 is 20 and the macroblock size is 16x16), the kernel size of Max Pooling may be an integer (for example, 2 in the above example) obtained by dividing the peripheral parameter by the size of one piece of the macroblock so that a conservatively wide area has the specified image quality.

[0165] The process of step S1404 may be performed immediately before the process of step S1402. This allows for correction so that the area of ​​the expanded region becomes more accurate. On the other hand, as shown in this embodiment, by performing the process of step S1404 after the process of step S1402, Max Pooling can be applied to a two-dimensional tensor having a resolution lower than that of the original frame, enabling faster processing.

[0166] The arithmetic device 310 generates compression level information of the encoder 125 for the selected object type based on the two-dimensional tensor corrected in step S1404 (step S1405). An example of such generation processing will be described below using the case where the encoder 125 is AVC.

[0167] A first example of such generation processing will be described taking as an example a case where the encoder 125 receives a QP value offset for each macroblock as compression level information. The calculation device 310 generates a QP value offset map as compression level information. For each element of a two-dimensional tensor, if its value is below a predetermined threshold (for example, if the value range of each element of the two-dimensional tensor is 0-1, the threshold is 0.5), the calculation device 310 assigns 0 as the QP value offset of the macroblock corresponding to that element; if its value is equal to or greater than the threshold, the calculation device 310 assigns a predetermined value determined from the image quality parameter acquired in step S1403 as the QP value offset of the macroblock corresponding to that element. For example, if the image quality parameter represents the QP value offset itself, the value of the image quality parameter acquired in step S1403 is set as the compression level information of the macroblock that is equal to or greater than the threshold. Alternatively, the QP value offset (compression level information) may be a value obtained by multiplying the image quality parameter acquired in step S1403 by a predetermined coefficient.

[0168] A second example of this generation process will be described using a case where the encoder 125 receives, as compression level information, information specifying the degree of image quality enhancement for each macroblock. For example, the encoder 125 receives four levels of designation for each macroblock: "High," "Mid," "Low," and "No enhancement." The arithmetic unit 310 generates compression level information in which, for each element of a two-dimensional tensor, if the value is below a predetermined threshold (e.g., if the value range of each element of the two-dimensional tensor is 0-1, the threshold is 0.5), "No enhancement" is assigned to the macroblock corresponding to that element. If the value is equal to or greater than the threshold, the arithmetic unit 310 assigns a value determined from the image quality parameter acquired in S1403 to the macroblock corresponding to that element. For example, if the image quality parameter is expressed in three levels: "High," "Mid," and "Low," the value of the image quality parameter acquired in S1403 is set to the macroblocks that are equal to or greater than the threshold.

[0169] Note that the method of converting the two-dimensional tensor calculated in step S1404 into compression level information is not limited to this, and any process may be used as long as it can generate compression level information using the result of step S1404. For example, the value of the compression level information may be dynamically controlled for each macroblock so as to improve the rate-distortion characteristics.

[0170] In step S1406, the arithmetic device 310 determines whether or not processing has been completed for all identified object types. If processing has not been completed for all identified object types, the arithmetic device 310 returns to step S1401 and executes the same processing.

[0171] When the processing is completed for all identified object types, the arithmetic device 310 merges the compression level information for each object type (step S1407).

[0172] For example, if a higher compression level means higher image quality for each compression unit, the calculation device 310 takes the maximum compression level among the object types for each compression unit, thereby enabling compression so that the image quality of the area where each object type exists is equal to or higher than the image quality represented by the image quality parameter 603 specified by the user.

[0173] The calculation for aggregating two-dimensional tensors of compression level information for multiple object types into one two-dimensional tensor of compression level information is not limited to calculating the maximum value, but may also be calculating an average value or the like.

[0174] As described above, according to the first embodiment, a standard codec is used, so that compatible compressed data can be generated at high speed. By using the image conversion unit 122, object type definition data conversion unit 120, and similarity calculation unit 123 generated by few-shot learning, it is not necessary to prepare learning data and re-learn when the definition of the object type is changed. In addition, by inputting processed frames that have been subjected to preprocessing such as reducing the resolution into the image conversion unit 122, it is possible to speed up processing.

[0175] Furthermore, by executing the processing of the image conversion unit 122 and the processing of the object type definition data conversion unit 120 independently at different timings, the number of times the neural network is executed in the compression process can be reduced, thereby speeding up the compression process. [Example]

[0176] In the second embodiment, the method of managing object type definition data is different from that in the first embodiment. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0177] FIG. 15 is a diagram illustrating an outline of a system according to the second embodiment.

[0178] The functional configuration of the compression unit 103 in the second embodiment is the same as that in the first embodiment. Moreover, the object type information 332 in the second embodiment has the same data structure as that in the first embodiment.

[0179] In the second embodiment, the data structure of the object type definition information 333 is different. The object type definition information 333 in the second embodiment includes an image 1501 and a mask image 1502 instead of the object type feature 602. The other field configurations are the same as those in the first embodiment. In the second embodiment, the image and mask image input via the object type definition interface 102 are managed as they are.

[0180] In the registration process of the object type definition information 333 in the second embodiment, the arithmetic device 310 registers the input object type definition data in the object type definition information 333 without executing the process of step S905.

[0181] The object type definition data conversion unit 120 calculates an object type vector using the image and the mask image at any timing.

[0182] For example, when the first frame is received from the data generation source 100, the object type definition data conversion unit 120 generates an object type vector 1510 from the object type definition data of the object type corresponding to the data generation source 100, and stores the object type vector in a cache (not shown). This allows the same data to be reused when compressing frames.

[0183] Alternatively, the following processing may be performed. As in the first embodiment, an object type feature 602 is provided in an entry of the object type definition information 333, and null is set as its initial value. In the compression processing, if the object type feature 602 of the entry corresponding to the data generation source 100 is null, the arithmetic device 310 causes the object type definition data conversion unit 120 to execute processing, stores an object type vector in the object type feature 602, and inputs the object type vector to the similarity calculation unit 123. If the object type feature 602 of the entry corresponding to the data generation source 100 is not null, the image conversion unit 122 inputs the object type vector set in the object type feature 602 to the similarity calculation unit 123.

[0184] The trigger for executing the calculation process of the object type vector is not limited to these.

[0185] According to the second embodiment, it is possible to flexibly respond to updates of the image and mask image. Also, by leaving the raw data (image and mask image) used to calculate the object type vector, it is possible to review various settings.

[0186] The present invention can be used to compress various types of multidimensional data, such as sensor data including sensor values ​​and time, in addition to images such as still images and moving images.

[0187] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments are provided to explain the present invention in detail, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0188] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, solid-state drives (SSDs), optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0189] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Python, and Java (registered trademark).

[0190] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

[0191] In the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines are necessarily shown in the product. All components may be interconnected. [Explanation of symbols]

[0192] 100 Data Sources 101 Object type setting interface 102 Object Type Definition Interface 103 Compression Unit 104 Compressed Data 120 Object Type Definition Data Conversion Unit 121 Pretreatment Unit 122 Image Conversion Unit 123 Similarity Calculation Unit 124 Compression Level Information Generation Unit 125 encoder 200 machine learning models 211 images 221 Object Type Definition Data 222 images 223 Mask Images 300 calculator 301 Management terminal 302 Storage Device 303 Network 310 Arithmetic equipment 311 Switch 312 memory 313 Front-End Interface 314 Backend Interface 330 Compression Program 331 Preprocessing parameter management information 332 Object Type Information 333 Object Type Definition Information 334 Image Feature Cache

Claims

1. 1. A computer system comprising: a processor, a storage device connected to the processor, and a connection device connected to the processor for connecting to an external device; The processor: a first process of calculating a first feature amount of an object type using object type definition data that defines an object type to be detected in a space defined by the dimensions of processed data obtained by cutting out multidimensional data to be compressed on a plane of any dimension; a second process of acquiring the processed data and calculating a second feature amount of the processed data; a third process of estimating an important region in which the object type exists in the space of the processed data by using the first feature amount and the second feature amount; a fourth process for generating compression level information including parameters for determining the data amounts of the important area and the area other than the important area; and a fifth process of generating the compressed data by converting the processed data into a data format that ensures compatibility with the destination of the compressed data through lossy compression using the compression level information.

2. 2. The computer system of claim 1, A computer system characterized in that, in the fifth process, the processor generates compressed data in which the amount of data in the important areas of the processed data is large and the amount of data in areas other than the important areas is small.

3. 2. The computer system of claim 1, The processor: In the first processing, the first feature amounts of a plurality of types of the object types are calculated; a computer system that executes the third process and the fourth process for each of the object types;

4. 4. The computer system according to claim 3, The processor determines the parameters for each area of ​​the space of the processing data using the compression level information for each type of object, and then executes the fourth processing.

5. 3. The computer system according to claim 2, The computer system is characterized in that, in the first process, the processor accepts the object type definition data, calculates the first feature amount from the object type definition data, and stores the first feature amount in the storage device.

6. 6. The computer system according to claim 5, The computer system is characterized in that the processor presents an interface for setting the object type definition data.

7. 7. The computer system of claim 6, the multidimensional data is a video; the processing data is a frame image, a computer system characterized in that the interface is an interface for setting a combination of a reference image including the object type and an annotation image indicating an area of ​​the object type included in the reference image as the object type definition data.

8. 8. The computer system of claim 7, the interface is capable of accepting an operation from a user to designate an area of ​​the object type included in the reference image, The processor generates the annotation image based on the operation.

9. 3. The computer system according to claim 2, The processor: before the first process is executed, the object type definition data is received and stored in the storage device; A computer system characterized in that, in the first processing, the first feature amount is calculated from the object type definition data stored in the storage device.

10. 3. The computer system according to claim 2, The processor corrects the important area in the fourth process.

11. 3. The computer system according to claim 2, In the third process, the processor A computer system comprising: a computer that calculates a probability that the object type exists in the space of the processing data based on the first feature amount and the second feature amount.

12. A program executed by a computer, the computer includes a processor, a storage device connected to the processor, and a connection device connected to the processor for connecting to an external device; The program a first process of calculating a first feature amount of an object type using object type definition data that defines an object type to be detected in a space defined by the dimensions of processed data obtained by cutting out multidimensional data to be compressed on a plane of any dimension; a second process of acquiring the processed data and calculating a second feature amount of the processed data; a third process of estimating an important region in which the object type exists in the space of the processed data by using the first feature amount and the second feature amount; a fourth process for generating compression level information including parameters for determining the data amounts of the important area and the area other than the important area; and a fifth process of generating the compressed data by converting the processed data into a data format that ensures compatibility with the destination of the compressed data through lossy compression using the compression level information.

13. 13. The program according to claim 12, A program characterized in that, in the fifth process, the computer generates compressed data in which the amount of data in the important area of ​​the processed data is large and the amount of data in areas other than the important area is small.

14. 13. The calculation program according to claim 12, In the first processing, the computer calculates the first feature amounts for a plurality of types of the object types; a program causing the computer to execute the third process and the fourth process for each of the object types;

15. 15. The program according to claim 14, A program that causes the computer to execute a process of determining the parameters of each area of ​​the space of the processed data using the compression level information for each type of object type, and then executes the fourth process.

16. 14. The program according to claim 13, a program for causing the computer to accept the object type definition data, calculate the first feature amount from the object type definition data, and store the first feature amount in the storage device, in the first processing;

17. 14. The program according to claim 13, before executing the first process, causing the computer to execute a process of receiving the object type definition data and storing it in the storage device; a program for causing the computer to calculate the first feature amount from the object type definition data stored in the storage device in the first processing;

18. 1. A data compression method executed by a computer system, comprising: the computer system includes a processor, a storage device connected to the processor, and a connection device connected to the processor for connecting to an external device; The data compression method includes: a first step in which the processor calculates a first feature of the object type using object type definition data that defines the object type to be detected in a space defined by the dimensions of processed data obtained by cutting out multidimensional data to be compressed on a plane of any dimension; a second step in which the processor acquires the processed data and calculates a second feature amount of the processed data; a third step in which the processor estimates an important region in which the object type exists in the space of the processed data using the first feature amount and the second feature amount; a fourth step in which the processor generates compression level information including parameters for determining the amount of data in each of the important area and the area other than the important area; and a fifth step in which the processor generates the compressed data by converting the processed data into a data format that is compatible with a destination of the compressed data through lossy compression using the compression level information.

19. 20. A data compression method according to claim 18, comprising: The fifth step is a data compression method characterized in that the processor generates compressed data in which the amount of data in the important areas of the processed data is large and the amount of data in areas other than the important areas is small.

20. 20. A data compression method according to claim 18, comprising: the first step includes a step of the processor calculating the first feature amounts for a plurality of types of the object types; The data compression method according to claim 1, wherein the third step and the fourth step are executed for each of the object types.

Images