Dimensional compression device, dimension compression method, dimension compression program, and data set generation method

Abstract

A dimension compression unit (112) performs dimensional compression on a data set comprising a plurality of pieces of labeled data. A correlation determination unit (113) determines, for each dimension, whether the correlation of the label distribution of the data set after the dimension compression with respect to the dimension direction of the dimension is low. When there is a low correlation dimension the correlation of which with the label distribution after dimension compression is determined to be low, a domain adaptation unit (114) performs domain adaptation on the data set after dimension compression along the dimension direction of the low correlation dimension, and a dimension compression unit (112) performs dimension compression on the data set after domain adaptation.

Description

Dimensional compression device, dimensional compression method, dimensional compression program, and data set generation method

[0001] This disclosure relates to dimensionality reduction of data such as images.

[0002] Patent Literature 1 discloses an evaluation device for evaluating multiple dimensionality reduction methods. The evaluation device includes a feature calculation unit, a feature similarity calculation unit, and an output unit. The feature calculation unit uses multiple feature extraction algorithms to extract a first feature of a data set before dimensionality reduction and a second feature of a data set after dimensionality reduction from a data set before dimensionality reduction and a data set after dimensionality reduction, for each of the multiple dimensionality reduction methods. The feature similarity calculation unit calculates a similarity between the first feature and the second feature using multiple feature similarity calculation algorithms corresponding to the multiple feature extraction algorithms, respectively. The output unit outputs the similarity calculated for each of the multiple dimensionality reduction methods.

[0003] The evaluation device described in Patent Document 1 evaluates the results of dimension reduction based on the correlation before and after dimension reduction. However, Patent Document 1 does not describe a means for preferentially retaining desired features from before dimension reduction. As a result, there may be a low correlation between the results of dimension reduction and pre-prepared labels. For example, the brightness dimension, which changes depending on the weather, is essentially unrelated to safety, but as a result of dimension reduction, the brightness dimension unrelated to safety may remain. In this case, it is not possible to confirm the trend of the desired label in the spatial domain, and it is not possible to visualize dimensions according to the desired feature.

[0004] Japanese Patent Application Laid-Open No. 2020-123294

[0005] The present disclosure aims to enable dimensionality reduction of a dataset by leaving dimensions that are highly correlated with the label distribution of the dataset.

[0006] The dimensionality reduction device of the present disclosure comprises: a dimensionality reduction unit that performs dimensionality reduction processing on a dataset consisting of a plurality of labeled data, a correlation determination unit that obtains correlation information of the label distribution of the dataset after dimensional reduction in the dimensional direction of each dimension, and a domain adaptation unit that performs domain adaptation processing on the dataset after dimensional reduction along the dimensional direction in accordance with the correlation information with the label distribution after dimensional reduction, wherein the dimensionality reduction unit performs the dimensionality reduction processing on the dataset after domain adaptation in accordance with the correlation information with the label distribution after dimensional reduction.

[0007] According to the present disclosure, it is possible to perform dimensionality reduction on a dataset while leaving dimensions that have a high correlation with the label distribution of the dataset.

[0008] FIG. 1 is a configuration diagram of a dimension reduction device 100 according to a first embodiment. FIG. 2 is a flowchart of a dimension reduction method according to the first embodiment. FIG. 3 is a schematic diagram of the dimension reduction method according to the first embodiment. FIG. 4 is a configuration diagram of a dimension reduction system 200 according to the first embodiment. FIG. 5 is a configuration diagram of a dimension reduction device 100 according to a second embodiment. FIG. 6 is a flowchart of a dimension reduction method according to the second embodiment. FIG. 7 is a diagram showing the relationship between the data density of a label distribution and the number of divisions according to the second embodiment. FIG. 8 is a configuration diagram of a dimension reduction system 200 according to the second embodiment. FIG. 9 is a configuration diagram of a dimension reduction device 100 according to a third embodiment. FIG. 10 is a flowchart of a dimension reduction method according to the third embodiment. FIG. 11 is a configuration diagram of a dimension reduction system 200 according to the third embodiment. FIG. 12 is a hardware configuration diagram of a dimension reduction device 100 according to an embodiment.

[0009] In the embodiments and drawings, the same or corresponding elements are denoted by the same reference numerals. The description of elements denoted by the same reference numerals as those already described will be omitted or simplified as appropriate. Arrows in the drawings primarily indicate the flow of data or the flow of processing.

[0010] First Embodiment A dimension reduction device 100 will be described with reference to FIGS.

[0011] ***Description of Configuration*** The configuration of the dimensionality reduction device 100 will be described with reference to Fig. 1. The dimensionality reduction device 100 is a computer including hardware such as a processor 101, a memory 102, an auxiliary storage device 103, a communication device 104, and an input / output interface 105. These pieces of hardware are connected to one another via signal lines.

[0012] The processor 101 is an IC that performs arithmetic processing and controls other hardware. For example, the processor 101 is a CPU, a DSP, or a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0013] The memory 102 is a volatile or non-volatile storage device. The memory 102 is also called a primary storage device or a main memory. For example, the memory 102 is a RAM. Data stored in the memory 102 is saved in the secondary storage device 103 as needed. RAM is an abbreviation for Random Access Memory.

[0014] The auxiliary storage device 103 is a non-volatile storage device. For example, the auxiliary storage device 103 is a ROM, a HDD, a flash memory, or a combination of these. Data stored in the auxiliary storage device 103 is loaded into the memory 102 as needed. ROM is an abbreviation for Read Only Memory. HDD is an abbreviation for Hard Disk Drive.

[0015] The communication device 104 is a receiver and a transmitter. For example, the communication device 104 is a communication chip or a NIC. The dimensionality reduction device 100 performs communication using the communication device 104. NIC is an abbreviation for Network Interface Card.

[0016] The input / output interface 105 is a port to which an input device and an output device are connected. For example, the input / output interface 105 is a USB terminal, the input devices are a keyboard and a mouse, and the output device is a display. Input and output to and from the dimensional reduction device 100 are performed via the input / output interface 105. USB is an abbreviation for Universal Serial Bus.

[0017] The dimensionality reduction device 100 includes elements such as a data acquisition unit 111, a dimensionality reduction unit 112, a correlation determination unit 113, a domain adaptation unit 114, and a data output unit 115. These elements are realized by software.

[0018] The auxiliary storage device 103 stores a dimension reduction program for causing the computer to function as a data acquisition unit 111, a dimension reduction unit 112, a correlation determination unit 113, a domain adaptation unit 114, and a data output unit 115. The dimension reduction program is loaded into the memory 102 and executed by the processor 101. The auxiliary storage device 103 also stores an OS. At least a portion of the OS is loaded into the memory 102 and executed by the processor 101. The processor 101 executes the dimension reduction program while running the OS. OS is an abbreviation for Operating System.

[0019] Data (input data, output data, etc.) of the dimensionality reduction program is stored in the storage unit 190. The memory 102 functions as the storage unit 190. However, a storage device such as the auxiliary storage device 103, a register in the processor 101, or a cache memory in the processor 101 may function as the storage unit 190 instead of or together with the memory 102.

[0020] The dimensionality compression program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or a flash memory.

[0021] ***Description of Operation*** The operation procedure of the dimension reduction device 100 corresponds to a dimension reduction method. Also, the operation procedure of the dimension reduction device 100 corresponds to a processing procedure by a dimension reduction program.

[0022] The dimensionality reduction method will be described with reference to Fig. 2. In step S110, the data acquisition unit 111 acquires a data set.

[0023] A dataset consists of multiple pieces of data, each of which is labeled. For example, a dataset might consist of multiple images, each of which shows a part of the facility being monitored. Each image is labeled with a label indicating its status at the time of capture (e.g., safe, caution, danger, etc.).

[0024] For example, a user prepares a dataset and inputs the dataset to the dimension reduction device 100. Then, the data acquisition unit 111 receives the input dataset.

[0025] In step S120, the dimensionality reduction unit 112 performs dimensionality reduction on the data set.

[0026] The method of dimensionality reduction to be performed does not matter. For example, methods such as PCA, ICA, t-SNE, and UMAP can be used for dimensionality reduction. PCA is an abbreviation for Principal Component Analysis. ICA is an abbreviation for Independent Component Analysis. t-SNE is an abbreviation for t-distributed Stochastic Neighbor Embedding. UMAP is an abbreviation for Uniform Manifold Approximation and Projection.

[0027] For example, if a dataset consists of multiple images, and each image has 100 pixels in each direction, the image has 10,000 pixels (10,000 dimensions), and the dimension-reduced image is compressed to an image with fewer than 10,000 pixels.

[0028] The dataset may also be a plurality of feature quantities obtained by transforming each of a plurality of images. The feature quantities may be, for example, discrete cosine transform coefficients. The feature quantities may be in a format that can be used for machine learning model training. The feature quantities may be in a format that can be input to a trained model generated by machine learning. The machine learning may be, for example, deep learning.

[0029] In step S130, the correlation determination unit 113 determines whether the correlation of the label distribution of the data set after dimensional compression is low in the dimensional direction for each dimension.

[0030] Possible criteria for determining whether a correlation is low include, for example, "when no statistically significant correlation is observed" or "when the correlation coefficient is close to 0." The criteria may be changed as appropriate.

[0031] The determination is performed for each dimension as follows. First, the correlation determination unit 113 calculates a correlation value of the label distribution of the dataset after dimensional compression. The correlation value is a value that represents the correlation in the dimension direction for the label distribution of the dataset after dimensional compression. When multiple pieces of data (data groups) with the same label are grouped together in the dimension direction and multiple pieces of data with different labels are arranged along the dimension direction, the correlation is high and the correlation value is large. Any method can be used to calculate the correlation value. For example, a correlation coefficient or correlation coefficient can be used as the correlation value. Then, the correlation determination unit 113 determines whether the correlation of the label distribution after dimensional compression in the dimension direction is low based on the calculated correlation value. For example, the correlation determination unit 113 compares the correlation value with a threshold value, and if the correlation value is smaller than the threshold value, determines that the correlation of the label distribution after dimensional compression in the dimension direction is low.

[0032] A dimension that has a low correlation with the label distribution after dimension reduction is called a low-correlation dimension.

[0033] If there is a low correlation dimension, the process proceeds to step S140. If there is no low correlation dimension, the process proceeds to step S150.

[0034] In step S140, the domain adaptation unit 114 performs domain adaptation on the dimension-reduced data set along the dimension direction of the low-correlation dimension.

[0035] That is, the domain adaptation unit 114 performs domain adaptation processing on the dimensionally compressed data set along the dimensional direction, according to correlation information with the label distribution after dimensional compression.

[0036] As long as domain adaptation is performed along the direction of the low-correlation dimension, the method of domain adaptation is not important. For example, the label distribution is divided into multiple areas along the direction of the low-correlation dimension, and domain adaptation is performed for each area.

[0037] The dataset after dimensionality reduction in which domain adaptation is performed along the dimension direction of the low-correlation dimension is referred to as the transformed dataset. That is, the dataset after domain adaptation is referred to as the transformed dataset.

[0038] After step S140, the process proceeds to step S120. In this case, the processes from step S120 onwards are performed on the transformed data set obtained in the immediately preceding step S140.

[0039] In step S150, the data output unit 115 outputs the dimension-reduced data set determined to have no low-correlation dimensions.

[0040] For example, the data output unit 115 displays the label distribution of the dataset after dimensionality reduction on a display.

[0041] ***Features of First Embodiment*** The features of the first embodiment will be described with reference to FIG. 3. A graph with X and Y coordinates shows the label distribution after dimensionality reduction of a dataset with three types of labels. The dimension reduction device 100 evaluates the distribution after dimensionality reduction (S120) based on the correlation with the labels before dimensionality reduction (S130), and performs domain adaptation (S140) depending on the strength of the correlation. Dimensionality reduction (S120) reduces the dimensions of the dataset (image). As a result, another dimension (axis) is visualized. In general, dimensionality reduction automatically selects (calculates) dimensions with high variance. Domain adaptation (S140) is performed for each cluster along the dimension. Therefore, features highly correlated with the current irrelevant dimension are lost. Domain adaptation (S140) transforms the dataset (image). Then, because features along the dimension are lost, those dimensions are less likely to remain in the next iteration. Repeating the flow increases the probability that features along the required dimension will ultimately remain.

[0042] ***Effects of First Embodiment*** In the past, there were cases where the correlation between the compression results and the labels prepared in advance was low, making visualization meaningless. On the other hand, with the first embodiment, it is possible to selectively leave dimensions that ultimately have a high correlation with the labels, making it possible to visualize dimensions in line with desired features.

[0043] ***Example of First Embodiment*** An example of the first embodiment will be described with reference to FIG. 4 . The dimension reduction system 200 is a system including the dimension reduction device 100. For example, the dimension reduction system 200 is used as a predictive maintenance system, and the dimension reduction device 100 is used as a predictive maintenance device. The dimension reduction device 100 can also perform dimensional compression on received images. Therefore, the dimension reduction device 100 can also be used as an image compression device. The dimension reduction system 200 includes an imaging device 201, a communication device 202, the dimension reduction device 100, a display device 203, and an input device 204. The imaging device 201 captures images of a monitoring target at each time, and the communication device 202 receives the images from the imaging device 201 and transmits the received images to the dimension reduction device 100. The dimension reduction device 100 further includes a preprocessing unit 121 and a postprocessing unit 122. The data acquisition unit 111 receives the images at each time and stores the received images in the storage unit 190. As a result, the image dataset is stored in the storage unit 190. The preprocessing unit 121 performs preprocessing on each image to reduce the dimension of each image. Examples of preprocessing include labeling, adjusting image size, and removing noise. The postprocessing unit 122 edits the dataset, the label distribution of which is displayed on the display unit 203, in accordance with the user's operation of the input device 204. For example, the postprocessing unit 122 changes the labels assigned to each image in accordance with the user's instructions.

[0044] Second Embodiment A mode for determining the number of divisions of the label distribution during domain adaptation will be described below, mainly with respect to the differences from the first embodiment, with reference to Figs.

[0045] ***Description of Configuration*** The configuration of the dimensionality reduction device 100 will be described with reference to Fig. 5. The dimensionality reduction device 100 further includes a division number determination unit 116. The dimensionality reduction program further causes the computer to function as the division number determination unit 116.

[0046] ***Description of Operation*** The dimensionality reduction method will be described with reference to Fig. 6. In step S210, the data acquisition unit 111 acquires a data set. Step S210 is the same as step S110 in the first embodiment.

[0047] In step S220, the dimensionality reduction unit 112 performs dimensionality reduction on the data set. Step S220 is the same as step S210 in the first embodiment.

[0048] In step S230, the correlation determination unit 113 determines whether there is a low-correlation dimension that has a low correlation with the label distribution. Step S230 is the same as step S130 in the first embodiment.

[0049] If there is a low correlation dimension, the process proceeds to step S240. If there is no low correlation dimension, the process proceeds to step S260.

[0050] In step S240, the division number determination unit 116 determines the division number of the label distribution after dimensional compression based on the data density per area when the label distribution after dimensional compression is divided into a plurality of areas.

[0051] The number of divisions of the label distribution after dimensionality compression is determined as follows. First, the division number determination unit 116 divides the label distribution after dimensionality compression into a standard number of areas. Next, the division number determination unit 116 calculates the data density per area. The data density is proportional to the number of data distributed in the area. For example, the division number determination unit 116 calculates the data density for each area and calculates statistical values ​​(average, maximum, minimum, etc.) of the data density for each area. The calculated statistical values ​​become the data density per area. Then, the division number determination unit 116 determines the number of divisions of the label distribution after dimensionality compression based on the data density per area. For example, the division number determination unit 116 compares the data density per area with a threshold value to determine the data density per area. If the data density per area is smaller than the threshold value, the data density per area is low density. If the data density per area is greater than the threshold value, the data density per area is high density. If the data density per area is the same as the threshold value, the data density per area is medium density. When the data density per area is low, the division number determination unit 116 determines the number of divisions of the label distribution after dimensional compression to be smaller than the reference division number. For example, the division number determination unit 116 determines the number of divisions of the label distribution after dimensional compression to be the number of divisions calculated by subtracting a constant (or variable) from the reference division number. The variable increases as the data density per area decreases, and decreases as the data density per area increases. When the data density per area is high, the division number determination unit 116 determines the number of divisions of the label distribution after dimensional compression to be the same as the reference division number or a number of divisions greater than the reference division number. For example, the division number determination unit 116 determines the number of divisions of the label distribution after dimensional compression to be the number of divisions calculated by adding a constant (or variable) to the reference division number. The variable increases as the data density per area increases, and decreases as the data density per area decreases. When the data density per area is medium, the division number determination unit 116 determines the number of divisions of the label distribution after dimensional compression to be the reference division number.

[0052] However, the division number determination unit 116 may determine the division number for each area according to the data density of the area. In other words, the division number may differ for each area.

[0053] In step S250, the domain adaptation unit 114 performs domain adaptation on the dimensionally reduced data set along the dimension direction of the low-correlation dimension. At this time, the domain adaptation unit 114 divides the dimensionally reduced label distribution into areas of the number of divisions determined in step S240, and performs domain adaptation for each divided area. If a different number of divisions is determined for each area, the domain adaptation unit 114 divides the dimensionally reduced label distribution into areas of the number of divisions determined in step S240, and performs domain adaptation for each divided area. Step S250 corresponds to step S140 in the first embodiment.

[0054] After step S250, the process proceeds to step S220.

[0055] In step S260, the data output unit 115 outputs the dimensionally compressed data set. Step S260 is the same as step S150 in the first embodiment.

[0056] ***Features of Embodiment 2*** Features of Embodiment 2 will be described with reference to FIG. 7 . A graph with X and Y coordinates shows the label distribution after dimensionality reduction of a dataset with three types of labels. When performing domain adaptation to eliminate features along the axes, it is assumed that division into multiple areas is performed. The dimension reduction device 100 determines the number of divisions during domain adaptation in accordance with the distribution after dimensionality reduction. The number of divisions may be increased or decreased. The number of divisions is determined in accordance with the density within each area previously divided after dimensionality reduction. For example, if the density within an area is low, the dimension reduction device 100 reduces the number of divisions. On the other hand, if the density within an area is high, the dimension reduction device 100 increases the number of divisions. For example, if there is a density difference between areas, the dimension reduction device 100 determines the number of divisions for each area in accordance with the density within the area. In other words, the number of divisions does not need to be uniform.

[0057] ***Effects of Second Embodiment*** According to the second embodiment, the number of divisions during domain adaptation can be changed. By reducing the number of divisions, the number of times domain adaptation is performed can be reduced. As a result, the time required to perform domain adaptation is reduced. By increasing the number of divisions, the accuracy of domain adaptation is improved.

[0058] ***Example of Second Embodiment*** Fig. 8 shows the configuration of a dimensionality reduction system 200. The dimensionality reduction device 100 further includes a division number determination unit 116. The other configuration is the same as the configuration of the dimensionality reduction system 200 in the first embodiment.

[0059] Third Embodiment A third embodiment of the present invention, in which information is added and a data set is stored after dimensional compression, will be described below, mainly with reference to the differences from the first embodiment, with reference to FIGS.

[0060] ***Description of Configuration*** The configuration of the dimensional compression device 100 will be described with reference to Fig. 9. The dimensional compression device 100 includes a data storage unit 117 instead of the data output unit 115. A dimensional compression program causes a computer to function as the data storage unit 117 instead of the data output unit 115.

[0061] ***Description of Operation*** The dimensionality reduction method will be described with reference to Fig. 10. Steps S310 to S340 are the same as steps S110 to S140 in the first embodiment.

[0062] If it is determined in step S330 that there is no low correlation dimension, the process proceeds to step S350.

[0063] In step S350, the data storage unit 117 stores the dimension-compressed data set, which is determined to have no low-correlation dimensions, together with additional information.

[0064] The additional information indicates at least one of the number of times dimension reduction (S320) has been performed and correlation information of the label distribution after dimension reduction. The correlation information is information obtained in step S330, and indicates, for example, a correlation value for each dimension.

[0065] However, the data storage unit 117 may add additional information to the dataset before domain application, which is the dataset before dimensionality reduction.

[0066] ***Features of Embodiment 3*** In Embodiment 3, the dimension reduction device 100 does not display the label distribution after dimension reduction on a display, but saves the dataset after dimension reduction. When saving the dataset after dimension reduction, the dimension reduction device 100 may save the number of times dimension reduction has been performed, the correlation coefficient, and the like in the properties of each data.

[0067] In the third embodiment, the dimension reduction device 100 does not display the label distribution after dimension reduction on a display, but saves the dataset before dimension reduction. When saving the dataset before dimension reduction, the dimension reduction device 100 may save the number of dimension reductions, the correlation coefficient, and the like in the properties of each data.

[0068] ***Effects of Third Embodiment*** The third embodiment provides the following effects. By recording information about a dimensionally reduced dataset without visualizing it, it is possible to record information about the processing of the dataset even in cases where visualization is not desired. Because the dimensionally reduced dataset is not visualized, the dataset can be processed and saved in the background, and the processed dataset can be referenced later.

[0069] The third embodiment provides the following advantages. By recording (adding) additional information obtained from information after dimensionality reduction to a data set before dimensionality reduction, an external device other than the dimensionality reduction device 100 can use the additional information, which is useful for analyzing the data set before dimensionality reduction. By having an external device analyze the data set before dimensionality reduction using the additional information, it is possible to reduce the number of processing steps for data analysis and improve the accuracy of data analysis. The external device is, for example, an image recognition device, an image compression device, an image encoding device, a machine learning device, or the like.

[0070] ***Example of Third Embodiment*** Fig. 11 shows the configuration of a dimensionality reduction system 200. The dimensionality reduction device 100 includes a data storage unit 117 instead of the data output unit 115. Other configurations of the dimensionality reduction device 100 are the same as those in the first embodiment.

[0071] *** Supplementary Note on Embodiment 3 *** Embodiment 3 may be implemented in combination with at least one of Embodiment 1 and Embodiment 2. That is, in Embodiment 3, the dimensional reduction device 100 may include at least one of the data output unit 115 and the division number determination unit 116.

[0072] Note that "saving" a dataset after dimensionality reduction may be read as "generating" a dataset after dimensionality reduction.

[0073] *** Supplementary Information of the Embodiment *** The hardware configuration of the dimensional reduction device 100 will be described with reference to Fig. 12. The dimensional reduction device 100 includes a processing circuit 109. The processing circuit 109 is hardware that realizes a data acquisition unit 111, a dimensional reduction unit 112, a correlation determination unit 113, a domain adaptation unit 114, a data output unit 115, a division number determination unit 116, and a data storage unit 117. The processing circuit 109 may be dedicated hardware, or may be a processor 101 that executes a program stored in the memory 102.

[0074] When the processing circuit 109 is dedicated hardware, the processing circuit 109 may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0075] The dimensionality reduction device 100 may include a plurality of processing circuits that replace the processing circuit 109 .

[0076] In the processing circuit 109, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0077] Thus, the functions of the dimensionality reduction device 100 can be realized by hardware, software, firmware, or a combination of these.

[0078] Each embodiment is an example of a preferred embodiment and is not intended to limit the technical scope of the present disclosure. Each embodiment may be implemented in part or in combination with other embodiments. Procedures described using flowcharts, etc. may be modified as appropriate.

[0079] The "part" of each element of the dimension reduction device 100 may be read as a "process," a "step," a "circuit," or a "circuitry."

[0080] Aspects of the present disclosure are described below as appendices. (Appendix 1) A dimension reduction device comprising: a dimension reduction unit that performs a dimensionality reduction process on a dataset consisting of a plurality of labeled data, a correlation determination unit that obtains correlation information of a label distribution of the dataset after dimension reduction in a dimensional direction of each dimension, and a domain adaptation unit that performs a domain adaptation process on the dataset after dimension reduction along the dimensional direction in accordance with the correlation information with the label distribution after dimension reduction, wherein the dimension reduction unit performs the dimensionality reduction process on the dataset after domain adaptation in accordance with the correlation information with the label distribution after dimension reduction.

[0081] (Supplementary Note 2) The dimension reduction device according to Supplementary Note 1, further comprising: a division number determination unit that determines the number of divisions of the label distribution after dimension reduction based on a data density per area when the label distribution after dimension reduction is divided into a plurality of areas; and the domain adaptation unit that divides the label distribution after dimension reduction into the determined number of areas and performs the domain adaptation for each divided area.

[0082] (Supplementary Note 3) The dimensionality reduction device according to Supplementary Note 2, wherein the division number determination unit determines the data density per area, and determines the division number to be smaller than a reference division number in accordance with the data density per area.

[0083] (Supplementary Note 4) The dimensionality compression device according to Supplementary Note 2 or Supplementary Note 3, wherein the division number determination unit determines the data density per area, and determines the division number to be greater than a reference division number in accordance with the data density per area.

[0084] (Supplementary Note 5) The dimensionality reduction device according to any one of Supplementary Note 2 to Supplementary Note 4, wherein the division number determination unit calculates a statistical value of data density of the plurality of areas as the data density per area, and determines the division number common to the plurality of areas based on the data density per area.

[0085] (Supplementary Note 6) The dimensionality reduction device according to any one of Supplementary Note 2 to Supplementary Note 4, wherein the division number determination unit calculates the data density per area for each area, and determines the division number for each area based on the data density per area.

[0086] (Supplementary Note 7) The dimensionality reduction device according to any one of Supplementary Note 1 to Supplementary Note 6, further comprising: a data storage unit; wherein the correlation determination unit acquires correlation information of the label distribution after domain-adapted dimensionality reduction for each dimension in the dimensional direction; and wherein the data storage unit stores the dataset after dimensionality reduction together with additional information indicating at least one of the number of times the dimensionality reduction has been performed and the correlation information of the label distribution after dimensionality reduction.

[0087] (Supplementary Note 8) The dimensionality reduction device according to any one of Supplementary Note 1 to Supplementary Note 6, further comprising: a data storage unit; wherein the correlation determination unit acquires correlation information of the label distribution after domain-adapted dimensionality reduction for each dimension in the dimensional direction; and wherein the data storage unit stores the dataset before dimensionality reduction together with additional information indicating at least one of the number of times the dimensionality reduction has been performed and the correlation information of the label distribution after dimensionality reduction.

[0088] (Supplementary Note 9) A dimension reduction method comprising: performing dimensionality reduction on a dataset consisting of a plurality of labeled data; obtaining, for each dimension, correlation information of label distribution of the dataset after dimensional reduction in the dimensional direction of the dimension; performing domain adaptation processing on the dataset after dimensional reduction along the dimensional direction according to the correlation information with the label distribution after dimensional reduction; and performing the dimensionality reduction on the dataset after domain adaptation according to the correlation information with the label distribution after dimensional reduction.

[0089] (Supplementary Note 10) A dimension reduction program for causing a computer to execute the following: a dimension reduction process for performing dimensional reduction on a dataset consisting of a plurality of labeled data, each of which is a dimension; a correlation determination process for obtaining correlation information of label distribution of the dataset after dimension reduction in the dimensional direction of the dimension for each dimension; and a domain adaptation process for performing domain adaptation on the dataset after dimension reduction along the dimensional direction in accordance with the correlation information with the label distribution after dimension reduction, wherein the dimension reduction process performs the dimensional reduction on the dataset after domain adaptation in accordance with the correlation information with the label distribution after dimension reduction.

[0090] (Supplementary Note 11) A dataset generation method comprising: performing dimensionality reduction on a dataset consisting of a plurality of labeled data; acquiring, for each dimension, correlation information of label distribution of the dataset after dimensional reduction in a dimensional direction of the dimension; performing domain adaptation processing on the dataset after dimensional reduction along the dimensional direction in accordance with the correlation information with the label distribution after dimensional reduction; performing the dimensional reduction on the dataset after domain adaptation; acquiring, for each dimension in the dimensional direction, correlation information of the label distribution after domain adaptation and dimension reduction; and generating the dataset after dimensional reduction together with additional information indicating at least one of the number of times the dimensional reduction is performed and the correlation information of the label distribution after dimensional reduction.

[0091] (Supplementary Note 12) A dataset generation method comprising: performing dimensionality reduction on a dataset consisting of a plurality of labeled data; acquiring, for each dimension, correlation information of label distribution of the dataset after dimensional reduction in a dimensional direction of the dimension; performing domain adaptation processing on the dataset after dimensional reduction along the dimensional direction in accordance with the correlation information with the label distribution after dimensional reduction; performing the dimensional reduction on the dataset after domain adaptation; acquiring, for each dimension in the dimensional direction, correlation information of the label distribution after domain adaptation and dimension reduction; and generating the dataset before dimensional reduction together with additional information indicating at least one of the number of times the dimensional reduction is performed and the correlation information of the label distribution after dimensional reduction.

[0092] 100 Dimensional compression device, 101 Processor, 102 Memory, 103 Auxiliary storage device, 104 Communication device, 105 Input / output interface, 109 Processing circuit, 111 Data acquisition unit, 112 Dimensional compression unit, 113 Correlation determination unit, 114 Domain adaptation unit, 115 Data output unit, 116 Division number determination unit, 117 Data storage unit, 121 Preprocessing unit, 122 Postprocessing unit, 190 Storage unit, 200 Dimensional compression system, 201 Imaging device, 202 Communication device, 203 Display device, 204 Input device.

Claims

1. A dimensionality reduction unit performs dimensionality reduction processing on a dataset consisting of multiple data points, each of which is labeled. A correlation determination unit obtains correlation information of the label distribution of the dataset after dimensionality reduction with respect to the dimensional direction of the said dimension for each dimension, A domain adaptation unit performs domain adaptation processing on the dimensionally compressed dataset along the dimensionality direction according to the correlation information with the label distribution after dimensionality compression, Equipped with, The dimensionality reduction unit performs the dimensionality reduction process on the domain-adapted dataset according to the correlation information with the label distribution after dimensionality reduction. Dimensional compression device.

2. The aforementioned dimensional compression device is A division number determination unit determines the number of divisions of the label distribution after dimensionality reduction based on the data density per area when the label distribution after dimensionality reduction is divided into multiple areas. Equipped with, The domain adaptation unit divides the label distribution after dimensionality reduction into a determined number of areas, and performs domain adaptation for each of the divided areas. The dimensional compression device according to claim 1.

3. The division number determination unit determines the data density per area and determines the number of divisions to be less than the reference number of divisions according to the data density per area. The dimensional compression device according to claim 2.

4. The division number determination unit determines the data density per area and determines the number of divisions to be greater than the reference number of divisions according to the data density per area. The dimensional compression device according to claim 2.

5. The division number determination unit calculates a statistical value of the data density of the multiple areas as the data density per area, and determines the division number common to the multiple areas based on the data density per area. The dimensional compression device according to claim 2.

6. The division number determination unit calculates the data density per area for each area and determines the division number for each area based on the data density per area. The dimensional compression device according to claim 2.

7. The aforementioned dimensional compression device includes a data storage unit, The correlation determination unit acquires correlation information of the domain-adapted dimensionality-reduced label distribution for each dimension in the dimensional direction, The dimensionality reduction device according to any one of claims 1 to 6, wherein the data storage unit stores the dataset after dimensionality reduction together with additional information indicating at least one of the number of dimensionality reduction operations and the correlation information of the label distribution after dimensionality reduction.

8. The aforementioned dimensional compression device includes a data storage unit, The correlation determination unit acquires correlation information of the domain-adapted dimensionality-reduced label distribution for each dimension in the dimensional direction, The dimensionality reduction device according to any one of claims 1 to 6, wherein the data storage unit stores the dataset before dimensionality reduction together with additional information indicating at least one of the number of dimensionality reduction operations and the correlation information of the label distribution after dimensionality reduction.

9. Dimensionality reduction is performed on a dataset consisting of multiple data points, each labeled. For each dimension, correlation information of the label distribution of the dimensionally compressed dataset is obtained with respect to the dimensional direction of that dimension. Domain adaptation processing is performed on the dimensionally compressed dataset along the dimensionality direction according to the correlation information with the label distribution after dimensionality reduction. Perform dimensionality reduction on the domain-adapted dataset according to the correlation information with the label distribution after dimensionality reduction. Dimensional compression method.

10. Dimensionality reduction is performed on a dataset consisting of multiple data points, each of which is labeled. A correlation determination process that obtains correlation information of the label distribution of the dataset after dimensionality reduction with respect to the dimensional direction of the said dimension for each dimension, A domain adaptation process is performed on the dimensionally compressed dataset along the dimensionality direction according to the correlation information with the label distribution after dimensionality compression, It is a dimensionality reduction program that allows a computer to perform the following: The dimensionality reduction process performs dimensionality reduction on the domain-adapted dataset according to the correlation information with the label distribution after dimensionality reduction. Dimensional compression program.

11. Dimensionality reduction is performed on a dataset consisting of multiple data points, each labeled. For each dimension, correlation information of the label distribution of the dimensionally compressed dataset is obtained with respect to the dimensional direction of that dimension. Domain adaptation processing is performed on the dimensionally compressed dataset along the dimensionality direction according to the correlation information with the label distribution after dimensionality reduction. The dimensionality reduction is performed on the dataset after domain adaptation. For each dimension, correlation information of the domain-adapted, dimensionally compressed label distribution is obtained with respect to the dimensional direction. The dimensionality-reduced dataset is generated along with additional information indicating at least one of the number of dimensionality reduction operations and the correlation information of the label distribution after dimensionality reduction. Dataset generation method.

12. Dimensionality reduction is performed on a dataset consisting of multiple data points, each labeled. For each dimension, correlation information of the label distribution of the dimensionally compressed dataset is obtained with respect to the dimensional direction of that dimension. Domain adaptation processing is performed on the dimensionally compressed dataset along the dimensionality direction according to the correlation information with the label distribution after dimensionality reduction. The dimensionality reduction is performed on the dataset after domain adaptation. For each dimension, correlation information of the domain-adapted, dimensionally compressed label distribution is obtained with respect to the dimensional direction. The dataset before dimensionality reduction is generated along with additional information indicating at least one of the number of dimensionality reduction operations and the correlation information of the label distribution after dimensionality reduction. Dataset generation method.