Information processing method, information processing device, and program

Abstract

The objective is to obtain an information processing method that can appropriately remove the influence of singular parts contained in the data to be normalized, thereby improving the estimation accuracy in user profile estimation using multiple data sets with different data attributes. [Solution] The information processing device acquires first data to be normalized, generates frequency distribution information for the first data, determines singular portions included in the frequency distribution information, generates second data by deleting singular data included in the singular portions of the first data, generates third data by normalizing the second data, and outputs the third data.

Description

Technical Field

[0001] The present disclosure relates to an information processing method, an information processing apparatus, and a program.

Background Art

[0002] In recent years, with the spread of IoT devices, technologies for profiling users themselves using data generated from user device operations and the like have become widespread. In this way, when analyzing data such as user profiling, normalization processing is used for the purpose of aligning the scales of different data and making it easier to compare feature quantities. For example, in Patent Document 1, for the purpose of making non-experts and experts match more effectively to improve their motivation, the feature quantities of non-experts and all experts are divided by the maximum value among them, so that all values are normalized to fall within the range of 0 to 1.

[0003] In Patent Document 2, in the statistical processing of biological data such as test results in the medical field, among various test items that act comprehensively, for the purpose of outputting a comprehensive evaluation such as the degree of aging, a technique for calculating a score serving as an evaluation criterion is disclosed by comparing the measured value of biological data with a reference value automatically selected from the distribution pattern.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] According to the technology disclosed in Patent Document 1, since each feature is normalized by dividing it by the maximum value, if the maximum value is an outlier, each feature that is not an outlier will become a value near 0 after the division, which may cause the data distribution to be biased and the user's characteristics to not be properly reflected.

[0006] According to the technology disclosed in Patent Document 2, a score is calculated based on a reference value automatically selected from a predetermined distribution pattern for measured values ​​of biological data. However, the trend of data distribution may differ depending on the data attribute, such as physical information or cyber information. Therefore, when fitting a distribution based on biological data, it may be difficult to maintain the user's characteristics for other data with different data attributes.

[0007] This disclosure aims to provide an information processing method, an information processing device, and a program that can appropriately remove the influence of singular parts contained in the data to be normalized, thereby improving the estimation accuracy in user profile estimation using multiple data sets with different data attributes. [Means for solving the problem]

[0008] An information processing method according to one aspect of the present disclosure involves an information processing device acquiring first data to be normalized, generating frequency distribution information relating to the first data, determining singular portions included in the frequency distribution information, generating second data by deleting singular data included in the singular portions of the first data, generating third data by normalizing the second data, and outputting the third data. [Effects of the Invention]

[0009] According to this disclosure, the influence of singular parts contained in the data to be normalized can be appropriately removed, thereby improving the estimation accuracy in user profile estimation using multiple data sets with different data attributes. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows a simplified example of the configuration of an information processing system according to the embodiment of this disclosure. [Figure 2] This diagram shows the contents of the memory unit. [Figure 3] This diagram shows a simplified representation of the processing unit's functional configuration. [Figure 4] This diagram shows the details of the normalization unit's function. [Figure 5] This flowchart shows the processing steps performed by the processing unit. [Figure 6] This figure shows the first example of a histogram. [Figure 7] This figure shows a second example of a histogram. [Figure 8] This figure shows a third example of a histogram. [Figure 9] This figure shows a fourth example of a histogram. [Figure 10] This flowchart shows the first example of the details of the process for deleting unusual data. [Figure 11] This flowchart shows a second example of the details of the process for deleting unusual data. [Figure 12] This flowchart shows the details of the normalization process. [Figure 13] This flowchart shows the first example of the details of the normalization parameter calculation process. [Figure 14] This flowchart shows a second example of the details of the calculation process for normalization parameters. [Modes for carrying out the invention]

[0011] (Knowledge that forms the basis of this disclosure) Techniques for recommending products and content such as videos are in practical use by estimating user profiles such as user preferences and trends based on screen operations and screen transitions of websites. Thus, it is possible to estimate user profiles from information such as operation logs, and further higher accuracy is expected by using not only cyber information such as websites but also physical information such as the number of steps and the range of movement of users.

[0012] When estimating a user profile using data that includes outliers, the estimation accuracy decreases. The data attributes obtained depending on the data acquisition source are different, such as cyber information obtained through activities on the web and physical information obtained through activities in a real environment such as the number of steps. Also, if the value ranges and number of digits are different between multiple data, those data have different data attributes. When using multiple data with different data attributes in combination, even if there are no outliers when looking at the data individually, there may be a case where a certain data becomes an outlier compared to other data when comparing multiple data. For example, cyber information such as the number of screen operations on a website is all on the order of about several times. On the other hand, the value range of physical information such as the number of steps (4 digits if several thousand steps) is significantly different from the value range of cyber information (1 digit if several times). Therefore, it is difficult to analyze the user profile using the relationship between data in data obtained by simply combining multiple data with different data attributes.

[0013] As a cause of the existence of outliers, there is a difference in the data acquisition source such as cyber information and physical information. Also, the number of digits of the data is considered as one of the factors. In the present disclosure, all things indicating the types of data such as the data acquisition source and the number of digits of these data are regarded as data attributes. In order to remove the influence of outliers, handle multiple data with different data attributes in combination, and appropriately perform user profile estimation, preprocessing of the data is necessary. Normalization, which is one of the preprocessings, and data processing based on the estimation result of the distribution to which the data applies are methods for processing by combining multiple data, but there are the following problems in the prior art methods.

[0014] This paper describes the challenges of user profile estimation using data that has different properties depending on the source. Considering that websites are designed with a specific purpose in mind, there are relationships between information related to screen transitions and click operations. Therefore, conventional profiling techniques integrate multiple cyber information sources. However, such a self-evident relationship does not exist between cyber and physical information. Conventional integration of cyber and physical information merely enumerates data, making accurate profile estimation difficult. This paper discusses the nature of the data and its impact on profiling in relation to this challenge.

[0015] This section explains the differences in the nature of cyber information and physical information. Under conditions where user permission has been obtained, cyber information can be obtained without any gaps in data acquisition, such as time periods or locations where data cannot be obtained from the user, and for view counts and click operations, unambiguous counts and truth values ​​can be obtained, respectively. On the other hand, with physical information such as step counts, the number of steps actually walked by the user does not match the step count of a smartphone, resulting in inaccuracies in observed values ​​and missing step count data due to the user not having a smartphone with them. Therefore, unambiguous cyber information and ambiguous physical information have different data properties.

[0016] This section explains the impact of the differences between cyber and physical information on profile estimation. When the complete cyber information described above is considered as a data distribution, as the amount of data increases, it converges to a simple data distribution that can be expressed with a few parameters, such as a normal distribution. However, physical information, which is ambiguous and contains missing data, does not converge in the same way as the normal distribution of cyber information. In other words, simply applying the above simple distribution to physical information does not accurately capture the data distribution, and as a result, the estimation accuracy of user profile estimation decreases. Furthermore, even with the complete cyber information described above, log data is generated due to operations that differ from normal use of the application (hereinafter abbreviated as "app") during application installation or tutorial viewing. The distribution of data when this log of operations that differ from normal use is mixed with logs of normal use is not a normal distribution. As a result, the data from logs of operations that differ from normal use become outliers, which can reduce the estimation accuracy of user profile estimation.

[0017] The following example illustrates the challenges of normalization due to the nature of the number of digits: The mean of data containing outliers, where the value "100" occurs in 3σ (standard deviation σ=1) of the standard normal distribution, corresponding to 1% of the number of observations, is "1.2," which is a significant deviation from the standard normal distribution mean of "0." Furthermore, the standard deviation of data containing outliers is "9.9," which is a significant deviation from the standard normal distribution standard deviation of "1."

[0018] Furthermore, the mean of the data containing outliers, where the value "10" exists at 3σ (standard deviation σ=1) of the standard normal distribution, corresponding to 1% of the number of observations, is "0.3," which is slightly different from the mean "0" of the standard normal distribution. On the other hand, the standard deviation of the data containing outliers is "1.0," which is no different from the standard deviation "1" of the standard normal distribution.

[0019] Therefore, if the majority of the data can be considered to follow a standard normal distribution, the influence of the outlier on the standard deviation of the observation distribution cannot be ignored when a value of "10," which is two orders of magnitude larger, is observed around three times the standard deviation, which is generally treated as an outlier. Furthermore, if a value of "100," which is three orders of magnitude larger, is observed, the impact on both the mean and the standard deviation is significant. Thus, in data where outliers exist that are 100 times or more the representative value of the observed values ​​(e.g., the mean), conventional methods cannot accurately normalize the data.

[0020] Furthermore, if the observed data follows a non-standard normal distribution, the values ​​described above will simply be approximate constant multiples and constant additions, essentially remaining the same. Therefore, even if only outliers less than 100 times the representative value of the observed values ​​are observed in the above explanation, conventional methods cannot accurately normalize the data if the data distribution containing the outliers has roughly the same shape.

[0021] The following describes the specific challenges in user profile estimation using data that contains a large number of data points with different properties.

[0022] One challenge is that data collected early in a user's app usage (for example, within the first week of use) includes app usage logs from installation and tutorial viewing. These logs may contain a large proportion of data collected for reasons unrelated to the app's intended use. If this data is used as is, the distribution of the data can be distorted by an excessive inclusion of certain parameters (e.g., "0" or "1"), obscuring characteristic data about the app user. As a result, the accuracy of user profile estimation using the obtained data decreases.

[0023] The second challenge is that app usage logs related to physical information and app usage logs that consist solely of cyber information are mixed together. In this case, the distribution of each data and the range of parameter values ​​may differ, and a uniform normalization that divides all data obtained from the app by the maximum value of the data may leave data that should be excluded as outliers.

[0024] The third challenge is that the distribution trends of biometric data and data obtained from apps differ. Biometric data has fewer outliers than expected, and the proportion of outliers in the total is small (e.g., a few percent), so they can be ignored. On the other hand, in data obtained from apps, outliers indicate specific user behavior within the app and are important information that represents user characteristics. Furthermore, outliers near "0" indicate the existence of users who are not using specific functions of the app, and are data that cannot be ignored. Therefore, normalization methods for biometric data that ignore the presence of outliers make it difficult to normalize while preserving user characteristics.

[0025] This disclosure is made to address these challenges and aims to provide a technology that automatically normalizes data while preserving user characteristics, even in data where data of different natures are mixed, by appropriately removing outliers that cause a decrease in accuracy, even for data acquired early from the start of data collection, in order to improve the estimation accuracy of user profile estimation.

[0026] Next, we will describe each aspect of this disclosure.

[0027] An information processing method according to a first aspect of this disclosure involves an information processing device acquiring first data to be normalized, generating frequency distribution information relating to the first data, determining singular portions included in the frequency distribution information, generating second data by deleting singular data included in the singular portions of the first data, generating third data by normalizing the second data, and outputting the third data.

[0028] According to the first embodiment, the influence of singular parts contained in the data to be normalized can be appropriately removed, thereby improving the estimation accuracy in user profile estimation using multiple data sets with different data attributes.

[0029] In the information processing method according to the second aspect of this disclosure, in the first aspect, in determining the singular portion, it is preferable to determine a predetermined singular frequency distribution shape among the frequency distribution information as the singular portion.

[0030] According to the second embodiment, by determining a predetermined unique frequency distribution shape among the frequency distribution information as a unique portion, it becomes possible to appropriately determine the unique portion.

[0031] In the third aspect of the information processing method of this disclosure, in the second aspect, the unique frequency distribution shape may include the peak shape of the frequency distribution near the minimum value of the first data.

[0032] According to the third embodiment, it becomes possible to appropriately determine the peak shape of the frequency distribution near the minimum value of the first data as a singular portion.

[0033] In the information processing method according to the fourth aspect of this disclosure, in the second aspect, the unique frequency distribution shape may include a flat shape of the frequency distribution.

[0034] According to the fourth embodiment, it becomes possible to appropriately determine the flat shape of the frequency distribution as a singular portion.

[0035] The information processing method relating to the fifth aspect of this disclosure is preferable in which, in any one of the first to fourth aspects, the singular data is deleted when a predetermined deletion condition is met during the generation of the second data.

[0036] According to the fifth aspect, it becomes possible to appropriately delete unusual data on the condition that predetermined deletion conditions are met.

[0037] In the sixth aspect of this disclosure, the information processing method, in the fifth aspect, may include at least one of the following with respect to the frequency distribution information: the number of peaks, the range of values, the data attributes, and the shape of the frequency distribution.

[0038] According to the sixth aspect, it becomes possible to set appropriate deletion conditions.

[0039] In the information processing method relating to the seventh aspect of this disclosure, in the sixth aspect, the data attributes may include physical information obtained through activities in real space and cyber information obtained through activities in virtual space.

[0040] According to the seventh aspect, it becomes possible to set appropriate deletion conditions according to physical information or cyber information.

[0041] The information processing method according to the eighth aspect of this disclosure is preferable in any one of the first to seventh aspects to generate the third data by estimating the data distribution of the second data from multiple data distributions, calculating parameters for normalization based on the data distribution of the second data, and normalizing the second data using the parameters.

[0042] According to the eighth aspect, by calculating parameters for normalization based on the data distribution of the second data, it becomes possible to appropriately normalize the second data using the calculated parameters.

[0043] In the information processing method according to the ninth aspect of this disclosure, in the eighth aspect, the plurality of data distributions may include discrete and continuous distributions.

[0044] According to the ninth aspect, it becomes possible to calculate appropriate parameters according to discrete or continuous distributions.

[0045] In the information processing method according to the tenth aspect of this disclosure, in the eighth aspect, the plurality of data distributions may include at least two of the following: a normal distribution, a uniform distribution, a Poisson distribution, binary data, a monotonically increasing distribution, and a monotonically decreasing distribution.

[0046] According to the tenth embodiment, it is possible to calculate appropriate parameters according to a normal distribution, uniform distribution, Poisson distribution, binary data, monotonically increasing distribution, or monotonically decreasing distribution.

[0047] In the information processing method according to the eleventh aspect of this disclosure, in the eighth aspect, the parameter may include at least one of a first threshold greater than the minimum value of the second data and a second threshold less than the maximum value of the second data.

[0048] According to the 11th embodiment, by normalizing the second data using a first threshold greater than the minimum value of the second data, or a second threshold less than the maximum value of the second data, the value of the normalized third data can be retained as a value greater than "1.0", making it possible to highlight data of characteristic users.

[0049] In the information processing method relating to the twelfth aspect of this disclosure, in any one of the first to eleventh aspects, the first data may include log data, image data, or audio data.

[0050] According to the 12th aspect, it becomes possible to appropriately normalize log data, image data, or audio data.

[0051] An information processing device according to a thirteenth aspect of this disclosure comprises a circuit configuration which acquires first data to be normalized, generates frequency distribution information relating to the first data, determines singular portions included in the frequency distribution information, generates second data by deleting singular data included in the singular portions of the first data, generates third data by normalizing the second data, and outputs the third data.

[0052] According to the 13th aspect, the influence of singular parts contained in the data to be normalized can be appropriately removed, thereby improving the estimation accuracy in user profile estimation using multiple data sets with different data attributes.

[0053] A program according to a 14th aspect of this disclosure is a program for causing an information processing device to perform processing, wherein the processing involves acquiring first data to be normalized, generating frequency distribution information relating to the first data, determining singular portions included in the frequency distribution information, generating second data by deleting singular data included in the singular portions of the first data, generating third data by normalizing the second data, and outputting the third data.

[0054] According to the 14th aspect, the influence of singular parts contained in the data to be normalized can be appropriately removed, thereby improving the estimation accuracy in user profile estimation using multiple data sets with different data attributes.

[0055] This disclosure can also be implemented as a program that causes a computer to execute each characteristic configuration included in such a method or apparatus, or as a system that operates using such a program. It goes without saying that such a computer program can be distributed via a computer-readable, non-temporary recording medium such as a CD-ROM, or via a communication network such as the Internet.

[0056] (Embodiments of the present disclosure) Embodiments of this disclosure will be described in detail below with reference to the drawings. Elements denoted by the same reference numeral in different drawings refer to the same or corresponding elements. Furthermore, the components, their arrangement, connection configurations, and operating sequences shown in the following embodiments are examples and are not intended to limit this disclosure. This disclosure is limited only by the claims. Accordingly, components in the following embodiments that are not described in the independent claims representing the highest-level concepts of this disclosure are described as constituting a more preferable configuration, even though they are not necessarily required to achieve the object of this disclosure.

[0057] Figure 1 is a simplified diagram showing an example configuration of an information processing system 1 according to an embodiment of this disclosure. The information processing system 1 comprises a terminal 11, a normalization device 12, and an estimation device 13.

[0058] Terminal 11 is, for example, a storage device that stores data to be normalized (hereinafter referred to as "first data D1"). First data D1 is, for example, a data file that exists in the local environment, but its form is not particularly limited. Terminal 11 inputs the first data D1, which is stored in a predetermined format, to the normalization device 12. The normalization device 12 may read first data D1 from a data file that exists in the local environment, or it may read first data D1 from log data that exists in the cloud.

[0059] The normalization device 12 is a device that normalizes the first data D1 input from the terminal 11 for use in user profile estimation. The normalization device 12 inputs the normalized data to the estimation device 13.

[0060] The normalization device 12 is configured using a computer that includes a processing unit 21, a storage unit 22, and a communication unit 23.

[0061] The processing unit 21 comprises the circuit configuration of an information processing device. The information processing device includes a processor such as a CPU.

[0062] The storage unit 22 is configured to include a storage device for storing data. The storage unit 22 includes a computer-readable non-volatile storage medium such as a hard disk drive, a solid-state drive, or flash memory.

[0063] The communication unit 23 is an arbitrary data input / output mechanism such as an interface circuit, and is configured to include, for example, a communication module that corresponds to the communication standard between the normalization device 12 and the terminal 11 and estimation device 13.

[0064] The estimation device 13 inputs the normalized data received from the normalization device 12 into a machine learning-based estimation model to estimate user profiles such as user preferences and tendencies.

[0065] Figure 2 shows the contents stored in the memory unit 22. The memory unit 22 stores the first data D1 input from the terminal 11. The memory unit 22 also stores the program 30.

[0066] The first data set, D1, includes log data related to user behavior acquired on the application. However, the first data set, D1, is not limited to log data; it may also include image data or audio data.

[0067] In the following section, we will explain the processing performed by the normalization device 12 using log data acquired on the application. However, the input data received by the normalization device 12 from the terminal 11 is not limited to log data; the normalization device 12 should perform processing corresponding to the data format, such as image data or audio data.

[0068] If the first data D1 is image data, the upper limits of its parameters and its data distribution are affected by the sensor used to acquire the data. To illustrate with an example using 8-bit image data, since the pixel value range for 8-bit images is "0" to "255", there is a high probability that noise and outliers will converge (saturate) at the upper limit "255" or the lower limit "0". Therefore, when using 8-bit image data, from the perspective of reducing the influence of outliers, it is sufficient to remove the data at the upper limit "255" and the lower limit "0". This removes the influence of outliers concentrated at the upper or lower limit of the pixel value range, allowing for proper normalization and enabling the unified handling of information from multiple data with different properties.

[0069] If the first data D1 is audio data, then data processing should be based on the dynamic range. For example, in the case of audio data heard by humans, it is generally known that, due to human characteristics, sounds above the upper limit of the dynamic range (approximately 120 dB for humans) tend to be painful, and sounds below the lower limit of the dynamic range (approximately 30 dB for humans) tend to be heard as noise. Therefore, values ​​that people find unpleasant or difficult to hear can be considered outliers, and by removing data corresponding to the upper or lower limit of the dynamic range as outliers, the impact of outliers can be mitigated.

[0070] Furthermore, since sound is a vibration of air, the audio data detected by sensors such as microphones will not be constant even when measuring sounds at the sensor's sensitivity limit. Therefore, when removing outliers from audio measurement data, it is sufficient to add a width of a few dB (for example, 2 dB) to the boundary where the outlier occurs and then remove the outlier. This removes the influence of outliers in audio data based on the human dynamic range, or outliers mixed into audio data acquired by sensors, and allows for proper normalization, enabling the unified handling of information from multiple data with different properties.

[0071] Figure 3 is a simplified diagram showing the functional configuration of the processing unit 21. The processing unit 21 has an acquisition unit 31, a generation unit 32, a determination unit 33, a deletion unit 34, a normalization unit 35, and an output unit 36, which are functions realized by the processor executing the program 30 read from the storage unit 22. Note that the functions shown in Figure 3 may also be configured using dedicated hardware circuits such as ASICs.

[0072] The acquisition unit 31 acquires the first data D1, which is the target of normalization, by reading the first data D1 from the storage unit 22.

[0073] The generation unit 32 generates frequency distribution information for the first data D1. The frequency distribution information is information that shows the distribution of the input data, and includes, for example, a histogram.

[0074] The determination unit 33 determines the singular portion included in the frequency distribution information. The singular portion is a unique data portion in the input data distribution that is not expected from the original purpose of using the application, and includes outliers, abnormal values, or irregular portions. In determining the singular portion, the determination unit 33 determines a predetermined singular frequency distribution shape in the frequency distribution information as a singular portion. The singular frequency distribution shape includes the peak shape of the frequency distribution near the minimum value of the first data D1. The singular frequency distribution shape also includes a flat shape of the frequency distribution. Details of the singular frequency distribution shape will be described later.

[0075] The deletion unit 34 generates the second data D2 by deleting the data included in the singular portion of the first data D1 (hereinafter referred to as "singular data"). In generating the second data D2, the deletion unit 34 deletes singular data if predetermined deletion conditions are met. The deletion conditions include at least one of the following related to frequency distribution information: number of peaks, range, data attributes, and frequency distribution shape. The data attributes include physical information obtained from activities in the real world and cyber information obtained from activities in the virtual world. Details of the singular data deletion process by the deletion unit 34 will be described later.

[0076] The normalization unit 35 generates the third data D3 by normalizing the second data D2 input from the deletion unit 34. If the first data D1 does not contain any singular parts, or if the first data D1 contains singular parts but does not satisfy the predetermined deletion conditions, the singular data is not deleted, and the normalization unit 35 may generate the third data D3 by normalizing the first data D1.

[0077] Figure 4 shows the details of the functions of the normalization unit 35. The normalization unit 35 includes an estimation unit 351, a calculation unit 352, and a normalization unit 353. The estimation unit 351 estimates the data distribution of the second data D2 from a plurality of pre-set data distributions. In other words, it applies one of the plurality of pre-set data distributions to the second data D2. The plurality of data distributions may include discrete distributions and continuous distributions. Alternatively, the plurality of data distributions may include at least two of the following: normal distribution, uniform distribution, Poisson distribution, binary data, monotonically increasing distribution, and monotonically decreasing distribution. The calculation unit 352 calculates parameters used in the normalization calculation (hereinafter referred to as "normalization parameters") based on the data distribution of the second data D2. The normalization parameters may include at least one of the following: a first threshold greater than the minimum value of the second data D2, and a second threshold less than the maximum value of the second data D2. The normalization unit 353 normalizes the second data D2 in a manner corresponding to the data distribution estimated by the estimation unit 351, and using the normalization parameters calculated by the calculation unit 352. Details of the normalization process by the normalization unit 35 will be described later.

[0078] Referring to Figure 3, the output unit 36 ​​outputs the third data D3 input from the normalization unit 35. The third data D3 is input to the estimation device 13 via the communication unit 23.

[0079] The configuration of the normalization device 12 is not particularly limited; for example, it may be configured using an edge server installed within a specific facility, or it may be configured using a cloud server. When the normalization device 12 is configured using an edge server, the terminal 11 and the normalization device 12 are connected via a local area network. When the normalization device 12 is configured using a cloud server, the terminal 11 and the normalization device 12 are connected via a wide-area communication network such as the Internet. In addition, a part of the normalization device 12 may be configured using an edge server and the other part using a cloud server.

[0080] Furthermore, the normalization device 12 does not necessarily have to be implemented using a single computer device, but may be implemented by a distributed processing system including a terminal device and a server device. In this case, the acquisition unit 31, generation unit 32, determination unit 33, deletion unit 34, and storage unit 22 may be provided in the terminal device, and the normalization unit 35 and output unit 36 ​​may be provided in the server device. In this case, the transmission and reception of data between the components may be performed via a wide-area communication network.

[0081] Figure 5 is a flowchart showing the processing steps performed by the processing unit 21.

[0082] First, in step S1, the acquisition unit 31 acquires the first data D1, which is the target of normalization, by reading the first data D1 from the storage unit 22.

[0083] Next, in step S2, the generation unit 32 generates frequency distribution information for the first data D1. The frequency distribution information is information that shows the distribution of the input data, and includes, for example, a histogram.

[0084] Figure 6 shows the first example of a histogram. The horizontal axis represents the number of clicks on the user profile screen on the app, which is cyber information, and the vertical axis represents the number of users, which is the frequency for each bin. The data distribution shows a normal distribution with peak shapes corresponding to bins where the horizontal axis values ​​are between "20" and "25".

[0085] Figure 7 shows a second example of a histogram. The horizontal axis represents the user's daily step count, which is physical information, and the vertical axis represents the number of users, which is the frequency for each bin. The data distribution shows a monotonically decreasing distribution with peak shapes corresponding to the bins near the minimum value where the horizontal axis value is between "0" and "2500".

[0086] Figure 8 shows a third example of a histogram. The horizontal axis represents the number of clicks on the user profile screen on the app, which is cyber information, and the vertical axis represents the number of users, which is the frequency for each bin. The data distribution shows a normal distribution with peak shapes corresponding to the bins near the minimum value where the horizontal axis value is between "0" and "5", and peak shapes corresponding to the bins where the horizontal axis value is between "20" and "25".

[0087] Figure 9 shows a fourth example of a histogram. The horizontal axis represents the number of clicks on the user profile screen on the app, which is cyber information, and the vertical axis represents the number of users, which is the frequency for each bin. The data distribution shows a normal distribution with a peak shape corresponding to the bins where the horizontal axis values ​​are "20" to "25", and a flat shape corresponding to the five bins where the horizontal axis values ​​are "35" to "60". The flat shape represents a distribution shape in which the data continues to be distributed at a constant frequency across multiple consecutive bins.

[0088] Referring to Figure 5, in step S3, the determination unit 33 determines the singular portion included in the histogram generated in step S2.

[0089] If the distribution of data obtained from an app contains an anomaly, the anomaly data within that anomaly distorts the overall distribution of the data. When using data containing this anomaly, the accuracy of user profile estimation decreases for users who use the app in accordance with its intended usage.

[0090] Furthermore, a given dataset may contain multiple data distributions based on different factors. In this case, the interaction of these multiple data distributions can cause the unique features of each data point to be obscured by singularities, leading to a decrease in the accuracy of user profile estimation.

[0091] Therefore, it is important to exclude data that negatively impacts the analysis from the data distribution, and the determination unit 33 has the effect of detecting data that should be excluded from the data distribution by determining the singular portion.

[0092] In determining the singular portion, the determination unit 33 determines that a predetermined singular frequency distribution shape in the histogram is a singular portion.

[0093] As a first example, the determination unit 33 determines that the peak shape of the frequency distribution near the minimum value of the first data D1 is a singular frequency distribution shape. The method for detecting peaks on the graph is to compare the frequency of a histogram bin with the frequencies of the preceding and succeeding bins. For example, if the difference between the frequency of a certain bin and the frequencies of the preceding and succeeding bins is greater than a set threshold (e.g., "5"), it can be determined to be a peak. For the bin containing the minimum value of the first data D1, if the difference between it and the frequency of the bin to its right is greater than a set threshold (e.g., "5"), it can be determined to be a peak. The determination unit 33 counts the number of peaks included in the histogram.

[0094] For the histogram shown in Figure 7, which has peaks only near the minimum value, the determination unit 33 determines that the peak shape of the frequency distribution near the minimum value of the first data D1, that is, the peak shape corresponding to the leftmost bin where the horizontal axis value is "0" to "2500", is a unique frequency distribution shape. The leftmost bin includes not only data of normal values ​​such as 1000 steps or 2000 steps per day, but also data of abnormal values ​​that are not reasonable for the number of steps per day, such as a few steps or a few tens of steps. The determination unit 33 determines that the peak shape corresponding to the bin near the minimum value of the first data D1 is a unique frequency distribution shape.

[0095] For the histogram with multiple peaks shown in Figure 8, the determination unit 33 determines that the peak shape of the frequency distribution near the minimum value of the first data D1, that is, the peak shape corresponding to the leftmost bin where the horizontal axis value is "0" to "5", is a unique frequency distribution shape. The histogram shown in Figure 8 contains data from users who continuously use the app and data from users who do not use the app after viewing the tutorial, resulting in multiple peaks on the graph. In the data from users who continuously use the app, peaks may appear at specific locations other than near the minimum value. On the other hand, in the data from users who do not use the app, peaks appear near the minimum value. The determination unit 33 determines that the peak shape corresponding to the bin near the minimum value of the first data D1 is a unique frequency distribution shape.

[0096] As a second example, the determination unit 33 determines that a flat shape included in a normal distribution is a singular frequency distribution shape. Targeting a histogram with a flat shape as shown in Figure 9, the determination unit 33 determines that a flat shape included in a normal distribution is a singular frequency distribution shape. Log data of users who frequently used the app in the past but do not currently use the app may remain on the histogram as a fixed flat shape. When a flat shape exists, the distribution of the entire data is distorted towards the side where the flat shape exists. For example, if the distribution of user app usage data according to the originally expected app usage method follows a normal distribution, the data in the normal distribution may be distorted by the flat shape, potentially obscuring the characteristics of app users and reducing the estimation accuracy of user profile estimation. Therefore, the determination unit 33 determines that a fixed flat shape included in a normal distribution is a singular frequency distribution shape. A method for detecting a flat shape on a graph is to compare the frequency of the histogram bin with the frequencies of the preceding and succeeding bins. For example, if the difference between the frequency of a given bottle and the frequencies of the preceding and succeeding bottles is less than a set threshold (e.g., "2"), then that bottle and the preceding and succeeding bottles can be determined to have the same frequency. In this frequency comparison with the preceding and succeeding bottles, if the number of consecutive bottles that are determined to have the same frequency as the preceding and succeeding bottles is equal to or greater than a threshold (e.g., "5"), then the shape can be determined to be flat.

[0097] Referring to Figure 5, in step S4, the deletion unit 34 generates the second data D2 by deleting the singular data included in the singular portion of the first data D1.

[0098] By removing outlier data, the remaining second data set D2 can be fitted to a specific data distribution, enabling effective normalization. The second data set D2 is likely to contain a high proportion of data from users who use the app in its intended way. Calculating normalization parameters based on this second data set D2 and performing normalization improves the accuracy of user profile estimation.

[0099] Figure 10 is a flowchart showing a first example of the details of the process for deleting outlier data in step S4. The first example corresponds to the process of deleting the peak shape near the minimum value of the first data D1.

[0100] First, in step S4A1, the deletion unit 34 obtains the number of peaks included in the histogram, which were counted by the determination unit 33 in step S3.

[0101] Next, in step S4A2, the deletion unit 34 determines whether there is a bin near the minimum value of the first data D1 among the bins that represent peaks on the histogram detected in step S3. In other words, it determines whether a bin near "0" on the horizontal axis of the first data D1 is a peak.

[0102] The determination method involves checking whether the data in the bins near the minimum value contains "0". Alternatively, the determination can be made by checking whether the smallest data in the bin closest to "0" among the bins determined to be peaks is below a threshold (e.g., "1"). If none of the detected bins contain a peak near the minimum value (step S4A2: NO), the process of deleting outlier data is terminated.

[0103] If the detected bins include a bin that has a peak near the minimum value (step S4A2: YES), then in step S4A3, the deletion unit 34 determines whether there are two or more peaks on the graph based on the number of peaks obtained in step S4A1.

[0104] If there are no more than two peaks on the graph (step SS4A3: NO), then in step S4A4, the deletion unit 34 determines whether the frequency difference between the bin near the minimum value and the adjacent bin is greater than or equal to a predetermined threshold. In other words, for a data distribution where peaks exist only near the minimum value, it determines whether there is an excessive amount of data mixed into the bin near the minimum value compared to what would be expected from the app usage logs of continuous app users.

[0105] The deletion unit 34 determines, for example, that if all input data are positive values, and the frequency difference between the bin near "0" and the bin to its right is greater than or equal to a threshold (for example, a difference equivalent to twice the value), then the bin near "0" is excessively populated with data. On the other hand, if the frequency difference between the bin near "0" and the bin to its right is less than the threshold, the deletion unit determines that the bin near "0" is not excessively populated with data and terminates the deletion process for the outlier data. The deletion unit 34 may also compare the frequency difference between the bin near "0" and the bin to its left if all input data are negative values, or it may compare the frequency difference between the bin near "0" and both the bins to its right and left if the input data includes both positive and negative values.

[0106] If there are two or more peaks on the graph (step SS4A3: YES), or if the frequency difference between a bin near "0" and an adjacent bin is greater than or equal to a threshold (step SS4A4: YES), then in step S4A5, the deletion unit 34 determines whether the maximum value on the horizontal axis of the input data is greater than or equal to a predetermined threshold (e.g., "7"). If the maximum value is greater than or equal to the threshold, the input data is likely to be physical information; if the maximum value is less than the threshold, the input data is likely to be cyber information. The threshold may be set to an arbitrary value in advance depending on the input data.

[0107] If the maximum value is less than the threshold (step S4A5: NO), the process of deleting outlier data is terminated. Cyber ​​information is, for example, the frequency of use of a specific function of an application over a week, and since the data distribution exists only within a narrow range, the impact on the overall data distribution is small even if outlier data is not deleted.

[0108] If the maximum value is greater than or equal to the threshold (step S4A5: YES), then in step S4A6, the deletion unit 34 determines whether the bins near "0" contain multiple values. For example, as shown in the histogram in Figure 7, if the bins near "0" contain multiple values ​​across a wide range such as "0" to "2500", then the bins near "0" contain not only data of normal values ​​such as 1000 or 2000 steps per day, but also data of abnormal values ​​that are not appropriate for a daily step count, such as a few steps or a few tens of steps.

[0109] If the bins near "0" contain multiple values ​​(step S4A6: YES), then in step S4A7, the deletion unit 34 deletes data below a predetermined threshold (e.g., "100") for the bins near "0", and terminates the process of deleting outlier data. The threshold may be set based on the maximum value of the input data, or an arbitrary value may be set in advance.

[0110] If the bins near "0" do not contain multiple values ​​(step S4A6: NO), then in step S4A8, the deletion unit 34 deletes the data contained in the bins near "0" and terminates the process of deleting singular data.

[0111] Figure 11 is a flowchart showing a second example of the details of the process for deleting outlier data in step S4. The second example corresponds to the process of deleting flat shapes that are included in a normal distribution.

[0112] First, in step S4B1, the deletion unit 34 determines whether the data distribution of the histogram of the input data fits a uniform distribution.

[0113] If the data distribution is uniform (Step S4B1: YES), the process of deleting outlier data is terminated. When the data distribution is uniform, the frequencies of each bin on the histogram will be approximately the same, and it will be judged as having a flat shape. However, this indicates a characteristic of the input data and should not be deleted as outlier data.

[0114] If the data does not fit a uniform distribution (step S4B1: NO), the deletion unit 34 then determines in step S4B2 whether the maximum value on the horizontal axis of the input data is greater than or equal to a predetermined threshold (e.g., "7"). If the maximum value is greater than or equal to the threshold, the input data is likely to be physical information; if the maximum value is less than the threshold, the input data is likely to be cyber information. The threshold may be set to an arbitrary value in advance depending on the input data.

[0115] If the maximum value is less than the threshold (step S4B2: NO), the process of deleting outlier data is terminated. Cyber ​​information is, for example, the frequency of use of a specific function of an application over a week, and since the data distribution exists only within a narrow range, the impact on the overall data distribution is small even if outlier data is not deleted.

[0116] If the maximum value is greater than or equal to the threshold (step S4B2: YES), then in step S4B3, the deletion unit 34 deletes flat shapes included in the histogram. Specifically, the deletion unit 34 deletes data from multiple consecutive bins where the difference between the frequency of a given bin and the frequencies of the preceding and succeeding bins is less than or equal to a set threshold (for example, "2") as singular data.

[0117] Referring to Figure 5, in step S5, the normalization unit 35 generates the third data D3 by normalizing the second data D2 input from the deletion unit 34. If the first data D1 does not contain any singular parts, or if the first data D1 contains singular parts but does not satisfy the predetermined deletion conditions, the singular data is not deleted, and the normalization unit 35 generates the third data D3 by normalizing the first data D1.

[0118] Figure 12 is a flowchart showing the details of the normalization process.

[0119] First, in step S51, the estimation unit 351 estimates the data distribution of the second data D2 from a plurality of pre-set data distributions. In other words, it applies one of the plurality of pre-set data distributions to the second data D2. The estimation of the distribution may involve determining whether it is a discrete distribution that is more likely to apply to cyber information (for example, the number of user accesses to a specific function) or a continuous distribution that is more likely to apply to physical information (for example, user biometric data). The determination of whether it is a discrete or continuous distribution may be made using a differential quantity based on the shape of the histogram or a determination method based on the probability of bin occurrence, but is not limited to these methods.

[0120] Furthermore, the estimation unit 351 may estimate whether the data distribution of the second data D2 fits a specific data distribution (e.g., a normal distribution). The specific data distribution may be arbitrarily set in advance, and there is no particular limit to the number of data distributions that can be set. To estimate whether it fits a specific data distribution, a test corresponding to each data distribution may be used, or the estimation may be based on the shape of the histogram. The target of the distribution estimation may be the second data D2 if the singularity removal process has been performed, or it may be the first data D1 if the singularity removal process has not been performed.

[0121] In this embodiment, five data distributions applicable to log data obtainable on the application were pre-set as specific data distributions. The first data distribution is a Poisson distribution, which is highly likely to apply to discrete user behaviors (such as access to specific functions) that occur frequently, such as the frequency of application use. The second data distribution is a normal distribution, which is highly likely to apply to biometric data as physical information. The third data distribution is a uniform distribution, which is highly likely to apply to data where a certain value appears with the same frequency. The fourth data distribution is binary data, which obtains binary information indicating whether or not a specific function was used. The fifth data distribution is a monotonically increasing distribution, which shows a monotonically increasing shape, such as the cumulative number of accesses to a specific function. A monotonically decreasing distribution, which shows a monotonically decreasing shape, may be used instead of a monotonically increasing distribution. Furthermore, the data distributions listed above are just examples, and other data distributions (e.g., exponential distributions) may be used as candidates for the applicable data distribution. As for methods for estimating these predefined data distributions, when determining whether or not the data fits a specific data distribution, a test corresponding to that data distribution (for example, the Shapiro-Wilk test for a normal distribution) may be used. Alternatively, when determining whether or not the data fits a binary data, monotonically increasing distribution, or monotonically decreasing distribution, the determination may be made based on the shape of the graph.

[0122] The data distributions obtained from cyber or physical information through early use of an application can be limited to at most a few types. If a suitable data distribution can be estimated, it is possible to appropriately perform normalization processing using a method or normalization parameters appropriate to that data distribution. For example, if it can be estimated that the data is binary, it can be immediately determined that the maximum and minimum values ​​of the input data should be used as normalization parameters. Also, if it can be estimated that the data is a normal distribution, for example, when normalizing to the range of "0" to "1", the input data can be preprocessed so that the peak is "0.5", and at that time, the mean value of the second data D2 can be obtained.

[0123] Referring to Figure 12, in step S52, the calculation unit 352 calculates the normalization parameters to be used in the normalization calculation based on the data distribution of the second data D2.

[0124] Figure 13 is a flowchart illustrating the details of the calculation process for the normalization parameters in step S52, as in the first example. The first example corresponds to the process when the estimated data distribution is discrete or continuous.

[0125] First, in step S52A1, the calculation unit 352 obtains the minimum and maximum values ​​of the horizontal axis of the second data D2.

[0126] Next, in step S52A2, the calculation unit 352 determines the normalization range based on the maximum and minimum values ​​obtained in step S52A1. The calculation unit 352 can determine the normalization range based on the signs of the maximum and minimum values. For example, if the signs of the maximum value and the minimum value are the same, the range from "0" to "1" can be determined as the normalization range. On the other hand, if the signs of the maximum value and the minimum value are different, the range from "-1" to "1" can be determined as the normalization range. Note that the normalization range is not limited to the above example; for example, the range from "-1" to "0" may also be determined as the normalization range.

[0127] Next, in step S52A3, the calculation unit 352 determines whether the data distribution of the second data D2 is a discrete distribution. The estimation result from the estimation unit 351 in step S51 can be used to determine the data distribution.

[0128] If the data distribution of the second data D2 is a discrete distribution (step S52A3: YES), then in step S52A4, the calculation unit 352 determines whether the maximum value of the second data D2 is less than a predetermined threshold (e.g., "7").

[0129] If the maximum value of the second data D2 is less than the threshold (step S52A4: YES), it is considered that there are no outliers in the second data D2. Therefore, the maximum and minimum values ​​of the second data D2 are set as normalization parameters, and the calculation process for the normalization parameters is terminated. If the second data D2 is a discrete distribution and the maximum value of the second data D2 is less than the threshold, it is highly likely that the data values ​​fall within a certain range and the maximum value is limited, such as the number of days a specific function of an app was used within a week. Therefore, if the maximum value of the second data D2 is less than the threshold, it is considered unlikely that there are outliers in the second data D2.

[0130] If the data distribution of the second data D2 is not discrete (step S52A3: NO), or if the maximum value of the second data D2 is greater than or equal to the threshold (step S52A4: NO), then in step S52A5, the calculation unit 352 calculates the interquartile range of the second data D2 to calculate the minimum value threshold and the maximum value threshold as outlier thresholds for the second data D2.

[0131] For example, the calculation unit 352 calculates the minimum threshold by subtracting the first quartile from the value obtained by multiplying the interquartile range by 1.5. The calculation unit 352 also calculates the maximum threshold by adding the value obtained by multiplying the interquartile range by 1.5 to the third quartile. Note that the multiplier for the interquartile range is not limited to 1.5; any multiplier may be used.

[0132] Next, in step S52A6, the calculation unit 352 sets the minimum value side normalization parameter and the maximum value side normalization parameter based on the minimum value side threshold and the maximum value side threshold calculated in step S52A5 of the second data D2.

[0133] The calculation unit 352 compares the minimum value of the second data D2 with the minimum value threshold calculated in step S52A5. If the minimum value of the second data D2 is less than the minimum value threshold, the calculation unit 352 sets the minimum value threshold as the minimum value normalization parameter. On the other hand, if the minimum value of the second data D2 is greater than or equal to the minimum value threshold, the calculation unit 352 sets the minimum value of the second data D2 as the minimum value normalization parameter.

[0134] The calculation unit 352 compares the maximum value of the second data D2 with the maximum value threshold calculated in step S52A5. If the maximum value of the second data D2 exceeds the maximum value threshold, the calculation unit 352 sets the maximum value threshold as the maximum value normalization parameter. On the other hand, if the maximum value of the second data D2 is less than or equal to the maximum value threshold, the calculation unit 352 sets the maximum value of the second data D2 as the maximum value normalization parameter.

[0135] The maximum value side of the second data set D2 may contain characteristic data indicating users who actively use the app. When the maximum value threshold of the second data set D2 is used as the maximum value normalization parameter, that characteristic data will have a value greater than "1.0" after normalization. This has the effect of highlighting information about characteristic users by displaying a value greater than "1.0".

[0136] Figure 14 is a flowchart illustrating a second example of the details of the calculation process for the normalization parameters in step S52. The second example corresponds to the process when the estimated data distribution is a Poisson distribution, a normal distribution, a uniform distribution, binary data, or a monotonically increasing distribution.

[0137] First, in step S52B1, the calculation unit 352 obtains the minimum and maximum values ​​of the horizontal axis of the second data D2.

[0138] Next, in step S52B2, the calculation unit 352 determines the normalization range based on the maximum and minimum values ​​obtained in step S52A1. The calculation unit 352 can determine the normalization range based on the signs of the maximum and minimum values. For example, if the signs of the maximum value and the minimum value are the same, the range from "0" to "1" can be determined as the normalization range. On the other hand, if the signs of the maximum value and the minimum value are different, the range from "-1" to "1" can be determined as the normalization range. Note that the normalization range is not limited to the above example; for example, the range from "-1" to "0" may also be determined as the normalization range.

[0139] Next, in step S52B3, the calculation unit 352 determines whether the data distribution of the second data D2 is a normal distribution. The estimation result from the estimation unit 351 in step S51 can be used to determine the data distribution.

[0140] If the data distribution of the second data D2 is a normal distribution (step S52B3: YES), then in step S52B4, the calculation unit 352 obtains the mean value of the second data D2. When the peak of the data after normalization is at the center of the normalization range, the estimation accuracy of user profile estimation using the normalization result is improved. In the case of a normal distribution, the peak of the data and the mean value coincide, so if the second data D2 is a normal distribution, the mean value of the data is obtained.

[0141] If the data distribution of the second data D2 is not a normal distribution (step S52B3: NO), the processing in step S52B4 is omitted.

[0142] Next, in step S52B5, the calculation unit 352 determines whether the data distribution of the second data D2 is binary data or a uniform distribution. The estimation result from the estimation unit 351 in step S51 can be used to determine the data distribution.

[0143] If the data distribution of the second data D2 is binary or uniform (step S52B5: YES), it is considered that there are no outliers in the second data D2. Therefore, the maximum and minimum values ​​of the second data D2 are set as normalization parameters, and the calculation process for normalization parameters is terminated.

[0144] If the data distribution of the second data D2 is neither binary nor uniform (step S52B5: NO), then in step S52B6, the calculation unit 352 calculates the interquartile range of the second data D2 and calculates the maximum value threshold as the outlier threshold for the second data D2.

[0145] For example, the calculation unit 352 calculates the maximum value threshold by adding a value obtained by multiplying the interquartile range by 1.5 to the third quartile. Note that the multiplier for the interquartile range is not limited to 1.5; any multiplier may be used.

[0146] Next, in step S52B7, the calculation unit 352 sets the maximum value normalization parameter based on the maximum value of the second data D2 and the maximum value threshold calculated in step S52B6.

[0147] The calculation unit 352 compares the maximum value of the second data D2 with the maximum value threshold calculated in step S52B6. If the maximum value of the second data D2 exceeds the maximum value threshold, the calculation unit 352 sets the maximum value threshold as the maximum value normalization parameter. On the other hand, if the maximum value of the second data D2 is less than or equal to the maximum value threshold, the calculation unit 352 sets the maximum value of the second data D2 as the maximum value normalization parameter.

[0148] The maximum value side of the second data set D2 may contain characteristic data indicating users who actively use the app. When the maximum value threshold of the second data set D2 is used as the maximum value normalization parameter, that characteristic data will have a value greater than "1.0" after normalization. This has the effect of highlighting information about characteristic users by displaying a value greater than "1.0".

[0149] Next, in step S52B8, the calculation unit 352 determines whether the data distribution of the second data D2 is a Poisson distribution or a monotonically increasing distribution. The estimation result from the estimation unit 351 in step S51 can be used to determine the data distribution.

[0150] If the data distribution of the second data set D2 is a Poisson distribution or a monotonically increasing distribution (step S52B8: YES), the minimum value of the second data set D2 is set as the minimum-side normalization parameter, and the calculation process for the normalization parameter is terminated. Here, in the Poisson distribution, the distribution becomes sparser as you move towards the maximum value, and there is a possibility that data of users with characteristics who use the app only casually may exist as outliers. On the other hand, data is concentrated towards the minimum value, but there is no data of users who actively use the app. Also, in the case of monotonically increasing distributions, it is thought that data of users who use the app casually is more abundant as you move towards the maximum value. For the above reasons, for Poisson distributions and monotonically increasing distributions, the process of obtaining the normalization parameter is performed only on the maximum value side.

[0151] If the data distribution of the second data D2 is neither a Poisson distribution nor a monotonically increasing distribution (step S52B8: NO), then in step S52B9, the calculation unit 352 calculates the interquartile range of the second data D2 and calculates the minimum value threshold as the outlier threshold for the second data D2.

[0152] For example, the calculation unit 352 calculates the minimum value threshold by subtracting the first quartile from a value obtained by multiplying the interquartile range by 1.5. Note that the multiplier for the interquartile range is not limited to 1.5; any multiplier may be used.

[0153] Next, in step S52B10, the calculation unit 352 sets the minimum value side normalization parameter based on the minimum value of the second data D2 and the minimum value side threshold calculated in step S52B9.

[0154] The calculation unit 352 compares the minimum value of the second data D2 with the minimum value threshold calculated in step S52B9. If the minimum value of the second data D2 is less than the minimum value threshold, the calculation unit 352 sets the minimum value threshold as the minimum value normalization parameter. On the other hand, if the minimum value of the second data D2 is greater than or equal to the minimum value threshold, the calculation unit 352 sets the minimum value of the second data D2 as the minimum value normalization parameter.

[0155] In the example shown in Figure 14, steps S52B9 and S52B10 are performed only if the second data D2 follows a normal distribution. Since a normal distribution may extend to the minimum value side, the minimum value normalization parameter is set by comparing it with the minimum value threshold.

[0156] Referring to Figure 12, in step S53, the normalization unit 353 performs a normalization operation on the second data D2 using the normalization parameters calculated in step S52.

[0157] If the data distribution of the second data D2 estimated in step S51 is a normal distribution, the normalization unit 353 performs processing according to the normalization range determined in step S52. If the normalization range is "0" to "1", the normalization unit 353 adds or subtracts the mean of the second data D2 obtained in step S52 to the second data D2 so that the mode after normalization is "0.5". Alternatively, if the normalization range is "-1" to "1", the normalization unit 353 adds or subtracts the mean of the second data D2 obtained in step S52 to the second data D2 so that the mode after normalization is "0".

[0158] The normalization unit 353 normalizes the second data D2 using the normalization parameters set in step S52.

[0159] If the normalization range is "0" to "1", the normalization unit 353 performs normalization using formula (1).

[0160]

number

[0161] If the normalization range is "-1" to "1", the normalization unit 353 performs normalization using equation (2).

[0162]

number

[0163] Here, a i a' refers to the i-th data point of the second data set, D2. i This represents the result of the normalization operation. max This represents the maximum-side normalization parameter. min This represents the minimum value normalization parameter.

[0164] By normalizing the data using minimum and maximum thresholds for the second data set D2, characteristic data points will exceed "1.0" after normalization. This has the effect of highlighting information about distinctive users by displaying values ​​above "1.0". Furthermore, since the normalization results in a distribution shape closer to that of users who continuously use the app, rather than a distribution skewed towards "0", it has the effect of improving the estimation accuracy of user profile estimation.

[0165] Referring to Figure 5, in step S6, the output unit 36 ​​outputs the normalized third data D3 input from the normalization unit 35. The data format and output destination of the third data D3 are arbitrary. The output destination may be a terminal in the local environment or a cloud server, etc.

[0166] Thus, according to this embodiment, the influence of singular parts contained in the first data D1, which is the target of normalization, can be appropriately removed, thereby improving the estimation accuracy in user profile estimation using multiple data sets with different data attributes.

[0167] The following describes specific examples of the application of the normalization process described in this disclosure.

[0168] As a first example, consider its application to a social networking service (SNS) app that revitalizes local communities. Users within this app belong to a specific community (for example, residents of a town). In this case, log data originating from diverse individuals with varying ages (from young people to the elderly) and occupations is likely to be collected through the app. Therefore, individual characteristics are widely distributed in the log data obtained through the app, and distinctive behaviors of certain users may be detected as outliers. In particular, data collected early in the user's life, when they have not been using the app for very long, may contain outliers such as operation logs during app installation or operation logs that are not expected from the intended use of the app due to unfamiliarity with it.

[0169] However, if outliers exist in the data used for user profiling, the estimation accuracy of user profiles using this data will be greatly affected by these outliers.

[0170] On the other hand, the normalization method described in this disclosure removes the influence of outliers even in data acquired early, enabling data preprocessing for highly accurate profile estimation. Identifying key figures within a region is crucial for regional revitalization, and log data is processed using the normalization method described in this disclosure. Highly accurate profile estimation using the normalized log data allows for the identification of key figures. For example, the user who has given the most "likes" to the activities of other users within the regional community can be identified as a key figure. Then, detailed information such as information about regional events can be explained to this key figure intensively, and this information can be effectively spread to various people in the region through their daily lives. In this way, this disclosure can ultimately be used for regional revitalization.

[0171] A specific example of the processing in the first example will be described in order, referring to the configuration of this disclosure. Assume that a user in a certain community is using a smartphone as terminal 11. Assume that an SNS application intended to revitalize the local community is already installed on terminal 11. The acquisition unit 31 acquires the user log of the user's SNS application. The storage unit 22 is thought to be a database built on the cloud, and the user log acquired from the smartphone is stored in the database.

[0172] Taking the example of a normalization process performed daily at 11 PM, this execution process is called as a batch process at 11 PM, and the determination unit 33 identifies anomalies in the data obtained from the database that are not expected from the intended use of the application. In the case of the SNS application example above, certain operation logs that are not expected from normal application operation occur frequently due to the operation during installation, and this is detected as a peak in the data distribution. The deletion unit 34 deletes the data included in the anomaly identified by the determination unit 33.

[0173] The estimation unit 351 estimates which of several predetermined data distributions the distribution of the data from which singular parts have been removed by the deletion unit 34 corresponds. In this example, the distribution is set based on the characteristics of the data source being an SNS app used by residents of a community, such as a Poisson distribution from the perspective of the number of times the app was used in discrete time and the number of times stamps representing "likes" were pressed, and a normal distribution from the perspective of the distribution of a large number of users belonging to a community.

[0174] The calculation unit 352 determines the normalization parameters necessary for normalization based on the data distribution estimated by the estimation unit 351. At this time, from the perspective of user log data, outlier data is generated due to the user's behavior on the app. For example, log data of a user who communicates exceptionally actively on the app. This data would normally be removed as an outlier, but it represents the characteristics of a user who acts actively on the app. If normalization is simply performed using the maximum value of the data, data that could be considered outliers will be rounded to "1.0", while other data will be rounded to a format closer to "0". At this time, the calculation unit 352 calculates parameters that allow normalization in a format that emphasizes the characteristics of the user, by leaving the data of the user who acts actively on the app at a value greater than "1.0", such as "1.3".

[0175] The normalization unit 353 normalizes the data acquired by the acquisition unit 31 based on the normalization parameters calculated by the calculation unit 352. The normalized data is then output from the output unit 36 ​​and used for user profile estimation. This normalized data excludes data that negatively impacts profile estimation, and retains data that might otherwise be excluded as outliers, retaining it as information about users with distinctive characteristics within the community. Therefore, it is possible to estimate the characteristics of users within a local community (for example, the characteristic of being a central figure within the community) with high accuracy through application logs.

[0176] A specific example of the processing in the second example will be explained with reference to the configuration of this disclosure. Assume that an employee working in an office is using a smartphone as terminal 11. Assume that an SNS application intended to revitalize interpersonal relationships within the office is already installed on terminal 11. The acquisition unit 31 acquires the user log of the user's SNS application. The storage unit 22 is thought to be a database built on the cloud, and the user log acquired from the smartphone is stored in the database.

[0177] Taking the example of a normalization process performed daily at 11 PM, this execution process is called as a batch process at 11 PM, and the determination unit 33 identifies anomalies in the data obtained from the database that are not expected from the intended use of the application. In the case of the SNS application example above, certain operation logs that are not expected from normal application operation occur frequently due to the operation during installation, and this is detected as a peak in the data distribution. The deletion unit 34 deletes the data included in the anomaly identified by the determination unit 33.

[0178] The estimation unit 351 estimates which of several predetermined data distributions the distribution of the data from which singular parts have been removed by the deletion unit 34 corresponds. In this example, the distribution is set based on the characteristics of the data source being log data from office employees, such as a Poisson distribution from the perspective of the number of times the app was used in discrete time and the number of times stamps representing "likes" were pressed, and a normal distribution from the perspective of the distribution of a large number of users working in an office.

[0179] The calculation unit 352 determines the normalization parameters necessary for normalization based on the data distribution estimated by the estimation unit 351. At this time, from the perspective of user log data, outlier data is generated due to the user's behavior on the app. For example, log data of a user who communicates exceptionally actively on the app. This data would normally be removed as an outlier, but it represents the characteristics of a user who acts actively on the app. If normalization is simply performed using the maximum value of the data, data that could be considered outliers will be rounded to "1.0", while other data will be rounded to a format closer to "0". At this time, the calculation unit 352 calculates parameters that allow normalization in a format that emphasizes the characteristics of the user, by leaving the data of the user who acts actively on the app at a value greater than "1.0", such as "1.3".

[0180] The normalization unit 353 normalizes the data acquired by the acquisition unit 31 based on the normalization parameters calculated by the calculation unit 352. The normalized data is then output from the output unit 36 ​​and used for user profile estimation. This normalized data excludes data that negatively impacts profile estimation, and data that might otherwise be excluded as outliers is retained as information about distinctive users among office employees. Therefore, the characteristics of office employees can be estimated with high accuracy through application logs. By analyzing the behavior of users identified as key figures in the office (working hours, frequency of real-world conversations with other users, chat application usage time, etc.) and recommending similar behaviors to other employees, it can lead to the revitalization of the entire office. [Industrial applicability]

[0181] This disclosure is broadly applicable to data normalization processes for purposes such as user profile estimation. [Explanation of symbols]

[0182] 12 Normalizer 21 Processing Unit 31 Acquisition Department 32 Generation part 33 Judgment section 34 Deleted section 35 Normalization section 36 Output section 351 Estimation Department 352 Calculation Unit 353 Normalization section

Claims

1. Information processing device, Obtain the first data to be normalized, Frequency distribution information is generated for the first data, The singular portion included in the aforementioned frequency distribution information is determined, Second data is generated by deleting the singular data included in the singular portion of the first data. By normalizing the second data mentioned above, a third data is generated. Output the third data mentioned above. Information processing methods.

2. In determining the singular portion, a predetermined singular frequency distribution shape among the frequency distribution information is determined to be the singular portion. The information processing method according to claim 1.

3. The aforementioned unique frequency distribution shape includes the peak shape of the frequency distribution near the minimum value of the first data. The information processing method according to claim 2.

4. The aforementioned unusual frequency distribution shape includes a flat shape of the frequency distribution. The information processing method according to claim 2.

5. In generating the second data, the singular data is deleted if a predetermined deletion condition is met. The information processing method according to claim 1.

6. The deletion condition includes at least one of the following with respect to the frequency distribution information: number of peaks, range, data attributes, and frequency distribution shape. The information processing method according to claim 5.

7. The aforementioned data attributes include physical information obtained through activities in the real world and cyber information obtained through activities in the virtual world. The information processing method according to claim 6.

8. In generating the third data, The data distribution of the second data is estimated from multiple data distributions. Based on the data distribution of the second data mentioned above, the parameters for normalization are calculated, The second data is normalized using the aforementioned parameters. The information processing method according to claim 1.

9. The aforementioned multiple data distributions include discrete and continuous distributions. The information processing method according to claim 8.

10. The aforementioned data distributions include at least two of the following: a normal distribution, a uniform distribution, a Poisson distribution, binary data, a monotonically increasing distribution, and a monotonically decreasing distribution. The information processing method according to claim 8.

11. The parameter includes at least one of a first threshold greater than the minimum value of the second data and a second threshold less than the maximum value of the second data. The information processing method according to claim 8.

12. The first data includes log data, image data, or audio data. The information processing method according to claim 1.

13. Equipped with a circuit configuration, The aforementioned circuit configuration is, Obtain the first data to be normalized, Frequency distribution information is generated for the first data, The singular portion included in the aforementioned frequency distribution information is determined, Second data is generated by deleting the singular data included in the singular portion of the first data. By normalizing the second data mentioned above, a third data is generated. Output the third data mentioned above. Information processing device.

14. A program that causes an information processing device to perform processing, The aforementioned process is, Obtain the first data to be normalized, Frequency distribution information is generated for the first data, The singular portion included in the aforementioned frequency distribution information is determined, Second data is generated by deleting the singular data included in the singular portion of the first data. By normalizing the second data mentioned above, a third data is generated. Output the third data mentioned above. program.

Images