Information processing device, information processing method, and program

Abstract

To provide a machine learning technique for estimating means of transportation of a user with high accuracy.SOLUTION: An information processing device obtains a movement path of a user, derives movement information indicating a feature relating to the movement, from the movement path, and uses a learning model to estimate means of transportation of the user from the movement path and the movement information. The learning model is configured from a first network comprising a first branch and a second branch, and a second network following the first network. The first branch generates a feature quantity of the movement path from the movement path; the second branch generates a feature quantity of the movement information from the movement information; and the second network is configured to generate a combined feature quantity by combining the feature quantity of the movement path and the feature quantity of the movement information, and to output data indicating the means of transportation of the user from the combined feature quantity.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program, and in particular to a technique for predicting a user's means of transportation. [Background technology]

[0002] Technology has been developed that uses machine learning to predict a user's mode of transportation (walking, car, bus, train, etc.) from sensor data acquired from a smartphone carried by the user (Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Ifigenia Drosouli, et al., “Transportation Mode Detection Using an Optimized Long Short-Term Memory Model on Multimodal Sensor Data”, Entropy, vol. 23, issue 11, p. 1457, November 2021 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Non-Patent Document 1 predicts a user's mode of transportation from six sensors (accelerometer, gyroscope, geomagnetic sensor, pressure sensor, GPS (altitude measurement), and temperature) installed in a smartphone. However, this method requires that all smartphones be equipped with the six sensors in order to predict the mode of transportation, and a more efficient method for predicting the mode of transportation is desired.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a machine learning technique for predicting a user's means of transportation efficiently and with high accuracy. [Means for solving the problem]

[0006] In order to solve the above problem, one aspect of an information processing device according to the present invention includes an acquisition unit that acquires a user's movement trajectory, a derivation unit that derives movement information indicating characteristics related to the movement from the movement trajectory, and an estimation unit that estimates the user's means of transportation from the movement trajectory and the movement information using a learning model, wherein the learning model is configured to include a first network consisting of a first branch and a second branch, and a second network following the first network, wherein the first branch generates features of the movement trajectory from the movement trajectory, the second branch generates features of the movement information from the movement information, and the second network is configured to combine the features of the movement trajectory and the features of the movement information to generate combined features, and output data indicating the user's means of transportation from the combined features.

[0007] The learning model is configured to output, for a plurality of means of transportation, a probability that the means of transportation is the user's means of transportation as data indicating the user's means of transportation, and the estimation unit can estimate the means of transportation with the highest probability as the user's means of transportation.

[0008] The movement trajectory may include the latitude and longitude of the user's location for each predetermined time period.

[0009] The movement information may include one or more of the speed, acceleration, jerk, direction, and direction difference between two points on the movement trajectory, and the speed difference, acceleration difference, average speed, average speed difference, and average acceleration between a plurality of two points.

[0010] The means of transportation may include one or more of a car, a train, a bus, a bicycle, walking, a boat or a ship.

[0011] The information processing device may further include an output unit that outputs a result of the estimation by the estimation unit. The output unit may generate and output an advertisement related to the transportation means estimated by the estimation unit.

[0012] In order to solve the above problems, one aspect of the information processing method of the present invention includes an acquisition process for acquiring a user's movement trajectory, a derivation process for deriving movement information indicating characteristics related to the movement from the movement trajectory, and an estimation process for estimating the user's means of transportation from the movement trajectory and the movement information using a learning model, wherein the learning model is configured to include a first network consisting of a first branch and a second branch, and a second network following the first network, wherein the first branch generates features of the movement trajectory from the movement trajectory, the second branch generates features of the movement information from the movement information, and the second network is configured to combine the features of the movement trajectory and the features of the movement information to generate combined features, and output data indicating the user's means of transportation from the combined features.

[0013] In order to solve the above problem, one aspect of the program of the present invention is an information processing program for causing a computer to execute information processing, the program causing the computer to execute processes including an acquisition process for acquiring a user's movement trajectory, a derivation process for deriving movement information indicating characteristics related to the movement from the movement trajectory, and an estimation process for estimating the user's means of transportation from the movement trajectory and the movement information using a learning model, the learning model being composed of a first network composed of a first branch and a second branch, and a second network following the first network, the first branch generating features of the movement trajectory from the movement trajectory, the second branch generating features of the movement information from the movement information, and the second network being configured to combine the features of the movement trajectory and the features of the movement information to generate combined features, and output data indicating the user's means of transportation from the combined features.

[0014] In order to solve the above problems, one aspect of the learning model of the present invention is configured to have a first network having a first branch that inputs a user's movement trajectory and generates features of the movement trajectory, and a second branch that inputs movement information regarding movement derived from the movement trajectory and generates features of the movement information, and a second network configured to combine the features of the movement trajectory and the features of the movement information to generate combined features, and to output data indicating the user's means of transportation from the combined features. Effect of the Invention

[0015] According to the present invention, a machine learning technique is provided for predicting a user's transportation mode efficiently and with high accuracy. The above-mentioned objects, aspects, and advantages of the present invention, as well as objects, aspects, and advantages of the present invention not described above, will be understood by those skilled in the art from the following detailed description of the invention by referring to the accompanying drawings and the claims. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 shows an example of the configuration of an information processing system. [Diagram 2] FIG. 2 illustrates an example of a functional configuration of an information processing device. [Diagram 3] FIG. 3 shows an example of a hardware configuration of an information processing device. [Figure 4] FIG. 4 shows a flowchart of the learning data generation process. [Diagram 5] FIG. 5 is a conceptual diagram for explaining the process of assigning soft labels. [Figure 6] Figure 6 shows a conceptual diagram of the input and output of the labeling function. [Figure 7] Figure 7 shows an example of a transportation mode prediction model architecture. [Figure 8] FIG. 8 shows a flowchart of the transportation mode estimation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment for carrying out the present invention will be described in detail with reference to the attached drawings. Among the components disclosed below, those having the same functions are given the same reference numerals, and their description will be omitted. Note that the embodiment disclosed below is an example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiment. In addition, not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present invention.

[0018] [Example of information processing system configuration] Fig. 1 shows an example of the configuration of an information processing system according to this embodiment. As an example, as shown in Fig. 1, this information processing system includes an information processing device 10 and multiple user devices 11-1 to 11-N (N>1) used by multiple arbitrary users 1 to N. In the following description, unless otherwise specified, user devices 11-1 to 11-N may be collectively referred to as user device 11. In the following description, the terms user device and user may be used synonymously.

[0019] The user device 11 is, for example, a device such as a smartphone or a tablet, and is configured to be able to communicate with the information processing device 10 via a public network such as LTE (Long Term Evolution) or a wireless communication network such as a wireless LAN (Local Area Network). The user device 11 has a display unit (display surface) such as a liquid crystal display, and each user can perform various operations using a GUI (Graphic User Interface) equipped on the liquid crystal display. The operations include various operations on content such as images displayed on the screen, such as tapping, sliding, and scrolling using a finger or a stylus. Note that the user device 11 is not limited to the device in the form shown in Fig. 1, and may be a device such as a tablet terminal or a notebook PC. Also, the user device 11 may be provided with a separate display screen.

[0020] The user device 11 can log in to a web service (internet-related service) provided from the information processing device 10 or from another device (not shown) via the information processing device 10, and can use the service. The web service can include an online mall, an online supermarket, or services related to communications, finance, real estate, sports, and travel, which are provided via the Internet. By using such a web service, the user device 11 can transmit information about the user of the user device 11 to the information processing device 10.

[0021] For example, the user device 11 can transmit to the information processing device 10 attribute information relating to the user device and the user (hereinafter, user attribute information), such as the IP (Internet Protocol) address of the user device 11, the user's address, and the user's name. In addition, the user device 11 performs positioning calculations based on signals received from GPS (Global Positioning System) satellites (not shown), and generates information such as latitude, longitude, and altitude obtained by the calculations as location information of the user device 11 and transmits it to the information processing device 10. The information processing device 10 acquires various information from the user device 11, and generates learning data for machine learning to predict the user's means of transportation based on the information, and performs processing to predict the means of transportation.

[0022] [Functional configuration of information processing device 10] The information processing device 10 according to this embodiment first acquires various information from the user devices 11-1 to 11-N, and generates learning data (teacher data) for training the transportation prediction model 111, which is a machine learning model for predicting the transportation of users. Next, the information processing device 10 trains the transportation prediction model 111 using the generated training data. Furthermore, the information processing device 10 predicts the transportation of an arbitrary user using the trained transportation prediction model 111.

[0023] FIG. 2 is a block diagram showing an example of a functional configuration of the information processing device 10 according to the present embodiment. 2 includes a user information acquisition unit 101, a map information acquisition unit 102, a movement information derivation unit 103, a learning data generation unit 104, a learning unit 105, an estimation unit 106, an output unit 107, a learning model storage unit 110, and a data storage unit 120. The learning model storage unit 110 is configured to be capable of storing a transportation prediction model 111. The data storage unit 120 is configured to be capable of storing a user trajectory 121, map information 122, movement information 123, a labeling function 124, and learning data 125.

[0024] The user information acquisition unit 101 acquires information on the user's movements from each of the user devices 11-1 to 11-N. Specifically, the user information acquisition unit 101 acquires a plurality of consecutive positional information within a certain period of time (hereinafter, referred to as a user trajectory) that indicates a movement trajectory. The certain period of time can be set arbitrarily. For example, the certain period of time can be a period during which the user's movement is continuous (does not stop for a long period of time). Each piece of positional information includes at least latitude and longitude to which a timestamp (date and time information or time information) is attached (has a timestamp). The user information acquisition unit 101 may also acquire user attribute information from each of the user devices 11-1 to 11-N. The user information acquisition unit 101 stores the acquired user trajectory in the data storage unit 120 as a user trajectory 121.

[0025] The map information acquisition unit 102 acquires map information from any service (such as a website or a database). As described later, the map information includes, for example, information on road networks, railroad networks, bus route networks, and borders of regional areas (such as prefectures and states) (hereinafter, regional area information). The map information acquisition unit 102 stores the acquired map information in the data storage unit 120 as map information 122.

[0026] The movement information derivation unit 103 derives features related to movement in the user trajectory 121 (hereinafter, movement information) from the user trajectory 121 for a certain time stored in the data storage unit 120. Specifically, the movement information derivation unit 103 derives the movement information using position information of each of two time-stamped points. In this embodiment, the movement information may include the speed, acceleration, jerk (rate of change of acceleration with respect to time), direction, and direction difference between the two points (between a first position (before movement) and a second position (after movement)).

[0027] The distance x between two points can be derived as shown in equation (1). TIFF0007620073000001.tif18158Here, θ1 and θ2 are the latitude (in radians) of the first location and the latitude (in radians) of the second location, respectively, λ1 and λ2 are the longitude (in radians) of the first location and the longitude (in radians) of the second location, respectively, and r is the radius (in meters) of the Earth (Sphere).

[0028] Using the distance x derived from equation (1), the user's velocity v between two points can be derived as shown in equation (2). TIFF0007620073000002.tif11158

[0029] In addition, using the distance x derived from equation (1) and the velocity v derived from equation (2), the acceleration a of the user between two points can be derived as shown in equation (3). TIFF0007620073000003.tif11158

[0030] In addition, the jerk j can be derived as shown in equation (4) using the distance x derived by equation (1), the velocity v derived by equation (2), or the acceleration a derived by equation (3). TIFF0007620073000004.tif11158

[0031] The bearing between two points can be derived as shown in equation (5). TIFF0007620073000005.tif8158Here, X and Y are expressed as equations (6) and (7), respectively. TIFF0007620073000006.tif8158 TIFF0007620073000007.tif8158Furthermore, the direction difference between multiple points can be derived as shown in equation (8) using the directions between multiple two points derived by equation (5). TIFF0007620073000008.tif8158

[0032] Furthermore, the movement information derivation unit 103 can derive the speed difference (Speed_diff), acceleration difference (Acceleration_diff), average speed (Avg_speed), average speed difference (Avg_speed_diff), and average acceleration (Avg_acceleration) as movement information from the speed v and acceleration a between a plurality of two points. Therefore, in this embodiment, the movement information can include one or more of the speed, acceleration, jerk, direction, and direction difference between two points, and the speed difference, acceleration difference, average speed, average speed difference, and average acceleration between a plurality of two points. The movement information derivation unit 103 stores the derived movement information in the data / function storage unit 120 as movement information 123 .

[0033] The learning data generation unit 104 generates a labeling function 124 and stores it in the data / function storage unit 120. The labeling function 124 will be described later. The learning data generation unit 104 also generates learning data (teacher data) for training the transportation prediction model 111, using the labeling function 124. The learning data generation unit 104 stores the generated learning data in the data / function storage unit 120 as learning data 125. The procedure for generating the learning data will be described later.

[0034] The learning unit 105 trains the transportation prediction model 111 using the training data 125 generated by the training data generation unit 104. The procedure for training the transportation prediction model 111 will be described later.

[0035] The estimation unit 106 estimates the transportation mode of a user corresponding to a user trajectory acquired from the user by using the transportation mode prediction model 111. The output unit 107 outputs the result of the user's transportation means estimated by the estimation unit 106 (estimated result). The output unit 107 may generate and output information (e.g., advertisement) based on the estimation result. The output may be any output process, and may be output to an external device via a communication I / F (communication I / F 37 in FIG. 3) or may be displayed on a display unit (display unit 36 ​​in FIG. 3).

[0036] [Hardware configuration of information processing device 10] FIG. 3 is a block diagram showing an example of a hardware configuration of the information processing device 10 according to the present embodiment. The information processing apparatus 10 according to the present embodiment can be implemented on any single or multiple computers, mobile devices, or any other processing platform. 3, the information processing device 10 is illustrated as being implemented in a single computer, but the information processing device 10 according to the present embodiment may be implemented in a computer system including multiple computers. The multiple computers may be connected to each other via a wired or wireless network so as to be able to communicate with each other.

[0037] 3, the information processing device 10 may include a CPU 31, a ROM 32, a RAM 33, a HDD 34, an input unit 35, a display unit 36, a communication I / F 37, and a system bus 38. The information processing device 10 may also include an external memory. A CPU (Central Processing Unit) 31 generally controls the operation of the information processing device 10, and controls each of the components (32 to 37) via a system bus 38, which is a data transmission path.

[0038] The ROM (Read Only Memory) 32 is a non-volatile memory that stores control programs and the like necessary for the CPU 31 to execute processing. Note that the programs may be stored in a non-volatile memory such as an HDD (Hard Disk Drive) 34 or an SSD (Solid State Drive) or in an external memory such as a removable storage medium (not shown). The RAM (Random Access Memory) 33 is a volatile memory and functions as a main memory, a work area, etc. of the CPU 31. That is, the CPU 31 loads necessary programs, etc. from the ROM 32 to the RAM 33 when executing processing, and realizes various functional operations by executing the programs, etc. The learning model storage unit 110 and the data / function storage unit 120 shown in FIG. 2 can be configured by the RAM 33.

[0039] The HDD 34 stores, for example, various data and various information required when the CPU 31 performs processing using a program. The HDD 34 also stores, for example, various data and various information obtained when the CPU 31 performs processing using a program. The input unit 35 is composed of a keyboard and a pointing device such as a mouse. The display unit 36 ​​is configured with a monitor such as a liquid crystal display (LCD), etc. The display unit 36 ​​may be configured in combination with the input unit 35 to function as a GUI (Graphical User Interface).

[0040] The communication I / F 37 is an interface that controls communication between the information processing device 10 and an external device. The communication I / F 37 provides an interface with a network and executes communication with an external device via the network. Various data, various parameters, etc. are transmitted and received between the external device and the communication I / F 37. In this embodiment, the communication I / F 37 may execute communication via a wired LAN (Local Area Network) or a dedicated line conforming to a communication standard such as Ethernet (registered trademark). However, the network available in this embodiment is not limited to this, and may be configured as a wireless network. This wireless network includes wireless PANs (Personal Area Networks) such as Bluetooth (registered trademark), ZigBee (registered trademark), and UWB (Ultra Wide Band). It also includes wireless LANs (Local Area Networks) such as Wi-Fi (Wireless Fidelity) (registered trademark), and wireless MANs (Metropolitan Area Networks) such as WiMAX (registered trademark). It also includes wireless WANs (Wide Area Networks) such as LTE / 3G, 4G, and 5G. Note that the network only needs to connect each device so that they can communicate with each other, and the communication standard, scale, and configuration are not limited to those described above.

[0041] At least some of the functions of each element of the information processing device 10 shown in Fig. 2 can be realized by the CPU 31 executing a program. However, at least some of the functions of each element of the information processing device 10 shown in Fig. 2 may be operated as dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 31.

[0042] [Hardware configuration of user device 11] The hardware configuration of the user device 11 shown in Fig. 1 may be the same as the hardware configuration shown in Fig. 3. That is, the user device 11 may include a CPU 31, a ROM 32, a RAM 33, a HDD 34, an input unit 35, a display unit 36, a communication I / F 37, and a system bus 38. The user device 11 can display various information provided by the information processing device 10 on the display unit 36, and perform processing corresponding to an input operation received from a user via a GUI (configured by the input unit 35 and the display unit 36).

[0043] [Procedure for generating training data] Next, a procedure for generating learning data according to this embodiment will be described. Fig. 4 shows a flowchart of the learning data generation process performed by the learning data generation unit 104. In this example, it is assumed that the user trajectory 121, map information 122, movement information 123, and labeling function 124 are stored in the data / function storage unit 120. For the sake of explanation, the information processing system shown in Fig. 1 will be referred to.

[0044] First, in S41, the learning data generation unit 104 acquires from the data / function storage unit 120 the map information 122 acquired by the map information acquisition unit 102. In S42, the learning data generation unit 104 acquires the user trajectories 121 of the users 1 to N acquired by the user information acquisition unit 101 from the data / function storage unit 120. In S43, the learning data generation unit 104 performs a spatial join operation between the map information 122 and the user trajectory 121, and assigns a soft label to the user trajectory 121 (soft labeling).

[0045] 5 shows a conceptual diagram for explaining the process of S43 (assignment of soft labels). In this embodiment, the map information 122 includes a road network 51, a railroad network 52, a bus route network 53, and regional area information 54. Note that the map information 122 shown in FIG. 5 is merely an example, and the map information 122 may include information on other maps.

[0046] The road network 51 is road information representing roads (all ground roads connecting one location to another). The railroad network 52 is information on railroad route maps. The bus route network 53 is information on route maps of route buses. The bus route network 53 may also include information on route maps of buses that operate on a temporary basis. The regional area information 54 is information on the boundaries of regional areas (e.g., prefectures and states), and also includes information on the boundaries of ponds, lakes, or oceans. The road network 51, the railway network 52, and the bus route network 53 are represented by lines representing the roads, railways, and bus routes, respectively, and the regional area information 54 can be represented by areas enclosed by any boundaries (lines).

[0047] The learning data generating unit 104 also prepares a soft label 55 corresponding to the map information 122. The soft label 55 is a label indicating traffic information corresponding to an area (region) or route (line) in the map information 122 that is consistent (highly likely to be consistent) with the user trajectory 121 when the user trajectory 121 is spatially combined (mapped) with the map information 122. The soft label 55 is also a potential label for a label 61 described later. The soft label 55 is "road", "railroad", "bus", or "boat / ship". Note that this is only an example, and other soft labels may be prepared. For example, the soft label 55 may include "other". If the user trajectory 121 does not pass through a line or region indicating "road", "railroad", "bus", or "boat / ship", "other" may be used.

[0048] The learning data generation unit 104 spatially combines the map information 122 and the user trajectory 121, and assigns a soft label 55 to the user trajectory 121. Specifically, the learning data generation unit 104 maps the user trajectory 121 to the road network 51, the railroad network 52, the bus route network 53, and the regional area information 54, and assigns a soft label 55 corresponding to the user trajectory 121. For example, when the user trajectory 121 is a trajectory that passes through a line (route) indicated by the bus route network 53, the learning data generation unit 104 assigns the soft label 55 of "bus" to the user trajectory 121. In addition, when the user trajectory 121 is a trajectory that moves within a lake area indicated by the regional area information 54, the learning data generation unit 104 assigns the soft label 55 of "boat / ship" to the user trajectory 121. In addition, if the user trajectory 121 does not match any of the road network 51, the railway network 52, the bus route network 53, and the regional area information 54, the learning data generation unit 104 may assign the user trajectory 121 a soft label 55 of “other.”

[0049] Next, in S44, the learning data generation unit 104 acquires the movement information 123 derived by the movement information derivation unit 103 from the data / function storage unit 120. As described above, the movement information 123 may include one or more of the speed, acceleration, jerk, direction, and direction difference between two points, and the speed difference, acceleration difference, average speed, average speed difference, and average acceleration between a plurality of two points.

[0050] In S45, the learning data generation unit 104 acquires the labeling function 124 from the data / function storage unit 120. The labeling function 124 includes a plurality of functions each configured to output a label from the movement information 123 and / or the soft label 55. The labeling function 124 is generated by the learning data generation unit 104 and stored in the data / function storage unit 120. Note that the labeling function 124 may be generated in advance in an information processing system and stored in the data / function storage unit 120.

[0051] 6 shows a conceptual diagram of the input and output of the labeling function 124. The label function 124 is a function that receives the soft label 55 and the movement information 123 as input, and outputs a label 61. In this embodiment, the label 61 corresponds to the user's (user's) means of transportation, and is assumed to be "car", "train", "bus", "bicycle", "walking", or "boat / ship". Note that this is just an example, and the label 61 may include other labels.

[0052] An example of the labeling function 124 is shown in [1] to [3]. [1] If the speed included in the movement information 123 is between 15 m / s and 33 m / s, and the soft label 55 is not "train," output the label 61 of "car." [2] If the speed included in the movement information 123 is greater than or equal to 2.5 m / s and less than or equal to 10 m / s, and the soft label 55 is “bus”, output the label 61 of “bus”. [3] The speed included in the movement information 123 is 1.4 m / s or less, and the acceleration included in the movement information 123 is 1.5 m / s 2 If less than 100, output label 61 for "walking" Note that this function is just an example, and the labeling function 124 may be configured to include a plurality of functions configured to output the labels 61 from the movement information 123 and / or the soft labels 55.

[0053] In S46, the learning data generation unit 104 applies the soft label 55 assigned in S43 and the movement information 123 acquired in S44 to the labeling function 124, and assigns a label 61 to the soft label 55 and the movement information 123. That is, the learning data generation unit 104 assigns the label 61 to a combination of the user trajectory 121 to which the soft label 55 has been assigned and the movement information 123.

[0054] The processes of S45 and S46 may be implemented by the Snorkel platform, which is a platform that automates labeling. Furthermore, the learning data generation unit 104 can modify the labeling function 124 based on the estimation result by the estimation unit 106. For example, the learning data generation unit 104 can change the value of the speed in the above equation [1] based on the estimation result.

[0055] The learning data generation unit 104 generates a dataset in which a label 61 is assigned to the user trajectory 121 and the movement information 123 to which the soft label 55 is assigned. That is, the learning data generation unit 104 generates a dataset in which the label 61 (correct answer data) indicating a means of transportation is associated with a combination of the user trajectory 121 and the movement information 123. The learning data generation unit 104 performs the dataset generation process multiple times to generate multiple datasets. The learning data generation unit 104 stores the multiple datasets in the data / function storage unit 120 as learning data 125.

[0056] In this way, the learning data generation unit 104 generates the learning data 125 based on the user trajectory 121. Specifically, the learning data generation unit 104 generates the learning data 125 by using the user trajectory 121 and the movement information 123 derived from the user trajectory 121. This makes it possible to generate learning data from the user trajectory 121, i.e., continuous position information of a user, and makes it possible to efficiently generate learning data without complicating the functions of the user terminal, such as by equipping the user terminal with a special sensor.

[0057] Furthermore, by setting the labeling function 124, it becomes possible to distinguish between cars and bicycles passing through the same road and to appropriately label them based on the user trajectory 121 and the movement information 123. For example, if the speeds or accelerations of two user trajectories passing through the same road are different, by setting the labeling function 124 with different speeds and / or accelerations as conditions, it becomes possible to distinguish between the two user trajectories and to appropriately label them.

[0058] [Training procedure for transportation mode prediction model 111] Next, a learning procedure of the transportation prediction model 111 will be described. Fig. 7 shows an example of the architecture of the transportation prediction model 111, which is a learning model for machine learning. The transportation prediction model 111 shown in Fig. 7 is a deep learning model using a neural network.

[0059] As shown in FIG. 7, the transportation mode prediction model 111 according to this embodiment is composed of a first network 71 consisting of a first branch 711 and a second branch 712, and a second network 72 following the first network 71. Each of the first network 71 and the second network 72 includes an input layer represented by a white box and multiple fully connected layers (Dense) represented by a shaded box. The second network 72 includes an output layer represented by a gray box. The numbers shown in each layer indicate the number of nodes (number of units). The number of layers is not limited to the number shown in FIG. 7.

[0060] The learning unit 105 trains the transportation prediction model 111 using the training data 125 generated by the training data generation unit 104. As described above, the training data 125 includes a plurality of data sets, and in each data set, the label 61 is associated with a combination of the user trajectory 121 and the movement information 123. Thus, the learning unit 105 trains the transportation prediction model 111 to learn the relationship between the user trajectory 121 and the movement information 123 and the label 61.

[0061] The learning unit 105 inputs the user trajectory 121 (data indicating the user trajectory 121) to the first branch 711, and inputs the movement information 123 (data indicating the movement information 123) to the second branch 712.

[0062] In the first branch 711, a feature amount (feature vector) of the user trajectory is generated (extracted) from the user trajectory 121. In addition, a compressed (encoded) feature amount of the user trajectory is generated via multiple fully connected layers.

[0063] In the second branch 712, a feature amount (feature vector) of the movement information is generated (extracted) from the movement information 123. In addition, the feature amount of the movement information compressed (encoded) is generated via multiple fully connected layers. As described above, the movement information 123 may include a plurality of values ​​among the speed, acceleration, jerk, direction, direction difference between two points, and the speed difference, acceleration difference, average speed, average speed difference, and average acceleration between a plurality of two points, and therefore the data size may be large. Therefore, in the architecture shown in Fig. 7, the second branch 712 has more layers than the first branch 711, and compressed feature amounts of the movement information may be generated.

[0064] In the second network 72, first, the feature of the user trajectory generated in the first branch 711 and the feature of the movement information generated in the second branch 712 are combined to generate a combined feature (combined feature vector). Furthermore, in the second network 72, the combined feature is compressed (encoded) through multiple fully connected layers to generate six features corresponding to the number of labels 61.

[0065] The output layer is configured to generate and output data (data indicating a means of transportation) representing the label 61 associated with the user trajectory 121 and the movement information 123 input to the transportation prediction model 111 from the six feature amounts. An activation function (such as a softmax function) is used in the output layer. In this embodiment, the output layer can be configured to output, as output data, the probability (0 to 1) that each of the six types of labels (car, train, bus, bicycle, walking, boat / ship) included in the training data 125 is correct data. Alternatively, the output layer can be configured to output, as output data, 1 for the label that has the highest probability of being correct data among the six types of labels, and 0 for the other labels.

[0066] The learning unit 105 calculates and / or adjusts parameters in the transportation prediction model 111 using a loss function to which the output data of the output layer and the correct answer data are applied, and trains the transportation prediction model 111. The learning unit 105 trains the transportation prediction model 111 using multiple data sets included in the training data 125. The learning unit 105 stores the trained transportation prediction model 111 in the training model storage unit 110.

[0067] In this way, the transportation prediction model 111 according to this embodiment inputs the user trajectory (raw data) acquired from the user and the movement information (derived data) derived from the user trajectory into different branches. Then, the model estimates the user's transportation from a combined feature generated by combining the features of the raw data and the derived data. By using two types of data, the raw data and the derived data representing the features of the raw data, it is possible to estimate the transportation with higher accuracy.

[0068] The learning process may be performed separately for urban areas and rural areas. In urban areas, people move around a lot, so learning can be done with a learning set 125 consisting of a large number of data sets. On the other hand, people move around less in rural areas, so there are fewer data sets for learning. Therefore, the learning accuracy can be higher in urban areas.

[0069] [Transportation method estimation procedure] Next, the transportation estimation procedure according to this embodiment will be described. Fig. 8 shows a flowchart of the transportation estimation process according to this embodiment. In this example, the trained transportation prediction model 111 trained as described above is stored in the trained model storage unit 110. For the sake of explanation, the information processing system shown in Fig. 1 will be referred to.

[0070] First, in S81, the user information acquisition unit 101 acquires a user trajectory of an arbitrary user (hereinafter, a target user) for a certain period of time. The target user may be any one of users 1 to N. The certain period of time may be set arbitrarily. For example, the certain period of time may be a period of time during which the movement of the user is continuous (does not stop for a long period of time).

[0071] In S82, the movement information derivation unit 103 derives movement information, which is a feature related to movement in the user trajectory, from the user trajectory of the target user acquired in S81. As described above, the movement information may include one or more of the speed, acceleration, jerk, direction, and direction difference between two points, and the speed difference, acceleration difference, average speed, average speed difference, and average acceleration between a plurality of two points.

[0072] In S83, the estimation unit 106 estimates the transportation of the target user using the trained transportation prediction model 111 based on the user trajectory acquired in S81 and the movement information derived in S82. Specifically, the estimation unit 106 inputs data indicating the user trajectory of the target user to the first branch 711 of the first network 71 of the transportation prediction model 111 shown in FIG. 7, and inputs data indicating the movement information of the target user to the second branch 722. From the input data, the transportation prediction model 111 outputs the probability (0 to 1) that each of the six types of labels (car, train, bus, bicycle, walking, boat / ship) is correct data (actual transportation) as described above. Alternatively, the transportation prediction model 111 outputs 1 to the label with the highest probability of being correct data among the six types of labels, and 0 to the other labels. The estimation unit 106 estimates the transportation means corresponding to the label that is most likely to be correct data as the transportation means of the target user.

[0073] Finally, in S84, the output unit 107 outputs information on the transportation means of the target user estimated in S83 (estimated result). For example, the output unit 107 outputs information in which the target user and the estimation result are associated with each other to an external device. The output unit 107 may generate and output information based on the estimation result. For example, the output unit 107 can generate and output an advertisement related to the transportation means estimated for the user trajectory. When the transportation means estimated from the ongoing user trajectory of the target user is a car, the output unit 107 can generate an advertisement for tourist spots and service information near the current location based on the current location of the target user, and provide it to the target user. When the transportation means estimated from the ongoing user trajectory of the target user is walking, the output unit 107 can generate an advertisement related to health improvement, and provide it to the target user.

[0074] In Figure 8, we have explained the transportation means of the target user, but by estimating the transportation means of many users using a similar process, we can estimate the traffic volume for each transportation means in any time period in any area. This estimation can be useful for appropriate transportation planning and traffic management.

[0075] Although specific embodiments have been described above, the embodiments are merely illustrative and are not intended to limit the scope of the present invention. The apparatus and method described in this specification can be embodied in forms other than those described above. Furthermore, the above-described embodiments can be appropriately omitted, substituted, and modified without departing from the scope of the present invention. Such omitted, substituted, and modified forms are included in the scope of the claims and their equivalents, and belong to the technical scope of the present invention.

[0076] The disclosure of this embodiment includes the following configuration. [1] A system including an acquisition unit that acquires a user's movement trajectory, a derivation unit that derives movement information indicating characteristics related to the movement from the movement trajectory, and an estimation unit that estimates the user's transportation means from the movement trajectory and the movement information using a learning model, wherein the learning model is composed of a first network composed of a first branch and a second branch, and a second network following the first network, wherein the first branch generates a feature amount of the movement trajectory from the movement trajectory, and the second branch generates a feature amount of the movement information from the movement information, The information processing device is characterized in that the second network is configured to combine features of the movement trajectory and features of the movement information to generate a combined feature, and to output data indicating the user's means of transportation from the combined feature.

[0077] [2] The information processing device described in [1], wherein the learning model is configured to output, for a plurality of means of transportation, a probability that the means of transportation is the user's means of transportation as data indicating the user's means of transportation, and the estimation unit estimates the means of transportation with the highest probability to be the user's means of transportation.

[0078] [3] The information processing device according to [1] or [2], characterized in that the movement trajectory includes the latitude and longitude of the user's location for each specified time period.

[0079] [4] The information processing device according to any one of [1] to [3], characterized in that the movement information includes one or more of the speed, acceleration, jerk, direction, and direction difference between two points on the movement trajectory, and the speed difference, acceleration difference, average speed, average speed difference, and average acceleration between a plurality of two points.

[0080] [5] The information processing device according to any one of [1] to [4], characterized in that the means of transportation include one or more of car, train, bus, bicycle, walking, boat or ship.

[0081] [6] The information processing device according to any one of [1] to [5], further comprising an output unit that outputs information on the means of transportation estimated by the estimation unit.

[0082] [7] The information processing device described in [6], wherein the output unit generates and outputs an advertisement related to the transportation means estimated by the estimation unit.

[0083] [8] An information processing method comprising: an acquisition process for acquiring a user's movement trajectory; a derivation process for deriving movement information indicating characteristics related to the movement from the movement trajectory; and an estimation process for estimating the user's means of transportation from the movement trajectory and the movement information using a learning model, wherein the learning model is composed of a first network composed of a first branch and a second branch, and a second network following the first network, wherein the first branch generates features of the movement trajectory from the movement trajectory, the second branch generates features of the movement information from the movement information, and the second network is configured to combine the features of the movement trajectory and the features of the movement information to generate combined features, and output data indicating the user's means of transportation from the combined features.

[0084] [9] An information processing program for causing a computer to execute information processing, the program causing the computer to execute processes including an acquisition process for acquiring a user's movement trajectory, a derivation process for deriving movement information indicating characteristics related to the movement from the movement trajectory, and an estimation process for estimating the user's means of transportation from the movement trajectory and the movement information using a learning model, the learning model being composed of a first network composed of a first branch and a second branch, and a second network following the first network, the first branch generating features of the movement trajectory from the movement trajectory, the second branch generating features of the movement information from the movement information, and the second network being configured to combine the features of the movement trajectory and the features of the movement information to generate combined features, and output data indicating the user's means of transportation from the combined features.

[0085]

[10] A learning model configured to have a first network having a first branch that inputs a user's movement trajectory and generates features of the movement trajectory, and a second branch that inputs movement information regarding movement derived from the movement trajectory and generates features of the movement information, and a second network configured to combine the features of the movement trajectory and the features of the movement information to generate combined features, and to output data indicating the user's means of transportation from the combined features. [Explanation of symbols]

[0086] 1 to N: user, 10: information processing device, 11-1 to 11-N: user device, 101: user information acquisition unit, 102: map information acquisition unit, 103: movement information derivation unit, 104: learning data generation unit, 105: learning unit, 106: estimation unit, 107: output unit, 110: learning model storage unit, 111: transportation prediction model, 120: data / function storage unit, 121: user trajectory, 122: map information, 123: movement information, 124: labeling function, 125: learning data

Claims

1. an estimation unit that estimates a means of transportation of the user by using a learning model from first information that is a movement trajectory of the user and second information that is movement information that indicates a feature related to the movement and is derived from the first information; The learning model is composed of a first network composed of a first branch and a second branch, and a second network following the first network, The first branch generates a feature amount of the first information from the first information, The second branch generates a feature amount of the second information from the second information, The information processing device is characterized in that the second network is configured to combine features of the first information and features of the second information to generate a combined feature, and to output data indicating the user's means of transportation from the combined feature.

2. The information processing apparatus according to claim 1 , further comprising a derivation unit that derives the second information from the first information.

3. The information processing device according to claim 1 , wherein the second information includes one or more of a speed, an acceleration, a jerk, a direction, and a direction difference between two points on the movement trajectory of the user which is the first information.

4. The information processing device according to claim 1 , wherein the second information includes one or more of a speed difference, an acceleration difference, an average speed difference, and an average acceleration between two points on the user's movement trajectory which is the first information.

5. the learning model is a learning model trained using learning data in which a label indicating the transportation means is associated with the first information and the second information, The information processing device according to claim 1, characterized in that the label is a soft label indicating traffic information corresponding to an area or route in the map information that is consistent with location information mapped to the map information, and is a label generated by applying a soft label assigned to the first information and the second information to a predetermined function.

6. An information processing method executed by an information processing device, an estimation step of estimating a means of transportation of the user using a learning model from first information which is a movement trajectory of the user and second information which is movement information indicating characteristics related to the movement derived from the first information; The learning model is composed of a first network composed of a first branch and a second branch, and a second network following the first network, The first branch generates a feature amount of the first information from the first information, The second branch generates a feature amount of the second information from the second information, The information processing method is characterized in that the second network is configured to combine features of the first information and features of the second information to generate combined features, and to output data indicating the user's means of transportation from the combined features.

7. An information processing program for causing a computer to execute information processing, the program comprising: The present invention is for executing an estimation process of estimating a means of transportation of a user using a learning model, from first information which is a movement trajectory of the user and second information which is movement information indicating characteristics related to the movement derived from the first information, The learning model is composed of a first network composed of a first branch and a second branch, and a second network following the first network, The first branch generates a feature amount of the first information from the first information, The second branch generates a feature amount of the second information from the second information, The second network is configured to generate a combined feature by combining the feature of the first information and the feature of the second information, and to output data indicating a transportation means of the user from the combined feature. Information processing program.

Images