Electronic device, storage medium, and topography determination method

Abstract

The present application provides an electronic device, a storage medium, and a terrain determination method. The electronic device includes a processor that acquires body motion information of a user in a user movement action, and determines a terrain in the user movement based on the acquired body motion information.

Description

Technical Field

[0001] This invention designs an electronic device, a storage medium, and a terrain determination method. Background Technology

[0002] Previously, techniques for acquiring and evaluating a user's body movement information during movement were known. For example, Japanese Patent Application Publication No. 2020-5766 describes such a technique. Japanese Patent Application Publication No. 2020-5766 describes a device that uses a user's movement data, self-evaluation data, and GPS data as teaching data to perform machine learning, predicting and calculating a self-evaluation for new movement data.

[0003] In Patent Document 1, various data such as motion data and terrain data are used to evaluate the user's running posture, etc. However, the terrain data is obtained from databases, etc., and there is room for improvement in obtaining more detailed information about the terrain the user is moving on. Summary of the Invention

[0004] The present invention was made in view of the following situation, and its purpose is to provide an electronic device, a storage medium, and a terrain determination method that can determine the terrain in which a user is moving with high accuracy.

[0005] The electronic device of the present invention is characterized in that it comprises: a processor, which acquires the user's body movement information during a user's movement (including both when the user is currently moving and when the user has previously moved), and determines the terrain during the user's movement (including both the terrain where the user is currently moving and the terrain where the user has previously moved) based on the acquired body movement information.

[0006] The storage medium of the present invention is a computer-readable storage medium that enables an electronic device to perform the following processing functions: acquiring the user's body movement information during the user's movement, and determining the terrain in which the user is moving based on the acquired body movement information.

[0007] The terrain determination method of the present invention is executed by an electronic device and includes: a processing step of obtaining the user's body movement information during the user's movement, and determining the terrain during the user's movement based on the obtained body movement information. Attached Figure Description

[0008] Figure 1 This is a system structure diagram illustrating the structure of a terrain determination system according to an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram illustrating an example of the use of a sensor device according to an embodiment of the present invention.

[0010] Figure 3This is a block diagram illustrating the hardware structure of a sensor device according to an embodiment of the present invention.

[0011] Figure 4 This is a block diagram illustrating the hardware structure of a user terminal according to an embodiment of the present invention.

[0012] Figure 5 This is a block diagram illustrating the hardware structure of a management server according to an embodiment of the present invention.

[0013] Figure 6 This is a functional block diagram illustrating a portion of the functional structure of a management server according to an embodiment of the present invention.

[0014] Figure 7 This is a flowchart illustrating an example of the process for constructing a learning model for a management server according to an embodiment of the present invention.

[0015] Figure 8 This is a flowchart illustrating an example of the process executed by a user terminal in the terrain determination process performed by the terrain determination system according to an embodiment of the present invention.

[0016] Figure 9 This is a flowchart illustrating an example of the terrain determination process performed by the management server of a terrain determination system according to an embodiment of the present invention. Detailed Implementation

[0017] The embodiments of the present invention are illustrated below using the accompanying drawings.

[0018] [Terrain Determination System]

[0019] Figure 1 This is a system structure diagram illustrating the terrain determination system S according to one embodiment of the present invention. For example... Figure 1 As shown, the terrain determination system S includes: a management server 1, which is an electronic device for performing terrain determination processing; a sensor device 2; and a user terminal 3.

[0020] Management server 1 and user terminal 3 can communicate with each other. Communication between management server 1 and user terminal 3 can be achieved, for example, through a network 4 consisting of the Internet, a LAN (Local Area Network), a mobile phone network, or a combination of these. Furthermore, user terminal 3 and sensor device 2 can also communicate with each other. Communication between user terminal 3 and sensor device 2 can be achieved, for example, through BLE (Bluetooth Low Energy). However, this is just an example; other communication methods can also be used for communication between management server 1 and user terminal 3, and between user terminal 3 and sensor device 2.

[0021] Sensor device 2 has the following functions: sensing function to sense the user's body movements when the user is walking, running, or performing other moving actions; positioning function to locate the position of sensor device 2; and communication function to send the sensing results to user terminal 3. Hereinafter, the various data acquired by sensor device 2 will be referred to as log data for explanation.

[0022] An example of using sensor device 2 will be described. Figure 2 This is a schematic diagram illustrating an example of the use of the sensor device 2 according to an embodiment of the present invention. For example... Figure 2 As shown, the sensor device 2 of this embodiment is equipped near the waist along the user's torso, where the user performs a given movement action on the ground surface 5 (in this embodiment, walking or running is described, but the movement action is not limited to these).

[0023] Sensor device 2 acquires motion data such as changes in the user's speed (acceleration) and direction (angular velocity) during walking and running through its sensing function. Sensor device 2 also acquires positioning data such as the user's movement trajectory and running distance during activities like walking and running through its positioning function. The data containing both motion and positioning data acquired by sensor device 2 will be referred to as log data below. Furthermore, in addition to the distance traveled, the posture during movement is calibrated using the angle between gravitational acceleration and the axis of the long side (given direction) of sensor device 2.

[0024] User terminal 3 is a communication device with computing and communication functions. User terminal 3 can be implemented, for example, through wearable devices such as smartphones, tablets, and smartwatches that can be carried by the user.

[0025] The user terminal 3 in this embodiment has: a communication function, which receives log data from the sensor device 2 and sends it to the management server 1, and receives information from the management server 1; a positioning function, which locates the position of the user terminal 3; and an output function, which displays the parsing results of the log data from the management server 1. Through the output function of the user terminal 3, the user can grasp the parsing results of the sensor information in a simple and intuitive way.

[0026] Management server 1 is a management device with computing and communication functions. Management server 1 is implemented, for example, through an electronic device such as a server device or a personal computer. Management server 1 obtains the user's movement information based on log data sent from sensor device 2 via user terminal 3, and determines the terrain the user is moving through. Furthermore, management server 1 parses positioning data to obtain path information containing information such as altitude, latitude, and longitude of the path the user has moved through, and corrects this path information as needed. The movement information, path information, and terrain determination results obtained by management server 1 are sent to user terminal 3. In addition, management server 1 is configured to receive log data from multiple sensor devices 2 via multiple user terminals 3.

[0027] [Hardware Structure]

[0028] Next, an example of the hardware structure of sensor device 2 will be described. Figure 3 This is a block diagram illustrating the hardware structure of the sensor device 2 according to an embodiment of the present invention. For example... Figure 3 As shown, the sensor device 2 includes a CPU (Central Processing Unit) 11-1, a ROM (Read Only Memory) 12-1, a RAM (Random Access Memory) 13-1, a bus 14-1, an input / output interface 15-1, a sensor unit 16-1, an input unit 17-1, an output unit 18-1, a storage unit 19-1, a communication unit 20-1, and a GNSS unit 21-1.

[0029] CPU 11-1 performs various processes by following the program recorded in ROM 12-1 or the program loaded from storage unit 19-1 into RAM 13-1.

[0030] RAM13-1 is also suitable for storing data required by CPU11-1 for various processing tasks.

[0031] CPU11-1, ROM12-1, and RAM13-1 are interconnected via bus 14-1. Input / output interface 15-1 is also connected to bus 14-1. Sensor unit 16-1, input unit 17-1, output unit 18-1, storage unit 19-1, communication unit 20-1, and GNSS unit 21-1 are connected to input / output interface 15-1.

[0032] The input section 17-1 consists of various buttons, etc., and allows users to input various information according to their instructions.

[0033] The output unit 18-1 consists of LED (Light Emitting Diode) lamps, a display, a speaker, etc., and outputs light, images, sound, etc.

[0034] The storage unit 19-1 consists of a hard disk or flash memory, etc., and stores various types of data.

[0035] The communications section 20-1, for example, controls communication between other devices based on wireless (Bluetooth Low Energy) or wired (USB) communication.

[0036] The sensor unit 16-1 comprises an accelerometer, a gyroscope, and other sensors used to measure the three-dimensional movement of the sensor device 2 itself. The accelerometer is a device that detects movement and acceleration in any direction. For example, the accelerometer is a capacitive or piezoresistive 3-axis sensor that detects acceleration generated in three mutually orthogonal axes. The gyroscope is a device that detects movement and angular velocity in any direction. For example, the gyroscope is a capacitive or piezoresistive 3-axis sensor that detects angular velocity generated in three mutually orthogonal axes.

[0037] Sensor unit 16-1 detects at least the acceleration and angular velocity of the sensor device 2 corresponding to the actions of the user equipped with sensor device 2, and stores this data as log data in storage unit 19-1. The log data is then sent to user terminal 3 via communication unit 20-1, and then to management server 1 via user terminal 3. Management server 1 obtains the body movement information of the user equipped with sensor device 2 by parsing the received log data. In this embodiment, the start / end of sensing is triggered by user operation of input unit 17-1.

[0038] The GNSS unit 21-1 performs positioning based on positioning satellite signals transmitted from positioning satellites. GNSS is an abbreviation for Global Navigation Satellite System, and the GNSS unit 21-1 utilizes satellite positioning systems such as GPS. In this embodiment, the GNSS unit 21-1 includes an antenna that receives positioning satellite signals from multiple positioning satellites and transmits the received positioning satellite signals to the CPU 11-1. The CPU 11-1 obtains positioning data, including the position information of the sensor device 2, based on the positioning satellite signals received from the GNSS unit 21-1.

[0039] In addition to the structure illustrated above, sensor device 2 may also include a drive suitable for equipping removable media such as disks, optical discs, optical disks, or semiconductor memories. Programs and data read from the removable media by the drive are installed into storage unit 19-1 as needed.

[0040] Next, the hardware structure of user terminal 3 will be described. Figure 4 This is a block diagram illustrating the hardware structure of a user terminal 3 according to an embodiment of the present invention. For example... Figure 4 As shown, the user terminal 3 includes a CPU 11-2, ROM 12-2, RAM 13-2, bus 14-2, input / output interface 15-2, input unit 17-2, output unit 18-2, storage unit 19-2, communication unit 20-2, GNSS unit 21-2, and camera unit 22-2. In the user terminal 3, the structure identical to that of the sensor device 2 is omitted from description.

[0041] The input unit 17-2 and the output unit 18-2 are user interfaces electrically connected to the input / output interface 15-2 via wired or wireless means. The input unit 17-2 consists of, for example, a keyboard, mouse, various buttons, microphone, etc., to input various information in response to user instructions. The output unit 18-2 consists of, for example, a monitor displaying images, a speaker amplifying sound, etc., to output images and sound.

[0042] Although not shown, the camera unit 22-2 includes an optical lens and an image sensor. The image data captured by the camera unit 22-2 is appropriately provided to the CPU 11-2, an image processing unit (not shown), etc.

[0043] The hardware structure of management server 1 will be described next. Figure 5 This is a block diagram illustrating the hardware structure of the management server 1 according to one embodiment of the present invention. For example... Figure 5 As shown, the management server 1 includes a CPU 11-3, ROM 12-3, RAM 13-3, bus 14-3, input / output interface 15-3, input unit 17-3, output unit 18-3, storage unit 19-3, communication unit 20-3, and meteorological information acquisition unit 23-3. Description of the same structure as the sensor device 2 is omitted in the description of the management server 1.

[0044] The Meteorological Information Acquisition Department 23-3 obtains meteorological information from meteorological data collection systems such as the Automatic Meteorological Data Collection System (AMeDAs) via the Communications Department 20-3. This meteorological information includes weather conditions such as sunny, cloudy, rainy, and snowy conditions.

[0045] The functional structure of the management server 1, which performs terrain determination processing, will be described next. Figure 6 This is a functional block diagram representing a part of the functional structure of management server 1.

[0046] like Figure 6 As shown, in one area of ​​storage unit 19-3, there are: a historical information storage unit 190 that stores log data obtained from multiple sensor devices 2, weather information associated with the log data, etc.; a machine learning information storage unit 191 that stores information related to the learning model described later and the teaching data used to build the learning model; and a communication information storage unit 192 that stores information related to communication with management server 1.

[0047] The processing unit 30, which performs various controls on the management server 1, is implemented by the CPU 11-3, which performs arithmetic processing. The processing unit 30 in this embodiment includes a communication processing unit 31, an identification information acquisition unit 32, a path information acquisition unit 33, a motion information acquisition unit 34, a weather information acquisition unit 35, a learning unit 36, a terrain determination unit 37, a path information correction unit 38, a posture analysis unit 39, and an output processing unit 40.

[0048] The communication processing unit 31 performs processing for communicating with external devices via the communication unit 20-3. For example, the communication processing unit 31 performs processing for sending and receiving various information with the user terminal 3 connected to the management server 1, and processing for sending and receiving various information with the sensor device 2 via the user terminal 3.

[0049] The identification information acquisition unit 32 acquires identification information from the sensor device 2, which is used to identify the source of the log data. The user equipped with the sensor device 2 can be identified using the identification information acquired from the identification information acquisition unit 32.

[0050] The path information acquisition unit 33 acquires path information of the path the user equipped with sensor device 2 is moving along, based on positioning data acquired by GNSS unit 21-1 and transmitted via communication unit 20-3. Path information refers to geographical location information, including altitude, latitude, longitude, and other information of the path the user is moving along.

[0051] The motion information acquisition unit 34 acquires the user's motion information based on motion data detected by the sensor unit 16-1 and transmitted from the sensor device 2 via the user terminal 3. Specifically, the motion information acquisition unit 34 acquires motion information by analyzing changes in the user's movement speed and direction during walking and running. Examples of the user's motion information include, for instance, stride frequency, stride length, pelvic tilt, vertical foot movement, foot contact time, acceleration during kicking, and deceleration during foot contact. Stride length refers to the amplitude of a single step during movement, and stride frequency refers to the number of steps per unit time. Furthermore, the motion information acquisition unit 34 acquires motion information as input data to the learning unit 36.

[0052] An example of how the motion information acquisition unit 34 acquires the pelvic tilt angle caused by the user's posture will be explained. The motion information acquisition unit 34 calibrates the tilt angle of the sensor device 2 in an upright posture to align with the vertical direction, so that the pelvic tilt angle corresponding to the upright posture of the user equipped with the sensor device 2 becomes a reference. In the calibration, the static state is maintained for a given time with the pelvic tilt angle corresponding to the upright posture, and the calibration is performed using the angle between the acceleration due to gravity and the axis of the long side of the device, so that the pelvic tilt angle in the upright posture becomes a reference. When the user equipped with the sensor device 2 tilts the pelvis to an angle equivalent to a forward tilt posture, the sensor device 2 equipped near the waist also tilts accordingly, and the tilt angle in the forward tilt posture is greater than that in the upright posture. Thus, the change in the user's pelvic tilt angle is acquired, and posture changes such as forward tilt posture are acquired.

[0053] The weather information acquisition unit 35 acquires weather information from the meteorological information acquisition unit 23-3. The processing unit 30 establishes a correspondence between the weather information acquired by the meteorological information acquisition unit 23-3 and the log data sent from the sensor device 2 and stores it. Specifically, the processing unit 30 correlates the identification information, path information, and movement information obtained by parsing the same log data with the corresponding weather information and stores them as a set of historical record information in the historical record information storage unit 190.

[0054] The learning unit 36 ​​constructs a learning model to determine the terrain the user is moving through by performing supervised learning. The learning unit 36 ​​includes an input data acquisition unit 361, a label acquisition unit 362, and a learning model construction unit 363.

[0055] The input data acquisition unit 361 acquires the motion information of multiple different users moving along a certain path on the terrain, such as a known path (hereinafter referred to as a known path), from the historical record information storage unit 190 as input data. Preferably, the input data acquisition unit 361 acquires input data for multiple known paths with different terrains, acquiring the motion information of multiple different users for each known path. This allows for a higher accuracy in determining the correlation between user motion information and terrain. The motion information acquired as input data is preferably information regarding sunny or cloudy weather. For example, in rainy or snowy weather, the ground surface 5 of the path becomes muddy due to the rain or snow, making it more difficult for users to move compared to sunny or cloudy weather. By using only the motion information of users who move when the ground surface 5 of the path is dry during sunny or cloudy weather as input data, changes in the state of the ground surface 5 caused by weather can be excluded. Therefore, a learning model that more accurately reflects the correlation between user motion information and terrain can be constructed.

[0056] The tag acquisition unit 362 acquires the terrain of the known path as a tag. Examples of terrain include, for example, the slope of the path relative to the horizontal plane, such as uphill, downhill, or flat ground, and the shape of the ground surface 5. Examples of the shape of the ground surface 5 include, for example, a flat shape, a stepped shape, or an uneven shape. Examples of terrain include: terrain that slopes upwards at a certain angle relative to the user's direction of travel and has a stepped ground surface 5; terrain that slopes downwards at a certain angle relative to the user's direction of travel and has a flat ground surface 5; and flat ground with an uneven ground surface 5. The terrain of the known path can be obtained by the user inputting terrain data via the input unit 17-3, or by obtaining it from a terrain database.

[0057] The learning model construction unit 363 constructs a terrain learning model for determining the path of an unknown terrain object by performing supervised learning on a set of input data obtained by the input data acquisition unit 361 and labels obtained by the label acquisition unit 362 as teaching data. Supervised learning involves providing a large amount of input data (input) and label (result) data sets to the learning unit 36 ​​to learn features located in these data sets, thereby inferring the result from the input. This can be implemented using neural networks, support vector machines (SVMs), etc.

[0058] The learning model construction unit 363 stores the constructed learning model in the machine learning information storage unit 191. Furthermore, the learning model construction unit 363 preferably uses teaching data from multiple known paths for different terrains to construct the learning model.

[0059] This section explains the relationship between the body movement information used as input data and the terrain used as labels. For example, when the user is moving uphill, the step frequency tends to decrease and the stride length tends to be shorter compared to when the user is moving on flat ground. Conversely, when the user is moving downhill, the step frequency tends to increase and the stride length tends to be longer. Furthermore, for example, when the terrain surface is stepped, compared to a flat surface, there is a tendency for the amount of foot movement in the vertical direction to be greater when moving uphill and less when moving downhill.

[0060] The terrain determination unit 37 determines the terrain that the user equipped with the sensor device 2 is moving on, for example, based on the motion information acquired by the motion information acquisition unit 34. Specifically, the terrain determination unit 37 determines the terrain that the user is moving on by inputting the user's motion information during movement along the target path into a learning model constructed by the learning unit 36.

[0061] The terrain determination unit 37 can select whether to determine the terrain of the target path based on motion information, depending on whether the motion information usage conditions are met. For example, it can be considered that the motion information usage conditions are met if all of (a) to (c) are met, or it can be considered that the motion information usage conditions are met if (a) and (b) are met, or it can be considered that the motion information usage conditions are met only if (c) is met. In the following description, the user who uses the motion information for the terrain determination process performed by the terrain determination unit 37 is designated as the "target user", and the user other than the target user is designated as the "other user". When describing the common content between the target user and the other user, only the term "user" will be used.

[0062] (a) The historical information of multiple users whose paths have been moved is stored in the historical information storage unit 190.

[0063] (b) Determine the common behaviors of the target user's body movement information and the body movement information of other users (a number greater than a given number) within a specific interval of the target user's path.

[0064] (c) Determine whether the weather information in the user's movement is sunny or cloudy.

[0065] A specific interval can be, for example, an area where the terrain changes from flat to uphill. Runners in their normal state, without accumulated fatigue, tend to decrease their cadence and stride length when moving from flat ground towards an uphill section. On the other hand, runners who have accumulated fatigue tend to have a lower cadence and shorter stride length even on flat ground, making it difficult for them to exhibit the same changes in body movement within a specific interval as runners in their normal state. In other words, by confirming whether the body movement information of the target user within such a specific interval shows behaviors common to other users, it is possible to determine whether the target user is in their normal state.

[0066] The terrain determination unit 37 can, for example, determine the terrain of the target path using motion information acquired by the motion information acquisition unit 34 only when the conditions for using motion information are met. On the other hand, the terrain determination unit 37 can determine the terrain of the target path without using motion information when the conditions for using motion information are not met; furthermore, it can choose not to perform terrain determination processing for the target path. When determining the terrain without using motion information, for example, the terrain of the target path can be set to flat land.

[0067] Here, it is considered that the user's body movement during movement is affected not only by terrain factors, but also by factors such as the user's fatigue and the state of the ground surface 5 where the user's feet touch the ground. For example, if a user accumulates fatigue while running, even on flat terrain, there is a possibility that the stride frequency will decrease and the stride length will become shorter, as if the terrain were uphill. In contrast, in this embodiment, body movement information is used to determine the terrain only when the target user's body movement information shows behavior common to that of other users within a specific interval. Thus, for example, since terrain determination using body movement information can be avoided when the target user's body movement changes due to fatigue, misjudgment of terrain can be prevented. Furthermore, for example, if the ground surface 5 becomes muddy due to rain or snow, it becomes difficult for the user to run, affecting the user's body movement. In this embodiment, since terrain determination using body movement information can be avoided in the case of rain or snow, misjudgment of terrain can be prevented.

[0068] The path information correction unit 38 corrects the path information of the target path obtained by the path information acquisition unit 33 based on the terrain determined by the terrain determination unit 37. Specifically, the path information correction unit 38 can estimate the height based on the terrain determined by the terrain determination unit 37, and determine the height of the target path based on the estimated height and the height contained in the path information obtained by the path information acquisition unit 33. For example, the path information correction unit 38 can determine the height of the location based on a comparison between the estimated height and the height contained in the path information obtained by the path information acquisition unit 33 at the same location where the estimated height was determined. Furthermore, the path information correction unit 38 can determine the height based on the maximum and minimum heights within the target path of the target path obtained by the path information acquisition unit 33, and the difference between the maximum and minimum estimated heights within the target path of the target path estimated from the terrain determined by the terrain determination unit 37. For example, if the path information acquisition unit 33 acquires path information where the difference between the maximum and minimum heights within the target path is set to 20m, and the difference between the estimated maximum and minimum heights within the target path is 10m, then the difference between the maximum and minimum heights within the target path is corrected to 15m. Thus, for example, even path information acquired based on positioning data with errors due to the influence of the radio wave environment can be corrected based on terrain determined using the motion information of a user directly grounded on the ground surface 5.

[0069] The posture analysis unit 39 determines the user's posture based on the user's body movement information. It judges the quality of the user's posture during walking, running, and other movement actions based on a pre-determined reference posture corresponding to the type of movement (walking, running, etc.) and the determined user's body movement information. For example, the posture analysis unit 39 can derive the user's posture by analyzing the user's pelvic tilt, stride frequency, stride length, and vertical foot movement amount acquired by the body movement information acquisition unit 34. For example, it can determine whether the user is in an upright or forward-leaning posture from the pelvic tilt information contained in the body movement information. Furthermore, the posture analysis unit 39 determines a reference posture based on the terrain determined by the terrain determination unit 37. For example, in the case of uphill terrain, a posture that leans more forward than in the case of flat terrain is set as the reference posture. Then, the posture analysis unit 39 determines the user's posture based on a comparison between the user's posture determined based on the body movement information and the determined reference posture. The posture analysis unit 39, for example, the higher the similarity between the user's posture and the reference posture, the better the posture is judged as a movement action.

[0070] The output processing unit 40 performs the following processing: outputting the path information of the target path obtained by the path information acquisition unit 33 or the path information of the target path corrected by the path information correction unit 38, the terrain of the target path determined by the terrain determination unit 37, and the analysis results of the user's movement posture obtained by the posture analysis unit 39.

[0071] [Construction and processing of learning models]

[0072] Next, the machine learning process in the terrain determination system S according to this embodiment will be explained. Figure 7 This is a flowchart illustrating an example of the process of constructing a learning model performed by the learning unit 36 ​​of the management server 1.

[0073] The input data acquisition unit 361 of the learning unit 36 ​​acquires the user's body movement information in the known path movement action from the body movement information stored in the historical record information storage unit 190, and uses it as input data (step S11).

[0074] The tag acquisition unit 362 acquires the terrain of the known path that is associated with the motion information acquired in step S11 from the terrain stored in the historical record information storage unit 190, and uses it as a tag (step S12).

[0075] The learning model construction unit 363 accepts the input data obtained in step S11 and the tag obtained in step S12 as teaching data (step S13).

[0076] The learning model building unit 363 uses the teaching data received in step S13 to perform machine learning (step S14).

[0077] After step S14, the learning model construction unit 363 determines whether to end machine learning or repeat machine learning (step S15). If the learning model construction unit 363 determines that machine learning should be repeated (step S15; no), the process returns to step S11. Then, the management server 1 repeats the same operation. On the other hand, if the learning model construction unit 363 determines that machine learning should be ended (step S15; yes), the process moves to step S16. Furthermore, the conditions for ending machine learning can be arbitrarily determined. For example, machine learning can be ended if it is repeated a predetermined number of times. Specifically, machine learning can be ended if machine learning is performed using teaching data obtained from different users with a predetermined number of decision-making individuals using known paths with a predetermined number of decision-making paths or more.

[0078] The learning unit 36 ​​stores the learning model in the machine learning information storage unit 191 (step S16). Therefore, when the learning model is requested from the terrain determination unit 37 for terrain determination, it can be retrieved from the machine learning information storage unit 191. Furthermore, when new teaching data is obtained, the learning model can be further machine learned.

[0079] [Terrain Determination and Processing]

[0080] Next, an example of terrain determination processing in terrain determination system S will be described with reference to the accompanying drawings. In this terrain determination processing, the case of a user running as a movement action will be used as an example for explanation.

[0081] First, refer to Figure 8 This is an example of the processing flow performed by user terminal 3. Figure 8 This is a flowchart illustrating an example of the processing flow performed by the user terminal 3 in the terrain determination process executed by the terrain determination system S.

[0082] First, the user carrying the user terminal 3 activates the sensor device 2 and begins running with the sensor device 2 attached near the waist (step S21). At this time, the user operation input unit 17-1 instructs the detection of log data from the sensor unit 16-1 and the GNSS unit 21-1 to begin.

[0083] User terminal 3 acquires and saves the log data detected by sensor device 2 (step S22). Specifically, firstly, CPU 11-1 of sensor device 2 acquires motion data representing the acceleration and angular velocity of sensor device 2 detected by sensor unit 16-1, and positioning data of sensor device 2 detected by GNSS unit 21-1, and sends this motion data and positioning data to user terminal 3. Then, user terminal 3 acquires the motion data and positioning data via communication unit 20-2 and saves them to storage unit 19-3.

[0084] The user ends the run (step S23). At this time, the user operation input unit 17-1 indicates that the detection of log data of sensor unit 16-1 and GNSS unit 21-1 has ended.

[0085] The CPU 11-2 of user terminal 3 retrieves the log data saved in step S22 from storage unit 19-2 and sends it to management server 1 via communication unit 20-2 (step S24). The processing performed by management server 1 will be described later.

[0086] The CPU 11-2 of user terminal 3 determines whether it has received the path information, terrain determination result, and running posture analysis result (described later) sent from management server 1 (step S25). If the CPU 11-2 has not received the path information, terrain determination result, and running posture analysis result from management server 1 (step S25; no), the process of step S25 is repeated. On the other hand, if the path information, terrain determination result, and running posture analysis result are received from management server 1 (step S25; yes), the process is transferred to step S26.

[0087] The CPU 11-2 of user terminal 3 displays the path information, terrain determination results, and running posture analysis results from management server 1 on the output unit 18-2, such as a display (step S26). After the processing in step S26, the processing performed by user terminal 3 ends in the terrain determination process.

[0088] Next, refer to Figure 9 This is an example illustrating the process performed by management server 1. Figure 9 This is a flowchart representing the terrain determination process of the management server 1 of the terrain determination system S.

[0089] like Figure 9 As shown, the communication processing unit 31 of the management server 1 receives the log data sent from the user terminal 3 in step S24 (step S31).

[0090] The processing unit 30 of the management server 1 obtains identification information, path information, and motion information from the log data received in step S31 (step S32). Specifically, the identification information acquisition unit 32 obtains identification information from the log data, the path information acquisition unit 33 obtains path information from the log data, and the motion information acquisition unit 34 obtains motion information from the log data. By obtaining the path information through the path information acquisition unit 33, the path of the terrain determination object determined by the terrain determination system S is determined.

[0091] The weather information acquisition unit 35 acquires weather information from the meteorological information acquisition unit 23-3 regarding the movement of the target user in the target path (step S33).

[0092] The terrain determination unit 37 determines whether the historical record information storage unit 190 contains historical record information obtained by other users besides the target user running on the target path (step S34). If the terrain determination unit 37 does not store historical record information of other users who have run on the target path (step S34; no), the process is transferred to step S37.

[0093] Then, the terrain determination unit 37 determines the terrain of the target path as flat land with a flat surface 5 (step S37). On the other hand, if the terrain determination unit 37 has stored historical information of other users who have run on the target path (step S34; yes), the process moves to step S35.

[0094] The terrain determination unit 37 determines whether there is a change in common body movement information between the target user and other users in a specific section within the target path (step S31). If there is no change in common body movement information between the target user and more than a given number of other users in the specific section within the target path (step S35; no), the terrain determination unit 37 moves the process to step S37. Then, the terrain determination unit 37 determines the terrain of the target path as flat land with a flat surface (step S37). On the other hand, if there is a change in common body movement information between the target user and more than a given number of other users in a specific section within the target path (step S35; yes), the terrain determination unit 37 moves the process to step S36.

[0095] The terrain determination unit 37 determines whether the weather information obtained in step S33 is sunny or cloudy (step S36). If the weather information obtained in step S33 is not sunny or cloudy (step S36; no), the terrain determination unit 37 moves the processing to step S37. Then, the terrain determination unit 37 determines the terrain of the target path as flat land with a flat surface 5 (step S37). On the other hand, if the weather information obtained in step S33 is sunny or cloudy, the terrain determination unit 37 moves the processing to step S38.

[0096] Next, the terrain determination unit 37 uses the learning model stored in the machine learning information storage unit 191 to determine the terrain of the target path (step S38). Specifically, the terrain determination unit 37 determines the terrain of the target path by inputting the motion information obtained in step S32 into the learning model stored in the machine learning information storage unit 191.

[0097] The path information correction unit 38 obtains the estimated height of the path of the determined object calculated based on the terrain determined in step S38 (step S39).

[0098] The path information correction unit 38 determines whether there is a difference between the estimated height obtained in step S39 and the height of the path information obtained in step S32 (step S40). If there is a difference between the estimated height and the height of the path information (step S40; yes), the path information correction unit 38 moves the process to step S41. Then, the path information correction unit 38 uses the estimated height obtained in step S39 to correct the path information obtained in step S32 (step S41), and moves the process to step S42. On the other hand, if there is no difference between the estimated height and the height of the path information (step S40; no), the path information correction unit 38 moves the process to step S42 without going through step S41.

[0099] The posture analysis unit 39 analyzes and determines the running posture of the target user based on the terrain determined in step S37 or S38, the body movement information obtained in step S32, and the reference posture determined based on the terrain determined in step S37 or S38, and determines the quality of the running posture (step S42). Specifically, the posture analysis unit 39 first determines the user's posture based on the user's body movement information obtained in step S32, and determines a predetermined reference posture based on the terrain determined in step S37 or S38. Then, the posture analysis unit 39 determines the quality of the user's posture in the movement by comparing the determined user posture with the determined reference posture.

[0100] The output processing unit 40 sends the path information obtained in step S32 or the corrected path information in step S41, the terrain determined in step S37 or step S38, and the analysis result of the running posture analyzed in step S42 to the user terminal 3 (step S43). After the processing in step S43, the processing performed by the management server 1 ends in the terrain determination processing.

[0101] As described above, the management server 1, which is an electronic device, includes a processing unit 30 that acquires the user's motion information when the user is moving, and determines the terrain the user is moving on based on the acquired motion information.

[0102] Furthermore, as the management server 1 for the electronic device, it can obtain the user's body movement information during the user's movement, or it can obtain the user's body movement information when the user has previously performed a movement. In addition, the processing unit 30 can determine the terrain during the user's movement, but it can also determine the terrain that the user has moved through.

[0103] Therefore, since the terrain is determined from motion information reflecting the movement of the user's feet in direct contact with the ground surface 5, the terrain can be determined with higher accuracy. Specifically, it can determine not only the inclination obtained from the height difference in a given interval, but also more detailed terrain including the stepped, uneven, and other shapes of the ground surface 5.

[0104] Furthermore, in this embodiment, the body movement information includes at least one of the following in the user's movement: step frequency, stride length, pelvic tilt, vertical foot movement, foot contact time, acceleration during kicking, and deceleration during foot contact.

[0105] This allows for the determination of detailed terrain information with greater accuracy.

[0106] Furthermore, in this embodiment, the processing unit 30 determines the user's posture based on body motion information, determines a reference posture based on the determined terrain, and determines the user's posture based on the reference posture and the user's posture.

[0107] Therefore, since the reference pose is determined based on the terrain with high accuracy, the quality of the user's pose in the movement corresponding to the terrain can be judged more accurately.

[0108] Furthermore, in this embodiment, the processing unit 30 obtains the motion information of multiple users moving along a certain path as input data, and obtains the terrain of a certain path as a label. Based on a learning model constructed by performing supervised learning by using the input data and the label as teaching data, and the motion information obtained separately from the teaching data, the processing unit 30 determines the terrain that the user is moving on.

[0109] Therefore, by using motion information from multiple users, a learning model that reflects the relationship between motion information and terrain can be constructed, enabling terrain determination with higher accuracy and efficiency.

[0110] Furthermore, in this embodiment, the processing unit 30 selects whether to determine the terrain based on the body movement information of the user in the movement action of the target path and the body movement information of other users in the movement action of the target path, whether they show common behavior in a specific interval within the target path.

[0111] Therefore, misjudgment of terrain can be prevented. For example, even when the target user is moving on the same terrain, their body movements can change due to fatigue. When determining terrain based on the body movement information of the target user accumulated from fatigue, changes in body movement information due to fatigue can lead to misjudgment of terrain. In this embodiment, since the selection of whether to use body movement information to determine terrain is based on whether the target user and other users exhibit common behaviors, misjudgment of terrain caused by changes in body movement due to fatigue of the target user can be prevented.

[0112] Furthermore, in this embodiment, the processing unit 30 selects whether to determine the terrain based on body movement information based on weather information during the user's movement.

[0113] This prevents misjudgments caused by changes in the state of the ground surface 5 due to weather conditions.

[0114] Furthermore, in this embodiment, the processing unit 30 determines the altitude of the location based on the estimated altitude determined by the terrain and the altitude of the location at the same location as the estimated altitude based on the positioning satellite signal received from the outside.

[0115] Therefore, the altitude of the path can be determined using the altitude estimated from the terrain, which is determined together with the altitude based on the positioning satellite signal and the motion information. Thus, for example, even if the path information is obtained based on positioning satellite signals that are subject to errors due to the radio wave environment, the altitude can be corrected to a more accurate level based on the terrain determination result obtained using the motion information of the user who actually moved along the path.

[0116] Furthermore, the present invention is not limited to the embodiments described above. Modifications and improvements within the scope of achieving the objectives of the present invention are included in the present invention.

[0117] In the above-described embodiments, the terrain determination unit 37 selects whether to determine the terrain of the target path based on motion information according to whether the motion information usage conditions are met. However, it may also determine the terrain based on motion information regardless of whether the motion information usage conditions are met.

[0118] Furthermore, in the above-described embodiment, the learning unit 36 ​​constructs a learning model for determining terrain by performing supervised learning using motion information as input data and terrain labels as teaching data. However, weather information may also be included as input data. That is, the processing unit 30 may further acquire weather information from the movement actions of multiple different users along a certain path, and use it as input data. Based on the learning model, the motion information of the user during movement actions on the target path, and the weather information during the user's movement actions on the target path, the processing unit 30 determines the terrain that the user is moving on. Specifically, the input data acquisition unit 361 may also acquire weather information along with motion information, and the learning model construction unit 363 may use the motion information and weather information as input data and terrain labels as teaching data to perform supervised learning, thereby constructing a learning model for determining the terrain of the target path. Then, the terrain determination unit 37 can input the weather information and motion information together into the learning model to determine the terrain of the target path.

[0119] Therefore, the state of the ground surface 5 along the path the user is moving on can be incorporated to determine the terrain. For example, if the ground surface 5 is wet and muddy due to rain or snow, it becomes difficult for the user to move on that surface 5, affecting their physical movement. To address this, since weather information is used in conjunction with physical movement information to build the learning model, conditions such as rain or snow causing the ground surface 5 to become muddy, making it difficult for the user to move, can be incorporated to determine the terrain.

[0120] Furthermore, for example, the learning unit 36 ​​may include information indicating the quality of the user's posture during movement, along with the terrain, as a label. That is, the processing unit 30 may also obtain comparison information between a reference posture corresponding to the terrain setting and the user's posture during movement on a certain path, as a label, and determine the terrain the user is moving on based on the learning model and the user's body motion information during movement on the target path, and determine the comparison information between the reference posture and the user's posture during movement on the target path. Specifically, the label acquisition unit 362 may obtain the comparison information between the reference posture and the user's posture during movement on a path with known terrain, along with the terrain, and the learning model construction unit 363 may use the body motion information as input data, the terrain as a label, and the aforementioned comparison information as teaching data to perform supervised learning, thereby constructing a learning model for judging the quality of the terrain of the target path and the posture of the user moving on that terrain. Then, the posture analysis unit 39 may input the user's body motion information into the learning model to determine the comparison information between the reference posture and the user's posture during movement.

[0121] Therefore, the terrain along the user's path can be determined efficiently, and the quality of the user's posture during movement can be assessed. For example, if the terrain determination unit 37 determines that the terrain is a stepped uphill slope, the quality of the user's running posture on that stepped uphill slope can be determined.

[0122] Alternatively, for example, the learning unit 36 ​​could acquire the user's motion information during movement as input data, obtain the terrain and an evaluation result indicating the quality of the user's posture as labels, and use the combination of the motion information as input data, the terrain as labels, and the posture evaluation result as teaching data for supervised learning, thereby constructing a learning model for judging the quality of the terrain of the target path and the posture of the user moving on that terrain. Then, the posture analysis unit 39 could also be configured to determine the terrain the user is moving on and the evaluation result of the user's posture at that time by inputting the user's motion information into the learning model.

[0123] Furthermore, in the above-described embodiment, the learning model is constructed by the management server 1, which is an electronic device performing terrain determination processing. However, the learning model can also be constructed by a device different from the management server 1 or other electronic devices. Moreover, the management server 1 can use the learning model received from the aforementioned different device to perform terrain determination.

[0124] Furthermore, in the above embodiments, the management server 1 is an electronic device equipped with a processing unit 30 for performing terrain determination processing. However, at least one of the sensor device 2 and the user terminal 3 may also be used as an electronic device equipped with a processing unit 30 for performing terrain determination processing. Alternatively, the management server 1, the sensor device 2, and the user terminal 3 may all be configured to perform terrain determination processing.

[0125] Furthermore, in the above-described embodiment, the terrain determination unit 37 determines the terrain based on motion information and a learning model when the conditions for using motion information are met. However, it may also determine the terrain by formulating a calculation based on the correlation between motion information and terrain without using a learning model. In this case, the terrain determination unit 37 can determine the terrain by formulating a calculation based on the correlation between motion information and terrain regardless of whether the conditions for using motion information are met.

[0126] The aforementioned series of processes can be executed either through hardware or software. In other words, Figure 6 The functional structure described is merely illustrative and not specifically limited. That is, it is sufficient for management server 1 to possess the capability to execute the aforementioned series of processes as a whole; the specific functional blocks used to implement this functionality are not particularly limited. Figure 6 Examples.

[0127] Furthermore, a functional block can be composed of a single hardware unit, a single software unit, or a combination thereof. The functional structure in this embodiment is implemented by a processor that performs computational processing. The processors used in this embodiment include not only processors composed of various processing devices such as single-processor, multi-processor, and multi-core processors, but also structures that combine these various processing devices with processing circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field-Programmable Gate Arrays).

[0128] When a series of processes are executed through software, the program constituting that software is installed from a network, recording medium, or other means onto a computer. The computer can also be a computer with dedicated hardware. Furthermore, a computer can be a computer capable of performing various functions by installing various programs, such as a general-purpose personal computer.

[0129] Recording media containing such programs can be categorized not only as removable media distributed separately from the device body for providing programs to users, but also as recording media provided to users pre-loaded into the device body. Removable media include, for example, disks (including floppy disks), optical discs, or optical disks. Optical discs include, for example, CD-ROMs (Compact Disk-Read Only Memory), DVDs (Digital Versatile Disk), and Blu-ray Discs. Optical disks include MDs (Mini-Disk). Furthermore, recording media provided to users pre-loaded into the device body, for example, are recording media containing programs... Figure 3 ROM12-1, Storage Unit 19-1, Figure 4 ROM12-2, Storage Unit 19-2, Figure 5 It consists of ROM12-3, hard disk included in storage unit 19-3, etc.

[0130] Furthermore, the steps of the program recorded on the recording medium described in this specification certainly include processing performed in a time sequence, but not necessarily in a time sequence, and also include parallel or individual processing.

[0131] The foregoing has described several embodiments of the present invention, but these embodiments are merely illustrative and do not limit the scope of the present invention. The present invention can be implemented in various other ways, and furthermore, various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and their variations are included within the scope and spirit of the invention as described in this specification, and are also included within the scope of the invention as described in the claims and its equivalents.

Claims

1. An electronic device, characterized in that, Include: The processing unit acquires the user's body movement information during the user's movement, and determines the terrain in which the user is moving based on the acquired body movement information. The processing unit performs the following processing: Obtain the body movement information of multiple users during movement along a certain path as input data, and obtain the terrain of the certain path as tags. A learning model is constructed by using the input data and the labels as teaching data for supervised learning. When the processing unit determines that the body movement information of the user in the target path movement action and the body movement information of other users in the target path movement action show common behavior in a specific interval within the target path, the obtained body movement information is input into the learning model to determine the terrain in the user's movement.

2. The electronic device according to claim 1, characterized in that, The body movement information includes at least one of the following in the user's movement: stride frequency, stride length, pelvic tilt, vertical foot movement, foot contact time, acceleration during kicking, and deceleration during foot contact.

3. The electronic device according to claim 1, characterized in that, The processing unit performs the following processing: The user's posture is determined based on the body movement information. Determine the reference pose based on the determined terrain. The user's posture is determined based on the reference posture and the user's posture.

4. The electronic device according to claim 1, characterized in that, The processing unit performs the following processing: Furthermore, weather information from the movement actions of multiple users along a certain path is obtained as input data. Based on the learning model, the user's body movement information during the object path movement action, and the user's weather information during the object path movement action, the terrain that the user is moving on is determined.

5. The electronic device according to claim 1, characterized in that, The processing unit performs the following processing: Further, comparison information is obtained between the baseline posture corresponding to the terrain setting and the user's posture during the movement along a certain path, which is used as the label. Based on the learning model and the user's body motion information during the object path movement action, the terrain that the user is moving on is determined, and the comparison information between the reference pose and the user's pose during the object path movement action is determined.

6. The electronic device according to claim 1, characterized in that, The processing unit performs the following processing: The user is asked to determine whether to base the terrain determination on the user's movement information based on the weather information during their movement.

7. The electronic device according to any one of claims 1 to 6, characterized in that, The processing unit performs the following processing: The altitude of the location is determined based on the estimated altitude based on the determined terrain and the altitude of the location at the same location as the estimated altitude, based on the altitude of the positioning satellite signal received from the outside.

8. A storage medium, which is a computer-readable storage medium, characterized in that, To enable electronic devices to perform the following processing functions: Obtain the user's body movement information during the user's movement actions. Based on the obtained body movement information, the terrain in which the user is moving is determined. Obtain the body movement information of multiple users during movement along a certain path as input data, and obtain the terrain of the certain path as tags. A learning model is constructed by using the input data and the labels as teaching data for supervised learning. If the body movement information of the user in the target path movement action and the body movement information of other users in the target path movement action show common behavior in a specific interval within the target path, the obtained body movement information is input into the learning model to determine the terrain in the user's movement.

9. A terrain determination method, executed by an electronic device, characterized in that, It includes the following processing steps: Obtain the user's body movement information during the user's movement actions. Based on the obtained body movement information, the terrain in which the user is moving is determined. Obtain the body movement information of multiple users during movement along a certain path as input data, and obtain the terrain of the certain path as tags. A learning model is constructed by using the input data and the labels as teaching data for supervised learning. If the body movement information of the user in the target path movement action and the body movement information of other users in the target path movement action show common behavior in a specific interval within the target path, the obtained body movement information is input into the learning model to determine the terrain in the user's movement.

Images