Information processing apparatus, and information processing program

Abstract

To provide an information processor and an information processing program which can provide a proper support to a white-cane user when the user is training for moving.SOLUTION: The information processor includes: an acquisition unit for acquiring the position of a moving route on which a white-cane user is moving and the position of the user on the moving route; and a notification unit for estimating the proficiency level of the white-cane user from moving information in which the acquired moving route is set and in which at least one of the displacement amount and the displacement angle from the moving route using the position of the user is set, and issuing a first notification to the user according to the proficiency level.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing program for supporting movement training using a white cane.

Background Art

[0002] Patent Document 1 discloses a server that specifies content available to a user according to the proficiency level represented based on acquired user information among pre-stored content, and transmits the content selected from the specified content to the user.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, an information processing apparatus for supporting movement training using a white cane has been proposed for users with visual impairments. The information processing apparatus, for example, determines the proficiency level from the acquired user information, specifies available content according to the proficiency level, and supports movement training by notifying the content selected from the specified content.

[0005] By the way, for example, when a notification such as "Please move to the right" is given to a user to guide the user on a predetermined route, the response to the notification may differ depending on whether the user moves to the predetermined route without difficulty, moves to the predetermined route with a large detour, or the user's response to the notification. In addition, by repeating the training, the user may improve the operation of the white cane and be able to move to the predetermined route without difficulty. That is, the user's response to the notification may differ depending on the user and the degree of training.

[0006] However, in the prior art, it only provides notifications corresponding to users, and it is not always possible to estimate the proficiency level corresponding to each user and the degree of training of the white cane for users using the white cane, and provide support suitable for the users during movement training.

[0007] An object of the present invention is to provide an information processing apparatus and an information processing program that can provide support suitable for a user using a white cane during movement training.

Means for Solving the Problems

[0008] The information processing apparatus according to claim 1 includes an acquisition unit that acquires a movement path along which a user using a white cane is moving and the position of the user on the movement path, and uses the acquired movement path and the position of the user to determine a deviation amount with respect to the movement path and at least one of a deviation angle. An estimation unit that estimates the proficiency level of the user's white cane from the set movement information, and a notification unit that notifies the user of a first notification according to the proficiency level.

[0009] The information processing apparatus according to claim 1 acquires a movement path along which a user is moving and the position of the user on the movement path, and based on the acquired movement path and the position of the user, uses movement information including at least one of a deviation amount with respect to the movement path and a deviation angle to estimate the proficiency level of the user's white cane and perform a notification according to the proficiency level. Thereby, support suitable for the user can be provided to the user using the white cane during movement training.

[0010] The information processing apparatus according to claim 2 is the information processing apparatus according to claim 1, wherein the acquisition unit further acquires user information regarding the user, and the estimation unit estimates the proficiency level using the acquired user information and the movement information.

[0011] According to the information processing apparatus according to claim 2, it is possible to perform a notification corresponding to the user in more detail in consideration of the information of each user.

[0012] The information processing apparatus according to claim 3 is the information processing apparatus according to claim 1 or claim 2, wherein the acquisition unit further acquires an operation status of operating the white cane, and the notification unit notifies a second notification according to the proficiency and the operation status to an external device.

[0013] According to the information processing apparatus according to claim 3, a supporter can grasp the operation status of the white cane by the user, and can support the training of the white cane according to each user.

[0014] The information processing apparatus according to claim 4 is the information processing apparatus according to any one of claims 1 to 3, wherein the estimation unit includes a learned model obtained by learning, based on statistics, a proficiency with respect to movement information acquired in the past, and the learned model outputs the proficiency estimated based on the movement information newly acquired by the acquisition unit.

[0015] According to the information processing apparatus according to claim 4, the proficiency can be estimated based on the past behavior of the user.

[0016] The information processing program according to claim 5 causes a computer to execute a process of acquiring a movement route along which a user using a white cane moves and a position of the user on the movement route, estimating a proficiency of the user's white cane from movement information in which at least one of a deviation amount and a deviation angle with respect to the movement route is set using the acquired movement route and the position of the user, and notifying the user of a first notification according to the proficiency.

[0017] A computer on which the information processing program according to claim 5 is executed acquires a movement route along which a user moves and a position of the user on the movement route, estimates a proficiency of the user's white cane using movement information including at least one of a deviation amount and a deviation angle with respect to the movement route based on the acquired movement route and the position of the user, and performs a notification according to the proficiency. According to the information processing program, it is possible to provide support suitable for the user during movement training to the user using the white cane.

Advantages of the Invention

[0018] According to the present invention, it is possible to provide support suitable for a user during movement training to a user using a white cane.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0020] Hereinafter, with reference to the drawings, a mode for carrying out the present disclosure will be described in detail. FIG. 1 is a schematic diagram showing an example of the configuration of an information processing system 1 according to this embodiment.

[0021] As shown in FIG. 1 as an example, the information processing system 1 includes an information processing apparatus 10, a supporter terminal 100, and a server 200. The information processing apparatus 10, the supporter terminal 100, and the server 200 are connected to each other via a network N.

[0022] The information processing device 10 is a device that supports training for a visually impaired user to move safely using a white cane. The information processing device 10 is mounted on the white cane held by the user, and notifies the user according to the user's current location and the movement route such as a sidewalk, and supports the training of moving using the white cane.

[0023] In addition, the information processing device 10 collects information related to the user's movement (hereinafter referred to as "movement information") and information related to the user (hereinafter referred to as "user information"), and transmits them to the supporter terminal 100 and the server 200. Further, the information processing device 10 transmits a notification to the supporter terminal 100 according to the acquired movement information.

[0024] The supporter terminal 100 is a terminal such as a personal computer, a tablet, and a mobile terminal owned by a supporter who supports the user's movement training. The supporter terminal 100 receives the movement information and the user information from the information processing device 10 and displays them on a monitor (not shown). Further, the supporter terminal 100 notifies the supporter according to the notification received from the information processing device 10.

[0025] The server 200 is a terminal such as a personal computer. The server 200 receives and stores the movement information and the user information from the information processing device 10. Further, the server 200 generates a learned model for performing machine learning to estimate the degree of proficiency (hereinafter referred to as "proficiency") of the white cane for each user using the stored past movement information and user information. The server 200 transmits the generated learned model to the information processing device 10. The server 200 uses the past movement information and user information as learning data, and executes machine learning using the proficiency corresponding to the past movement information and user information as teacher data to generate a learned model. The learned model estimates the proficiency from the movement information and the user information using statistics such as simple regression and multiple regression, for example. Note that the learned model according to the present embodiment will be described in a form that estimates the proficiency using regression. However, it is not limited to this. The learned model may estimate the proficiency using classification or may estimate the proficiency using clustering.

[0026] The information processing apparatus 10 receives a learned model from the server 200, estimates the proficiency level for each user using the learned model, and issues a notification for assisting movement according to the proficiency level.

[0027] Next, with reference to FIG. 2, the hardware configuration of the information processing apparatus 10 will be described.

[0028] As shown in FIG. 2 as an example, the information processing apparatus 10 according to the present embodiment includes a CPU (Central Processing Unit) 10A, a ROM (Read Only Memory) 10B, a RAM (Random Access Memory) 10C, a storage 10D, an input / output interface (input / output I / F) 10E, and a communication interface (communication I / F) 10F. Each of the CPU 10A, the ROM 10B, the RAM 10C, the storage 10D, the input / output I / F 10E, and the communication I / F 10F is interconnected by a bus 10G. Further, the information processing apparatus 10 is connected to a GPS (Global Positioning System) device 12A, a camera 12B, a sensor 12C, and a speaker 12D via the input / output I / F 10E.

[0029] The CPU 10A overall controls the information processing apparatus 10. The ROM 10B stores various programs and data. The RAM 10C is a memory used as a work area during the execution of various programs. The CPU 10A estimates the proficiency level for each user and performs a process of issuing a notification according to the proficiency level by expanding and executing the program stored in the ROM 10B in the RAM 10C.

[0030] The storage 10D is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory. The storage stores map information, user information, a learned model, and various programs.

[0031] The input / output I / F 10E is an interface for connecting to devices. The communication I / F 10F is a communication module for communicating with the supporter terminal 100, the server 200, etc. For this wireless communication module, communication standards such as 5G, LTE, Wi-Fi (registered trademark), etc. are used.

[0032] The GPS device 12A is a device that receives position information representing the current location of the information processing device 10 measured by a plurality of GPS satellites. The GPS device 12A includes an antenna (not shown) for receiving position information from GPS satellites. In this embodiment, a form of acquiring position information using the GPS device 12A will be described. However, it is not limited to this. Position information may be acquired using a GNSS (Global Navigation Satellite System) device, or the position of the user may be measured using an image taken by a camera, Wi-Fi, Bluetooth (registered trademark), etc.

[0033] The camera 12B is a device that images the user's surrounding situation, including the tip of the white cane held by the user. The sensor 12C is an acceleration sensor and a gyro sensor that detect vibrations in the information processing device 10, the acceleration, and the angular velocity of the white cane on which the information processing device 10 is mounted. The speaker 12D outputs sound related to notifications to the user. In this embodiment, the image data captured by the camera 12B and the detection data detected by the sensor 12C will be described as information indicating the user's situation (hereinafter referred to as "situation information").

[0034] Next, with reference to FIG. 3, the functional configuration of the information processing device 10 will be described. FIG. 3 is a block diagram showing an example of the functional configuration of the information processing device 10 according to this embodiment.

[0035] 3, the information processing device 10 includes an acquisition unit 21, a setting unit 22, an estimation unit 23, a determination unit 24, a storage unit 25, a notification unit 26, and a transmission unit 27. The CPU 10A executes an information processing program to function as the acquisition unit 21, the setting unit 22, the estimation unit 23, the determination unit 24, the storage unit 25, the notification unit 26, and the transmission unit 27.

[0036] The acquisition unit 21 acquires location information of the information processing device 10 and map information corresponding to the location information using the GPS device 12A. The acquisition unit 21 also acquires situation information of the user using the camera 12B and the sensor 12C.

[0037] The setting unit 22 uses the position information and the map information to derive the distance by which the user deviates from the travel route and the angle by which the user deviates from the travel route, and sets these as travel information. As an example, as shown in FIG. 4, the setting unit 22 derives a travel route along a sidewalk using the map information, and calculates the distance from the user's current location to the travel route (hereinafter referred to as "deviation amount") d r The setting unit 22 derives the deviation amount d r If the deviation amount d is greater than the predetermined distance, the setting unit 22 sets a destination point on the travel route and derives the angle θ between the user's traveling direction at the current location and the direction toward the destination point (hereinafter referred to as the "deviation angle"). r , and the deviation angle θ are set as movement information. Here, the user is proceeding along the movement route (deviation amount d r is equal to or less than a predetermined distance), the deviation amount may be set to 0. The traveling direction of the user may be derived from the difference in position information, may be derived from an image captured by the camera 12B, or may be derived using the acceleration and angular velocity obtained by the sensor 12C.

[0038] The estimation unit 23 estimates the proficiency of the user's white cane using the derived movement information and the user information received in advance. Specifically, the estimation unit 23 uses a trained model that has performed machine learning for estimating proficiency, takes the movement information and the user information as inputs, and outputs the proficiency of the user. The proficiency is represented by the following mathematical formula.

[0039]

Equation

[0040] Here, Score is the score indicating proficiency, a is the user's personality, b is the user's gender, c is the user's height, d is the user's weight, e is the degree of the user's disability, f is the user's place of residence, g is the user's moving location, h is the user's frequency of going out, i is the user's living environment, j is the user's risk literacy, k is the weather, l is the time zone of movement, and z is other information related to the user. Also, A is a correction coefficient, which is set by the judgment of the supporter. The proficiency may be estimated and updated every predetermined period while walking, or may be estimated and updated during a period specified by the user such as the training mode.

[0041] The determination unit 24 determines the situation of the user using the position information, the map information, the movement information, and the situation information. Specifically, the determination unit 24 determines whether the user can move safely by operating the white cane appropriately using the situation information. For example, the determination unit 24 detects and determines a situation approaching a corner, a situation deviating from the movement path, a situation of falling, and a situation where the user cannot operate the white cane appropriately (for example, the tip of the white cane cannot contact the wall).

[0042] As an example, as shown in FIG. 5, the determination unit 24 generates a graph showing the user's walking trajectory for each time series using the position information, map information, movement information, and situation information, and determines whether the user has significantly deviated from the moving route. The determination unit 24 determines whether the walking trajectory has deviated from a predetermined range (amount of deviation d r becomes larger than a predetermined distance), it is determined that the user is not moving safely. Here, the predetermined range corresponds to the dashed line shown in FIG. 5 as an example.

[0043] 6, the determination unit 24 uses the situation information to generate a graph showing acceleration over time, and determines whether the tip of the white cane is in contact with a wall using image data corresponding to the time when the acceleration peaks. If the tip of the white cane is not in contact with the wall at the time when the acceleration peaks, the determination unit 24 determines that the user is not operating the white cane appropriately.

[0044] The storage unit 25 stores, in chronological order, the position information, map information, movement information, user information, proficiency level, situation information, and determination results.

[0045] The notification unit 26 notifies the user according to the proficiency level, the movement information, and the determination result. For example, if the deviation amount exceeds 0, the notification unit 26 notifies the user that the user has deviated from the movement route and notifies the user of the direction in which the user should move (to return to the movement route). If the notification unit 26 determines as a determination result that the user is approaching a corner or the like, the notification unit 26 notifies the user of the direction in which the user should move.

[0046] Furthermore, the notification unit 26 notifies the supporter terminal 100 of the user's situation. If the notification unit 26 determines that the user is not operating the white cane appropriately or is not moving safely, it notifies the supporter terminal 100 of a warning.

[0047] Here, the notification unit 26 controls the notification so that the number of notifications is reduced as the proficiency level is higher. In other words, the notification unit 26 controls the number of notifications so that the number of notifications is increased as the proficiency level is lower. In the present embodiment, the form of controlling the number of notifications according to the proficiency level has been described. However, this is not limiting. The length of the notification period may also be controlled according to the proficiency level. For example, the notification period may be controlled so that the higher the proficiency level is, the longer the notification period.

[0048] The transmitter 27 transmits the stored position information, map information, movement information, user information, proficiency level, situation information, and judgment results to the supporter terminal 100. Here, the supporter terminal 100 presents the position information, map information, movement information, user information, proficiency level, situation information, and judgment results, allowing the user's supporter to check the user's white cane operation status. Here, the supporter terminal may present the position information, map information, movement information, user information, proficiency level, situation information, and judgment results in a display, or may present an audio notification or warning.

[0049] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of information processing according to this embodiment. The information processing shown in Fig. 7 is executed by the CPU 10A reading and executing an information processing program from the ROM 10B or the storage 10D. The information processing shown in Fig. 7 is executed, for example, when an instruction to execute information processing is input.

[0050] In step S101, CPU 10A acquires position information and situation information.

[0051] In step S102, CPU 10A obtains map information corresponding to the location information.

[0052] In step S103, the CPU 10A derives a travel route using the position information and map information, and calculates the deviation amount d as travel information. r , and the deviation angle θ are derived and set.

[0053] In step S104, CPU 10A determines whether or not to estimate the user's proficiency. If the user's proficiency is to be estimated (step S104: YES), CPU 10A proceeds to step S105. On the other hand, if the user's proficiency is not to be estimated (step S104: NO), CPU 10A proceeds to step S108.

[0054] In step S105, CPU 10A acquires user information that has been accepted in advance.

[0055] In step S106, CPU 10A estimates the proficiency level using the user information and movement information.

[0056] In step S107, CPU 10A stores the estimated skill level.

[0057] In step S108, CPU 10A acquires the stored skill level and sets it as the user's skill level.

[0058] In step S109, CPU 10A determines the situation of the user using the position information, map information, movement information, and situation information.

[0059] In step S110, CPU 10A notifies the user of the determination result and the level of proficiency.

[0060] In step S111, CPU 10A notifies supporter terminal 100 of the user's situation according to the determination result and the proficiency level.

[0061] In step S112, CPU 10A stores each piece of information such as position information, map information, movement information, user information, skill level, situation information, and determination result.

[0062] In step S113, CPU 10A transmits each piece of information such as position information, map information, movement information, user information, skill level, situation information, and determination result.

[0063] In step S114, the CPU 10A determines whether to end the process. If it is determined to end the process (step S114: YES), the CPU 10A ends the process. On the other hand, if it is determined not to end the process (step S114: NO), the CPU 10A proceeds to step S115.

[0064] In step S115, the CPU 10A acquires the position information and the situation information and proceeds to step S103.

[0065] As described above, according to the present embodiment, it is possible to provide support suitable for the user during the movement training for the user using the white cane.

[0066] In the above embodiment, the form in which the information processing device 10 is mounted on the white cane held by the user has been described. However, the present invention is not limited to this. The information processing device 10 may be a wearable device worn by the user.

[0067] Further, in the above embodiment, the form in which the determination unit 24 determines that the user has deviated from the movement path and that the user cannot operate the white cane appropriately has been described. However, the present invention is not limited to this. The determination unit 24 may use the photographed image data to determine the presence or absence of a step, a depression, or other locations that the user should pay attention to. For example, the determination unit 24 may be a learned model that has learned locations that the user should pay attention to from the image data photographed in the past, detect locations that the user should pay attention to from the image data photographed by the camera 12B, and determine the presence or absence of locations that the user pays attention to. Further, when the determination unit 24 detects a location that the user should pay attention to, the notification unit 26 may notify the user or may notify the supporter terminal 100.

[0068] In the above embodiment, the mobile training is performed together with a supporter, and the supporter sets the correction coefficient A. However, this is not limiting. The user may perform the mobile training alone. When the user performs the mobile training alone, the correction coefficient A in the above-described formula (1) may be set to 1, or a predetermined value may be set as the correction coefficient.

[0069] In the above embodiment, the travel route is set using map information. However, the present invention is not limited to this. A predetermined location such as a training ground may be set.

[0070] Furthermore, the information processing device 10 according to the above embodiment has been described as transmitting various pieces of information to the supporter terminal 100 and presenting the information immediately. However, this is not limiting. Various pieces of information may be transmitted from the information processing device 10 to the supporter terminal 100 when training is completed. This allows the supporter to understand the tendency of the user's movement using a white cane and then provide advice.

[0071] Furthermore, the subject of the above embodiment has been described as a visually impaired user. However, the subject is not limited to this. It may also be an infant or a child. For example, the information processing device 10 may be worn as a wearable device to train infants and children to walk.

[0072] [remarks] In addition, in the above-described embodiment, various processes executed by the CPU 10A by reading software (program) may be executed by various processors other than the CPU. Examples of the processor in this case include a PLD (Programmable Logic Device) whose circuit configuration can be changed after manufacturing, such as an FPGA (Field-Programmable Gate Array), and a dedicated electric circuit which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit). Further, each of the above-described processes may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). Moreover, the hardware structure of these various processors is, more specifically, an electric circuit formed by combining circuit elements such as semiconductor elements.

[0073] Also, in the above-described embodiment, each program has been described in a mode where it is pre-stored (installed) in a non-transitory computer-readable recording medium. For example, the information processing program in the information processing apparatus 10 is pre-stored in the ROM 10B. However, this is not limiting, and each program may be provided in a form recorded in a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. Further, the program may be in a form downloaded from an external device via a network.

[0074] The flow of the processes described in the above-described embodiment is an example, and unnecessary steps may be deleted, new steps may be added, or the order of the processes may be changed within a range not departing from the gist.

Explanation of Reference Numerals

[0075] 10 Information processing apparatus 21 Acquisition unit 23 Estimation unit 26 Notification unit

Claims

1. An acquisition unit that acquires a movement path along which a user using a white cane is moving and the position of the user on the movement path; An estimation unit that estimates the proficiency of the user's white cane from movement information in which at least one of a deviation amount and a deviation angle with respect to the movement path is set, using the acquired movement path and the position of the user; A notification unit that notifies the user of a first notification according to the proficiency; An information processing apparatus comprising the above.

2. The acquisition unit further acquires user information regarding the user; The estimation unit estimates the proficiency using the acquired user information and the movement information. The information processing apparatus according to Claim 1.

3. The acquisition unit further acquires an operation status of operating the white cane; The notification unit notifies an external device of a second notification according to the proficiency and the operation status. The information processing apparatus according to Claim 1.

4. The estimation unit includes a learned model that has learned, based on statistics, the proficiency with respect to movement information acquired in the past; The learned model outputs the proficiency estimated based on the movement information newly acquired by the acquisition unit. The information processing apparatus according to Claim 1.

5. Acquire a movement path along which a user using a white cane is moving and the position of the user on the movement path; Estimate the proficiency of the user's white cane from movement information in which at least one of a deviation amount and a deviation angle with respect to the movement path is set, using the acquired movement path and the position of the user; Notify the user of a first notification according to the proficiency; An information processing program for causing a computer to execute the above process.

Images