Advice presentation system, information terminal, tool terminal, program, and advice presentation method

Abstract

To improve the quality of advice for improving the skill of a user of a bicycle.SOLUTION: An advice presentation system 1 comprises an information terminal 2 and an instrument terminal 3. The instrument terminal 3 comprises: a first imaging unit 31a which captures a user image including a user; and a second imaging unit 31b which captures an instrument image including a bicycle 4. The information terminal 2 comprises: an image acquisition unit 231 which acquires the user image and the instrument image from the instrument terminal 3; a first state specification unit 232 which specifies the state of the user on the basis of the user image and specifies the state of the bicycle on the basis of the instrument image; a first advice creation unit 233 which creates advice on the basis of one or more advice candidates associated with the state of the user and the state of the bicycle specified by the first state specification unit 232 among the plurality of advice candidates associated with the state of the user and the state of the bicycle; and a first advice presentation unit 234 which presents advice created by the first advice creation unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an advice providing system, an information terminal, a tool terminal, a program, and an advice providing method for providing advice to bicycle users and instructors who instruct bicycle users. [Background technology]

[0002] 2. Description of the Related Art Conventionally, electronic devices that provide advice to a driver of a vehicle have been disclosed. For example, Patent Document 1 proposes a technology that provides advice to a driver of a vehicle according to the driver's driving proficiency. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-228725 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, the driver's proficiency level is calculated using historical information on the driver's movements and attribute information on the driver's physical characteristics, and advice according to the driver's proficiency level is presented. However, when using a bicycle or a skateboard, even if the user's movements are the same, the appropriateness of the user's movements differs depending on the relationship between the state of the bicycle and the user's movements. For this reason, there is a problem that the accuracy of advice may be insufficient if only the user's movements are used.

[0005] The present invention has been made in consideration of these points, and has an object to improve the quality of advice for improving the skills of bicycle users. [Means for solving the problem]

[0006] The advice presentation system of a first aspect of the present invention is an advice presentation system comprising an equipment terminal attached to a bicycle and an information terminal that presents advice to a user of the bicycle to improve their skills in using the bicycle, the equipment terminal having a first imaging unit that captures a user image including the user and a second imaging unit that captures an equipment image including the bicycle, the information terminal having an image acquisition unit that acquires the user image and the equipment image from the equipment terminal, a first state identification unit that identifies the condition of the user based on the user image and identifies the condition of the bicycle based on the equipment image, a first advice creation unit that creates the advice based on one or more advice candidates associated with the user's condition and the condition of the bicycle identified by the first state identification unit out of a plurality of advice candidates associated with the condition of the user and the condition of the bicycle, and a first advice presentation unit that presents the advice created by the first advice creation unit.

[0007] The first state identification unit may identify at least one of the user's body posture and movement as the state of the user based on the user image, and may identify at least one of the inclination, speed, and acceleration of the bicycle as the state of the bicycle based on the equipment image.

[0008] The first state identifying unit may identify a facial expression of the user as the state of the user based on the user image.

[0009] The first condition identifying unit may further identify a condition of the surroundings of the bicycle based on the instrument image, and the first advice creation unit may create the advice based on one or more advice candidates associated with the condition of the user, the condition of the bicycle, and the condition of the surroundings of the bicycle identified by the first condition identifying unit, out of a plurality of advice candidates associated with the condition of the user, the condition of the bicycle, and the condition of the surroundings of the bicycle.

[0010] The first advice creation unit may create the advice based on the one or more advice candidates for straight roads when the condition around the bicycle is a straight road, and may create the advice based on the one or more advice candidates for curved roads when the condition around the bicycle is a curved road.

[0011] The information terminal may further have a third imaging unit that captures an image including the bicycle and the user, the first state identification unit identifying a relative positional relationship between a body position of the user and positions of components that make up the bicycle based on the image including the bicycle and the user, and the first advice creation unit creating the advice based on the one or more advice candidates associated with the relative positional relationship.

[0012] The tool terminal may have a second condition identification unit that identifies a condition of the user based on the user image and identifies a condition of the bicycle based on the tool image, a second advice creation unit that creates the advice based on one or more advice candidates associated with the condition of the user and the condition of the bicycle identified by the second condition identification unit out of a plurality of advice candidates associated with the condition of the user and the condition of the bicycle, and a second advice presentation unit that presents the advice created by the second advice creation unit.

[0013] The second advice creating unit may create the advice different from the advice created by the first advice creating unit.

[0014] The information terminal may store, in a storage unit, images of the user image for a period selected based on the timing at which the first advice creating unit created the advice.

[0015] The bicycle may be a bicycle with detachable pedals, and the first advice creation unit may create the advice encouraging the user to attach the pedals based on the state of the user and the state of the bicycle identified by the first state identification unit based on the user image and the equipment image captured while the bicycle is being used with the pedals not attached.

[0016] An information terminal of a second aspect of the present invention is an information terminal that presents advice to a user of a bicycle to improve their skills in using the bicycle, and has an image acquisition unit that acquires a user image including the user and an equipment image including the bicycle, a first state identification unit that identifies the state of the user based on the user image and identifies the state of the bicycle based on the equipment image, a first advice creation unit that creates the advice based on one or more advice candidates associated with the state of the user and the state of the bicycle identified by the first state identification unit out of a plurality of advice candidates associated with the state of the user and the state of the bicycle, and a first advice presentation unit that presents the advice created by the first advice creation unit.

[0017] An equipment terminal of a third aspect of the present invention is attached to a bicycle and presents advice to a user of the bicycle to improve their skills in using the bicycle, and has a first imaging unit that captures a user image including the user, a second imaging unit that captures an equipment image including the bicycle, a second state identification unit that identifies a condition of the user based on the user image and identifies a condition of the bicycle based on the equipment image, a second advice creation unit that creates the advice based on one or more advice candidates associated with the user's condition and the condition of the bicycle identified by the second state identification unit out of a plurality of advice candidates associated with the condition of the user and the condition of the bicycle, and a second advice presentation unit that presents the advice created by the second advice creation unit.

[0018] A fourth aspect of the program of the present invention is a program for causing a computer to execute the steps of: acquiring a user image including the user and an equipment image including the bicycle; identifying the condition of the user based on the user image and identifying the condition of the bicycle based on the equipment image; creating the advice based on one or more advice candidates associated with the identified condition of the user and the condition of the bicycle from among a plurality of advice candidates associated with the condition of the user and the condition of the bicycle; and presenting the created advice.

[0019] A fifth aspect of the present invention is a program that is attached to a bicycle and causes a computer that presents advice to a user of the bicycle to improve their skills in using the bicycle to execute the following steps: capturing a user image including the user; capturing an equipment image including the bicycle; identifying the condition of the user based on the user image and identifying the condition of the bicycle based on the equipment image; creating the advice based on one or more advice candidates associated with the identified condition of the user and the condition of the bicycle out of a plurality of advice candidates associated with the condition of the user and the condition of the bicycle; and presenting the created advice.

[0020] A sixth aspect of the advice presentation method of the present invention is a computer-implemented method for presenting advice to a user of a bicycle to improve their skills in using the bicycle, the method comprising the steps of capturing a user image including the user, capturing an equipment image including the bicycle, acquiring the user image and the equipment image, identifying the condition of the user based on the user image and identifying the condition of the bicycle based on the equipment image, creating the advice based on one or more advice candidates associated with the identified condition of the user and the condition of the bicycle from among a plurality of advice candidates associated with the condition of the user and the condition of the bicycle, and presenting the created advice.

[0021] The seventh aspect of the present invention is an advice presentation system comprising an equipment terminal attached to a bicycle and an information terminal that presents advice to a user of the bicycle to improve their skills in using the bicycle, the equipment terminal having an imaging unit that captures a user image including the user, and a detection unit that detects the state of the bicycle and outputs state information indicating the detected state, the information terminal having an acquisition unit that acquires the user image and the state information from the equipment terminal, a first state identification unit that identifies the state of the user based on the user image and identifies the state of the bicycle based on the state information, a first advice creation unit that creates the advice based on one or more advice candidates associated with the state of the user and the state of the bicycle identified by the first state identification unit out of a plurality of advice candidates associated with the state of the user and the state of the bicycle, and a first advice presentation unit that presents the advice created by the first advice creation unit. Effect of the Invention

[0022] According to the present invention, it is possible to improve the quality of advice for improving the skills of a bicycle user. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a diagram for explaining an overview of an advice presentation system. [Diagram 2] FIG. 2 is a diagram illustrating a functional configuration of the information terminal. [Diagram 3] FIG. 2 is a diagram illustrating a functional configuration of the instrument terminal. [Figure 4] FIG. 13 is a diagram showing examples of advice candidates. [Diagram 5] 13 is a flowchart showing a process flow for presenting advice on an information terminal. [Figure 6] FIG. 13 is a perspective view showing a modified example of the instrument terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] [Outline of the advice presentation system] Fig. 1 is a diagram for explaining an overview of the advice presentation system 1. The advice presentation system 1 is a system for presenting advice to a user of a bicycle 4 to improve the skill of using the bicycle 4. The advice presentation system 1 has an information terminal 2 and an equipment terminal 3.

[0025] In the advice presentation system 1, the information terminal 2 presents advice to a user of the bicycle to improve the skill of using the bicycle 4, based on images captured by an equipment terminal 3 attached to the bicycle 4. In this way, for example, an instructor who provides instruction to a user of the bicycle 4 can use the advice presented on the information terminal 2 to provide instruction to the user.

[0026] The tool terminal 3 is a terminal capable of capturing images of the state of the user of the bicycle 4 and the state of the bicycle 4, and is, for example, a smartphone or tablet, but may also be a terminal that is worn on the user's body (for example, a terminal in the shape of glasses). The tool terminal 3 can transmit the captured images to the information terminal 2.

[0027] In Fig. 1, the bicycle 4 is a training bicycle mainly for learning how to ride a bicycle, and the pedals can be detached. In Fig. 1, the bicycle 4 is shown as a bicycle with detachable pedals, but the bicycle 4 may be other bicycles to which the tool terminal 3 can be attached, such as a bicycle with fixed pedals, a bicycle without pedals, a skateboard, a horizontal bar, etc.

[0028] [Overview of Information Terminal 2] Fig. 2 is a diagram showing the functional configuration of the information terminal 2. The information terminal 2 is a terminal that presents advice to a user of a bicycle 4 to improve the skill of using the bicycle 4. The information terminal 2 has an imaging unit 21, a communication unit 22, a control unit 23, a storage unit 24, a display unit 25, and an audio output unit 26. The control unit 23 has an image acquisition unit 231, a first state identification unit 232, a first advice creation unit 233, and a first advice presentation unit 234.

[0029] The imaging unit 21 has a lens and an imaging element, and generates a captured image. Specifically, the imaging unit 21 captures an image including the bicycle 4 and the user, and generates the captured image.

[0030] The communication unit 22 is a communication interface for transmitting and receiving data to and from other electronic devices such as computers, and includes, for example, a LAN (Local Area Network) controller. The communication unit 22 is connected to a network such as the Internet or a LAN, or is directly connected to the appliance terminal 3 via wireless communication, and inputs an image received from the appliance terminal 3 via the network to the image acquisition unit 231.

[0031] The control unit 23 includes, for example, a CPU (Central Processing Unit), and executes a program stored in the storage unit 24 to function as an image acquiring unit 231, a first state identifying unit 232, a first advice creating unit 233, and a first advice presenting unit 234. The control unit 23 may include other circuits together with or instead of the CPU.

[0032] The storage unit 24 includes storage media such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 24 stores programs executed by the control unit 23. The storage unit 24 also stores a plurality of advice candidates associated with the state of the user and the state of the bicycle. The storage unit 24 also stores various algorithms for the first state identification unit 232 to identify the state of the user from the user image and for the first state identification unit 232 to identify the state of the bicycle 4 from the equipment image. The storage unit 24 also stores various algorithms for identifying the surrounding state from an image including the bicycle and the user.

[0033] The display unit 25 is a display for displaying images. The display unit 25 displays display data input from the control unit 23. The audio output unit 26 is, for example, a speaker, and outputs audio data input from the control unit 23 as audio.

[0034] The image acquisition unit 231 acquires, via the communication unit 22, a user image including the user and an appliance image including the bicycle 4 captured by the imaging unit of the appliance terminal 3. The image acquisition unit 231 may also acquire an image including the bicycle 4 and the user generated by the imaging unit 21.

[0035] The first condition identifying section 232 identifies the condition of the user based on the user image acquired by the image acquiring section 231, and identifies the condition of the bicycle based on the implement image acquired by the image acquiring section 231.

[0036] Specifically, the first condition identifying unit 232 identifies at least one of the posture and movement of the user's body as the user's condition based on the user image. The first condition identifying unit 232 detects feature points of multiple parts of the user's body from the user image using, for example, an algorithm for detecting the posture and movement of a human body stored in the storage unit 24.

[0037] Then, the first condition identifying unit 232 refers to information that associates the feature points of multiple body parts with the body posture and / or movement stored in the storage unit 24, and identifies the body posture and / or movement associated with the detected feature points of the multiple body parts of the user as the body posture and / or movement of the user. The first condition identifying unit 232 may identify both the body posture and movement, or may identify either one of the body posture and movement.

[0038] Furthermore, the first state identification unit 232 identifies at least one of the inclination, speed, and acceleration of the bicycle 4 as the state of the bicycle 4 based on the equipment image.

[0039] The first condition identification unit 232 detects an image of the handlebars of the bicycle 4 from the equipment image using, for example, an object detection algorithm stored in the memory unit 24, and detects the inclination of the handlebars of the bicycle 4 by comparing the detected image with a reference image in which the handlebars of the bicycle 4 are in a horizontal state. The first condition identification unit 232 then identifies the detected inclination of the handlebars of the bicycle 4 as the state of the bicycle 4. The inclination of the bicycle 4 identified by the first condition identification unit 232 from the equipment image may be the inclination of the handlebars of the bicycle 4, the inclination of the tires of the bicycle 4, or the inclination of the entire body of the bicycle 4.

[0040] Furthermore, for example, if markings have been applied to the tires of the bicycle 4 in advance, the first state identification unit 232 detects images of the markings in an image of the rotating tire using an image processing algorithm stored in the storage unit 24. The first state identification unit 232 then refers to information stored in the storage unit 24 that associates the images of the markings with the speed of the bicycle 4, and identifies the speed associated with the identified image of the markings as the speed of the bicycle 4. The first state identification unit 232 can also identify the acceleration of the bicycle 4 in the same manner as the speed of the bicycle 4.

[0041] It is difficult to detect the marked areas from an image of a tire that is rotating at high speed, such as that of an automobile, but if the tire of the bicycle 4 rotates slowly (for example, if the bicycle 4 is a small bicycle for children), it is possible to use an image processing algorithm to detect the image of the markings in the image of the rotating tire.

[0042] The first state identification unit 232 may also identify the state of the user and the facial expression of the user based on the user image. The first state identification unit 232 identifies the facial expression of the user from the user image, for example, using a face recognition algorithm stored in the storage unit 24. The identified facial expression is an expression related to the use of the bicycle 4, such as relaxed, nervous, laughing, crying, etc. By identifying the facial expression of the user in this way, the state of the user can be appropriately identified.

[0043] Furthermore, the first condition identifying unit 232 may further identify the condition of the surroundings of the bicycle 4 based on the equipment image. For example, the first condition identifying unit 232 detects the condition of the road surface and objects around the bicycle 4 from the equipment image using an object detection algorithm stored in the storage unit 24, and identifies the detected condition of the road surface and the surrounding objects as the condition of the surroundings of the bicycle 4. The condition of the surroundings of the bicycle 4 may be, for example, that the road surface on which the bicycle is traveling is a curved road, a straight road, an uphill slope, a downhill slope, the presence of an obstacle (for example, a pylon) on the road surface, etc.

[0044] The first condition specifying unit 232 may also specify a relative positional relationship between the position of the user's body and the positions of the members constituting the bicycle 4, based on an image including the bicycle 4 and the user captured by the imaging unit 21. The first condition specifying unit 232, for example, obtains a distance between the center of gravity of the detected buttocks of the user's body and the center of gravity of the saddle of the bicycle 4. The first condition specifying unit 232 then determines whether the distance between the centers of gravity is a certain distance or more, and specifies that the distance between the centers of gravity is a certain distance or more or less than the certain distance as a relative positional relationship.

[0045] The first advice creation unit 233 creates advice based on one or more advice candidates associated with the user's condition and the condition of the bicycle 4 identified by the first condition identification unit 232, from among a plurality of advice candidates associated with the user's condition and the condition of the bicycle 4 stored in the memory unit 24.

[0046] For example, when the first condition identification unit 232 identifies "body straight" as the condition of the user and "tires leaning significantly to the right" as the condition of the bicycle 4, the first advice creation unit 233 selects an advice candidate "Your balance is off. Practice kicking the ground so that you can move forward in a straight line" that is associated with the condition of the user and the condition of the bicycle. Then, the first advice creation unit 233 creates image data and / or audio data for the selected advice candidate.

[0047] Furthermore, when the first condition identification unit 232 identifies the user's condition as "the facial expression is tense" and the condition of the bicycle 4 as "the tires are leaning significantly to the right," the first advice creation unit 233 selects an advice candidate "He looks nervous. Speak to him." which is associated with the user's condition and the condition of the bicycle. Then, the first advice creation unit 233 creates image data and / or audio data for the selected advice candidate.

[0048] When the first condition identification unit 232 further identifies the condition in the vicinity of the bicycle 4, the first advice creation unit 233 may create advice based on one or more advice candidates associated with the user's condition, the condition of the bicycle 4, and the condition in the vicinity of the bicycle 4 identified by the first condition identification unit 232, out of a plurality of advice candidates associated with the user's condition, the condition of the bicycle 4, and the condition in the vicinity of the bicycle 4.

[0049] For example, if the first condition identification unit 232 identifies "body straight" as the condition of the user, "tires leaning significantly to the right" as the condition of the bicycle 4, and "there is an obstacle on the road surface" as the surrounding condition, the first advice creation unit 233 selects an advice candidate "Proceed carefully without hitting the obstacle" that is associated with the condition of the user and the condition of the bicycle. Then, the first advice creation unit 233 creates image data and / or audio data of the selected advice candidate.

[0050] By identifying the state of the vicinity of the bicycle 4 in this manner, the advice providing system 1 can appropriately identify the state of the bicycle 4 and create advice.

[0051] Furthermore, when the condition around the bicycle 4 is a straight road, the first advice creating unit 233 creates advice based on one or more advice candidates for a straight road. Furthermore, when the condition around the bicycle 4 is a curved road, the first advice creating unit 233 creates advice based on one or more advice candidates for a curved road.

[0052] First Advice For example, when the first condition identification unit 232 identifies "body leaning to the right" as the condition of the user, "tires leaning to the right" as the condition of the bicycle 4, and "curved road" as the surrounding condition, the first advice creation unit 233 selects an advice candidate corresponding to the curved road, "You're turning clockwise curves well. Next, practice turning counterclockwise." Then, the first advice creation unit 233 creates image data and / or audio data for the selected advice candidate.

[0053] In this way, by creating advice depending on whether the surrounding condition is a straight road or a curved road, the advice providing system 1 can create more appropriate advice for the user to improve their skills.

[0054] In addition, when the first condition identification unit 232 identifies the relative positional relationship between the position of the user's body and the positions of the components that make up the bicycle 4, the first advice creation unit 233 creates advice based on one or more advice candidates associated with the relative positional relationship identified by the first condition identification unit 232.

[0055] For example, when the first condition identification unit 232 identifies the relative positional relationship as a certain distance or more between the center of gravity of the user's body's buttocks and the center of gravity of the saddle of the bicycle 4, the first advice creation unit 233 selects one or more advice candidates "Your buttocks are floating in the air. Instruct the user to ride with a correct posture" associated with the relative positional relationship identified by the first condition identification unit 232. Then, the first advice creation unit 233 creates image data and / or audio data of the selected advice candidates.

[0056] By identifying the relative positional relationship between the position of the user's body and the positions of the components that make up the bicycle 4 in this manner, the advice presentation system 1 can create advice from the state of the user and the bicycle 4 that cannot be identified from a single image captured by the equipment terminal 3.

[0057] In addition, if the bicycle 4 is a bicycle with detachable pedals, the first advice creation unit 233 creates advice encouraging the user to attach the pedals based on the state of the user and the state of the bicycle 4 identified by the first state identification unit 232 based on user images and equipment images captured while the bicycle is being used with the pedals not attached.

[0058] For example, when the first advice creation unit 233 identifies the state of the user and the state of the bicycle 4 that indicate that the bicycle 4 is being ridden correctly when no pedals are attached to the bicycle 4, it creates advice encouraging the user to attach the pedals, "Next, try attaching the pedals."

[0059] Here, the state of the user and the state of the bicycle 4 that indicate that the bicycle 4 is being ridden correctly include a plurality of combinations in which the relationship between the state of the user and the state of the bicycle 4 is an appropriate relationship that indicates that it is possible to start practicing pedaling. Such combinations are, for example, the state of the user "body is straight" and the state of the bicycle 4 "tires are horizontal and the speed is above a certain level", or the state of the user "body is leaning to the right" and the state of the bicycle 4 "tires are leaning to the right", etc.

[0060] In this way, the advice providing system 1 can create advice including the appropriate timing for fastening the pedals based on the user image and the equipment image.

[0061] The first advice presenting unit 234 presents the advice created by the first advice creating unit 233. Specifically, the first advice presenting unit 234 outputs image data, text data, and the like of the advice to the display unit 25 and causes it to be displayed on the display unit 25. In addition, the first advice presenting unit 234 outputs audio data of the advice to the audio output unit 26, and the audio output unit 26 outputs the audio.

[0062] The first condition identification unit 232 and the first advice creation unit 233 may exist on a server connected via the Internet. In this case, data obtained by the appliance terminal 3 is sent to the server via the Internet, condition identification and advice creation are performed on the so-called cloud, and advice or a signal for issuing advice is sent from the server to the information terminal 2, and the advice is presented.

[0063] [Outline of Instrument Terminal 3] Fig. 3 is a diagram showing a functional configuration of the instrument terminal 3. The instrument terminal 3 shown in Fig. 3 is attached to the bicycle 4, and in addition to an imaging function, has a function of presenting advice to a user of the bicycle 4 to improve the skill of using the bicycle 4. The instrument terminal 3 has a first imaging unit 31a, a second imaging unit 31b, a communication unit 32, a control unit 33, a storage unit 34, a display unit 35, and a voice output unit 36. The control unit 33 has a second state identification unit 331, a second advice creation unit 332, and a second advice presentation unit 333.

[0064] The first imaging unit 31a and the second imaging unit 31b each have a lens and an imaging element, and generate an image. Specifically, the first imaging unit 31a captures a user image including a user. The second imaging unit 31b captures an equipment image including a bicycle. The first imaging unit 31a and the second imaging unit 31b may capture the user image and the equipment image simultaneously, or may capture the user image and the equipment image sequentially in a time-division manner. The first imaging unit 31a may capture the user image over a first period (e.g., a period for one revolution around a predetermined area), and the second imaging unit 31b may capture the equipment image over a second period (e.g., a period for one revolution around a predetermined area after the first period).

[0065] The communication unit 32 is a communication interface for transmitting and receiving data to and from other electronic devices such as computers, and includes, for example, a LAN (Local Area Network) controller. The communication unit 32 is connected to a network such as the Internet or a LAN, and outputs a user image and an instrument image to the information terminal 2 via the network.

[0066] The control unit 33 includes, for example, a CPU, and functions as a second state identification unit 331, a second advice creation unit 332, and a second advice presentation unit 333 by executing a program stored in the storage unit 34. The control unit 33 may include other circuits together with or instead of the CPU.

[0067] The storage unit 34 includes storage media such as a ROM, a RAM, and a hard disk. The storage unit 34 stores a program executed by the control unit 33. The storage unit 34 also stores a plurality of advice candidates associated with the state of the user and the state of the bicycle. The storage unit 34 also stores various algorithms for the second state identification unit 331 to identify the state of the user from the user image and for the second state identification unit 331 to identify the state of the bicycle 4 from the equipment image.

[0068] The display unit 35 is a display for displaying images. The display unit 35 displays display data input from the control unit 33. The audio output unit 36 ​​is, for example, a speaker, and outputs audio data input from the control unit 33 as audio.

[0069] The second state identification unit 331, like the first state identification unit 232 of the information terminal 2, identifies the state of the user based on the user image, and identifies the state of the bicycle 4 based on the equipment image. Specifically, the second state identification unit 331 identifies at least one of the posture and movement of the user's body as the state of the user based on the user image. The second state identification unit 331 identifies at least one of the inclination, speed, and acceleration of the bicycle 4 as the state of the bicycle 4 based on the equipment image.

[0070] The second condition specifying unit 331 may also specify the condition of the user and the facial expression of the user based on the user image. The second condition specifying unit 331 may also specify the condition of the surroundings of the bicycle 4 based on the equipment image.

[0071] The second state specifying section 331 may also specify the relative positional relationship between the position of the user's body and the positions of the components that make up the bicycle 4, based on an image captured by the imaging section 21 that includes the bicycle 4 and the user.

[0072] The second advice creation unit 332, like the first advice creation unit 233 of the information terminal 2, creates advice based on one or more advice candidates associated with the user's condition and the condition of the bicycle 4 identified by the second condition identification unit 331, out of a plurality of advice candidates associated with the user's condition and the condition of the bicycle 4.

[0073] Furthermore, the second advice creating unit 332 may create advice different from the advice created by the first advice creating unit 233. For example, the second advice creating unit 332 of the equipment terminal 3 creates voice data for outputting a voice instructing the timing of when the user should pedal, such as "one, two, one, two." The second advice creating unit 332 may create voice data for outputting a voice instructing the timing of when to pedal at a time interval based on the state of the user shown by the user image including the user and the state of the bicycle 4 shown by the equipment image including the bicycle 4.

[0074] Meanwhile, the first advice creating unit 233 of the information terminal 2 creates image data and / or audio data for the instructor to output an image and / or audio showing the user's learning state, such as "You are not pedaling smoothly." Since advice for the user and advice for the instructor can be created in this way, the user can improve his / her skills in using the bicycle 4.

[0075] The second advice presenting section 333 presents the advice created by the second advice creating section 332. Specifically, the second advice presenting section 333 outputs image data, text data, and the like of the advice to the display section 35 and causes it to be displayed on the display section 35. In addition, the second advice presenting section 333 outputs audio data of the advice to the audio output section 36 and causes the audio output section 36 to output audio.

[0076] [Recording of captured images] In addition, the control unit 23 of the information terminal 2 may store in the memory unit 24 images of a period selected based on the timing at which the first advice creation unit 233 created the advice from among the multiple user images captured by the first imaging unit 31a of the instrument terminal 3.

[0077] For example, the control unit 23 edits, from among the multiple user images captured by the first imaging unit 31a, images from a period selected based on the timing at which the first advice creating unit 233 created the advice, in chronological order into an album format, and stores the edited data in the storage unit 24. In this way, when the bicycle user is a child, the advice presentation system 1 can keep the stored images as a growth record of the child.

[0078] [Examples of advice candidates] 4 is a diagram showing examples of advice candidates. As shown in FIG. 4, the advice candidates are associated with the state of the user and the state of the bicycle, and are stored in the storage unit 24 of the information terminal 2 and the storage unit 34 of the tool terminal 3.

[0079] For example, if the first state identification unit 232 of the information terminal 2 identifies the user's state as "body straight" and the state of the bicycle 4 as "tire axle horizontal, speed above a certain level," the first advice creation unit 233 selects an advice candidate "You're moving straight. Next, practice turning a curve." that is associated with the user's state and the state of the bicycle. Then, the first advice creation unit 233 creates image data and / or audio data for the selected advice candidate.

[0080] Also, the tool terminal 3 creates advice from the advice candidates by performing the same process as the information terminal 2. Note that, although an example in which one advice candidate is associated with a pair of the user's state and the state of the bicycle 4 is shown in Fig. 4, a plurality of advice candidates may be associated with a pair of the user's state and the state of the bicycle 4.

[0081] [Advice presentation process flow] 5 is a flowchart showing the flow of a process for presenting advice on the information terminal 2. First, the image acquisition unit 231 acquires a user image and an equipment image from the equipment terminal 3 via the communication unit 22 (S1). The first condition identification unit 232 identifies the condition of the user based on the user image, and identifies the condition of the bicycle 4 based on the equipment image (S2).

[0082] The first advice creating unit 233 creates advice based on one or more advice candidates associated with the user's condition and the condition of the bicycle 4 identified by the first condition identifying unit 232, out of a plurality of advice candidates associated with the user's condition and the condition of the bicycle 4 (S3). The first advice presenting unit 234 presents the advice created by the first advice creating unit 233 (S4).

[0083] [Variation 1] In the above description, the advice presentation system 1 identifies the state of the bicycle 4 using the instrument image captured by the second imaging unit 31b of the instrument terminal 3, but the configuration of the instrument terminal 3 is not limited to this. For example, the instrument terminal 3 may have a first imaging unit 31a that captures a user image including the user, and a detection unit that detects the state of the bicycle 4 and outputs state information indicating the detected state. In addition, the first imaging unit 31a, the second imaging unit 31b, and the detection unit do not have to be physically built into one instrument terminal 3. If the first imaging unit 31a, the second imaging unit 31b, and the detection unit are connected to the instrument terminal 3 by wire or wirelessly, the instrument terminal 3 can be interpreted as having the first imaging unit 31a, the second imaging unit 31b, and the detection unit.

[0084] In this case, the detection unit includes, for example, an acceleration sensor, an inclination sensor or gyro sensor, a distance sensor using GPS, etc. For example, if the detection unit is an acceleration sensor, the detection unit detects the acceleration of the bicycle 4 to detect the inclination of the body of the bicycle 4 as the state of the bicycle 4. Also, if the detection unit is a distance sensor, the detection unit detects the speed, distance, and running stability from the movement trajectory of the bicycle 4 as the state of the bicycle 4. The detection unit may be provided separately from the tool terminal 3, and for example, a wheel rotation speed detection device, an inclination sensor provided on a helmet, etc. can be used as the detection unit by being wirelessly connected to the tool terminal 3.

[0085] Furthermore, the information terminal 2 has an acquisition unit that acquires a user image and status information from the tool terminal 3, and a first status identification unit 232 that identifies the status of the user based on the user image and identifies the status of the bicycle 4 based on the status information.

[0086] The information terminal 2 also has a first advice creation unit 233 that creates advice based on one or more advice candidates associated with the user's condition and the state of the bicycle 4, which are identified by the first state identification unit 232 based on the user image and state information, from among a plurality of advice candidates associated with the user's condition and the state of the bicycle 4, and a first advice presentation unit 234 that presents the advice created by the first advice creation unit 233.

[0087] With this configuration, the advice presentation system 1 can detect the state of the bicycle 4 without using images, and can therefore appropriately identify the state of the bicycle 4 even in an environment where it is difficult to identify the state of the bicycle 4 using an imaging unit. Furthermore, by combining this with image information, it can point out mismatches, such as the body of the bicycle leaning to the left while the driver's eyes are looking to the right.

[0088] More specifically, an inclination sensor and a camera built into information terminal 2 detect and determine whether the user is able to right the vehicle by himself / herself; the size of the stride length when using the balance bike and the difference between the left and right are detected using a camera while the speed is detected using GPS to determine whether it exceeds a threshold; and when riding in a figure eight, the camera, GPS and inclination sensor detect and determine whether there is any difference in the way the user turns on the left and right sides. If the threshold is exceeded, the user is prompted to proceed to the next step, for example, the step of putting on the pedals.

[0089] Furthermore, when the user pedals and rides the bicycle, for example, if the user can pedal 10 times with the feet off the ground, it is determined that the user has been able to ride the bicycle, and the in-camera of the information terminal 2 captures the user's face, etc., and sends a signal to the information terminal 1 to issue an alert. In this way, it is possible to record the facial expression at the decisive moment when the user has been able to ride the bicycle on video, preventing parents from missing this moment. To determine whether the user has been able to ride the bicycle, the out-camera of the information terminal 2 captures the pedals and the user's feet to determine whether the user has been pedaled 10 times in succession, and a tilt sensor as a detection unit determines whether the user is balanced, and when these data exceed a threshold, it is determined that the user has been able to ride the bicycle.

[0090] [Variation 2] Also, in the above explanation, it is assumed that the information terminal 2 is located at a distance from the bicycle 4, but for example, the information terminal 2 may be attached to the bicycle 4 in the same manner as the equipment terminal 3. Specifically, in the advice presentation system 1, the equipment terminal 3 may be attached near the handlebars of the bicycle, and the information terminal 2 may be attached near the saddle of the bicycle.

[0091] With this configuration, in the advice presentation system 1, the information terminal 2 is attached to the bicycle 4, so that the instructor can give advice to the user in the vicinity of the user while receiving advice from the information terminal 2.

[0092] [Variation 3] Furthermore, in the above explanation, it is essential to have the information terminal 2 and the tool terminal 3, but this is not limiting, and the tool terminal 3 alone may be provided. Specifically, in the advice presentation system 1, the tool terminal 3 is simply attached near the handlebars of the bicycle, and advice is created from status information obtained from the detection unit and the imaging unit, and the advice may be provided directly to the user by voice from the audio output unit 36 ​​of the tool terminal 3.

[0093] [Variation 4] Furthermore, in the advice presentation system 1, the tool terminal 3 attached to the bicycle 4 may function as the information terminal 2. In this case, the control unit 33 of the tool terminal 3 functions as the image acquisition unit 231, the first state identification unit 232, the first advice creation unit 233, and the first advice presentation unit 234 of the information terminal 2. With this configuration, the advice presentation system 1 can realize the functions of the information terminal 2 and the tool terminal 3 using the tool terminal 3 alone.

[0094] [Variation 5] In the above description, the case where the appliance terminal 3 has the first imaging unit 31a and the second imaging unit 31b has been exemplified, but the appliance terminal 3 may have an omnidirectional camera capable of capturing an image in a 360-degree range, and the user image and the appliance image may be captured by the omnidirectional camera. In this case, the omnidirectional camera functions as the first imaging unit and the second imaging unit. The user image and the appliance image may be captured by the omnidirectional camera connected to the appliance terminal 3.

[0095] [Variation 6] In addition, when the first imaging unit 31a is a so-called in-camera of a smartphone and captures a user image including the user, and the second imaging unit 31b is a so-called out-camera of a smartphone and captures an equipment image including the bicycle, a wide-angle lens, a prism, a mirror, or other attachments may be attached to the equipment terminal 3, which is a smartphone, so that the user and the bicycle can be captured at an appropriate angle of view. In FIG. 6, a reflecting member 31aP having a built-in mirror shown in dashed lines is attached to the in-camera 31a as the first imaging unit of the smartphone 3 as the equipment terminal, and a reflecting member 31bP having a built-in mirror shown in dashed lines is attached to the out-camera 31b as the second imaging unit to adjust the angle of view. In this way, when the equipment terminal 3 is attached to the handlebars of the bicycle, it is possible to operate the equipment terminal 3 such that the out-camera 31b mainly captures the body of the bicycle while the in-camera 31a mainly captures the user.

[0096] [Effects of Advice Presentation System 1] As described above, the first condition identifying unit 232 identifies the condition of the user based on the user image, and identifies the condition of the bicycle 4 based on the equipment image. Then, the first advice creating unit 233 creates advice based on one or more advice candidates associated with the condition of the user and the condition of the bicycle identified by the first condition identifying unit 232, out of a plurality of advice candidates associated with the condition of the user and the condition of the bicycle.

[0097] Then, the first advice presenting unit 234 presents the advice created by the first advice creating unit 233. With the information terminal 2 having such a configuration, the information terminal 2 can present advice based on the state of the user and the state of the bicycle 4, thereby improving the quality of the advice for improving the skills of a user of the bicycle 4. In particular, even if the instructor who provides instruction on the bicycle 4 to the user has little instruction experience, the algorithm presented to the information terminal 2 can be used to provide appropriate advice to the user.

[0098] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0099] 1. Advice Presentation System 2. Information terminal 3. Instrument terminal 4. Bicycles 21 Imaging unit 22 Communications Department 23 Control Unit 24 Memory section 25 Display section 26 Audio output section 31a First imaging unit 31b Second imaging unit 32 Communications Department 33 Control Unit 34 Storage section 35 Display section 36 Audio output section 231 Image Acquisition Unit 232 First state identification unit 233 First Advice Creation Department 234 First Advice Presentation Section 331 Second State Identification Unit 332 Second Advice Creation Department 333 Second Advice Presentation Section

Claims

1. An advice presentation system comprising: an equipment terminal attached to a bicycle; and an information terminal that presents advice to a user of the bicycle to improve a skill in using the bicycle, The instrument terminal includes: A first imaging unit that captures a user image including the user; a second imaging unit that captures an equipment image including the bicycle, or a detection unit that detects a state of the bicycle and outputs state information indicating the detected state, The information terminal includes: an image acquisition unit that acquires the user image and the appliance image or the status information from the appliance terminal; a first status identification unit that identifies a status of the user based on the user image and identifies a status of the bicycle based on the instrument image or the status information; a first advice creation unit that creates the advice based on one or more advice candidates associated with the user's condition and the bicycle condition identified by the first condition identification unit, out of a plurality of advice candidates associated with the user's condition and the bicycle condition; a first advice presentation unit that presents the advice created by the first advice creation unit, The first state identification unit identifies a facial expression of the user as the state of the user based on the user image. Advice presentation system.

2. The information terminal further includes a third imaging unit that captures an image including the bicycle and the user, the first state specifying unit specifies a relative positional relationship between a body position of the user and positions of components constituting the bicycle based on an image including the bicycle and the user; the first advice creation unit creates the advice based on the one or more advice candidates associated with the relative positional relationship; The advice presentation system according to claim 1 .

3. The instrument terminal includes: a second status identification unit that identifies a status of the user based on the user image and identifies a status of the bicycle based on the equipment image or the status information; a second advice creation unit that creates advice for the user based on one or more advice candidates associated with the user's condition and the bicycle condition identified by the second condition identification unit, out of a plurality of advice candidates associated with the user's condition and the bicycle condition; a second advice presentation unit that presents the advice created by the second advice creation unit; having the second advice creation unit creates the advice different from the advice created by the first advice creation unit. The advice presentation system according to claim 1 .

4. 4. The advice presentation system according to claim 3, wherein the advice for the instructor created by the first advice creation unit includes linguistic information, and the advice for the user created by the second advice creation unit does not include linguistic information and is composed of any one of sound, light, and vibration, or a combination of these.

5. the information terminal causes an image of the user for a period selected based on a timing at which the first advice creating unit created the advice to be stored in a storage unit. The advice presentation system according to any one of claims 1 to 4.

6. The bicycle is a bicycle with detachable pedals, intended for practicing handlebar operation with the pedals removed and then practicing pedal operation with the pedals attached, the first advice creation unit creates the advice instructing the timing to attach the pedals based on a state of the user and a state of the bicycle identified by the first state identification unit based on the user image and the equipment image captured while the bicycle is being used with the pedals not attached or on the state information. The advice presentation system according to any one of claims 1 to 5.

7. 5. The advice providing system according to claim 4, wherein the first advice creating unit indicates, as the advice for the instructor, a position of the instructor relative to the bicycle.

8. The advice providing system according to claim 1 , wherein the first state identifying unit identifies, as the state of the user, a movement of the user's feet when the user pedals the bicycle, based on the user image.

9. A tool terminal that is attached to a bicycle and enables a user of the bicycle to improve a skill of using the bicycle, A first imaging unit that captures a user image including the user; A second imaging unit that captures an equipment image including the bicycle or a detection unit that detects a state of the bicycle and outputs state information indicating the detected state; a second state identification unit that identifies a state of the bicycle based on the instrument image or the state information, When the tool image captured by the second imaging unit or the status information output by the detection unit satisfies a certain condition, the second status identification unit determines that the user has been able to ride the bicycle, The first imaging unit captures an image of a user.

10. The device is attached to a bicycle and is used by a user of the bicycle to improve the user's skill in using the bicycle; A first imaging unit that captures a user image including the user; A second imaging unit that captures an equipment image including the bicycle or a detection unit that detects a state of the bicycle and outputs state information indicating the detected state; a second state identification unit that identifies a state of the bicycle based on the instrument image or the state information, When the tool image captured by the second imaging unit or the status information output by the detection unit satisfies a certain condition, the second status identification unit determines that the user has been able to ride the bicycle, a first imaging unit that images a user; and an information terminal used at a location away from the bicycle; when the second state identification unit determines that the user is able to ride the bicycle, a signal is sent from the tool terminal to the information terminal, and an alert is issued at the information terminal.

Images