Information processing method, information processing device, program, and information processing system

Abstract

To provide support matched to an object person.SOLUTION: In an information processing method, a computer specifies a voice feature inducing a change in state value on the basis of correspondence information in which a change in state value indicating action states of an object person before and after each transmission timing of a plurality of voices transmitted to the object person during an action, and each feature of the plurality of voices are associated. An information processing device includes a processing part for specifying a voice feature inducing a change in state value on the basis of correspondence information in which a change in state value indicating action states of an object person before and after each transmission timing of a plurality of voices transmitted to the object person during an action, and each feature of the plurality of voices are associated.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to an information processing method, an information processing device, a program, and an information processing system. [Background technology]

[0002] Conventionally, there is a technology that supports the actions of a subject wearing a terminal device by outputting various sounds from the terminal device while the subject is moving. For example, Patent Document 1 discloses a technology that supports the exercise of a subject by outputting sounds such as guidance and encouragement from a terminal device to a subject who is exercising. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-42620 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the characteristics of a voice that are effective for providing support vary from person to person and from subject to subject. For this reason, there is a problem that it is not necessarily easy to provide effective support using a voice with characteristics that suit the subject.

[0005] The present invention aims to provide support suited to each individual. [Means for solving the problem]

[0006] In order to solve the above problems, an information processing method according to the present invention comprises: The computer Based on correspondence information that associates a change in a state value indicating the behavioral state of a subject before and after each of the timings at which a plurality of sounds are sent to the subject during behavior with the characteristics of each of the plurality of sounds, the characteristics of the sounds that induce the change in the state value are identified. Effect of the Invention

[0007] According to the present invention, it is possible to provide support suited to a target person. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an information processing system. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of a server. [Diagram 3] FIG. 2 is a block diagram showing a functional configuration of an instructor terminal. [Figure 4] FIG. 2 is a block diagram showing a functional configuration of an exerciser terminal. [Diagram 5] FIG. 13 is a diagram showing the contents of an analysis setting screen. [Figure 6] FIG. 11 is a diagram showing the contents of analysis setting data. [Figure 7] FIG. 4 is a diagram showing the contents of exercise data. [Figure 8] FIG. 11 is a diagram showing the contents of voice transmission result data. [Figure 9] FIG. 13 is a diagram illustrating a stride change rate. [Figure 10] FIG. 13 is a diagram showing the contents of an analysis result screen. [Figure 11] 13 is a flowchart showing a control procedure for instruction effect analysis processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] (Configuration of Information Processing System) FIG. 1 is a diagram showing a configuration of an information processing system 1. As shown in FIG. The information processing system 1 includes a server 10 (information processing device), an instructor terminal 20 (first terminal device), an exerciser terminal 30 (second terminal device), a sensor device 40 (motion sensor), and earphones 50 (output unit). The instructor terminal 20 and the exerciser terminal 30 are connected to the server 10 via a network N so as to be able to communicate data with each other. The network N is, for example, the Internet, but is not limited to this. Each exerciser terminal 30 is connected to the sensor device 40 and the earphones 50 so as to be able to communicate data with each other by short-range wireless communication such as Bluetooth (registered trademark).

[0011] The user of the information processing system 1 is, for example, an athlete (subject) who performs an exercise (behavior), and an instructor (vocalizer) who gives voice guidance to the athlete. The exercise performed by the athlete may be training such as running, or a competitive event such as a marathon. In addition, the exercise does not necessarily have to involve movement, and may be strength training, etc. The instructor may be an advisor for motion analysis, a friend, etc. The athlete uses an athlete terminal 30, a sensor device 40, and an earphone 50. The instructor uses an instructor terminal 20. In addition, when multiple athletes use the information processing system 1, the information processing system 1 may include multiple athlete terminals 30, multiple sensor devices 40, and multiple earphones 50 corresponding to the multiple athletes. In addition, when multiple instructors use the information processing system 1, the information processing system 1 may include multiple instructor terminals 20 corresponding to the multiple instructors.

[0012] FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG. The server 10 includes a CPU 11 (Central Processing Unit), a RAM 12 (Random Access Memory), a storage unit 13, and a communication unit 14. The components of the server 10 are connected to each other via a communication path such as a bus.

[0013] The CPU 11 is a processor that reads and executes the program 131 stored in the storage unit 13 and performs various arithmetic processing to control the operation of each unit of the server 10. The server 10 may have multiple processors (e.g., multiple CPUs), and the multiple processes executed by the CPU 11 of this embodiment may be executed by the multiple processors. In this case, the multiple processors correspond to processing units. In this case, the multiple processors may be involved in a common process, or the multiple processors may independently execute different processes in parallel.

[0014] The RAM 12 provides a working memory space for the CPU 11 and stores temporary data.

[0015] The storage unit 13 is a non-transitory recording medium readable by the CPU 11 as a computer, and stores the program 131 and various data. The storage unit 13 includes a non-volatile memory such as a hard disk drive (HDD) or a solid state drive (SSD). The program 131 is stored in the storage unit 13 in the form of a computer-readable program code. The storage unit 13 stores analysis setting data 132, motion data 133, voice transmission result data 134 (corresponding information), a machine learning model 135, and the like. Details of these data and the like will be described later.

[0016] The communication unit 14 performs a communication operation in accordance with a predetermined communication standard. The communication unit 14 transmits and receives data to and from the instructor terminal 20 and the exerciser terminal 30 via the network N through this communication operation.

[0017] FIG. 3 is a block diagram showing the functional configuration of the instructor terminal 20. As shown in FIG. The instructor terminal 20 of this embodiment is assumed to be a tablet terminal. However, the present invention is not limited to this, and the instructor terminal 20 may be a smartphone, a notebook PC, or the like. The instructor terminal 20 includes a CPU 21, a RAM 22, a storage unit 23, a display unit 24, an operation unit 25, a microphone 26, a communication unit 27, and the like. Each unit of the instructor terminal 20 is connected via a communication path such as a bus.

[0018] The CPU 21 is a processor that controls the operation of each part of the instructor terminal 20 by reading and executing the program 231 stored in the storage unit 23 and performing various arithmetic processing. The RAM 22 provides a working memory space for the CPU 21 and stores temporary data. The storage unit 23 stores a program 231 and various data. The storage unit 23 includes a non-volatile memory such as an HDD or SSD.

[0019] The display unit 24 displays various information display screens such as an analysis setting screen 60 (see FIG. 5) and an analysis result screen 70 (see FIG. 10) based on a control signal transmitted from the CPU 21. As the display unit 24, for example, a liquid crystal display device can be used.

[0020] The operation unit 25 accepts an input operation by the user (instructor) and outputs an input signal corresponding to the input operation to the CPU 21. The operation unit 25 includes a touch panel provided over the display screen of the display unit 24, and detects contact of the user's finger or the like as an input operation by the touch panel. The operation unit 25 may also include hardware buttons in addition to or instead of the touch panel.

[0021] The microphone 26 converts the input voice into voice data and outputs it to the CPU 21 .

[0022] The communication unit 27 performs a communication operation in accordance with a predetermined communication standard. The communication unit 27 transmits and receives data to and from the server 10 via the network N through this communication operation.

[0023] FIG. 4 is a block diagram showing the functional configuration of exerciser terminal 30. As shown in FIG. The exerciser terminal 30 is a device worn or carried by an exerciser during exercise, and in this embodiment, it is a smart watch worn on the wrist. The exerciser terminal 30 may be a wearable or portable device such as a smartphone or an activity meter. The exerciser terminal 30 includes a CPU 31, a RAM 32, a storage unit 33, a display unit 34, an operation unit 35, a sound output unit 36, a pulse wave sensor 37, a position information acquisition unit 38, a communication unit 39, etc. Each unit of the exerciser terminal 30 is connected via a communication path such as a bus.

[0024] The CPU 31 is a processor that controls the operation of each part of the exerciser terminal 30 by reading and executing the program 331 stored in the storage unit 33 and performing various arithmetic processing. The RAM 32 provides a working memory space for the CPU 31 and stores temporary data. The storage unit 33 stores a program 331 and various data. The storage unit 33 includes a non-volatile memory such as a flash memory.

[0025] The display unit 34 displays various operation screens, information display screens, etc. under the control of the CPU 31. As the display unit 34, for example, a liquid crystal display device that performs display using a dot matrix method can be used, but is not limited to this.

[0026] The operation unit 35 accepts an input operation by the user (a person exercising) and outputs an input signal corresponding to the input operation to the CPU 31. The operation unit 35 includes a touch panel provided over the display screen of the display unit 34, and detects contact of the user's finger or the like as an input operation by the touch panel. The operation unit 35 may also include hardware buttons together with or instead of the touch panel.

[0027] The sound output unit 36 ​​includes a speaker, and outputs voice or notification sound in accordance with a control signal transmitted from the CPU 31 .

[0028] The pulse wave sensor 37 detects the pulse wave of the exerciser. The pulse wave sensor 37 is a biosensor that is indirectly worn by the exerciser by wearing the exerciser terminal 30. The pulse wave sensor 37 includes a light-emitting element that emits green light that is easily absorbed by hemoglobin in the blood from the surface of the exerciser terminal 30 that contacts the wrist when worn, and a light-receiving element that receives the light emitted from the light-emitting element and reflected by the exerciser's skin. Since a part of the light irradiated to the skin is absorbed by the blood in the blood vessels, the amount of light reflected from the skin and received by the light-receiving element changes over time according to the change in blood flow rate associated with the pulsation of the heart. The pulse wave sensor 37 detects the pulse wave based on this change in the amount of received light, and outputs a waveform corresponding to the detected pulse wave to the CPU 31. The CPU 31 derives the heart rate (pulse rate) as bioinformation based on the waveform.

[0029] The position information acquisition unit 38 receives and decodes radio waves transmitted from a positioning satellite of a Global Positioning Satellite System (GNSS) such as a Global Positioning System (GPS) to calculate a current position. The position information acquisition unit 38 calculates the current position under the control of the CPU 31 and outputs the result to the CPU 31. Note that a position information acquisition unit having the same function as the position information acquisition unit 38 may be provided in the sensor device 40.

[0030] The communication unit 39 performs a communication operation according to a predetermined communication standard. Through this communication operation, the communication unit 39 transmits and receives data to and from the server 10 via the network N. The communication unit 39 also transmits and receives data to and from the sensor device 40 and the earphones 50 by short-range wireless communication.

[0031] The sensor device 40 shown in Fig. 1 is a wearable terminal that is attached to the body (for example, the waist) of an athlete. The sensor device 40 has a CPU (not shown), a triaxial acceleration sensor that detects acceleration in three orthogonal axial directions, a triaxial angular velocity sensor that detects angular velocity around three orthogonal axes, and a communication unit. The CPU of the sensor device 40 derives index values ​​of various motion indices (motion index value Ib: see Fig. 7) related to the body motion of the athlete wearing the sensor device 40 based on the detection results of the triaxial acceleration sensor and the triaxial angular velocity sensor, and transmits the derivation results to the athlete's terminal 30 via the communication unit. Alternatively, the sensor device 40 may transmit the detection results of the triaxial acceleration sensor and the triaxial angular velocity sensor to the athlete's terminal 30, and the CPU 31 of the athlete's terminal 30 may derive the motion index value Ib.

[0032] The earphones 50 shown in Fig. 1 are worn on the ears of an athlete during exercise. The earphones 50 include a CPU, a speaker, and a communication unit (not shown). When the CPU of the earphones 50 receives audio data from the exerciser's terminal 30 via the communication unit, it outputs voice or an alarm sound from the speaker based on the audio data.

[0033] (Operation of information processing system) Next, the operation of the information processing system 1 will be described. The information processing system 1 can analyze the instruction voice (voice) sent from the instructor to the athlete and identify the characteristics of the instruction voice that are effective for the instruction of the athlete. The outline of the operation related to the analysis of effective instruction voice is as follows. The CPU 21 of the instructor terminal 20 converts the instruction voice sent from the instructor to the athlete into audio data and sends it to the server 10. The CPU 11 of the server 10 analyzes the features (multiple feature items) of the instruction voice related to the received audio data. Meanwhile, the CPU 31 of the exerciser terminal 30 repeatedly sends to the server 10 status information Ia (see FIG. 7) indicating the exercise state (behavior state) of the athlete while the athlete is exercising. The CPU 11 of the server 10 generates and updates audio transmission result data 134 (see FIGS. 2 and 8) in which a change in a certain exercise index value Ib (status value) (see FIG. 7) of the status information Ia before and after the timing of sending the instruction voice is associated with multiple feature items of the sent instruction voice. Based on this audio transmission result data 134, the CPU 11 identifies the feature items of the instruction voice (hereinafter referred to as "effective feature items") that induce a change in the exercise index value Ib in a desired direction after sending, and their tendency. The CPU 11 of the server 10 transmits the analysis result of the effective instruction voice, including this identification result, to the instructor terminal 20, and causes it to be displayed on the instructor terminal 20. From this analysis result, the instructor can grasp the characteristics of the instruction voice that is effective for the athlete (effective characteristic items and their tendencies), and can reflect this in the next instruction voice.

[0034] When analyzing an effective teaching voice, an analysis setting screen 60 for setting the analysis is displayed on the instructor terminal 20. The analysis setting screen 60 is displayed on the display unit 24 when a predetermined operation is performed on the operation unit 25 by the instructor.

[0035] FIG. 5 is a diagram showing the contents of the analysis setting screen 60. As shown in FIG. The analysis setting screen 60 displays a text box 61, drop-down lists 62 and 63, and an analysis start button 64. The text box 61 is used to specify an exerciser who is the target of training. The exerciser's exerciser ID is input into the text box 61. The exerciser ID is a unique code that is assigned in advance to an exerciser. A drop-down list 62 is used to specify a motion index to be analyzed. In Fig. 5, "stride" is selected as the motion index to be analyzed. The drop-down list 63 is used to specify whether the motion index to be analyzed selected in the drop-down list 62 is a larger-the-better characteristic, in which the larger the better, a smaller-the-better characteristic, in which the smaller the better, or a nominal characteristic, in which the closer the value is to a specific target value, the better. In FIG. 5, "nominal" is selected, which indicates a larger-the-better characteristic. When the nominal characteristic is selected, a text box (not shown) for inputting a target value is further displayed. When the analysis start button 64 is selected, the input contents in the text box 61 and the data of the selection contents in the drop-down lists 62, 63 at that time are transmitted from the instructor terminal 20 to the server 10, and the analysis setting data 132 is generated in the server 10 based on the transmitted data. In the following, in this embodiment, as shown in Fig. 5, an operation will be described by way of example in the case where "stride", which is a larger-than-expected characteristic, is specified as the motion index to be analyzed.

[0036] FIG. 6 is a diagram showing the contents of the analysis setting data 132. As shown in FIG. The analysis setting data 132 registers the specified motion index of the analysis target and its classification of larger-than-expected characteristic, smaller-than-expected characteristic, and best-before characteristic. The analysis setting data 132 also registers a body part (hereinafter also referred to as a "first part") directly related to the motion index of the analysis target. For example, when the motion index is "stride", the first part directly related to it is the "legs". The first part directly related to the motion index is predetermined for each motion index and stored in the storage unit 13.

[0037] In response to selection of the analysis start button 64, analysis of effective instruction audio begins. When the analysis begins, the CPU 11 of the server 10 begins receiving the status information Ia from the exerciser terminal 30. The status information Ia is repeatedly transmitted from the exerciser terminal 30 at a predetermined timing (for example, at predetermined time intervals or each time the exerciser runs a predetermined distance). The CPU 11 of the server 10 registers the received status information Ia in the exercise data 133.

[0038] FIG. 7 is a diagram showing the contents of the exercise data 133. As shown in FIG. Each data row of the exercise data 133 corresponds to state information Ia that indicates the state of the exerciser at a certain point during exercise. The "exerciser ID" in each piece of status information Ia is the exerciser ID of the exerciser currently exercising. "Time" indicates the date and time when the status information Ia was generated. "Location" indicates the location (here, latitude and longitude) of the exerciser's terminal 30 at the time the status information Ia was generated. The "Location" data is acquired by the location information acquisition unit 38 of the exerciser's terminal 30. "Distance" indicates the distance traveled by the exerciser from the start of exercise to the point in time when the status information Ia was recorded. The "distance" data is derived based on the transition of the positioning results obtained by the position information acquisition unit . "Speed" indicates the speed of the athlete at the time when the state information Ia was recorded. Pace may be used instead of speed. “Heart rate” indicates the heart rate of the athlete at the time when the status information Ia was recorded. As described above, the “heart rate” data is derived based on the athlete's pulse waves detected by the pulse wave sensor 37. The "motion index" is a type of index that indicates the motion state of an athlete, and here, "stride," "vertical movement," and "ground contact time" are exemplified. As described above, the value of each motion index (motion index value Ib) is derived based on the detection results of acceleration and the like by the sensor device 40. The motion index is not limited to those exemplified in FIG. 7, and may include, for example, the backward inclination angle of the trunk, sinking of the waist, rotation of the pelvis, left-right impact, push-off time, landing impact, push-off acceleration, deceleration amount, and the like.

[0039] On the other hand, when the analysis start button 64 is selected, the CPU 21 of the instructor terminal 20 transmits a control signal to the microphone 26, and transitions to a state in which input of an instruction voice from the instructor is accepted. After that, when an instruction voice is input to the microphone 26, the instruction voice is converted by the microphone 26 into voice data and output to the CPU 31. The CPU 31 transmits the voice data to the server 10.

[0040] The CPU 11 of the server 10 transmits the received audio data to the exerciser terminal 30, thereby outputting (sending) an instructional audio to the exerciser via the earphone 50 or the sound output unit 36. In more detail, here, the CPU 31 of the exerciser terminal 30 that has received the audio data transmits the audio data to the earphone 50 or outputs it to the sound output unit 36, thereby outputting the instructional audio via the earphone 50 or the sound output unit 36. In addition, when the instructor and the athlete are close to each other, or when the athlete can directly hear the instructor's instruction voice, the transmission of audio data from the server 10 to the instructor terminal 20 and the output operation of the instruction voice by the earphone 50 and the sound output unit 36 ​​may be omitted.

[0041] Furthermore, the CPU 11 of the server 10 analyzes the characteristics of the instruction voice related to the voice data received from the instructor terminal 20 by various known analysis methods, and registers the analysis results in the voice transmission result data 134 .

[0042] FIG. 8 is a diagram showing the contents of the voice transmission result data 134. As shown in FIG. Each data row of the voice transmission result data 134 corresponds to the transmitted voice information Ic corresponding to one instruction voice transmitted to the exerciser. The "voice ID" in each piece of transmitted voice information Ic is a unique code assigned to each instruction voice. The “target athlete ID” is the athlete ID of the athlete to be trained, specified in the text box 61 on the analysis setting screen 60 . "Time" is the time when the instruction voice for that data line was sent to the exerciser (sending timing). The "speech feature items" include a plurality of feature items that represent the features of the analyzed teaching speech. Here, the plurality of feature items include "volume," "emotional strength," "inflection range," and "semantic content." "Volume" is the loudness of the instruction voice, expressed as an integer on five levels ranging from the smallest "-2" to the largest "+2." "Emotional strength" is expressed as a five-level integer ranging from the weakest "-2" to the strongest "+2" that represents the strength of the emotion conveyed in the instructional audio. The "range of intonation change" is expressed as a five-level integer ranging from the smallest "-2" to the largest "+2" that indicates the range of change in intonation (pitch) of the instruction voice.

[0043] "Semantic content" further includes the sub-items "Instructional method," "Positive / Negative," and "Message." The "instruction method" indicates whether the instruction method of the instruction voice is "direct" or "indirect". In the case of "direct", the value is "+2", and in the case of "indirect", the value is "-2". Here, "direct" indicates that the instruction is a direct instruction for a first part that is directly related to the motion index of the instruction target. Also, "indirect" indicates that the instruction is a instruction for a second part that is correlated with the first part and different from the first part. When the motion index of the instruction target is "stride", the first part is the legs as described above, and the second part is, for example, the arms. In this case, the direct instruction is an instruction to directly increase the stride for the legs, such as "raise your legs higher". On the other hand, the indirect instruction is an instruction to indirectly increase the stride by making the user conscious of the movement of the arms, such as "swing your arms more". "Positive / Negative" indicates whether the instructional audio is positive or negative. An example of a positive instructional audio is audio that does not include negative expressions, such as "Raise your legs high." An example of a negative instructional audio is audio that includes negative expressions, such as "Your back is not straight." In the case of "positive," the value is "+2," and in the case of "negative," the value is "-2." The "message" is the instruction voice converted into text data.

[0044] The CPU 11 of the server 10 analyzes the voice data of the instruction voice by various known analysis methods capable of identifying the contents of each of the above-mentioned characteristic items of the voice transmission result data 134. That is, based on the analysis results of the volume and its changes, the pitch and its changes, etc. of the instruction voice, the "volume", "intensity of emotion" and "intonation change range" are identified. Also, words included in the instruction voice are extracted from the voice data, and from the meaning of the words, it is identified whether the instruction target is the first part or the second part, and the contents of the "intonation method" are identified. Also, from the meaning of the words, the contents of "positive / negative" are identified. The method of analyzing the voice data of the instruction voice is not particularly limited, but may include, for example, a process of identifying at least one of the "volume", "intensity of emotion" and "intonation change range" based on the amplitude and time-series changes of the voice waveform in the voice data. Also, the method of analyzing the voice data may include a process of identifying the "intensity of emotion" and / or the "intonation change range" by decomposing the voice waveform in the voice data into frequency components by Fourier transform and analyzing the obtained spectrum. The method may also include a process of inputting voice data to a machine learning model that has been trained to output text data of a teaching voice in response to input of voice data (i.e., to perform voice recognition), thereby acquiring text data of the teaching voice, and identifying the contents of the "teaching method" and / or "positive / negative" based on the acquired text data. The method may also include a process of using a machine learning model that has been trained to directly output the contents of the "teaching method" and / or "positive / negative" from the machine learning model, thereby acquiring the contents of the "teaching method" and / or "positive / negative" directly from the machine learning model. The method may also include inputting voice data to a machine learning model that has been trained to output the features in response to input of voice data, thereby acquiring the contents of the feature items of "volume", "emotional strength", and "inflection range" from the machine learning model. The machine learning model used for analyzing the voice data may be provided in the storage unit 13 of the server 10, or may be provided in an external device that is connected to the server 10 directly or via the network N.

[0045] Each piece of transmitted voice information Ic in the voice transmission result data 134 further includes data on "stride change rate." "Stride change rate" indicates the rate of change in the stride motion index value Ib before and after the timing of transmitting the instruction voice. Although the rate of change is used here, the amount of change may be used instead. The rate of change or amount of change in stride corresponds to a "change in state value."

[0046] FIG. 9 is a diagram illustrating the rate of stride change. The graph shown in FIG. 9 plots the time series change of the stride value in the exercise data 133. In addition, times T1, T2, T3, and T4 are the times when the instruction voices with the voice IDs "M001", "M002", "M003", and "M004" in the voice transmission result data 134 in FIG. 8 were transmitted, respectively. As can be seen from the graph, the stride value measured first after time T1 is reduced by 3% compared to the stride value measured last before time T1. In other words, the stride value is reduced by 3% before and after time T1. Similarly, the stride value is increased by 10% before and after time T2, increased by 4% before and after time T3, and reduced by 2% before and after time T4. This increase rate or decrease rate is registered in the "stride change rate" of the voice transmission result data 134. In this way, the "stride change rate" indicates the degree to which the stride is increased or decreased by the instruction voice of the transmitted voice information Ic. The stride value used to derive the increase rate and the decrease rate may not be the value first recorded after the instruction voice is sent, but may be the value first recorded after a predetermined time has elapsed after the sending time. In other words, the increase rate and the decrease rate may be derived based on the exercise index value Ib before and after a predetermined period including the instruction voice is sent.

[0047] When multiple pieces of transmitted voice information Ic are registered in the voice transmission result data 134, the CPU 11 of the server 10 identifies, according to a predetermined analysis algorithm, at least one feature item among the multiple feature items that induces an increase in stride (a change in a desired direction) and the tendency of that feature item (an effective feature item and its tendency).

[0048] As an analysis algorithm, for example, multiple regression analysis can be used. In the following, the stride change rate before and after a time Tn (n is a natural number from 1 to N, and N is the number of transmitted voice information Ic included in the voice transmission result data 134) is expressed as R Tn When multiple regression analysis is used, the CPU 11 calculates the stride change rate R Tn The predicted value of r Tn When expressed by the following multiple regression equation (1), the coefficient w is determined so that the objective function L expressed by equation (2) is minimized. 0 ~w 5 Derive: r Tn =w 1 x 1 +w 2 x 2 +w 3 x 3 +w 4 x 4 +w 5 x 5 +w 0 x 0 (1) L=(R T1 -r T1 ) 2 +(R T2 -r T2 ) 2 +···+(R Tn -r Tn ) 2 (2) where x 0 ~x 5 are explanatory variables, w 0 ~w 5 is the coefficient of the explanatory variable. Coefficient w 0 ~w 5 is the explanatory variable x 0 ~x 5 This can also be rephrased as the weight of the explanatory variable x 1 ~x 5 corresponds to the five feature items in the transmitted voice information Ic. 1 is the value of "volume" in the transmitted audio information Ic corresponding to time Tn. 2is the value of "emotional intensity" in the transmitted voice information Ic corresponding to time Tn. 3 is the value of the "inflection range" in the transmitted voice information Ic corresponding to time Tn. 4 is the value of the "instruction method" in the transmitted voice information Ic corresponding to the time Tn. 5 is the value of "positive / negative" in the transmitted audio information Ic corresponding to time Tn. 0 is a constant "1", and "w 0 x 0 The term " is the coefficient w 0 Therefore, the coefficient w 0 corresponds to the bias in multiple regression equation (1).

[0049] CPU11 is the derived coefficient w 1 ~w 5 Among them, coefficients that satisfy a predetermined condition related to the magnitude of the absolute value of the coefficient are identified, and explanatory variables corresponding to the identified coefficients are identified as effective feature items. The predetermined condition related to the magnitude of the absolute value can be determined appropriately. For example, the predetermined condition may be that the absolute value of the coefficient is equal to or greater than a predetermined threshold value. In addition, the predetermined condition may be that the coefficient w 1 ~w 5 Alternatively, it may be within a predetermined number of positions when the absolute values ​​of the respective items are arranged in descending order.

[0050] Further, the CPU 11 identifies the tendency of the effective feature item that induces a change in the stride in a desired direction (here, an increase) from the sign of the identified coefficient. Here, the desired direction corresponds to the setting of "Larger / Smaller / Aim" in the analysis setting data 132, where "Larger" is the direction of increase, "Smaller" is the direction of decrease, and "Aim" is the direction toward the target value. When a certain coefficient is positive, it indicates that the stride change rate tends to increase as the explanatory variable (feature item) corresponding to that coefficient increases. Also, the larger the absolute value of the coefficient, the more likely it is that the stride change rate will increase as the explanatory variable increases. Moreover, when a certain coefficient is negative, it indicates that the stride change rate tends to increase in response to a decrease in the explanatory variable (feature item) corresponding to that coefficient. Also, the larger the absolute value of the coefficient, the more likely it is that the stride change rate will increase in response to a decrease in the explanatory variable.

[0051] For example, when the voice transmission result data 134 shown in FIG. 8 is analyzed by multiple regression analysis, the explanatory variable x 1 The coefficient w corresponding to 1 is positive. This is because, when the transmitted voice information Ic with the voice IDs "M002" and "M003" is focused on, there is a correlation that the stride change rate increases when the "volume" value is large. For this reason, the CPU 11 sets the coefficient w 1 If the above-mentioned predetermined condition is satisfied and "volume" is identified as an effective feature item, a tendency is identified regarding the feature item "volume" that "the larger the value, the more likely it is that the stride change rate will increase." In addition, when the above analysis was performed, the explanatory variable x 4 The coefficient w corresponding to 4 becomes negative. This is because, when the transmitted voice information Ic with the voice IDs "M002" and "M003" is focused on, there is a correlation that the stride change rate becomes large when the value of "instruction method" is small (i.e., indirect instruction). For this reason, the CPU 11 sets the coefficient w 4 If the above-mentioned specified conditions are satisfied and the "instruction method" is identified as an effective characteristic item, a tendency is identified that "when the value of the "instruction method" characteristic item is small (i.e., the instruction method is 'indirect'), the stride change rate is likely to increase."

[0052] In this embodiment, each explanatory variable x 1 ~x 5 The range of possible values ​​for (characteristic items) is set to -2 to +2, but the derived coefficient w 1 ~w 5 Each explanatory variable x is chosen so that the absolute value of 1 ~x 5 The range of possible values ​​of may be adjusted in advance. In addition, if an analysis based on the same analysis setting data 132 has been performed in the past, in addition to the transmitted audio information Ic generated during the current analysis, the transmitted audio information Ic used in the past analysis may be added to the audio transmission result data 134 and used for the multiple regression analysis.

[0053] The effective feature items and their tendencies may be specified using a machine learning model 135. The machine learning model 135 is machine-trained to output information related to the effective feature items and their tendencies by inputting the voice transmission result data 134 (multiple pieces of transmitted voice information Ic). The CPU 11 inputs the voice transmission result data 134 including the latest generated transmitted voice information Ic to the machine learning model 135, and can specify the effective feature items and their tendencies based on the output from the machine learning model 135. The algorithm by which the machine learning model 135 specifies the effective feature items and their tendencies may be the one using the multiple regression analysis described above, or may be an algorithm separate from the above. The configuration of the machine learning model 135 is not particularly limited, and may be, for example, one using a neural network or a support vector machine. The machine learning model 135 may be provided in an external device connected to the server 10 via the network N.

[0054] When the effective characteristic items and their tendencies are identified, the CPU 11 transmits data of the analysis result including the identification result to the instructor terminal 20 and causes the display unit 24 of the instructor terminal 20 to display the analysis result screen 70.

[0055] FIG. 10 is a diagram showing the contents of the analysis result screen 70. As shown in FIG. 10 illustrates an example of an analysis result based on the voice transmission result data 134 shown in FIG. 8. Based on the result of the multiple regression analysis described above, the analysis result screen 70 displays advice to the effect that the stride change rate can be effectively increased by increasing the "volume" of the voice and setting the "instruction method" to "indirect." Note that the analysis result screen 70 may also display examples of specific instruction voice words in accordance with the above advice. The timing for displaying the analysis result screen 70 can be determined appropriately. For example, the analysis result screen 70 may be displayed every time the instructor transmits a teaching voice. Alternatively, the analysis result screen 70 may be displayed when the number of registered pieces of transmitted voice information Ic in the voice transmission result data 134 reaches or exceeds the lower limit number at which a valid analysis result is obtained.

[0056] Next, a teaching effect analysis process executed by the CPU 11 of the server 10 to realize the above-mentioned operation will be described. FIG. 11 is a flowchart showing a control procedure for the instruction effect analysis process. The instruction effect analysis process is started, for example, in response to a predetermined operation being performed on the instructor terminal 20 to start analysis of effective instruction voice, and a predetermined notification being sent from the instructor terminal 20 to the server 10.

[0057] When the instruction effect analysis process is started, the CPU 11 of the server 10 repeatedly determines whether or not data of the setting contents on the analysis setting screen 60 has been received from the instructor terminal 20 (step S101). If the CPU 11 determines that the data has been received ("YES" in step S101), it generates the analysis setting data 132 based on the received data (step S102).

[0058] The CPU 11 repeatedly determines whether or not the exerciser has started exercising (step S103). For example, the CPU 11 determines that the exerciser has started when it receives predetermined notification data from the exerciser terminal 30. When it determines that the exerciser has started ("YES" in step S103), the CPU 11 starts acquiring the status information Ia repeatedly transmitted from the exerciser terminal 30 (step S104).

[0059] The CPU 11 repeatedly determines whether or not the voice data of the coaching voice of the instructor is received from the instructor terminal 20 (step S105). When it is determined that the voice data of the coaching voice is received ("YES" in step S105), the CPU 11 analyzes each feature item of the coaching voice according to the setting contents of the analysis setting data 132, and registers it in the voice transmission result data 134 (step S106). For example, when the motion index to be analyzed is set to "stride" in the analysis setting data 132, the CPU 11 determines whether the feature item of the "instruction method" is "direct" or "indirect" based on whether the instruction contents of the coaching voice are targeted at the legs. The CPU 11 also refers to the motion data 133, and specifies the change in the motion index value to be analyzed (in this embodiment, the stride change rate) before and after the transmission time of the coaching voice, and registers it in the voice transmission result data 134 (step S107).

[0060] The CPU 11 executes multiple regression analysis using the values ​​of each feature item as explanatory variables using the method described above, and identifies effective feature items and their tendencies based on the magnitude of the coefficient of each obtained explanatory variable (step S108).

[0061] The CPU 11 determines whether it is a predetermined timing for outputting the analysis result (step S109), and if it is not the output timing ("NO" in step S109), the process returns to step S105. If it is the output timing ("YES" in step S109), the CPU 11 transmits data related to the analysis result to the instructor terminal 20, and causes the display unit 24 of the instructor terminal 20 to display the analysis result screen 70 (step S110).

[0062] The CPU 11 determines whether the exerciser has finished exercising (step S111), and if it determines that the exercise has not finished ("NO" in step S111), returns the process to step S105. If it determines that the exercise has finished ("YES" in step S111), the CPU 11 determines whether an operation to end the analysis of the instruction voice has been performed on the instructor terminal 20 (step S112). If the CPU 11 determines that the operation has not been performed ("NO" in step S112), it returns the process to step S103, and if it determines that the operation has been performed ("YES" in step S112), it ends the instruction effect analysis process.

[0063] (effect) As described above, in the information processing method according to the present embodiment, the CPU 11 of the server 10 identifies the characteristics of the instruction voice that induces a change in the exercise index value Ib, based on the voice transmission result data 134 in which the change in the exercise index value Ib indicating the exercise state of the athlete before and after each transmission timing of the instruction voices transmitted to the athlete during exercise is associated with each characteristic of the instruction voices. This makes it possible to present to the instructor the characteristics of the instruction voice that enable effective instruction (support) to the athlete. As a result, it becomes possible to provide support that is suited to the athlete. For example, it becomes possible for the instructor to provide effective instruction using a voice having characteristics that are suited to the athlete.

[0064] Furthermore, by the CPU 11 identifying the characteristics of the voice that induces a change in the exercise index value Ib in a desired direction, it is possible to present to the instructor the characteristics of the instructional voice that can have the intended effect on the exerciser.

[0065] Moreover, the voice transmission result data 134 includes information on a plurality of characteristic items that express the characteristics of each of the instruction voices, which are identified based on the voice data of each of the instruction voices, and the CPU 11 identifies at least one characteristic item that induces the above-mentioned change and the tendency of the characteristic item from among the plurality of characteristic items, based on the voice transmission result data 134. This makes it possible to present the instructor with specific and appropriate advice, such as which characteristic item of the instruction voice should be in what form to be effective in instructing the exerciser.

[0066] The plurality of characteristic items include at least one of the following: the volume of the instruction voice, the strength of the emotion of the speaker (instructor) of the instruction voice, the range of change in intonation of the instruction voice, and the meaning of the instruction voice. This makes it possible to clearly show the instructor what kind of instruction voice is effective.

[0067] In addition, by using the exercise index value Ib, which is a exercise index that represents the body movement of an athlete performing exercise, as a state value representing the behavioral state, accurate analysis results based on changes in the actual body movement of the athlete in response to the instruction audio can be presented.

[0068] The multiple feature items include an item of the meaning of the instruction voice, which indicates whether the instruction is directed to a first part of the body directly related to the motion index, or a second part different from the first part that is correlated with the movement of the first part, and includes an item of "instruction method." This makes it possible to specify whether an instruction voice directed to a first part (e.g., legs) directly related to the motion index (e.g., stride) to be analyzed is effective, or an instruction voice that indirectly instructs a second part (e.g., arms) different from the first part is effective, and present this to the instructor.

[0069] Furthermore, the CPU 11 may specify effective feature items and their tendencies by inputting the voice transmission result data 134 to a machine learning model 135 that has been trained to output information relating to effective feature items and their tendencies in response to the input of the voice transmission result data 134. This allows the analysis result to be obtained by a simple process of inputting data to the machine learning model 135.

[0070] Furthermore, the information processing method according to the present embodiment identifies the characteristics of the instruction voice that induces the change in the exercise index value Ib based on the voice transmission result data 134 in which the change in the exercise index value Ib, which indicates the exercise state of the athlete before and after each transmission timing of the instruction voices transmitted to the athlete during exercise, is associated with each of the characteristics of the instruction voices. This makes it possible to provide support suited to the athlete.

[0071] Moreover, the program 131 according to the present embodiment causes the CPU 11 to execute a process of identifying the characteristics of the instruction voice that induces a change in the exercise index value Ib, based on the voice transmission result data 134 in which the change in the exercise index value Ib, which indicates the exercise state of the athlete before and after each transmission timing of the instruction voices transmitted to the athlete during activity, is associated with each of the characteristics of the instruction voices. This makes it possible to provide support suited to the athlete.

[0072] The information processing system 1 according to the present embodiment includes an instructor terminal 20 that receives input of a plurality of instructional voices to be sent to an athlete during exercise, an athlete terminal 30 that is worn or carried by the athlete and that obtains an exercise index value Ib that indicates the athlete's exercise state before and after each of the timings of sending the plurality of instructional voices, and a server 10 having a CPU 11 that identifies the characteristics of the instructional voice that induce a change in the exercise index value Ib based on voice sending result data 134 that associates the change in the exercise index value Ib with the characteristics of each of the plurality of instructional voices. This makes it possible to provide support that is suited to the athlete.

[0073] (others) The present invention is not limited to the above-described embodiment, but may be modified in various ways. For example, the processing executed by the CPU 11 of the server 10 in the above embodiment may be executed by the CPU 21 of the instructor terminal 20. In this case, the instructor terminal 20 corresponds to the information processing device, and the CPU 21 corresponds to the processing unit. Also, the server 10 can be omitted.

[0074] In the above embodiment, exercise is exemplified as one aspect of the behavior, and an athlete who exercises is exemplified as one aspect of the target person who is in the behavior, but the present invention is not limited to this. For example, the target person may be a learner who performs learning as an action, and in this case, the characteristics of the instruction voice that improve the state value representing the learning effect of the learner may be specified.

[0075] In addition, the motion indices to be analyzed are not limited to "form indices" that indicate the form of exercise, such as stride, vertical oscillation, and ground contact time, and any item included in the status information Ia (e.g., speed and heart rate) may be used as the motion indices to be analyzed.

[0076] In the above embodiment, an example is described in which a single motion index (stride) is the object of analysis, but the present invention is not limited to this, and multiple motion indexes may be the object of analysis. For example, a single evaluation value (state value) for multiple motion indexes may be derived by substituting the values ​​of each motion index into a predetermined function with multiple motion indexes as variables, and a feature of the voice that induces a change in this evaluation value in a desired direction may be specified. For example, it is preferable that the above function is determined such that the larger the value of the motion index of the larger-the-better characteristic, the larger the value of the function, the smaller the value of the motion index of the smaller-the-better characteristic, and the closer the value of the motion index of the nominal-theory characteristic, the larger the value of the function.

[0077] In addition, in the above embodiment, effective analysis of instructional audio is performed while the athlete is exercising. However, instead, analysis may be performed with desired analysis settings based on previously acquired audio data of instructional audio and exercise data 133 at any time other than during exercise.

[0078] In the above embodiment, the effective feature items and their trends are identified using a multiple regression analysis technique, but the analysis method is not limited to this. For example, multivariate analysis other than multiple regression analysis may be used.

[0079] In the above description, an example has been disclosed in which the HDD or SSD of the storage unit 13 is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. As other computer-readable media, information recording media such as flash memory and CD-ROM can be applied. In addition, a carrier wave is also applied to the present invention as a medium for providing data of the program according to the present invention via a communication line.

[0080] Furthermore, it goes without saying that the detailed configuration and detailed operation of each component of the information processing system 1 in the above embodiment can be modified as appropriate without departing from the spirit of the present invention.

[0081] Although the embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0082] 1. Information Processing Systems 10 Server (information processing device) 11 CPU (processing unit) 132 Analysis setting data 133 Exercise Data 134 Voice transmission result data (support information) 135 Machine Learning Models 20 Instructor terminal (first terminal device) 26. Mike 27 Communications Department 30 Exercise terminal (second terminal device) 40 Sensor device 50 Earphones Ia Status Information Ib Movement index value (state value) Ic Audio transmission information

Claims

1. The computer Identifying a feature of a sound that induces a change in the state value based on correspondence information in which a change in a state value that indicates a behavioral state of the target person before and after each of the transmission timings of a plurality of sounds transmitted to the target person during behavior is associated with each feature of the plurality of sounds. Information processing methods.

2. The change in the state value is a change in a desired direction. The information processing method according to claim 1 .

3. the correspondence information includes information related to a plurality of feature items that represent features of each of the plurality of voices, the information being identified based on the voice data of each of the plurality of voices; identifying at least one feature item that induces the change and a tendency of the feature item from among the plurality of feature items based on the correspondence information; The information processing method according to claim 2 .

4. The plurality of feature items include at least one of a volume of the voice, a strength of an emotion of a speaker of the voice, a range of change in intonation of the voice, and a semantic content of the voice. The information processing method according to claim 3.

5. the behavior is exercise; The state value is a motion index value of a motion index representing a body movement of the subject performing the exercise. The information processing method according to claim 1 .

6. the behavior is exercise; the state value is a motion index value of a motion index representing a body motion of the subject performing the exercise, the voice is an instruction voice for changing the exercise index value of the subject performing the exercise in the desired direction, The plurality of feature items are items of the meaning of the voice, and include an item that indicates whether the instruction is directed to a first body part that is directly related to the movement index, or a second body part that is different from the first body part and is correlated with the movement of the first body part. The information processing method according to claim 3.

7. inputting the correspondence information into a machine learning model that has been trained by machine learning to output information relating to the at least one feature item that induces the change in the state value and a tendency of the feature item, thereby identifying the at least one feature item and a tendency of the feature item; The information processing method according to claim 3 , 4 , or 6 .

8. An information processing device having a processing unit that identifies audio characteristics that induce a change in the state value based on correspondence information that corresponds a change in a state value representing the behavioral state of a subject before and after each of the timings of sending multiple sounds to the subject while the subject is in action, and each of the characteristics of the multiple sounds.

9. On the computer, A process of identifying a feature of a sound that induces a change in the state value based on correspondence information in which a change in a state value that represents a behavioral state of a target person before and after each transmission timing of a plurality of sounds transmitted to the target person during behavior is associated with each feature of the plurality of sounds; A program that executes the following.

10. A first terminal device that receives input of a plurality of voices to be transmitted to a target person who is in motion; a second terminal device that is worn or carried by the subject and that acquires state values ​​that represent behavioral states of the subject before and after each of the output timings of the plurality of sounds; an information processing device having a processing unit that identifies a feature of a sound that induces a change in the state value based on correspondence information in which the change in the state value and each feature of the plurality of sounds are associated with each other; An information processing system comprising:

Images