Information processing apparatus, information processing method, and program

Abstract

To solve the problem in which it is unable to appropriately recognize motions of a person.SOLUTION: A motion recognition device of the present disclosure includes: a transforming unit for transforming first motion data into a first symbol string including a sequence of symbols; a recognizing unit for recognizing a motion of the first motion data based on the first symbol string; and a deforming unit for generating a third symbol string in which the first symbol string is deformed based on a second symbol string in which second motion data corresponding to the motion recognized in the first motion data is transformed into a sequence of symbols. Consequently, a motion recognition model can be machine-learned using the generated third symbol string for example, and decision making based on the recognized motion can be supported.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 describes recognizing a person's movements from video. Specifically, Patent Document 1 recognizes a person's basic movements from skeletal information of the person in each frame of video, and further recognizes higher-level movements that are combinations of the basic movements. In this case, examples of basic movements include raising a hand and looking down, and higher-level movements include work behavior and suspicious behavior. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-3434 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 requires a large amount of learning data for higher-level actions. Therefore, it is difficult to prepare a large amount of learning data for higher-level actions that are unique to a location or environment, and new higher-level actions cannot be recognized. As a result, there is a problem in that a person's actions cannot be properly recognized.

[0005] Therefore, one of the objectives of the present disclosure is to solve the above-mentioned problem of being unable to properly recognize the movements of a person. [Means for solving the problem]

[0006] An information processing device according to an embodiment of the present disclosure includes: a conversion unit that converts the first motion data into a first symbol string consisting of an array of symbols; a recognition unit that recognizes the action of the first action data based on the first symbol string; a transformation unit that transforms the first symbol string based on a second symbol string obtained by converting second action data corresponding to the action recognized in the first action data into a symbol array, and generates a third symbol string by transforming the first symbol string; Equipped with The structure is as follows. Furthermore, an information processing method according to an embodiment of the present disclosure includes: converting the first motion data into a first symbol string consisting of an array of symbols; Recognizing the action of the first action data based on the first symbol string; generating a third symbol string by transforming the first symbol string based on a second symbol string obtained by converting second motion data corresponding to the motion recognized in the first motion data into a symbol array; The structure is as follows. Furthermore, a program according to an embodiment of the present disclosure includes: converting the first motion data into a first symbol string consisting of an array of symbols; Recognizing the action of the first action data based on the first symbol string; generating a third symbol string by transforming the first symbol string based on a second symbol string obtained by converting second motion data corresponding to the motion recognized in the first motion data into a symbol array; Have the computer perform the process, The structure is as follows. [Effects of the Invention]

[0007] With the above-described configuration, the present disclosure can appropriately recognize the movements of a person. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example of a configuration of a data expansion device according to the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example of data related to the present disclosure. [Figure 3]FIG. 1 is a diagram illustrating an example of data related to the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of a process performed by a data extension device according to the present disclosure. [Figure 5] 10 is a flowchart illustrating an example of a processing operation of the data extension device according to the present disclosure. [Figure 6] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an information processing device according to the present disclosure. [Figure 7] 1 is a block diagram illustrating an example of a configuration of an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] First Embodiment A first embodiment of the present disclosure will be described with reference to the drawings, which may be relevant to any embodiment.

[0010] [composition] As an example, the data expansion device 10 of the present disclosure is used to generate training data used in machine learning of an action recognition model that recognizes a person's actions from the person's action data. Specifically, in this embodiment, it is assumed that basic actions are recognized from the person's action data, and higher-level actions are recognized from combinations of such basic actions, and in particular, it is assumed that higher-level actions are recognized from combinations of basic actions using an action recognition model. The action recognition model is generated by machine learning of training data in which combinations of basic actions are previously associated with higher-level actions, and the combinations of basic actions that make up such training data are generated by the data expansion device 10 of the present disclosure.

[0011] However, the data generated by the data expansion device 10 of the present disclosure is not necessarily limited to being used as learning data when machine learning an action recognition model as described above, and may be used for any purpose.

[0012] Here, a specific example of the action recognition assumed in the above-described embodiment will be described. First, a higher-level action of a person to be recognized in action recognition is, for example, a nursing action by a nurse for a patient, and examples of such nursing actions are "assistance to sit up" and "position change." As shown in FIG. 2, combinations of basic actions constituting the higher-level action "assistance to sit up" are, in order, "1. raise knees," "2. place the patient's hands on the stomach," "3. place the patient in a lateral position (turn the patient over)," "4. put a hand in the gap around the neck," and "5. raise the patient." At this time, as more specific basic actions of a person's body parts corresponding to each basic action, actions of the "left hand" and "right hand" are defined as shown in FIG. 4. Furthermore, as shown in FIG. 3, basic actions constituting the higher-level action "position change" are, in order, "1. place arms on chest," "2. bend knees," and "3. place the patient in a lateral position." At this time, the movements of the "left hand" and "right hand" as shown in FIG. 3 are defined as more specific basic movements of human body parts corresponding to each basic movement.

[0013] In the above-described situation, during action recognition, a combination of a series of basic actions in chronological order is first recognized from the person's action data. Then, a higher-level action is recognized from the recognized combination of basic actions. FIG. 4 shows an example of a combination of basic actions corresponding to a higher-level action. In the upper diagram of FIG. 4, a combination of a series of basic actions corresponding to the higher-level action "assistance in sitting up" is surrounded by a dotted line. In this example, the basic action is represented by "characters" such as "turn palms up." In this embodiment, such "characters" are referred to as "action words," and a sequence of combinations of a series of "action words" is referred to as an "action word sequence." In other words, as described below, in this embodiment, a basic action is recognized as an "action word" from the person's action data, and a higher-level action is recognized from an "action word sequence," which is a word combination consisting of a sequence of such "action words."

[0014] In this embodiment, an example is given in which a basic action is represented by a sentence consisting of multiple meaningful characters that represent the content of the action, called an "action word." However, the "action word" is not limited to being represented by multiple meaningful characters that represent the content of the action, and may be represented by multiple meaningless characters. Furthermore, the "action word" corresponding to the basic action is not limited to being represented by Japanese characters, and may be represented by any symbol, including letters, numbers, and symbols of any language. Furthermore, the "action word" is not limited to being represented by multiple symbols, and may be represented by a symbol such as a single character.

[0015] The data expansion device 10 is composed of one or more information processing devices each having a calculation device and a storage device. As shown in FIG. 1, the data expansion device 10 includes an action word generation unit 11, a higher-level action recognition unit 12, a data extraction unit 13, an action word frequency analysis unit 14, and a data transformation unit 15. The functions of the action word generation unit 11, the higher-level action recognition unit 12, the data extraction unit 13, the action word frequency analysis unit 14, and the data transformation unit 15 can be realized by the calculation device executing a program for realizing each function stored in the storage device. An operation terminal 20 is connected to the data expansion device 10. The operation terminal 20 is an information processing terminal operated by an operator who checks the data generated by the data expansion device 10.

[0016] Then, new action data V of the person is input to the data extension device 10. The new action data V is, for example, data that has not been used as learning data when machine learning the action recognition model. In this embodiment, as an example, the new action data V is data corresponding to a higher-level action of "posture change". However, the new action data V may be data that was used when machine learning the action recognition model, and in this case, it is a target for further machine learning.

[0017] In this case, the motion data may be, for example, acceleration data of a person's body parts, such as acceleration data measured by a wearable device such as a smart watch worn on the person's wrist. However, the motion data may be any data representing the person's motion obtained from the person. For example, the motion data may be data such as the position, speed, and acceleration of the person's joints obtained by analyzing video.

[0018] Furthermore, an action dataset X consisting of multiple action data of a person and a higher-order action label Y corresponding to the action data of the action dataset X are input to the data extension device 10. The action data of the action dataset X and the higher-order action label Y are, for example, training data used during machine learning of an action recognition model or data used for verifying the machine-learned action recognition model. In this embodiment, as an example, the action data of the action dataset X is data corresponding to the higher-order action label Y of "assistance in sitting up". However, the action dataset X and the higher-order action label Y are not limited to training data or verification data of an action recognition model, and may be any data.

[0019] The new action data V, the action data set X, and the higher-level action label Y may be stored in a storage device included in the data extension device 10, or may be stored in an external storage device.

[0020] The action wording unit 11 (conversion unit) acquires new action data V (first action data) in time series, converts the action data for each predetermined unit time into "action words" (symbols) that represent basic actions, and outputs an "action word string Vword" (first symbol string) consisting of a series of "action words" arranged in time series. As a result, the action wording unit 11 can convert and output the new action data V into an "action word string Vword" consisting of a series of "action words" such as ["turn palm up", "raise arm up", ...] as shown in the lower diagram of Figure 4, for example.

[0021] The action wording unit 11 converts new action data V into an action word string Vword and outputs it by inputting the new action data V into a basic action recognition model, which is a machine learning model, for example. The basic action recognition model is constructed by machine learning learning data in which action data is associated with "action words" that represent basic actions corresponding to the action data. However, the action wording unit 11 may convert action data into action words using any method.

[0022] Furthermore, the action wording unit 11 acquires each action data (second action data) of the action dataset X, converts the action data for each predetermined unit time into "action words" (symbols) representing basic actions for each action data, and outputs an "action word string Xword" (second symbol string) consisting of a sequence of a series of "action words" along a time series. As a result, as shown in the upper diagram of FIG. 4, for example, the action wording unit 11 can convert each action data of the action dataset X into an "action word string Xword" consisting of a sequence of a series of "action words", such as ["turn palm up", "raise arm", ...], ..., ["raise arm", "lower arm", ...], and output it. Note that the upper diagram of FIG. 4 shows the case of data corresponding to the higher-level action label Y of "assistance to sit up" as the higher-level action, and action word strings Xword are also output for the action dataset X with other higher-level action labels Y.

[0023] The action word generator 11 converts each action data into an action word string Xword by inputting each action data of the action dataset X into a basic action recognition model, which is a machine learning model as described above, and outputs the converted action data. At this time, the conversion from the action dataset X to the action word string Xword by the action word generator 11 may be performed at any timing, and the action word string Xword converted from the action dataset X may be stored in association with the higher-order action label Y during machine learning of the action recognition model, etc.

[0024] The higher-level action recognition unit 12 (recognition unit) acquires the action word string Vword converted from the new action data V, and outputs an inference result Wp that recognizes the higher-level action of the new action data V from the action word string Vword. The higher-level action recognition unit 12 outputs a higher-level action label that represents the higher-level action as the inference result. In this embodiment, for example, in the example of new action data V shown in the lower diagram of Figure 4, the new action data V is actually data corresponding to the higher-level action of "changing position", but the higher-level action recognition unit 12 recognizes it as the higher-level action of "assistance in getting up", and outputs the inference result Wp.

[0025] The higher-level action recognition unit 12 outputs the recognized higher-level action labels as the inference result Wp by inputting an action word string to an action recognition model, which is a machine learning model, for example. The action recognition model is constructed by machine learning learning data in which the action word string and the higher-level action labels are associated with each other. However, the higher-level action recognition unit 12 may infer the higher-level action labels from the action word string by any method.

[0026] The data extraction unit 13 (transformation unit) acquires an action word sequence Xword of the action data set X, a higher-order action label Y corresponding to the action data set X, and an inference result Wp recognized from the action word sequence Vword of the new action data V. Then, the data extraction unit 13 extracts and outputs the action word sequence Xword of the action data set X corresponding to the same higher-order action label Y as the higher-order action label of the inference result Wp of the new action data V. In other words, the data extraction unit 13 extracts only the action word sequence Xword of the higher-order action label Y that is determined to be most similar to the new action data V. In this embodiment, as an example, it is assumed that a plurality of action word sequences Xword corresponding to the higher-order action label Y of "assistance in sitting up" shown in the upper diagram of FIG. 4 are extracted.

[0027] The action word frequency analysis unit 14 (transformation unit) acquires the extracted action word strings Xword and compares and analyzes the action word strings Xword. Specifically, the action word frequency analysis unit 14 calculates the appearance position and appearance frequency, i.e., which action word appears in which position and how many times in the action word string Xword, and outputs the distribution of appearance frequency and appearance position for each action word. For example, in the example in the upper diagram of Figure 4, it is analyzed that the action word "raise arm" appears n times in the action word string Xword at the second position along the time series.

[0028] Furthermore, as an example of analysis, the action word frequency analysis unit 14 determines whether the occurrence frequency of an action word is low. Here, low frequency means, for example, that the occurrence rate is below a preset threshold, such as "occurrence rate < 20%" at a certain occurrence position. Note that the action word frequency analysis unit 14 may also determine whether an action word is low in frequency based on the occurrence rate of the action word in the entire action word string, regardless of the occurrence position. For example, in the example in the upper diagram of Figure 4, the action word "lower arm" is determined to be low in frequency. Note that such action words determined to be low in frequency are assumed to be action words with little meaning.

[0029] As an example of analysis, the action word frequency analysis unit 14 determines whether the occurrence frequency of an action word is high. Here, a high frequency is defined as a predetermined threshold or higher, such as an "occurrence rate ≥ 80%" at a certain occurrence position. The action word frequency analysis unit 14 may also determine whether an action word is high in frequency based on the occurrence rate of the action word in the entire action word string, regardless of the occurrence position. As described above, an action word that appears frequently at a specific occurrence position is assumed to be an important word that has meaning and characterizes the corresponding higher-level action. In this case, it is also determined whether there is a pattern in which there are multiple important words and their occurrence positions are interchanged. On the other hand, an action word that appears frequently without a specified occurrence position is assumed to be a meaningless word that can be ignored as a stop word.

[0030] As an example of analysis, the action word frequency analysis unit 14 determines whether the vector expressions of the action words themselves are similar. For example, it calculates the distance when the action words are expressed as vectors, and determines that action words whose distance is close within a threshold are synonyms.

[0031] Furthermore, the action word frequency analysis unit 14 may perform the above-described analysis not in units of one action word but in units of multiple action words. For example, the frequency of appearance of two or three consecutive action words may be determined.

[0032] The data transformation unit 15 (transformation unit) transforms the action word sequence Vword of the new action data V according to the analysis results of the action word frequency analysis unit 14 described above, to generate expanded data Vword', which is a new action word sequence. Specifically, as a result of comparing the action word sequences Xword as described above, the data transformation unit 15 assumes that action words determined to have a low frequency of occurrence are not very meaningful action words, and therefore, even if they are added to or deleted from the action word sequence Vword, they are unlikely to affect the action recognition of the original new action data. For this reason, the data transformation unit 15 regards action words determined to have a low frequency of occurrence as noise, and generates new expanded data Vword' by adding or deleting them from the action word sequence Vword as data to which the same action recognition label as the original action word sequence Vword is assigned. For example, in the example of Figure 4, if the action word ``lower arms'' is determined to be low frequency among the action words in the action data set X, the action word ``lower arms'' is added to the action word string Vword of the new action data V to generate new data, or the action word ``lower arms'' is deleted from the action word string Vword of the new action data V to generate new data, which is called the expanded data Vword'.

[0033] Furthermore, when the data transformation unit 15 compares the action word sequences Xword with each other and determines that an action word has a high frequency of occurrence at a specific occurrence position, the action word is assumed to be important, and therefore, even if a part of the action word is deleted from the action word sequence Vword, it is unlikely to have an adverse effect. Therefore, the data transformation unit 15 replaces a part, such as half of the action words determined to have a high frequency of occurrence at a specific occurrence position in the action word sequence Vword, with other action words to generate the expanded data Vword'. At this time, if there is a pattern in which there are multiple important action words and their occurrence positions are swapped, the data transformation unit 15 swaps the appearance positions of the action words in the action word sequence Vword to generate the expanded data Vword'. Furthermore, the data transformation unit 15 assumes that action words determined to have a high frequency of occurrence regardless of their occurrence position are meaningless words that can be ignored as stop words, and therefore considers such action words to be noise, and adds or deletes them from the action word sequence Vword to generate the expanded data Vword'.

[0034] Furthermore, if, as a result of the above-mentioned analysis, there is a synonym that has a similar vector expression to the action word itself, the data transformation unit 15 replaces the action word in the action word string Vword with the synonym and generates the expanded data Vword'.

[0035] The data transformation unit 15 may perform the process of deleting, adding, or changing action words from the action word string Vword as described above in units of two or three consecutive action words.

[0036] In this way, the data transformation unit 15 can generate expanded data Vword' and increase the number of data items without changing the essence of the data content by adding or deleting action words that are considered to be noise, replacing some of the important action words, or replacing action words with synonyms from the action word string of the new action data V. For example, in the example of the action word string of the new action data V shown in the lower diagram of Figure 4, the number of expanded data Vword' can be increased without changing the essence of the data. Then, these generated expanded data Vword' are associated with the original higher-level action label "posture change" of the original new action data V, and can be used as training data for machine learning an action recognition model that recognizes higher-level actions.

[0037] Here, the data transformation unit 15 may output the expanded data Vword' generated as described above so as to be displayed on the screen of the operation terminal 20. In response to this, the operation terminal 20 accepts input from the operator regarding the appropriateness of the displayed expanded data Vword'. At this time, the operator of the operation terminal 20, for example, checks the content of the action word sequence that is the displayed expanded data Vword', that is, the meaning of the sentence made up of the action word sequence as shown in FIG. 4, and inputs an instruction to apply the action word sequence if it is consistent with the higher-level action, or an instruction not to apply the action word sequence if it is contradictory. Then, the data transformation unit 15 can select the action word sequence that has received the input to apply from the operation terminal 20 as the expanded data Vword' and use it as learning data.

[0038] [Operation] Next, the processing operation by the data extension device 10 will be described. First, the data extension device 10 acquires new action data V and converts it into an "action word string Vword" consisting of a sequence of a series of "action words" (step S1 in FIG. 5). For example, as shown in the lower diagram in FIG. 4, the new action data V is converted into an "action word string Vword" consisting of a sequence of a series of "action words" such as ["turn palm up", "raise arm up", ...].

[0039] The data extension device 10 may also acquire a label action dataset X to which a higher-level action label Y has been assigned, and convert the label action dataset X into an "action word string Xword" consisting of a sequence of a plurality of "action words." As a result, as shown in the upper diagram of FIG. 4, for example, each action data of the action dataset X is converted into an "action word string Xword" consisting of a sequence of a series of "action words," such as ["turn palm up," "raise arm,"...],..., ["raise arm," "lower arm,"...]. These action word strings Xword are then stored in association with the higher-level action label Y of the higher-level action "assistance to sit up." However, the data extension device 10 may convert the action dataset X into an action word string Xword at any timing and store the converted data, or may acquire an action word string Xword that has already been converted and stored in a predetermined storage device.

[0040] Next, the data extension device 10 recognizes higher-level actions of the new action data V from the action word string Vword converted from the new action data V, and outputs the higher-level action label, which is the inference result Wp (step S2 in FIG. 5). For example, in the example shown in the lower diagram in FIG. 4, the new action data V corresponds to the higher-level action of "changing body position", but in a situation where such higher-level actions have not been learned by machine learning, it is assumed that the action word string Vword of the new action data V is recognized as the higher-level action of "assistance in getting up".

[0041] Next, the data extension device 10 acquires an action word string Xword of the action data set X corresponding to the same higher-order action label Y as the higher-order action label that is the inference result Wp of the new action data V (step S3 in FIG. 5). For example, in the example shown in FIG. 4, the new action data V is recognized as the higher-order action "assistance to get up", so multiple action word strings Xword associated with the same higher-order action label Y "assistance to get up" are extracted.

[0042] Next, the data extension device 10 compares and analyzes the extracted action word strings Xword (step S4 in FIG. 5). For example, the data extension device 10 calculates the appearance position and appearance frequency of a specific action word in the action word string Xword. Then, the data extension device 10 generates extended data Vword' by modifying the action word string of the new action data V according to the analysis results such as the appearance frequency of the action word (step S5 in FIG. 5). As an example, if the action word "lower arm" is determined to have a low frequency in the example in the upper diagram of FIG. 4, the data extension device 10 regards the action word as noise and adds or deletes it from the action word string Vword to generate new extended data Vword'. Note that, as described above, the data extension device 10 also performs various data modifications according to the analysis results.

[0043] As described above, the data extension device 10 increases the number of data by applying various modifications to the action word strings of the new action data V. This increases the number of action word strings corresponding to higher-level actions of the new action data V, and increases the amount of learning data used for machine learning of the action recognition model that recognizes the higher-level actions. As a result, the action recognition model can be machine-learned with high accuracy, and people's actions can be appropriately recognized.

[0044] In particular, the data extension device 10 extends the data by modifying the action word string of the new action data V according to the analysis results such as the appearance frequency of action words in the action word string of the action data set X corresponding to the top action recognized by the action word string of the new action data V. For example, by adding or deleting action words that are determined to be noise or stop words by the analysis, the data can be extended by modifying the action word string while suppressing the influence on action recognition without changing the essence of the data.

[0045] In the present disclosure, as an example, it is possible to generate training data that can be used for machine learning of an action recognition model for recognizing nursing actions performed by nurses. As a result, the accuracy of the action recognition model is improved, and it is possible to support medical professionals such as nurses and doctors in making decisions about treatment for the recognized nursing actions.

[0046] However, the higher-level actions that can be recognized as described above are not limited to the nursing actions by nurses described above, nor are they limited to actions in the fields of medicine and healthcare, and may be any actions. Accordingly, the basic actions that make up the higher-level actions are also not limited to the basic actions listed above, and may be any actions.

[0047] <Second embodiment> Next, a second embodiment of the present disclosure will be described with reference to the drawings. In this embodiment, an outline of the data extension device etc. described in the above embodiment is shown. Note that the drawings may be relevant to any embodiment.

[0048] First, a description will be given of the hardware configuration of the information processing device 100 in the present disclosure. The information processing device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, as an example, as shown in FIG. ·CPU(Central Processing Unit)101(Arithmetic unit) ROM (Read Only Memory) 102 (storage device) RAM (Random Access Memory) 103 (storage device) Programs 104 loaded into RAM 103 A storage device 105 for storing a group of programs 104 A drive device 106 that reads and writes from a storage medium 110 external to the information processing device A communication interface 107 that connects to a communication network 111 outside the information processing device Input / output interface 108 for inputting and outputting data Bus 109 connecting each component

[0049] 6 shows an example of the hardware configuration of the information processing device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with a part of the above-described configuration, such as not including the drive device 106. Furthermore, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof, instead of the above-described CPU.

[0050] The information processing device 100 can then configure and include the conversion unit 121, recognition unit 122, and transformation unit 123 shown in Fig. 7 by having the CPU 101 acquire and execute the program group 104. The program group 104 is stored in advance in, for example, the storage device 105 or the ROM 102, and is loaded into the RAM 103 and executed by the CPU 101 as needed. The program group 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, with the drive device 106 reading out the programs and supplying them to the CPU 101. However, the conversion unit 121, recognition unit 122, and transformation unit 123 described above may be configured using dedicated electronic circuits for realizing such means.

[0051] The conversion unit 121 converts the first action data into a first symbol string consisting of an array of symbols. The recognition unit 122 recognizes the action of the first action data based on the first symbol string. The transformation unit 123 generates a third symbol string by transforming the first symbol string based on a second symbol string obtained by converting second action data corresponding to the action recognized in the first action data into an array of symbols.

[0052] With the above-described configuration, the present disclosure can generate third motion data by transforming a symbol string of the first motion data using a symbol string of second motion data that can correspond to the new first motion data when the new first motion data is acquired. This allows the symbol string of the first motion data corresponding to the new motion to be transformed and expanded, and used as training data for a motion recognition model that recognizes the new motion. As a result, by machine learning the motion recognition model, the motion of a person can be appropriately recognized by the motion recognition model.

[0053] At least one of the functions of the conversion unit 121, recognition unit 122, and transformation unit 123 described above may be executed by an information processing device installed and connected anywhere on the network, that is, may be executed by so-called cloud computing.

[0054] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program can also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0055] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each of the above-described embodiments can be combined with other embodiments as appropriate.

[0056] <Additional Notes> Some or all of the above embodiments may be described as follows: The following provides an overview of the configurations of an information processing device, an information processing method, and a program according to the present disclosure. However, the present disclosure is not limited to the configurations described in the following supplementary notes. Note that the configurations described in Supplements 2 to 8.2 that are dependent on Supplementary Note 1 above and some or all of the functions of the configurations may also be dependent on other Supplements 9 and 10 in the same dependent relationship as Supplements 2 to 8.2. Furthermore, not limited to Supplements 1, 9, and 10, but also within the scope of each of the above-mentioned embodiments, similar hardware, software, various recording means for recording software, or systems may similarly be made to be dependent on the configurations described as Supplements and some or all of the functions of the configurations. (Appendix 1) a conversion unit that converts the first motion data into a first symbol string consisting of an array of symbols; a recognition unit that recognizes the action of the first action data based on the first symbol string; a transformation unit that transforms the first symbol string based on a second symbol string obtained by converting second action data corresponding to the action recognized in the first action data into a symbol array, and generates a third symbol string by transforming the first symbol string; An information processing device comprising: (Appendix 2) 10. The information processing device according to claim 1, the transformation unit generates the third symbol string by transforming a part of the first symbol string based on a part of the second symbol string. Information processing device. (Appendix 3) 10. The information processing device according to claim 2, the transformation unit generates the third symbol string by adding or deleting a symbol constituting the second symbol string to or from the first symbol string. Information processing device. (Appendix 3.1) 10. The information processing device according to claim 2, the transformation unit generates the third symbol string by adding or deleting symbols that are similar to symbols constituting the second symbol string based on a preset criterion to the first symbol string. Information processing device. (Appendix 4) 10. The information processing device according to claim 1, the transformation unit generates the third symbol string by transforming the first symbol string based on a comparison of the second symbol strings obtained by transforming the second motion data. Information processing device. (Appendix 5) 5. The information processing device according to claim 4, the transformation unit generates the third symbol string by transforming the first symbol string based on the frequency of appearance of a predetermined symbol in the plurality of second symbol strings; Information processing device. (Appendix 6) 6. The information processing device according to claim 5, the transformation unit generates the third symbol string by adding or deleting, from the first symbol string, a symbol whose appearance frequency in the plurality of second symbol strings is low based on a preset criterion. Information processing device. (Appendix 6.1) 6. The information processing device according to claim 5, the transformation unit generates the third symbol string by adding or deleting, from the first symbol string, a symbol whose appearance frequency in the plurality of second symbol strings is higher than a predetermined standard. Information processing device. (Appendix 6.2) 5. The information processing device according to claim 4, the transformation unit generates the third symbol string by transforming the first symbol string based on appearance positions of a predetermined symbol in the plurality of second symbol strings. Information processing device. (Appendix 7) 5. The information processing device according to claim 4, the transformation unit generates the third symbol string by transforming the first symbol string based on the appearance positions and appearance frequencies of predetermined symbols in the plurality of second symbol strings; Information processing device. (Appendix 7.1) 8. The information processing device according to claim 7, the transformation unit generates the third symbol string by adding or deleting, from the first symbol string, a symbol that appears at the same position in a plurality of the second symbol strings and has a high frequency of appearance based on a preset criterion. Information processing device. (Appendix 8) 10. The information processing device according to claim 1, the recognition unit recognizes the action of the first action data from the first symbol string using a machine learning model; Information processing device. (Appendix 8.1) 10. The information processing device according to claim 1, Each of the symbols constituting the symbol string is composed of one symbol or a symbol group of multiple symbols. Information processing device. (Appendix 8.2) 10. The information processing device according to claim 1, Each of the symbols in the sequence constituting the symbol string is made up of a plurality of characters. Information processing device. (Appendix 9) converting the first motion data into a first symbol string consisting of an array of symbols; Recognizing the action of the first action data based on the first symbol string; generating a third symbol string by transforming the first symbol string based on a second symbol string obtained by converting second action data corresponding to the action recognized in the first action data into an array of symbols; Information processing methods. (Appendix 10) converting the first motion data into a first symbol string consisting of an array of symbols; Recognizing the action of the first action data based on the first symbol string; generating a third symbol string by transforming the first symbol string based on a second symbol string obtained by converting second action data corresponding to the action recognized in the first action data into an array of symbols; A program that causes a computer to perform a process. [Explanation of symbols]

[0057] 10 Data expansion device 11 Action Vocabulary 12 Upper action recognition unit 13 Data Extraction Section 14 Action word frequency analysis section 15 Data transformation section 20 Operation terminal 100 Information processing device 101 CPU 102 ROM 103 RAM 104 Programs 105 Storage device 106 Drive device 107 Communication Interface 108 Input / Output Interface 109 Bus 110 Storage medium 111 Communication Network 121 Conversion Unit 122 Recognition part 123 Deformed part

Claims

1. a conversion unit that converts the first motion data into a first symbol string consisting of an array of symbols; a recognition unit that recognizes the action of the first action data based on the first symbol string; a transformation unit that transforms the first symbol string based on a second symbol string obtained by converting second action data corresponding to an action recognized from the first action data into an array of symbols, and generates a third symbol string by transforming the first symbol string; An information processing device comprising:

2. 2. The information processing device according to claim 1, the transformation unit generates the third symbol string by transforming a part of the first symbol string based on a part of the second symbol string. Information processing device.

3. 3. The information processing device according to claim 2, the transformation unit generates the third symbol string by adding or deleting a symbol constituting the second symbol string to or from the first symbol string. Information processing device.

4. 2. The information processing device according to claim 1, the transformation unit generates the third symbol string by transforming the first symbol string based on a comparison of the second symbol strings obtained by transforming the second motion data. Information processing device.

5. 5. The information processing device according to claim 4, the transformation unit generates the third symbol string by transforming the first symbol string based on the frequency of appearance of a predetermined symbol in the plurality of second symbol strings; Information processing device.

6. 6. The information processing device according to claim 5, the transformation unit generates the third symbol string by adding or deleting, from the first symbol string, a symbol whose appearance frequency in the plurality of second symbol strings is low based on a preset criterion. Information processing device.

7. 5. The information processing device according to claim 4, the transformation unit generates the third symbol string by transforming the first symbol string based on the appearance positions and appearance frequencies of predetermined symbols in the plurality of second symbol strings; Information processing device.

8. 2. The information processing device according to claim 1, the recognition unit recognizes the action of the first action data from the first symbol string using a machine learning model; Information processing device.

9. converting the first motion data into a first symbol string consisting of an array of symbols; Recognizing the action of the first action data based on the first symbol string; generating a third symbol string by transforming the first symbol string based on a second symbol string obtained by converting second action data corresponding to the action recognized in the first action data into an array of symbols; Information processing methods.

10. converting the first motion data into a first symbol string consisting of an array of symbols; Recognizing the action of the first action data based on the first symbol string; generating a third symbol string by transforming the first symbol string based on a second symbol string obtained by converting second action data corresponding to the action recognized in the first action data into an array of symbols; A program that causes a computer to perform a process.

Images