System, information processing method, and program

Abstract

[Problem] To perform a motion data search that more closely matches a user's intention. [Solution] Provided is a system comprising a search unit that searches for motion data similar to the motion data of a user on the basis of the motion data of the user that was captured during a capture period set by a capture start trigger and a capture end trigger.

Description

System, Information Processing Method, and Program

[0001] The present disclosure relates to a system, an information processing method, and a program.

[0002] In recent years, various applications using motion data have become widespread. Further, for example, as disclosed in Patent Document 1, a technique for searching for motion data similar to the motion data based on motion data capturing a user's operation has been developed.

[0003] International Publication No. 2022 / 123800

[0004] When performing the above search, it is required to accurately capture the operation as a search key.

[0005] According to an aspect of the present disclosure, there is provided a system including a search unit that searches for motion data similar to the motion data of a user based on the motion data of the user captured during a capture period defined by a capture start trigger and a capture end trigger.

[0006] Further, according to another aspect of the present disclosure, there is provided an information processing method including a processor searching for motion data similar to the motion data of a user based on the motion data of the user captured during a capture period defined by a capture start trigger and a capture end trigger.

[0007] Further, according to another aspect of the present disclosure, there is provided a program that causes a computer to function as an information processing apparatus including a search unit that searches for motion data similar to the motion data of a user based on the motion data of the user captured during a capture period defined by a capture start trigger and a capture end trigger.

[0008] This figure shows an example of the functional configuration of a system 1 according to one embodiment of the present disclosure. This figure illustrates an example of a capture system 10 according to the same embodiment. This block diagram shows an example of the functional configuration of an information processing terminal 20 according to the same embodiment. This block diagram shows an example of the functional configuration of an information processing server 30 according to the same embodiment. This flowchart shows an example of the capture processing flow according to the same embodiment. This flowchart shows an example of the search processing flow according to the same embodiment. This flowchart shows an example of the display control flow of search result data according to the same embodiment. This figure illustrates an example of the display of search result data according to the same embodiment. This figure illustrates the detection of a variable capture period and a search termination trigger according to the same embodiment. This figure shows an example of the transition of search results when a search is performed for each capture period according to the same embodiment. This flowchart shows an example of the search flow for each of multiple capture periods according to the same embodiment. This flowchart shows an example of the repetitive processing flow according to the same embodiment. This figure illustrates an example of the display of captured data according to the same embodiment. This figure illustrates an example of accepting trimming for captured data 71 according to the same embodiment. This block diagram shows an example of the hardware configuration of an information processing device 90 according to the same embodiment.

[0009] Preferred embodiments of this disclosure will be described in detail below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0010] Furthermore, in this specification and drawings, when describing multiple identical components to distinguish them, letters or other symbols may be added to the end of the reference numerals. On the other hand, when there is no need to distinguish multiple identical components, the letters or other symbols may be omitted, and a description common to all identical components may be provided.

[0011] The explanation will be presented in the following order: 1. Embodiments 1.1. Example Functional Configuration 1.2. Functional Details 2. Example Hardware Configuration 3. Summary

[0012] <1. Embodiments> <<1.1. Example of Functional Configuration>> First, an example of the functional configuration of System 1 according to one embodiment of the present disclosure will be described. Figure 1 is a diagram showing an example of the functional configuration of System 1 according to one embodiment of the present disclosure.

[0013] As shown in Figure 1, the system 1 according to this embodiment includes a capture system 10, an information processing terminal 20, and an information processing server 30.

[0014] (Capture System 10) The capture system 10 according to this embodiment is equipped with various sensors for capturing the user's movements.

[0015] Figure 2 is a diagram illustrating an example of the capture system 10 according to this embodiment. As shown in Figure 2, the capture system 10 may include a plurality of inertial sensors 15.

[0016] In the example shown in Figure 2, the capture system 10 includes inertial sensors 15A to 15F that are attached to the user's head, waist, right hand, left hand, right foot, and left foot, respectively.

[0017] Each of the inertial sensors 15A to 15F transmits the sensed acceleration and angular velocity information to the information processing terminal 20 via wireless communication. For this purpose, each of the inertial sensors 15A to 15F is equipped with an antenna or the like to realize wireless communication compliant with a predetermined communication standard.

[0018] In Figure 2, an example is shown where the capture system 10 is equipped with six inertial sensors 15A to 15F. However, the number of inertial sensors 15 included in the capture system 10 according to this embodiment, the mounting locations of the inertial sensors 15, etc., are not limited to this example.

[0019] Furthermore, the capture system 10 according to this embodiment is not limited to an inertial sensor type, but may be a system that realizes optical motion capture.

[0020] (Information Processing Terminal 20) The information processing terminal 20 according to this embodiment is an information processing device operated by a user.

[0021] The information processing terminal 20 according to this embodiment may be, for example, a smartphone, a tablet, a PC (Personal Computer), etc.

[0022] Furthermore, the information processing terminal 20 according to this embodiment communicates information with the capture system 10 and the information processing server 30, respectively.

[0023] (Information Processing Server 30) The information processing server 30 according to this embodiment is an information processing device that searches for motion data similar to the captured motion data.

[0024] Furthermore, the information processing server 30 according to this embodiment controls the user interface related to the search for motion data.

[0025] Next, with reference to Figures 3 and 4, an example of the functional configuration of the information processing terminal 20 and the information processing server 30 according to this embodiment will be described in more detail.

[0026] Figure 3 is a block diagram showing an example of the functional configuration of the information processing terminal 20 according to this embodiment.

[0027] As shown in Figure 3, the information processing terminal 20 according to this embodiment may include a control unit 210, an operation reception unit 220, a display unit 230, an audio output unit 240, a communication unit 250, and the like.

[0028] (Control Unit 210) The control unit 210 according to this embodiment controls the operation of each component of the information processing terminal 20.

[0029] Furthermore, the control unit 210 according to this embodiment cooperates with the information processing server 30 to control the user interface related to the retrieval of motion data.

[0030] The functions of the control unit 210 according to this embodiment are realized through the cooperation of various processors and memory.

[0031] (Operation Reception Unit 220) The operation reception unit 220 according to this embodiment receives various operations from the user. For this purpose, the operation reception unit 220 according to this embodiment is equipped with input devices such as a touch panel, keyboard, buttons, mouse, and microphone.

[0032] (Display Unit 230) The display unit 230 according to this embodiment displays various information in accordance with the control unit 210. For this purpose, the display unit 230 according to this embodiment is equipped with a display.

[0033] (Audio Output Unit 240) The audio output unit 240 according to this embodiment outputs various types of audio according to the control unit 210. For this purpose, the audio output unit 240 according to this embodiment is equipped with a speaker, amplifier, etc.

[0034] (Communication Unit 250) The communication unit 250 according to this embodiment communicates information with the capture system 10 and the information processing server 30, respectively. For this purpose, the communication unit 250 according to this embodiment is equipped with various communication devices.

[0035] For example, the communication unit 250 receives acceleration and angular velocity information from the capture system 10.

[0036] Furthermore, for example, the communication unit 250 transmits the acceleration and angular velocity information to the information processing server 30.

[0037] Furthermore, for example, the communication unit 250 receives information related to the control of the user interface from the information processing server 30.

[0038] Furthermore, for example, the communication unit 250 transmits information related to user operations using the user interface to the information processing server 30.

[0039] Figure 4 is a block diagram showing an example of the functional configuration of the information processing server 30 according to this embodiment.

[0040] As shown in Figure 4, the information processing server 30 according to this embodiment may include an interface control unit 310, a search unit 320, a motion database 330 (also referred to as motion DB330), and a communication unit 340, etc.

[0041] (Interface Control Unit 310) The interface control unit 310 according to the present embodiment controls a user interface related to the search for motion data.

[0042] For example, the interface control unit 310 may provide the user interface as a web application. The user accesses the user interface using the information processing terminal 20.

[0043] The functions of the interface control unit 310 according to the present embodiment are realized by the cooperation of various processors and memories. Details of the functions of the interface control unit 310 according to the present embodiment will be described later.

[0044] (Search Unit 32) The search unit 320 according to the present embodiment searches the motion DB 330 for motion data similar to the captured user motion data based on the captured user motion data.

[0045] The functions of the search unit 320 according to the present embodiment are realized by the cooperation of various processors and memories. Details of the functions of the search unit 320 according to the present embodiment will be described later.

[0046] (Motion DB 330) The motion DB 330 according to the present embodiment stores motion data, 3D model data to which the motion data is applied, and the like.

[0047] (Communication Unit 340) The communication unit 340 according to the present embodiment performs information communication with the information processing terminal 20. For this purpose, the communication unit 340 according to the present embodiment includes various communication devices.

[0048] For example, the communication unit 340 receives information on acceleration and angular velocity from the information processing terminal 20.

[0049] Also, for example, the communication unit 340 transmits information related to the control of the user interface to the information processing terminal 20.

[0050] Also, for example, the communication unit 340 receives information related to the user's operation using the user interface from the information processing terminal 20.

[0051] The above describes an example of the functional configuration of System 1 according to this embodiment. Note that the above functional configuration described with reference to Figures 1 to 4 is merely an example, and the functional configuration of System 1 according to this embodiment is not limited to this example.

[0052] For example, the above example illustrates a case where the information processing server 30 includes an interface control unit 310, a search unit 320, and a motion database 330. On the other hand, the interface control unit 310, the search unit 320, and the motion database 330 may each be provided in different information processing devices.

[0053] The functional configuration of System 1 according to this embodiment can be flexibly modified according to specifications, operation, etc.

[0054] <<1.2. Functional Details>> Next, the functions of System 1 according to this embodiment will be described in more detail.

[0055] As mentioned above, in recent years, a technology has been developed that uses motion data captured from a user's movements as a search key to search for motion data similar to that data.

[0056] With the technologies described above, for example, users can intuitively search for desired motion data by having the system capture their own movements.

[0057] However, when performing searches as described above, if the captured motion data (search key) includes unnecessary movements, the motion data obtained as search results may differ from the user's intent.

[0058] Here, we assume that the start and end of the capture are instructed by pressing buttons or other means on the user interface. Furthermore, in this case, we assume, for example, that the user wants to search for motion data that includes a hand-waving motion.

[0059] In this case, the user signals the start of the capture by pressing a button located on the user interface, then signals the end of the capture by waving their hand and pressing the button again.

[0060] In this case, the captured motion data includes the actions from immediately after pressing the button that signals the start of capture until the hand waving motion, and the actions from when the hand waving motion ends until immediately before pressing the button that signals the end of capture.

[0061] Therefore, motion data obtained using the above-mentioned motion data as a search key (search result data) may reflect actions related to button operations, and this may also result in motion data including hand waving being not obtained as search results.

[0062] The technical concept of one embodiment of this disclosure was conceived with the above-mentioned points in mind, and aims to ensure the quality of motion data used as a search key and to realize motion data search that better matches the user's intent.

[0063] To this end, one of the features of the search unit 320 according to this embodiment is that it searches for motion data similar to the user's motion data based on the user's motion data captured during the capture period determined by the capture start trigger and the capture end trigger.

[0064] Figure 5 is a flowchart showing an example of the capture process flow according to this embodiment.

[0065] In the example shown in Figure 5, the search unit 320 first determines whether or not a search start instruction has been detected (S101).

[0066] The command to start a search may be implemented, for example, by pressing a button located on the user interface. In this case, the interface control unit 310 detects the command to start a search based on the fact that the button has been pressed.

[0067] However, the command to start a search is not limited to pressing a button. The command to start a search may also be given by voice, a predetermined gesture, or the like.

[0068] If no search start instruction is detected (S101: NO), the search unit 320 repeatedly performs the determination in step S101.

[0069] On the other hand, if the search unit 320 detects a search start instruction (S101: YES), it then determines whether or not a capture start trigger has been detected (S102).

[0070] If no capture start trigger is detected (S102: NO), the search unit 320 repeatedly performs the determination in step S102.

[0071] On the other hand, if the search unit 320 detects a trigger for starting a capture (S102: YES), it starts the capture process (S103).

[0072] After starting the capture process in step S103, the search unit 320 then determines whether or not a capture termination trigger has been detected (S104).

[0073] If no event triggering the end of capture is detected (S104: NO), the search unit 320 repeatedly performs the determination in step S104.

[0074] On the other hand, if the search unit 320 detects an event that triggers the termination of the capture process (S104: YES), it terminates the capture process (S104).

[0075] As described above, the search unit 320 in this embodiment starts the capture process based on the detection of a capture start trigger and ends the capture process based on the detection of a capture end trigger.

[0076] The capture process according to this embodiment may also be a process that generates motion data based on acceleration and angular velocity information detected by the capture system 10.

[0077] For example, the search unit 320 generates motion data based on acceleration and angular velocity information acquired from the time the capture start trigger is detected to the time the capture end trigger is detected.

[0078] In this case, the acceleration and angular velocity information acquired by the capture system 10 is transmitted to the information processing server 30 via the information processing terminal 20.

[0079] On the other hand, the capture process according to this embodiment may be performed by the information processing terminal 20. For example, the control unit 210 of the information processing terminal 20 may generate motion data based on acceleration and angular velocity information acquired from the time when the capture start trigger is detected to the time when the capture end trigger is detected, and transmit the generated motion data to the information processing server 30 via the communication unit 250.

[0080] Alternatively, the control unit 210 may generate motion data regardless of the capture start trigger and capture end trigger, and transmit the generated motion data to the information processing server 30. In this case, the search unit 320 trims the received motion data based on the capture start trigger and capture end trigger to obtain motion data based on acceleration and angular velocity information acquired from the time the capture start trigger is detected to the time the capture end trigger is detected.

[0081] In this embodiment, the trigger for starting and ending the capture may be detected based on, for example, a timer, voice command, or a predetermined operation.

[0082] For example, the interface control unit 310 may detect a capture start trigger based on the elapsed time determined by a timer after detecting a search start instruction, and then detect a capture end trigger based on the elapsed time determined by a timer after detecting the capture start trigger.

[0083] When detecting the start and end of capture triggers using a timer, information regarding the timing of the capture start and end, information regarding the capture period determined by the capture start and end triggers, etc., may be presented to the user via the display unit 230, the audio output unit 240, etc.

[0084] According to this, users can improve search accuracy by understanding the capture period based on the information presented and avoiding unnecessary actions during the capture period.

[0085] Furthermore, for example, the interface control unit 310 may, after detecting a search start instruction, detect a capture start trigger based on the detection of a predetermined voice or predetermined gesture (which may include stillness for a predetermined period of time or longer), and after detecting a capture start trigger, detect a capture end trigger based on the detection of a predetermined voice or predetermined gesture.

[0086] According to this, users can improve search accuracy by explicitly specifying the capture period themselves and avoiding unnecessary actions during that period.

[0087] Next, with reference to Figure 6, the search process according to this embodiment will be described. Figure 6 is a flowchart showing an example of the flow of the search process according to this embodiment.

[0088] In the following, motion data captured during the capture period will be referred to as "capture data." Motion data pre-stored and accumulated in the motion DB330 will be referred to as "accumulated data."

[0089] In the example shown in Figure 6, the search unit 320 first calculates the feature quantities of the captured data (S201).

[0090] Next, the search unit 320 acquires the feature quantities of each stored data (S202). The feature quantities of each stored data may be stored in the motion DB 330 in advance. If the feature quantities of the stored data are not stored in the motion DB 330 in advance, the search unit 320 may calculate the feature quantities of the stored data and have the calculated feature quantities recorded in the motion DB 330 in advance.

[0091] Next, the search unit 320 calculates the similarity (similarity distance) between the captured data and each of the stored data based on the feature quantities of the captured data calculated in step S201 and the feature quantities of each of the stored data acquired in step S202 (S203).

[0092] Next, the search unit 320 acquires the accumulated data whose similarity, calculated in step S203, exceeds the threshold as a search result (S204).

[0093] Next, the search unit 320 identifies similar intervals and similar regions with respect to each of the captured data and the accumulated data (hereinafter also referred to as search result data) acquired as search results in step S204 (S205).

[0094] The above-mentioned similarity interval refers to a temporal interval (frame interval) in which the capture data and search result data have a high degree of similarity. Similarity intervals may be defined using information such as the sequential number (number), start time (time_start), and end time (time_end) related to the frame of the search result data.

[0095] Furthermore, the above-mentioned similarity region refers to a spatial region (frame region) where the capture data and the search result data have a high degree of similarity. The similarity region may be defined, for example, using information such as the display position coordinates (x,y) and display width (Wx,Wy) related to the frame of the search result data.

[0096] In other words, the search unit 320 identifies temporal / spatial elements in the search result data in which motion similar to that of the captured data occurs.

[0097] The search unit 320 may identify similar intervals / regions by comparing the similarity between the captured data and the search result data temporally and spatially. Alternatively, the search unit 320 may identify similar intervals / regions by estimating intervals / regions with large changes in feature quantities in the search result data as intervals / regions where characteristic behavior is observed.

[0098] The above describes an example of the search process flow according to this embodiment. Note that if sufficient search result data cannot be obtained, the search unit 320 may perform processing such as lowering the threshold used for similarity comparison and performing a re-search. Furthermore, the search unit 320 may store the search result data in association with similarity and index.

[0099] Next, we will describe the display control of search result data according to this embodiment. Figure 7 is a flowchart showing an example of the flow of display control of search result data according to this embodiment.

[0100] In the example shown in Figure 7, the interface control unit 310 first acquires search result data and information related to similar intervals / similar regions (S301).

[0101] Next, the interface control unit 310 plays back each of the search result data acquired in step S301 from the beginning (S302).

[0102] The interface control unit 310 may start playback of the search result data based on a playback instruction from the user. The interface control unit 310 may also display the search result data in descending order of similarity to the captured data, based on the similarity and index acquired along with the search result data.

[0103] Next, the interface control unit 310 highlights the similar intervals / similar regions based on the information related to similar intervals / similar regions acquired in step S301 (S303).

[0104] Figures 8 and 9 illustrate an example of how search result data is displayed according to this embodiment.

[0105] Figures 8 and 9 show an example of a search results screen 60 that displays search result data. The search results screen 60 may include a thumbnail display area 61 containing thumbnails of the search result data, and a selected data display area 62 that displays the search result data (selected data) selected by the user in the thumbnail display area 61 in an enlarged view. In Figures 8 and 9, the selected data is indicated by dots.

[0106] Figure 8 shows an example of the search results screen 60 before the playback of the search results data begins.

[0107] In this example, we illustrate a scenario where a user performs a hand-raising motion during the user capture period, and a search is performed based on the captured data of that motion.

[0108] In this case, if the search results data does not include a motion of raising one hand at the beginning, the user cannot determine whether each of the search results data is similar to the captured data before playback, nor can they judge the validity of the search.

[0109] On the other hand, Figure 9 shows an example of the search results screen 60 after the playback of the search results data has started.

[0110] The interface control unit 310 according to this embodiment may control the simultaneous playback of multiple search result data.

[0111] Furthermore, as shown in Figure 9, the interface control unit 310 according to this embodiment may highlight frame intervals (similar intervals) / frame regions (similar regions) that are similar to the captured data when simultaneously playing back multiple search result data.

[0112] Through the control described above, users can intuitively grasp the temporal and spatial elements similar to the captured data for each piece of search result data, and can also judge the validity of the search.

[0113] The above describes an example of displaying search result data according to this embodiment. The interface control unit 310 may control the display of thumbnails based on information regarding similar intervals / similar regions.

[0114] Specifically, the interface control unit 310 may display frames containing similar intervals / similar regions as thumbnails. In this case, the user can intuitively judge the validity of the search by viewing the thumbnails.

[0115] Next, we will describe the search based on the capture of repeated operations according to this embodiment.

[0116] As described above, the system 1 according to this embodiment captures the user's actions and performs a search based on the generated captured data.

[0117] In this scenario, if the user performs an action only once, the accuracy of the search will depend on the accuracy of that single action. In other words, if the user fails to perform the intended action correctly on the first attempt, they may not obtain the desired search results.

[0118] Therefore, the search unit 320 according to this embodiment may search for motion data similar to the captured data based on the captured data, which is the user's actions that were repeatedly performed during the capture period.

[0119] In this way, by requesting users to repeat actions and capturing those repeated actions, it is expected that the accuracy of the user's actions will improve, and consequently, the accuracy of the search will also improve.

[0120] The search unit 320 may also perform searches based on operations that are repeatedly executed within a single capture period.

[0121] In this case, the search unit 320 may identify the temporal interval of the captured data to be used as the search key from the transition of the feature quantities of the captured data.

[0122] Specifically, if the feature quantities of the captured data change periodically, the search unit 320 may use the captured data trimmed to one period as the search key.

[0123] On the other hand, multiple capture periods may be provided according to this embodiment. In this case, the user repeats the operation for each capture period.

[0124] In this case, the search unit 320 according to this embodiment may perform the search based on the motion data captured in each of the multiple capture periods.

[0125] For example, if a fixed capture period is set for seven times, the search unit 320 may average the feature quantities of the seven motion data captured during each of the seven capture periods and perform a search based on the averaged feature quantities.

[0126] Furthermore, the search unit 320 according to this embodiment may repeatedly perform a search for motion data similar to the motion data captured for each of the multiple capture periods.

[0127] Furthermore, the number of capture periods according to this embodiment may be variable. In this case, the search unit 320 according to this embodiment may terminate the search based on the issued search termination trigger.

[0128] Figure 10 is a diagram illustrating the detection of a variable capture period and a search termination trigger according to this embodiment.

[0129] Figure 10 illustrates several capture periods 1 to 7. Each capture period is also accompanied by an interval.

[0130] As shown in Figure 10, the user performs the desired action that they want to obtain as a search result during each capture period.

[0131] In such cases, the accuracy of the user's actions is expected to improve with each repetition.

[0132] For example, in the initial stages (e.g., capture periods 1-3), there may be unnecessary actions due to trial and error, but the operation may stabilize afterward (e.g., capture periods 4-6).

[0133] Figure 11 shows an example of the transition of search results when a search is performed for each capture period. The interface control unit 310 according to this embodiment may control the presentation of acquired search results for each of the multiple capture periods.

[0134] The upper part of Figure 11 shows an example of how search results obtained in the initial period (e.g., capture periods 1 to 3) are displayed, while the lower part of Figure 11 shows an example of how search results obtained in the subsequent period (e.g., capture periods 4 to 6) are displayed.

[0135] As shown in Figure 11, when user behavior is captured and searches are performed at each capture period, it is expected that more search result data will be obtained as the user's behavior becomes more stable.

[0136] Furthermore, if a search is performed and search result data is presented at each capture period, the user may be able to instruct the system to end the search if they determine that sufficient search result data has been obtained.

[0137] The search unit 320 terminates the search based on the user's instruction to end the search, i.e., the detection of a trigger for ending the search.

[0138] For example, the user may be able to signal the end of the search by voice, a predetermined action, or other means.

[0139] Additionally, for example, the user may signal the end of the search by stopping the repeated operation.

[0140] In the example shown in Figure 10, the user did not perform the hand-raising action during capture period 7, which was repeatedly performed during capture periods 1 to 6.

[0141] In this case, the search unit 320 may detect that a deviation from the repetitive operation has occurred during the capture period 7, and may terminate the search by discarding the information acquired during the capture period 7.

[0142] The search unit 320 in this embodiment may detect a trigger for ending the search based on the similarity between repeatedly captured motion data.

[0143] More specifically, the search unit 320 may detect when the similarity between repeatedly captured motion data falls below a threshold, which can be used as a trigger to terminate the search.

[0144] Hereinafter, an example of the search flow for each of the multiple capture periods according to this embodiment will be described with reference to Figures 12 and 13.

[0145] In the example shown in Figure 12, the interface control unit 310 determines whether the capture period and interval have been set (S401).

[0146] If the interface control unit 310 determines that the capture period and interval have not been set (S401: NO), it displays the screen for setting the capture period and interval (S402) and prompts the user to set them.

[0147] Users may be able to set the capture period and interval depending on the action they are performing.

[0148] On the other hand, if the interface control unit 310 determines that the capture period and interval have been set (S401: YES), the search unit 320 determines whether or not a search start instruction has been detected (S403).

[0149] If no search start instruction is detected (S403: NO), the search unit 320 repeatedly performs the determination in step S403.

[0150] On the other hand, if a search start instruction is detected by the search unit 320 (S403: YES), it performs a repetitive process (S404).

[0151] Figure 13 is a flowchart showing an example of the iterative processing flow in step S404.

[0152] In the example shown in Figure 13, the search unit 320 first performs a search process (S501). The search process in step S501 may be equivalent to the processes in steps S102 to S105 shown in Figure 5.

[0153] Next, the search unit 320 performs a comparison of the captured data (S502).

[0154] For example, the search unit 320 may calculate the similarity by comparing the captured data obtained in the previous step S501 with the captured data obtained in the previous iteration.

[0155] Alternatively, for example, the search unit 320 may calculate the similarity by comparing the captured data acquired in the previous step S501 with the average of the captured data acquired in the previous iterations.

[0156] If this is the first iteration of the loop, the search unit 320 may skip step S502.

[0157] Next, the search unit 320 determines whether or not a search termination trigger has been detected (S503).

[0158] If the search unit 320 determines that no search termination event has been detected (S503: NO), it performs the search process (S504) and returns to step S501. Note that the search process in step S504 may be equivalent to the processes in steps S210 to S205 shown in Figure 6.

[0159] On the other hand, if the search unit 320 determines that a trigger for ending the search has been detected (S503: YES), it terminates the search.

[0160] As described above, the trigger for terminating the search according to this embodiment can be detected based on voice, a predetermined action, the similarity between captured data, etc.

[0161] For example, if the similarity calculated in step S502 falls below a threshold, the search unit 320 may determine that the user's actions have deviated from repetition and detect this as a trigger for terminating the search.

[0162] The above describes an example of the search flow for each of the multiple capture periods according to this embodiment. By following the processing described above, high-precision motion search can be achieved.

[0163] Furthermore, as mentioned above, when multiple capture periods are provided, the user can more clearly specify the search key compared to when the search unit 320 identifies an action to be used as a search key from an action that is repeatedly performed during a single capture period.

[0164] For example, suppose we want to use the action of a user waving their hand twice as the search key. In this case, we assume that there is one capture period, and during that capture period, the user repeats the action of waving their hand twice three times.

[0165] In this case, the search unit 320 may have difficulty determining whether the action of waving once was repeated six times, the action of waving twice was repeated three times, or the action of waving three times was repeated twice.

[0166] On the other hand, when multiple capture periods are set, users can clearly specify their search key by waving their hand twice during each capture period, allowing them to obtain search result data that better matches their intent.

[0167] Next, the display control of the captured data according to this embodiment will be described. The interface control unit 310 according to this embodiment may present the acquired captured data to the user in addition to the search result data.

[0168] Figure 14 is a diagram illustrating an example of displaying captured data according to this embodiment. Figure 14 shows an example of a capture result screen 70 that displays the captured data 71.

[0169] The capture results screen 70 may also display the captured data 71 and various buttons used for operations such as playing and stopping the captured data 71.

[0170] Furthermore, if multiple capture periods are provided, the capture results screen 70 may display multiple capture data 71 acquired during each capture period.

[0171] Furthermore, in this case, the user may specify the captured data 71 to be used as a search key.

[0172] Furthermore, the interface control unit 310 according to this embodiment may accept trimming of the captured data 71.

[0173] Figure 15 is a diagram illustrating an example of receiving trimming for the capture data 71 according to this embodiment.

[0174] As shown in Figure 15, the user may specify the range to be trimmed by entering a start time and an end time.

[0175] Furthermore, the user may specify the range to be trimmed by referring to the frame data 72 of the capture data 71 shown in chronological order.

[0176] In this embodiment, the search unit 320 performs trimming of the capture data 71 based on the range specified as described above, and performs a search based on the trimmed capture data 71.

[0177] The above processing makes it possible to obtain search result data that better matches the user's intent.

[0178] <2. Hardware Configuration Example> Next, a hardware configuration example common to the information processing terminal 20 and information processing server 30 according to one embodiment of this disclosure will be described.

[0179] Figure 16 is a block diagram showing an example of the hardware configuration of an information processing device 90 according to one embodiment of the present disclosure. The information processing device 90 may be a device having a hardware configuration equivalent to that of the above-described devices.

[0180] As shown in Figure 16, the information processing device 90 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, a storage device 880, a drive 881, a connection port 882, and a communication device 883. Note that the hardware configuration shown here is just an example, and some of the components may be omitted. Furthermore, the information processing device 90 may include components other than those shown here.

[0181] (Processor 871) The processor 871 functions, for example, as an arithmetic processing unit or a control unit, and controls the overall operation or part thereof of each component based on various programs recorded in the ROM 872, RAM 873, storage 880, or removable storage medium 901.

[0182] (ROM 872, RAM 873) ROM 872 is a means for storing programs loaded into the processor 871 and data used for calculations. RAM 873 temporarily or permanently stores, for example, programs loaded into the processor 871 and various parameters that change as needed when executing those programs.

[0183] (Host bus 874, bridge 875, external bus 876, interface 877) The processor 871, ROM 872, and RAM 873 are interconnected via, for example, the host bus 874, which is capable of high-speed data transmission. On the other hand, the host bus 874 is connected to the external bus 876, which has a relatively low data transmission speed, via, for example, the bridge 875. The external bus 876 is also connected to various components via the interface 877.

[0184] (Input device 878) The input device 878 may include, for example, a mouse, keyboard, touch panel, buttons, switches, and levers. Furthermore, the input device 878 may also include a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves. The input device 878 may also include an audio input device such as a microphone.

[0185] (Output device 879) The output device 879 is a device that can visually or audibly notify the user of acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. The output device 879 according to this disclosure also includes various vibration devices capable of outputting tactile stimuli.

[0186] (Storage 880) Storage 880 is a device for storing various types of data. Examples of storage devices used for storage 880 include magnetic storage devices such as hard disk drives (HDDs), semiconductor storage devices, optical storage devices, or magneto-optical storage devices.

[0187] (Drive 881) Drive 881 is a device that reads information recorded on a removable storage medium 901, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, or writes information to the removable storage medium 901.

[0188] (Removable storage medium 901) The removable storage medium 901 is, for example, DVD media, Blu-ray® media, HD DVD media, various semiconductor storage media, etc. Of course, the removable storage medium 901 may also be, for example, an IC card equipped with a contactless IC chip, or an electronic device, etc.

[0189] (Connection port 882) Connection port 882 is a port for connecting external devices 902, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal.

[0190] (External connected device 902) The external connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.

[0191] (Communication device 883) The communication device 883 is a communication device for connecting to a network, and is, for example, a communication card for wired or wireless LAN, Bluetooth®, or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0192] <3. Summary> As described above, System 1 according to one embodiment of the present disclosure includes a search unit 320 that searches for motion data similar to the user's motion data based on the user's motion data captured during a capture period determined by the capture start trigger and the capture end trigger.

[0193] The above configuration makes it possible to perform motion data searches that better match the user's intentions.

[0194] While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may be conceived within the scope of the technical ideas described in the claims, and these will naturally also fall within the technical scope of the present disclosure.

[0195] Furthermore, each step of the process described in this disclosure does not necessarily have to be processed chronologically in the order shown in the flowchart or sequence diagram. For example, each step of the process for each device may be processed in an order different from the order described, or may be processed in parallel.

[0196] Furthermore, the series of processes performed by each device described in this disclosure may be implemented by a program stored on a non-transitory computer-readable storage medium. Each program is, for example, loaded into RAM when executed by a computer and executed by a processor such as a CPU. The storage medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or flash memory. Alternatively, the program may be distributed without using a storage medium, for example, via a network.

[0197] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the technology relating to this disclosure may produce other effects that are obvious to those skilled in the art from the description herein, in addition to or instead of the effects described herein.

[0198] Furthermore, the following configurations also fall within the technical scope of this disclosure: (1) A system comprising: a search unit that searches for motion data similar to the motion data of a user based on the motion data of the user captured during a capture period determined by a capture start trigger and a capture end trigger. (2) The system according to (1), wherein the search unit searches for motion data similar to the motion data based on the motion data of the user's actions that were repeatedly performed during the capture period. (3) The system according to (2), wherein the search unit performs a search based on the motion data captured in each of the multiple capture periods. (4) The system according to (3), wherein the search unit repeatedly performs a search for motion data similar to the motion data based on the motion data captured for each of the multiple capture periods. (5) The system according to (4), wherein the search unit terminates the search based on a detected search end trigger. (6) The system according to (5), wherein the search end trigger is detected based on the similarity between repeatedly captured motion data. (7) The system according to (6), wherein the trigger for ending the search is that the similarity between repeatedly captured motion data falls below a threshold. (8) The system according to any one of (1) to (7), wherein the trigger for starting the capture and the trigger for ending the capture are detected based on a timer, voice command, or predetermined operation. (9) The system according to any one of (1) to (8), further comprising an interface control unit that controls the presentation of search results by the search unit. (10) The system according to (9), wherein the interface control unit highlights and indicates similar frame intervals with respect to captured motion data and motion data acquired as search results. (11) The system according to any one of (9) or (10), wherein the interface control unit highlights and indicates similar frame regions with respect to captured motion data and motion data acquired as search results.(12) The system according to (10) or (11), wherein the interface control unit controls the simultaneous playback of a plurality of motion data acquired as search results. (13) The system according to (9), wherein the interface control unit controls the presentation of acquired search results for each of the plurality of capture periods. (14) The system according to any one of (9) to (13), wherein the interface control unit controls the presentation of captured motion data. (15) The system according to any one of (9) to (14), wherein the interface control unit accepts trimming of captured motion data. (16) The system according to (15), wherein the search unit performs a search based on the trimmed motion data. (17) The system according to any one of (1) to (16), wherein the user's motion data is captured using an inertial sensor worn by the user. (18) An information processing method comprising: a processor searching for motion data similar to the user's motion data based on the user's motion data captured during a capture period determined by a capture start trigger and a capture end trigger. (19) A program that causes a computer to function as an information processing device, comprising: a search unit that searches for motion data similar to the user's motion data based on the user's motion data captured during a capture period determined by a capture start trigger and a capture end trigger.

[0199] 1 System 10 Capture System 15 Inertial Sensor 20 Information Processing Terminal 210 Control Unit 220 Operation Reception Unit 230 Display Unit 240 Audio Output Unit 250 Communication Unit 30 Information Processing Server 310 Interface Control Unit 320 Search Unit 330 Motion DB 340 Communication Unit

Claims

1. A system comprising: a search unit that searches for motion data similar to the user's motion data based on the user's motion data captured during a capture period determined by the capture start trigger and capture end trigger.

2. The system according to claim 1, wherein the search unit searches for motion data similar to the motion data captured based on the user's actions that were repeatedly performed during the capture period.

3. The system according to claim 2, wherein the search unit performs a search based on motion data captured in each of the plurality of capture periods.

4. The system according to claim 3, wherein the search unit repeatedly performs a search for motion data similar to the motion data captured for each of the multiple capture periods.

5. The system according to claim 4, wherein the search unit terminates the search based on the detected search termination trigger.

6. The system according to claim 5, wherein the trigger for terminating the search is detected based on the similarity between repeatedly captured motion data.

7. The system according to claim 6, wherein the trigger for terminating the search is that the similarity between repeatedly captured motion data falls below a threshold.

8. The system according to claim 1, wherein the capture start trigger and the capture end trigger are detected based on a timer, voice command, or predetermined operation.

9. The system according to claim 1, further comprising an interface control unit for controlling the presentation of search results by the search unit.

10. The system according to claim 9, wherein the interface control unit highlights and indicates similar frame intervals with respect to captured motion data and motion data obtained as search results.

11. The system according to claim 9, wherein the interface control unit highlights and displays similar frame regions with respect to captured motion data and motion data obtained as search results.

12. The system according to claim 10, wherein the interface control unit controls the simultaneous playback of a plurality of motion data obtained as search results.

13. The system according to claim 9, wherein the interface control unit controls the presentation of acquired search results for each of the plurality of capture periods.

14. The system according to claim 9, wherein the interface control unit controls the presentation of captured motion data.

15. The system according to claim 9, wherein the interface control unit accepts trimming of captured motion data.

16. The system according to claim 15, wherein the search unit performs a search based on trimmed motion data.

17. The system according to claim 1, wherein the user's motion data is captured using an inertial sensor worn by the user.

18. An information processing method comprising: a processor searching for motion data similar to the user's motion data based on the user's motion data captured during a capture period determined by a capture start trigger and a capture end trigger.

19. A program that causes a computer to function as an information processing device, comprising a search unit that searches for motion data similar to the user's motion data based on the user's motion data captured during a capture period determined by the capture start trigger and the capture end trigger.

Images