Throwing Instrument Hit Location Estimation with Multimodal Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for estimating the location of a throwing instrument in a throwing game, such as darts, rely solely on image data, which limits accuracy due to the lack of consideration of physical phenomena like sounds and impacts upon contact.
Innovation Solution
A hit location estimation system that incorporates both image data and detection data, such as audio and impact waveforms, to improve estimation accuracy using a machine learning model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only image data is used as input to the machine learning model, then the system complexity is low, but the location estimation accuracy is insufficient
Solution Approach 1:
The patent combines multiple data sources (image data from cameras, audio data from microphones, and impact data from sensors) into a unified machine learning model input. This merging of diverse data modalities enables more accurate hit location estimation by leveraging complementary information from different sensing mechanisms, directly resolving the contradiction between accuracy and complexity.
Solution Approach 2:
The patent transitions from two-dimensional image data alone to multi-dimensional data integration by incorporating temporal (audio waveforms, impact timing) and spatial (sensor positions, impact locations) dimensions. This dimensional expansion allows the system to capture physical phenomena beyond visual information, improving accuracy while managing complexity through structured data fusion.
2Measurement precision
If multiple data modalities are integrated, then the location estimation accuracy improves, but the data processing complexity increases
Solution Approach 1:
The patent introduces a central control device that acts as an intermediary to receive, synchronize, and process data from multiple sources (image capturing devices, audio capturing devices, impact detection devices). This intermediary coordinates the integration of heterogeneous data formats and timing, managing processing complexity while enabling accurate multi-modal analysis.
Solution Approach 2:
The patent transforms diverse physical phenomena (light reflections, sound waves, impact forces) into standardized digital parameters that can be processed uniformly by the machine learning model. By converting different data modalities into compatible parameter formats, the system reduces processing difficulty while maintaining the benefits of multi-modal integration.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A location hit by a throwing instrument is estimated with higher accuracy. A hit location estimation system includes a storage unit that stores a machine-trained machine learning model configured to output, when image data generated on the basis of capturing of an image of a target upon throwing of a throwing instrument in a throwing game and detection data generated on the basis of detection of contact of the thrown throwing instrument with the target or another object is input to the machine learning model, a result of estimating a location on the target hit by the throwing instrument, an image capturing unit that generates the image data on the basis of the capturing of the image of the target upon the throwing of the throwing instrument in the throwing game, a detection unit that generates the detection data on the basis of the detection of the contact of the thrown throwing instrument with the target or the other object, an acquisition unit that acquires the result of estimating the location on the target hit by the throwing instrument, the result being output by inputting the image data and the detection data to the machine learning model, and a display unit that displays the estimation result.