Computer Vision Scene Estimation Using Dual Machine Learning Models

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing computer vision systems for estimating scene types in sports match videos face accuracy issues due to the uniform length of training periods, which does not account for the varying lengths of different scene types, such as goal scenes and corner kick scenes, leading to decreased estimation accuracy.

Innovation Solution

A computer vision system that utilizes two machine learning models, one trained for shorter and one for longer scene types, to generate estimation data, which is then used by a determination data generation unit to accurately classify the scene type based on the output from both models, taking into account the specific length of each scene type.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single machine learning model is trained using moving images of periods having a uniform length, then the training process is simplified, but the estimation accuracy decreases because the varying lengths of different scene types are not accounted for

Engineering Contradiction:
Improvetraining process complexityVSAvoidscene type estimation accuracy
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The patent divides the estimation task into multiple segments by creating separate machine learning models for different scene type lengths. Instead of using a single uniform model, the system segments the problem into: (1) a first machine learning model for short scene types (e.g., goal scenes), (2) a second machine learning model for long scene types (e.g., corner kick scenes), and (3) a determination unit that selects the appropriate model based on scene characteristics. This segmentation resolves the contradiction by maintaining training simplicity through dedicated models while improving accuracy through length-specific training data.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies local quality by training different machine learning models with different characteristics suited to specific scene type lengths. The first machine learning model is trained on short period data with specific feature extraction parameters, while the second machine learning model is trained on long period data with different parameters. This allows each model to have optimized local quality for its specific scene type, resolving the contradiction between uniform training simplicity and accurate local estimation.

Inventive Principle:
Principle #3Local quality

2Measurement precision

If the length of each scene type is taken into account during estimation, then the scene type identification accuracy improves, but the system complexity increases due to multiple machine learning models being required

Engineering Contradiction:
Improvescene type identification accuracyVSAvoidsystem complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the estimation system into multiple specialized machine learning models, each handling specific scene type lengths. The determination unit then segments the classification decision by selecting the appropriate model based on detected scene characteristics. This segmentation approach improves accuracy by dedicating specific models to specific tasks while managing complexity through a clear hierarchical structure.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces dynamics by making the model selection process adaptive rather than static. The determination unit dynamically selects which machine learning model to use based on the detected scene characteristics and length. This dynamic approach allows the system to optimize accuracy for each specific scene type while managing overall system complexity through intelligent model routing rather than requiring all models to handle all cases.

Inventive Principle:
Principle #15Dynamics

3Measurement precision

If multiple machine learning models trained for different scene lengths are used, then the accuracy of scene type estimation improves, but the training data requirements and processing time increase

Engineering Contradiction:
Improvescene type estimation accuracyVSAvoidtraining time and data processing load
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent segments the training process into separate, focused tasks for different scene type lengths. Each machine learning model is trained independently on specialized data, which allows for more efficient training compared to a single model trying to handle all scene types. The determination unit then segments the inference process by selecting the appropriate pre-trained model, reducing real-time processing time.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies preliminary action by pre-training multiple machine learning models with different length specifications before actual estimation is needed. This allows the system to prepare specialized models in advance, so that during actual scene type estimation, the system only needs to select the appropriate pre-trained model rather than training new models, significantly reducing inference time while maintaining high accuracy.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS20240362915A1Computer vision system, computer vision method and computer vision program
Publication Date: 2024.10.31 RAKUTEN GROUP INC
  • US20240362915A1 patent drawing
  • US20240362915A1 patent drawing
  • US20240362915A1 patent drawing

AI summary

A computer vision system, with at least one processor configured to: acquire, from a sports match video, moving image data of a first period and moving image data of a second period; by using a first machine learning model, generate, based on the moving image data of the first period, first estimation data and second estimation data for an estimation period; by using a second machine learning model, generate, based on the moving image data of the second period, the first estimation data and the second estimation data for the estimation period; and generate determination data based on the first estimation data and the second estimation data that are output from the first machine learning model and the first estimation data and the second estimation data that are output from the second machine learning model.