Sports Video Scene Estimation With Salience-Weighted Frame Groups

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing machine learning models for estimating scene types in sports match videos, such as those using recurrent neural networks (RNNs), are prone to gradient vanishing during learning due to sequential processing of consecutive data.

Innovation Solution

A computer vision system employing a machine learning model that acquires and processes consecutive image data by calculating salience degrees and weighting features to reduce gradient vanishing, allowing separate determinations before and after events in sports match videos.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If RNN is used to estimate scene type by sequentially processing consecutive data, then correlation between pieces of image data can be considered, but gradient vanishing occurs during learning

Engineering Contradiction:

Improveestimation accuracyVSAvoidgradient vanishing

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The consecutive image data is divided into multiple groups, with each group containing a plurality of pieces of consecutive image data. This segmentation allows the model to process groups independently while still capturing temporal correlations within each group, thereby reducing the propagation of gradient vanishing issues across the entire sequence.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a group dimension in addition to the temporal dimension of consecutive image data. By organizing data into groups and processing them separately, the model adds a structural dimension that breaks the continuous temporal dependency chain, reducing gradient vanishing while preserving essential correlations through group-level features.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Device complexity

If only image data of ball being kicked toward goal is used to estimate goal scene, then processing is simplified, but estimation result is likely to be incorrect

Engineering Contradiction:

Improveprocessing complexityVSAvoidestimation accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The model processes more image data than the minimal single frame by incorporating multiple consecutive image data within groups. This partial excessive action captures additional contextual information (such as ball trajectory, player positions, and goal net movement) that improves estimation accuracy without requiring the full complexity of processing all possible video data.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The model performs preliminary processing by extracting features from multiple consecutive image data frames before making the final scene type estimation. This preliminary action of gathering and processing multiple frames in advance allows the model to make more accurate determinations based on accumulated evidence rather than single-frame snapshots.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12555356B2Computer vision system, computer vision method, computer vision program, and learning method

Publication Date: 2026.02.17 RAKUTEN GROUP INC

AI summary

A computer vision system, with at least one processor configured to: acquire, from a sports match video, a plurality of pieces of consecutive image data indicating a portion of the sports match video, the plurality of pieces of consecutive image data including a plurality of pieces of first consecutive image data that are consecutive; and execute an estimation, by using a machine learning model, of whether the portion is of a predetermined scene type.