Neural Network Semantic Search for Response Relevance

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current systems face challenges in determining the context of user requests, often resulting in the return of irrelevant responses due to difficulties in understanding the semantic meaning behind the requests.

Innovation Solution

The implementation of a computing system that processes user inputs by mapping each word into a hidden semantic space using neural networks, such as recurrent neural networks (RNN) and long short-term memory (LSTM) cells, to determine a semantic representation, and then matches this representation with responses to provide semantically relevant outputs, regardless of language barriers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional category-based context matching is used, then the system is simple to operate, but the relevance of responses deteriorates due to difficulty in determining request context

Engineering Contradiction:
Improveresponse relevanceVSAvoidsemantic processing complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces traditional mechanical category-matching systems with neural network-based semantic processing. The system uses recurrent neural networks (RNNs) and long short-term memory (LSTMs) to automatically process and understand the semantic meaning of user requests, transforming the manual category-based approach into an intelligent, automated semantic analysis system that significantly improves response relevance.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent changes the fundamental parameter of context determination from categorical labels to continuous semantic representations. By encoding user requests and candidate responses as vectors in a high-dimensional semantic space, the system can perform sophisticated similarity calculations that capture nuanced meanings, thereby improving response relevance while managing complexity through learned parameter optimization.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If semantic representation matching is implemented, then response relevance is improved, but computational complexity increases due to neural network processing

Engineering Contradiction:
Improvesemantic similarity accuracyVSAvoidneural network processing complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the semantic processing task into distinct neural network components: embedding layers for word representation, recurrent neural networks for sequential processing, and long short-term memory units for capturing contextual dependencies. This segmentation allows each component to specialize in specific aspects of semantic understanding, improving measurement precision while making the overall system more manageable through modular architecture.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces semantic vector representations as intermediary structures between user input and response selection. These dense vector embeddings serve as mediators that capture the semantic meaning of requests and responses, enabling precise similarity calculations through vector operations. This intermediary layer significantly improves semantic similarity accuracy while the use of efficient vector operations helps manage computational complexity.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Measurement precision

If click-through data is used for training, then semantic matching accuracy is improved, but data processing requirements increase

Engineering Contradiction:
Improvesemantic matching accuracyVSAvoiddata processing volume
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent performs preliminary processing of click-through data during the training phase, where historical user interactions are used to pre-train the neural network models. The system learns semantic relationships and user preferences in advance, storing this knowledge in the trained model parameters. This preliminary action allows the system to achieve high semantic matching accuracy during deployment without requiring extensive real-time data processing, thereby reducing the data processing volume during operational use.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS10133729B2Semantically-relevant discovery of solutions
Publication Date: 2018.11.20 MICROSOFT TECHNOLOGY LICENSING LLC
  • US10133729B2 patent drawing
  • US10133729B2 patent drawing
  • US10133729B2 patent drawing

AI summary

Systems, methods, and computer-readable media for providing semantically-relevant discovery of solutions are described herein. In some examples, a computing device can receive an input, such as a query. The computing device can process each word of the input sequentially to determine a semantic representation of the input. Techniques and technologies described herein determine a response to the input, such as an answer, based on the semantic representation of the input matching a semantic representation of the response. An output including one or more relevant responses to the request can then be provided to the requestor. Example techniques described herein can apply machine learning to train a model with click-through data to provide semantically-relevant discovery of solutions. Example techniques described herein can apply recurrent neural networks (RNN) and/or long short term memory (LSTM) cells in the machine learning model.