Reinforcement Learning for Automated Negotiation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Automated agents interacting with users typically have a limited range of responses, leading to superficial interactions and missed opportunities for deeper, personalized engagement.

Innovation Solution

A system utilizing reinforcement learning (RL) techniques analyzes user profiles and historical data to determine desirable outcomes, constructs models for simulating interaction steps, and selects actions based on expected outcomes, refining actions in real-time to achieve desired results.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If automated agents use a limited range of predefined responses, then the system complexity is reduced and ease of operation is improved, but the depth and personalization of user interactions deteriorate

Engineering Contradiction:

Improveease of operationVSAvoidinteraction depth

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamic response generation where the automated agent adapts its responses in real-time based on user feedback and interaction history. The system transitions from static predefined responses to dynamic generated responses that evolve during the interaction, allowing the agent to personalize communications while maintaining operational simplicity through automated learning.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system employs self-service mechanisms where the automated agent autonomously learns from interactions and improves its response capabilities without human intervention. Through reinforcement learning, the agent automatically adjusts its behavior patterns, selects optimal responses, and personalizes interactions based on accumulated experience, eliminating the need for manual programming of each interaction scenario.

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

If automated agents use reinforcement learning to generate personalized responses, then interaction depth and personalization are improved, but the computational complexity and resource requirements increase

Engineering Contradiction:

Improvepersonalization capabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the reinforcement learning system into distinct functional modules: a profile analysis module that processes user data, a response generation module that creates personalized responses, and a feedback processing module that updates the learning model. This modular architecture reduces overall system complexity by allowing each component to be developed, tested, and optimized independently while maintaining clear interfaces between them.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by pre-processing user profiles and historical data before interactions occur. The reinforcement learning model is trained in advance on accumulated user data, and response options are pre-generated based on predicted user preferences. This preliminary preparation reduces the computational burden during actual interactions, balancing personalization capability with system complexity.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If the system performs real-time simulation and selection of multiple possible actions, then the quality of interaction outcomes is improved, but the processing time and computational resources increase

Engineering Contradiction:

Improveoutcome qualityVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent implements partial action by selecting a limited subset of the most promising responses for simulation and evaluation rather than exhaustively analyzing all possible responses. The reinforcement learning model ranks potential responses and selects only the top candidates for detailed simulation, achieving high outcome quality while significantly reducing processing time compared to complete enumeration of all possibilities.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20230325845A1Reinforcement learning for automated individualized negotiation and interaction

Publication Date: 2023.10.12 AT&T INTELLECTUAL PROPERTY I L P

AI summary

Aspects of the subject disclosure may include, for example, a method in which a processing system analyzes data including a user profile and historical data relating to previous interactions between an automated agent and equipment of the user. The method also includes determining a desirable outcome of an interaction between the automated agent and the user equipment; constructing a model for generating an expected outcome of a step of the interaction; using the model to perform a simulation of a next step of the interaction by generating an expected outcome for each of a plurality of possible actions, resulting in a plurality of expected outcomes; and selecting a next action for the next step of the interaction. If the desirable outcome is not obtained, the system can refine the plurality of possible actions to perform a simulation of a subsequent step of the interaction. Other embodiments are disclosed.