Kernel Memory Buffer Page Fault Segmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Modern computer systems are vulnerable to malicious attacks that can stall the kernel by intentionally triggering page faults, leading to denial-of-service situations, as malicious software exploits system calls to access restricted memory areas.

Innovation Solution

The kernel generates a memory buffer in user space for system calls, validates its size and address range, maps a portion of RAM to this buffer, and disables page fault handling within this range, preventing malicious software from stalling the kernel by ensuring valid system calls do not require page faults to access data from a hard disk.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If the kernel allows user space processes to access memory resources through system calls, then system functionality and user space operation are improved, but the kernel becomes vulnerable to stalling attacks via intentional page fault triggering

Engineering Contradiction:
Improvesystem call functionalityVSAvoidkernel responsiveness
Core Design Contradiction:
Ease of operationVSReliability

Solution Approach 1:

The patent segments the address space into distinct regions with different page fault handling behaviors. A first address space region is designated where page faults are handled normally, while a second address space region is designated where page faults are not handled. This segmentation allows the kernel to maintain normal system call functionality in the first region while protecting against stalling attacks in the second region by preventing page fault-triggered stalls.

Inventive Principle:
Principle #1Segmentation

2Adaptability or versatility

If the kernel handles page faults by retrieving data from hard disk, then memory access flexibility is improved, but malicious software can exploit this to stall the kernel

Engineering Contradiction:
Improvememory access flexibilityVSAvoidkernel stalling vulnerability
Core Design Contradiction:
Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The patent applies local quality by making page fault handling behavior location-dependent within the address space. In the first address space region, page faults are handled with full flexibility including retrieving data from hard disk. In the second address space region, page fault handling is disabled or restricted, preventing malicious software from exploiting the flexible page fault mechanism to stall the kernel. This local differentiation maintains versatility where needed while eliminating vulnerability in critical areas.

Inventive Principle:
Principle #3Local quality

Data Source

PatentUS11586727B2Systems and methods for preventing kernel stalling attacks
Publication Date: 2023.02.21 RED HAT LLC
  • US11586727B2 patent drawing
  • US11586727B2 patent drawing
  • US11586727B2 patent drawing

AI summary

Systems and methods for preventing kernel stalling attacks. An example method may comprise receiving, by a kernel, an address range associated with a data store of an application program; mapping, by the kernel, a portion of random access memory (RAM) to the address range; disabling page fault handling with respect to addresses falling within the address range; and responsive to receiving, from the application program, a memory access request specifying an address outside of the address range, returning a memory access error to the application program.