Methods, devices, and systems for efficient multinomial distribution sampling
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-02-05
- Publication Date
- 2026-08-13
Smart Images
Figure EP2026053015_13082026_PF_FP_ABST
Abstract
Description
[0001] METHODS, DEVICES, AND SYSTEMS FOR EFFICIENT MULTINOMIAL DISTRIBUTION SAMPLING BACKGROUND
[0002] Multinomial sampling represents a cornerstone operation in modern computational systems, particularly within artificial intelligence (Al) and machine learning (ML) frameworks, telecommunications systems, image processing pipelines, quantum communication systems, and industrial control applications, that is to say, within Al and ML modules or cores, network processing units, signal processing systems, and distributed sensor networks. This fundamental statistical operation underlies critical processes in probabilistic modeling, statistical simulations, and decision-making systems. The growing sophistication of Al applications has elevated efficient multinomial sampling from a desirable optimization to an essential requirement for system performance.
[0003] Multinomial sampling extends beyond AI / ML into diverse technical domains. Telecommunications systems employ sampling for adaptive modulation schemes, quality-of-service packet scheduling in network routers, and load balancing across transmission paths. Image processing pipelines utilize sampling for adaptive filtering, multi-camera sensor fusion, and real-time color space transformations. Industrial control systems leverage sampling for multi-sensor data fusion, multi-criteria decision making in process control, and distributed fault detection. In quantum communication systems, particularly Quantum Key Distribution (QKD) protocols, sampling operations determine basis selections and state preparations that directly impact secure key generation rates, where entropy efficiency and unpredictability constitute fundamental security requirements.
[0004] In contemporary Al deployments, the impact of sampling efficiency manifests directly in system performance metrics. Deep learning models, especially in reinforcement learning contexts, demonstrate a 20-30% reduction in training time when equipped with optimized sampling mechanisms. This improvement stems from the high-frequency sampling operations required during training, where probabilistic actions are sampled millions of times per iteration. Similarly, generative Al systems, which rely on rapid sampling for real-time content generation, show marked improvements in user interaction latency when sampling operations are optimized. Operational data from large-scale deployments indicates that optimized multinomial sampling can enhance Graphics Processing Unit (GPU) utilization by approximately 15%, enabling concurrent operation of multiple models on existing hardware infrastructure.Traditional approaches to multinomial sampling exhibit several fundamental limitations that impede their effectiveness in modern applications. The prevalent Inverse Cumulative Distribution Function (CDF) method necessitates computationally expensive sorting and searching operations, creating performance bottlenecks that scale poorly with distribution dimensionality. These methods become particularly problematic in scenarios requiring dynamic probability updates or real-time processing. Hierarchical sampling techniques, while theoretically elegant, introduce additional latency in decision tree traversal and probabilistic graphical model operations, rendering them suboptimal for contemporary low-latency applications.
[0005] The challenge of unnormalized probabilities presents another significant limitation in practical implementations. Current methods typically assume normalized probability distributions, an assumption that rarely holds in real-world applications. The requisite normalization preprocessing introduces substantial computational overhead, particularly in high-dimensional spaces or scenarios with dynamically changing probabilities. This limitation becomes acute in reinforcement learning policies and probabilistic graphical models, where probability distributions evolve continuously. Furthermore, the normalization process can introduce numerical instability when scaling factors vary significantly across dimensions.
[0006] Parallel processing architectures, including GPUs and Tensor Processing Units (TPUs), present both opportunities and challenges for multinomial sampling implementation. While these platforms excel at parallel computations, traditional sampling methods often contain sequential dependencies that inhibit effective parallelization. This architectural mismatch becomes particularly evident in distributed Al training and inference tasks, where sampling operations can create processing bottlenecks. Hybrid systems incorporating Field-Programmable Gate Arrays (FPGAs) and GPUs encounter additional complexity due to disparate processing paradigms, necessitating careful optimization to maintain system efficiency.
[0007] The limitations of current sampling methods become particularly apparent in time-sensitive applications. Autonomous vehicles requiring rapid probabilistic decision-making, real-time critical applications, and healthcare diagnostic tools all demand sampling operations that complete within strict temporal constraints. Alternative approaches, such as sampling from non-Gumbel distributions (including Weibull or Cauchy distributions), have proven inadequate in matching the precision and scalability requirements of these applications.
[0008] Quantum Key Distribution (QKD) systems present a particularly compelling application domain for efficient multinomial sampling. In QKD protocols such as BB84, bases and states must beselected according to specific probability distributions, typically involving small categorical sets of 2-4 elements. The security guarantees of QKD fundamentally depend on the unpredictability of these selections, necessitating high-quality random number sources. Furthermore, the highspeed operation of modern QKD systems, operating at gigahertz rates, creates stringent requirements for sampling throughput while simultaneously demanding minimal entropy consumption per sample. Traditional sampling methods that require multiple random bits per sample become prohibitively expensive when the random source is a certified Quantum Random Number Generator (QRNG), where each bit represents a precious resource derived from quantum physical processes.
[0009] The intersection of these challenges creates a clear need for advanced multinomial sampling methods that address multiple constraints simultaneously. These methods must efficiently handle unnormalized probabilities, support parallel processing architectures, and maintain accuracy while operating under strict temporal constraints. The solution to these challenges would significantly impact the broader field of Al and ML, enabling more responsive and scalable intelligent systems across diverse applications.
[0010] Contemporary research has explored various approaches to address these limitations, including modified tree-based sampling methods and hybrid hardware-software solutions. However, these attempts have not fully resolved the fundamental tension between sampling efficiency and implementation practicality. The persistence of these challenges, coupled with the growing demands of modern Al systems, underscores the need for innovative approaches to multinomial sampling that can overcome these established limitations while maintaining computational efficiency and numerical stability.
[0011] SUMMARY
[0012] The present disclosure relates to methods, devices and systems for multinomial distribution sampling, leveraging adaptive transformation frameworks (i.e., modules or cores) and hardware-optimized implementations to achieve efficient, parallel sampling operations with reduced computational overhead and improved numerical stability.
[0013] A device or system that performs multinomial distribution sampling, comprising at least one processing unit, at least one transformation framework, and at least one memory unit. The transformation framework is configured to efficiently transform input random variables into output samples following a specified multinomial distribution through adaptive reparametrization techniques, and the processing unit is configured to execute the transformation operations with hardware utilization.For the sake of simplicity and clarity, embodiments of the disclosure will be simply described hereinafter as a device, although a device as described herein includes also systems comprising for instance two or more elements, apparatuses, or devices (these terms being used interchangeably), those two or more elements being functionally related to each other.
[0014] A transformation framework, according to the present disclosure, refers to one or more functional and structural components implemented in hardware, software, firmware, or combinations thereof. In embodiments in which the transformation framework refers to multiple components, i.e., more than one component, these components are configured to work together to transform input random variables into multinomial samples according to specified probability distributions.
[0015] In some embodiments, the at least one transformation framework comprises one or more FPGAs, and / or one or more ASICs, and / or one or more CPUs, and / or one or more GPUs.
[0016] In the following, even if referring to “the transformation framework”, it will be understood that said term refers to one, some or all transformation frameworks of the at least one transformation network of the device or system.
[0017] In some embodiments, the transformation framework is configured to utilize one or more processing components that perform the computational operations required for sampling. These processing components may include arithmetic logic units (ALUs) configured for logarithmic operations, multiplexers for data path selection, comparators for argmax and decision operations, accumulators for probability sum tracking, and random number interface circuits. Configuration and interconnection of these one or more processing components may be dependent on the particular embodiment and target hardware platform.
[0018] In some embodiments, the transformation framework is configured to utilize one or more memory components that store intermediate values, lookup tables, and configuration parameters. These memory components may include lookup tables (LUTs) for logarithm approximations, registers for intermediate value storage, buffers for probability value caching, and configuration registers for operational parameters. The memory architecture is optimized based on the access patterns and throughput requirements of the specific implementation.
[0019] In some embodiments, control logic within the transformation framework is configured to coordinate the operation of the processing and memory components. In some embodiments, the transformation framework comprises a prefactor computation unit configured to generate scaling factors based on desired output distribution properties, a multiplication unit configured to apply scaling factors to transformed random values, and an output unit configured toproduce scaled transformed random values suitable for comparison operations. These units may be implemented as dedicated hardware blocks in some embodiments in which the transformation framework comprises one or more FPGAs and / or the transformation framework comprises one or more ASICs, as compiled software functions in some embodiments in which the transformation framework comprises one or more CPUs, or as shader operations in some embodiments in which the transformation framework comprises one or more GPUs. The specific implementation of these units depends on the target hardware platform and application requirements. This control logic may implement state machines coordinating sampling stages, pipeline control for sequential operations, synchronization logic for parallel sampling paths, and mode selection logic that switches between standard and tail sampling approaches. The control logic ensures proper sequencing of operations and maintains data coherency across multiple processing stages.
[0020] In some embodiments in which the transformation framework comprises one or more FPGAs, the transformation framework comprises logic blocks implementing binary logarithm operations, block RAM configured as lookup tables for storing logarithm approximations, digital signal processing (DSP) slices performing multiplication operations, and distributed registers forming pipeline stages that enable high-throughput operation. The FPGA fabric allows flexible configuration of these components to match specific application requirements, enabling trade-offs between resource utilization, throughput, and precision.
[0021] In some embodiments in which the transformation framework comprises one or more ASICs, the transformation framework comprises logarithm approximation circuits, preferably dedicated logarithm approximation circuits, that are preferably optimized for the specific bit widths and precision predetermined requirements of the application, multiplier blocks, preferably hardened multiplier blocks, that provide high-speed arithmetic operations with minimal power consumption, comparison trees implementing argmax operations, and Random Access Memory (RAM), preferably Static RAM (SRAM), preferably on-chip, for probability storage that provides low-latency access to distribution parameters. An ASIC implementation may enable maximum performance and energy efficiency for high-volume applications.
[0022] In some embodiments in which the transformation framework comprises one or more GPUs, the transformation framework comprises shader programs implementing logarithmic transformations that execute on streaming multiprocessors, thread blocks executing parallel Bernoulli machines, shared memory implementing accumulator operations that enable efficient communication between threads, and texture units providing accelerated access tolookup tables. The GPU architecture naturally supports the parallel processing patterns required for high-dimensional sampling operations.
[0023] In some embodiments in which the transformation framework comprises one or more CPUs and / or software, the transformation framework comprises compiled functions configured to implement cascaded logarithm operations, vector instructions such as, e.g., AVX-512 or ARM NEON that enable parallel processing of multiple probability values, cache-optimized data structures for probability arrays that minimize memory access latency, and operating system threads that distribute sampling workload across multiple processor cores. Software implementations provide high or maximum flexibility and ease of deployment across diverse computing platforms.
[0024] In some embodiments, the transformation framework is configured to interface with components or apparatuses external to the device or system, and / or that are part of the device or system, through defined input and output ports. Input ports may receive probability values that may be normalized or unnormalized, random data from either hardware or software random number generators, and control signals specifying operational parameters such as sampling mode or precision requirements. Output ports may provide sampled indices or values, probability sums for use in hierarchical sampling structures, and status signals indicating sampling completion or error conditions.
[0025] In some embodiments, the transformation framework implements, and is thus configured to provide, hierarchical sampling techniques optimized for specific hardware architectures, comprising at least one hierarchical decomposition module configured to break down highdimensional sampling operations into efficient sub-operations.
[0026] In some embodiments, the transformation framework employs, and is thus configured to use, hardware-specific optimizations including, but not limited to, one, a combination of or all of: binary logarithmic transformations for digital circuits, piecewise linear approximations for reduced memory footprint, custom fixed-point arithmetic for improved throughput, low-precision number representations for enhanced parallelism, and hybrid approaches combining multiple optimization techniques based on operational requirements.
[0027] In some embodiments, the transformation framework utilizes, and is thus configured to use, advanced probability transformations based on extreme value theory, including but not limited to Gumbel, Frechet, and Weibull distributions. A specific transformation is, in some embodiments, dynamically selected based on input distribution characteristics and sampling requirements.In some embodiments, the transformation framework implements, and is thus configured to provide, one or more parallel sampling architectures capable of simultaneously generating multiple samples from either distinct multinomial distributions or different segments of a single large-scale distribution, enabling efficient scaling for high-dimensional sampling spaces.
[0028] In some embodiments, the transformation framework includes one or more adaptive preprocessing modules configured to handle unnormalized probability distributions, preferably directly, eliminating the computational overhead of explicit normalization steps while maintaining sampling accuracy.
[0029] It is an object of at least some embodiments of this disclosure to provide a computationally efficient framework for multinomial sampling that reduces processing overhead by at least 20% compared to traditional methods while maintaining sampling accuracy within specified error bounds. For example, this improvement is achieved, partially or completely, through binary logarithm operations that align naturally with digital hardware architectures, eliminating the computational overhead of natural logarithm calculations and enabling efficient implementation using bit manipulation operations in FPGAs and ASICs.
[0030] It is an object of at least some embodiments of this disclosure to enable real-time multinomial sampling operations in latency-sensitive applications through hardware-optimized implementations that preferably achieve sub-millisecond sampling times for high-dimensional distributions. For example, parallel processing capabilities enabled by the hierarchical decomposition architecture allow multiple probability pairs to be processed simultaneously, while pipelined data paths eliminate idle cycles between successive sampling operations.
[0031] It is an object of at least some embodiments of this disclosure to support parallel sampling operations that scale efficiently with distribution dimensionality, enabling simultaneous sampling from multiple distributions or distribution segments while maintaining numerical stability and accuracy. For example, a cascaded Bernoulli machine architecture progressively reduces problem dimensionality while maintaining statistical correctness, with each processing stage operating independently to maximize throughput in parallel processing environments.
[0032] It is an object of at least some embodiments of this disclosure to provide entropy-efficient sampling methods suitable for quantum communication systems, wherein the sampling operations consume minimal random bits per sample while maintaining the unpredictability requirements essential for cryptographic security. The binary decision tree structure enables four-way selection using only two binary sampling stages, minimizing entropy consumptionfrom resource-constrained Random Number Generators. For a given unpredictability requirement or set of security requirements, sampling operations by devices and systems according to some embodiments consume fewer random bits per sample than devices providing sampling operations as known in the art, thereby enabling higher key generation rates in Quantum Key Distribution systems where RNG throughput directly limits system performance.
[0033] BRIEF DESCRIPTION OF THE DRAWINGS
[0034] To complete the description and to provide a better understanding of the disclosure, a set of drawings is provided. Said drawings form an integral part of the description and illustrate embodiments of the disclosure, which should not be interpreted as restricting the scope of the disclosure but just as examples of how the disclosure can be carried out. The drawings comprise the following figures:
[0035] Figure 1 shows a structure of a sampling system, in accordance with some embodiments, that transforms input probabilities and random numbers into sampled output data through a central transformation core.
[0036] Figure 2 shows a binary choice sampling system, in accordance with some embodiments, that processes two unnormalized probabilities and uses a binomial decision maker to generate both summed probabilities and final choices.
[0037] Figure 3 shows a binary choice architecture by sampling from a set of sampled elements in accordance with some embodiments.
[0038] Figure 4 shows a Gumbel-based sampling approach, in accordance with some embodiments, that combines Gumbel distribution sampling with accumulation and maximum-finding operations to generate optimized choices.
[0039] Figure 5 shows a hardware-efficient implementation for generating Gumbel-distributed samples using binary base-2 logarithm operations and scaling factors in accordance with some embodiments.
[0040] Figure 6 shows a Gumbel sampling architecture incorporating tail sampling capabilities, allowing for more accurate sampling in the distribution's tail regions in accordance with some embodiments.
[0041] Figure 7 shows multiple binary choice modules combined hierarchically, in accordance with some embodiments, to handle larger choice sets while maintaining computational efficiency.Figure 8 shows a parallel, pipelined processing approach that uses cascaded Bernoulli machines together with Gumbel distribution sampling to handle high-dimensional input efficiently in accordance with some embodiments.
[0042] DETAILED DESCRIPTION
[0043] Figure 1 shows a general scheme of operation of a device or system, in accordance with some embodiments, that enables efficient multinomial sampling. The device or system comprises a transformation core 101, distribution probabilities 102, e.g., stored in at least one memory of the device or system, a randomness source 103, and at least one sampled data 104, e.g., stored in at least one memory of the device or system.
[0044] In some embodiments, the transformation core 101 comprises at least one processing unit configured to transform input random variables into output samples. In some embodiments, the at least one processing unit is a general-purpose processor, such as a CPU or GPU. In some embodiments, the at least one processing unit is a specialized hardware accelerator, such as an FPGA, an Application Specific Integrated Circuit (ASIC), or other custom silicon. In some embodiments, the at least one processing unit comprises multiple processing elements configured to work in parallel or in a pipelined fashion.
[0045] In some embodiments, the distribution probabilities 102 comprise a set of unnormalized probability values. In some embodiments, these probabilities are represented in floating-point format. In some embodiments, these probabilities are represented in fixed-point format for hardware efficiency. In some embodiments, these probabilities are dynamically updated during system operation.
[0046] In some embodiments, the randomness source 103 provides random or pseudo-random numbers to the transformation core 101. In some embodiments, the source 103 comprises a hardware random number generator. In some embodiments, the source 103 comprises a pseudo-random number generator. In some embodiments, the source 103 combines multiple random sources to improve randomness quality or generation speed. In some embodiments, the randomness source 103 is configured to provide the random numbers uniformly distributed. In some embodiments, the random numbers follow other distributions to optimize the sampling process.
[0047] The sampled data 104 represents the output of the transformation process. In some embodiments, the output of the sampled data 104 comprises indices indicating selected choices from the input distribution. In some embodiments, the output of the sampled data 104 comprises probability values associated with the selected choices. In some embodiments, theoutput of the sampled data 104 includes additional metadata about the sampling process, such as confidence scores or sampling statistics.
[0048] In some embodiments, the transformation core 101 implements, and is thus configured to provide, adaptive processing based on the characteristics of the input probabilities 102 or requirements of the sampling task, e.g., predetermined requirements associated with a sampling task. For example, the transformation core 101 is configured to switch between different sampling algorithms depending on the distribution's dimensionality or sparsity. In some embodiments, the transformation core 101 includes, and is thus configured to provide, error detection and correction mechanisms to ensure sampling accuracy.
[0049] In some embodiments, the transformation core 101 implements a transformation framework utilizing one or more of the architectural components previously described. As aforementioned, in some embodiments, the specific implementation depends on the target hardware platform and application requirements.
[0050] In some embodiments, the transformation core 101 is implemented as an FPGA fabric containing logarithm approximation lookup tables, multiplication units, and comparator trees, which may be configured according to the architecture detailed in Figure 5, for example.
[0051] In some embodiments, the transformation core 101 is implemented as a GPU shader program with thread blocks executing parallel Bernoulli sampling operations, which may be configured according to the architecture detailed in Figure 8, for example.
[0052] In some embodiments, the transformation core 101 is implemented as an ASIC with hardened arithmetic circuits optimized for binary logarithm operations, providing high or maximum performance and energy efficiency. In some embodiments, the transformation core 101 is implemented as software executing on a general-purpose processor, utilizing vector instructions to achieve parallel processing of probability values.
[0053] In some embodiments, the transformation core 101 utilizes control logic that coordinates data flow between the probability inputs 102, the randomness source 103, and the sampling output 104. This control logic may implement any one of the sampling methods described in Figures 2 through 8, selecting processing paths based on input characteristics and operational requirements. In some embodiments, the control logic dynamically switches between different sampling approaches, such as standard Gumbel sampling and tail sampling, based on the properties of the input probability distribution. In some embodiments, the control logic implements pipelining to enable continuous processing of sampling requests with minimal latency between successive samples.In some embodiments, the transformation core 101 implements different levels of parallelism depending on the embodiment. In some embodiments, the transformation core 101 processes a single sampling request at a time using sequential operations. In some embodiments, the transformation core 101 implements parallel processing of multiple probability pairs simultaneously using, for example, the hierarchical architecture shown in Figure 7. In some embodiments, the transformation core implements pipelined parallel processing, for example as shown in Figure 8, where multiple stages operate concurrently on different portions of the sampling problem. The degree of parallelism affects both the throughput and the hardware resource requirements of the implementation.
[0054] Figure 2 illustrates a device or system implementing a binary choice sampling architecture, which represents a building block for efficient multinomial sampling. This architecture processes two (potentially unnormalized) probability values pa201 and pb202 along with random data 205 to generate both a probability sum 207 and a binary choice output 208.
[0055] The device implements, for instance with a transformation core thereof (such as, e.g., the transformation core 101 shown in Figure 1), a binary sampling by combining addition operations 203 with inverse transformations 204 and a specialized binomial decision maker 206. In some embodiments, the unnormalized probability values pa201 and pb202 represent logits from a neural network output layer. In some embodiments, these values represent unnormalized probabilities from a statistical model. In some embodiments, these values are derived from signal strength measurements in telecommunications systems, pixel intensity values in image processing applications, or sensor confidence scores in multi-sensor fusion systems.
[0056] In some embodiments, the transformation core of the device implements the addition operation 203 using floating-point arithmetic units. In some embodiments, the transformation core performs the addition using fixed-point arithmetic for improved hardware efficiency. In some embodiments, the transformation core provides preprocessing steps such as bit alignment or normalization as part of the addition operation. In some embodiments, the transformation core incorporates error-checking mechanisms to detect numerical overflow or underflow conditions during the addition operation.
[0057] The inverse operation 204 transforms the summed probability into a scale factor for the binomial decision maker 206. In some embodiments, the transformation core implements the inverse operation 204 as a division operation. In some embodiments, the transformation core uses look-up tables to approximate the inverse function. In some embodiments, the transformation core employs iterative methods such as Newton-Raphson approximation. Insome embodiments, the inverse operation is combined with additional transformations to improve numerical stability or computational efficiency. In some embodiments, the device or system does not provide the inverse operation 204 or bypasses the inverse operation 204 when inputs to the inverse operation 204 are known to add up to 1.
[0058] The binomial decision maker 206 converts random input data into a binary choice based on the processed probability values. In some embodiments, the decision maker 206 implements a threshold comparison between the random data and the normalized value coming from the inverse operation 204. In some embodiments, the decision maker 206 employs an entropyefficient binary decision process, for example, but without limitation, a binary decision process as disclosed in European patent No. 4235400B1.
[0059] The device or system produces two outputs: a sum of the input probabilities 207 and a binary choice 208. In some embodiments, the probability sum is used for normalization in subsequent processing stages. In some embodiments, the binary choice output includes additional metadata such as confidence scores or statistical measures. In some embodiments, the system maintains historical data about previous choices to enable temporal consistency in sequential sampling tasks.
[0060] Figure 3 shows a binary choice architecture as shown, for example, in the embodiments of Figure 2, and which includes selection values sa308 and sb309 that represent the actual elements being sampled from the distribution. Such architecture enables the device or system to not only perform the sampling process but also directly output the sampled elements.
[0061] As in the previous figure, the device or system is configured to process two unnormalized probability values pa301 and pb302 to determine sampling probabilities. However, in this architecture, instead of just indicating which probability was selected, there is an output of the actual sampled element from the distribution.
[0062] In some embodiments, the values saand sbrepresent indices into an array or database containing the actual distribution elements. In some embodiments, these values are the distribution elements themselves, such as category labels, network nodes, or action choices. In some embodiments, they represent pointers or references to more complex data structures that contain the full information about each distribution element.
[0063] The addition operation 303 and inverse operation 304 function identically to Figure 2, processing the unnormalized probabilities to enable proper sampling. The binomial decision maker 306 continues to use random data 305 to make probabilistic choices based on the input probabilities.The device or system comprises a multiplexer component 310 that selects between saand sbbased on the binomial decision maker's output. In some embodiments, the multiplexer implements direct selection of the chosen element. In some embodiments, the multiplexer includes validation logic to ensure the selected element meets certain criteria. In some embodiments, the multiplexer can handle cases where the selected element requires additional processing or transformation before output.
[0064] The device or system produces both a probability sum pa+ pb307 and the selected distribution element: either saor sb311. In some embodiments, the selected element is used directly by downstream processes. In some embodiments, the selected element serves as a key or index for retrieving additional information. In some embodiments, the output includes both the selected element and its associated probability for further processing or analysis.
[0065] This architecture provides a complete sampling solution that not only determines which probability to select but also outputs the corresponding distribution element, enabling direct integration with systems that need to work with the sampled elements rather than just their probabilities. Said systems or applications include QKD systems selecting quantum states for transmission, network routers selecting transmission paths, and image processing systems selecting filter coefficients.
[0066] Figure 4 shows a device or system that uses the Gumbel distribution to enable efficient multinomial sampling. The device or system is configured to transform input probabilities into optimized sampling choices while maintaining computational efficiency and numerical stability.
[0067] The device or system is configured to process two types of inputs: a set of (potentially unnormalized) probability values {p 401 and corresponding selection values {s 402. These inputs, combined with random data 403, flow through a specialized architecture designed to generate both aggregated probabilities and optimal choices.
[0068] In some embodiments, a Gumbel distribution sampler 404 comprised in the device or system is configured to transform input random variables into Gumbel-distributed samples. The Gumbel distribution's properties enable efficient sampling from categorical distributions. In some embodiments, the Gumbel distribution sampler 404 employs direct transformation methods. In some embodiments, the Gumbel distribution sampler 404 uses approximation techniques optimized for hardware implementation. In some embodiments, the Gumbel distribution sampler 404 includes adaptive mechanisms to handle varying input distributions.
[0069] The device or system comprises an accumulator (ACC) 405 that maintains running statistics of the sampling process. In some embodiments, the accumulator 405 tracks statistics neededfor probability estimation. In some embodiments, the accumulator 405 includes mechanisms for dealing with numerical overflow or underflow conditions that might arise during extended operation.
[0070] An addition operation 406 combines the Gumbel-distributed samples with the input probabilities. In some embodiments, the operation is implemented with standard floating-point addition. In some embodiments, the operation is implemented with arithmetic units, preferably specialized arithmetic units optimized for the particular numerical ranges involved in Gumbel sampling. In some embodiments, the addition includes preprocessing steps to ensure numerical stability.
[0071] The device or system also provides an argmax operation 407, which determines a choice, preferably the optimal choice, based on the combined probability and Gumbel samples. In some embodiments, the argmax operation 407 implements a simple maximum finder. In some embodiments, the argmax operation 407 includes comparison mechanisms that account for numerical precision issues, for example comparison trees as previously described with reference to some embodiments in which the transformation framework comprises one or more ASICs. In some embodiments, the argmax operation maintains state information to ensure consistency across sequential samples.
[0072] The device or system produces two outputs: an aggregated probability sum iPi 408 and the selected elements; from a choice set 409. In some embodiments, the probability sum serves as input for normalization in larger sampling systems. In some embodiments, the selected element output includes additional metadata about the selection process. In some embodiments, both outputs are used for monitoring and adjusting the sampling behavior of the system.
[0073] This Gumbel-based architecture provides several advantages over traditional multinomial sampling approaches, particularly in terms of computational efficiency and the ability to handle unnormalized probabilities directly. The device’s or system's design enables parallel processing and efficient hardware implementation while maintaining the statistical properties required for accurate sampling.
[0074] Figure 5 shows a device or system with a hardware-efficient implementation, in accordance with some embodiments, for generating samples that enable Gumbel-based multinomial sampling. The device or system minimizes computational complexity and resource requirements.The device or system is configured to take random data as input 501 and process it through a series of transformations. Cascaded binary logarithm operations 502, 503 are provided, which provide an efficient path to transform uniform random variables into a form suitable for Gumbel-based sampling. The binary logarithm operations are logic blocks implementing binary logarithm operations as previously described with reference to some embodiments in which the transformation framework comprises one or more FPGAs.
[0075] While the standard Gumbel distribution used in some other embodiments is typically generated using natural logarithms, a device or system according to this implementation uses binary logarithms (Iog2) instead. This transformation introduces a constant scaling factor, as well as a constant bias in the output distribution compared to the standard Gumbel distribution. However, since the sampling process ultimately relies on an argmax operation to select among options, this constant scaling factor and bias affects all samples equally and thus does not change the relative ordering of the samples. Therefore, the bias is, in some embodiments, safely ignored without affecting the sampling results, and the scaling factor is, in some embodiments, corrected via a simple multiplication 504 by a known, constant prefactor 505. The multiplication 504 may correspond to DSP slices performing multiplication operations as previously described with reference to some embodiments in which the transformation framework comprises one or more FPGAs.
[0076] In some embodiments, the transformation core (such as, e.g., the transformation core 101 shown in Figure 1) implements a first binary logarithm operation 502 that performs the transformation -Iog2(x) on the input random data. The use of base-2 logarithms is particularly advantageous in digital hardware, as it aligns naturally with binary arithmetic and can be implemented efficiently using bit manipulation operations. In some embodiments, this operation uses lookup tables to approximate the logarithm. In some embodiments, it employs piecewise linear approximations optimized for hardware implementation.
[0077] The transformation core cascades to a second binary logarithm operation 503, which completes the distribution transformation. The second logarithm stage can maintain the essential statistical properties required for multinomial sampling while using computationally efficient binary logarithms throughout.
[0078] The multiplication operation 504 implemented by the transformation core combines the transformed random values with a prefactor 505. In some embodiments, the prefactor 505 is constant. In some embodiments, the prefactor 505 is dynamically adjusted based on the desired properties of the output distribution. Examples of these properties include, but are notlimited to, sampling temperature. In some embodiments, constant scaling introduced using logarithms is absorbed into this prefactor 505, further simplifying the implementation.
[0079] The system produces output samples 506 that, while not strictly Gumbel-distributed, preserve the property that their argmax follows the same distribution as true Gumbel samples would, thereby making this implementation particularly suitable for high-performance sampling systems.
[0080] Figure 6 shows a device or system with a sampling architecture that enhances the Gumbel-based approach with specialized tail sampling capabilities in accordance with some embodiments. The device or system combines multiple sampling stages to achieve improved accuracy, particularly for sampling rare events or elements with very low probabilities in the distribution.
[0081] The device or system is configured to receive a random data stream 603 along with probability values {pt} 601 and selection values {s 602. The architecture comprises at least two cooperating sampling modules: a Gumbel distribution sampler 604 and a Gumbel tail sampler 610.
[0082] The Gumbel distribution sampler 604 implements the base sampling functionality like Figure 4. The device or system also comprises a binomial decision maker 611 that works in conjunction with the tail sampler 610. This combination enables the device or system to efficiently and adaptively switch between standard and tail sampling modes based on the characteristics of the input probabilities.
[0083] The binomial decision maker 611 is configured to determine when to engage the tail sampling mechanism. In some embodiments, this decision is based on probability thresholds, for example probabilities below 10'6or below 10'9. In some embodiments, the decision maker implements adaptive thresholds that adjust based on the device's or system's performance metrics.
[0084] The Gumbel tail sampler 610 is configured to handle the challenges of sampling from the tail of the distribution. This becomes particularly important in applications where rare events must be sampled accurately, such as in anomaly detection in network intrusion detection systems, extreme value analysis in telecommunications link failure prediction, or rare quantum state sampling in QKD protocols. In some embodiments, the tail sampler 610 implements importance sampling techniques. In some embodiments, the tail sampler610 uses specialized numerical methods optimized for handling very small probabilities, for example, probabilities below 10’4, or below 10’5, or below 10’6, or below 10'9.The device or system comprises an accumulator ACC 605 that tracks statistics across both standard and tail sampling operations. This unified accumulation enables the device or system to maintain consistent probability estimates regardless of which sampling mode is active. In some embodiments, the accumulator implements separate tracking for standard and tail samples to enable mode-specific optimization.
[0085] The addition operation 606 and argmax component 607 work together to combine the outputs of both sampling paths into a coherent final selection. The device or system produces both an aggregated probability sum iPi 608 and a selected element st609 from the choice set. In some embodiments, these outputs include additional metadata indicating whether the selection came from standard or tail sampling modes.
[0086] This dual-mode sampling architecture provides significant advantages over single-mode approaches, particularly in handling distributions with heavy tails or rare events, or in domains where the default implementation of the distribution lacks accuracy. The device’s or system's ability to adaptively switch between sampling modes enables it to maintain accuracy across a wider range of probability distributions while still preserving computational efficiency.
[0087] Figure 7 shows binary choice sampling modules, in accordance with some embodiments, as described in earlier figures. The binary choice sampling module are, in some embodiments, hierarchically combined to construct efficient samplers for larger choice sets. This figure illustrates a device or system configured to handle four-way choices by combining two binary choice modules in a tree-like structure.
[0088] The device or system receives multiple sets of inputs: unnormalized probabilities paand pbfor a first module 701, pcand pdfor a second module 702, along with their corresponding selection values sa,sb,scand sd. Each module 701, 702 is also configured to receive its own random data input to ensure independent sampling decisions.
[0089] First-level modules 701 , 702 operate in parallel, each performing binary sampling as described in Figure 3. The first module 701 processes the pair (pa, pb) and their corresponding selection values (sa,sb), while the second module handles (pc, pd) and (sc,sd). Each module 701, 702 is configured to produce both a summed probability output and a selected element from its respective pair.
[0090] A third binary choice module 703 combines the outputs from the first and second modules 701, 702 to make the final selection. This hierarchical arrangement enables the device or system to process four-way choices while maintaining the computational efficiency and numerical stability advantages of binary sampling. In some embodiments, the third module703 implements additional logic to account for the accumulated probabilities from the previous stages. In some embodiments, the third module 703 includes mechanisms to ensure uniform sampling probability across all four possible outcomes.
[0091] The device’s or system's outputs maintain both the probability information (pa+ pb+ pc+ pd) and the final selected element from the set {sa,sb,sc,sd}. This hierarchical approach scales efficiently to larger choice sets by adding more levels to the tree structure. In some embodiments, the device or system balances the tree to minimize sampling latency. In some embodiments, the device or system implements dynamic path selection to optimize sampling based on the input probability distribution.
[0092] This hierarchical architecture provides a practical solution for extending binary sampling to larger choice sets while preserving the computational advantages of binary operations. Applications include four-state QKD protocols (BB84), four-level intensity modulation in telecommunications, and quaternary decisions in industrial control systems.
[0093] Figure 8 shows a device or system, in accordance with some embodiments, with a parallel, pipelined architecture for efficient sampling from high-dimensional distributions. The device or system is configured to process n-dimensional inputs {p (801) and {s (802) through a series of specialized processing stages, each optimized for parallel operation.
[0094] In these embodiments, the architecture of the device or system is configured to employ cascaded Bernoulli machines 803, 804 that progressively reduce the dimensionality of the sampling problem. For example, thread blocks may execute parallel Bernoulli machines 803, 804 as previously described with reference to some embodiments in which the transformation framework comprises one or more GPUs. A first stage corresponding to a first Bernoulli machine 803 processes the full n-dimensional input, splitting it into two n / 2-dimensional problems. A second stage corresponding to a second Bernoulli machine 804 further reduces these to n / 4-dimensional problems. This progressive dimensionality reduction continues until the problem becomes manageable for direct Gumbel-based sampling 805.
[0095] Each Bernoulli machine 803, 804 implements an efficient binary sampling operation, but unlike the embodiments of previous figures, these machines 803, 804 are specifically designed for parallel operation. In some embodiments, the Bernoulli machines 803, 804 employ shared random number generators to reduce hardware complexity. In some embodiments, the Bernoulli machines 803, 804 implement specialized synchronization mechanisms to maintain sampling coherence across parallel paths.The final stage employs Gumbel distribution sampling 805 to make the ultimate selection, producing a choice output 806 that represents the selected element Sj. This combination of parallel Bernoulli processing and Gumbel sampling provides an efficient solution for handling high-dimensional sampling problems while maintaining both computational efficiency and sampling accuracy.
[0096] The significance of this architecture lies in its ability to process high-dimensional inputs with reduced latency compared to sequential approaches. By leveraging both parallelism and pipelining, the device or system achieves substantially higher throughput than traditional sampling methods while maintaining the statistical properties required for accurate multinomial sampling.
[0097] In more general embodiments, each of the pipelining stages comprises either a set of Gumbel distribution samplers, a set of binary-sampling cores, or a suitable combination of both.
[0098] Application to Quantum Key Distribution Systems
[0099] The sampling architectures described herein find, for example but without limitation, particular application in Quantum Key Distribution (QKD) systems, where multinomial sampling operations must satisfy stringent security and performance requirements. In these applications, the sampling process directly impacts the cryptographic security of the key distribution protocol.
[0100] In some embodiments, the device or system is configured to sample basis selections for QKD protocols such as, e.g., BB84. In the BB84 protocol, each transmission requires selection from two possible bases (rectilinear and diagonal), with each basis containing two possible states. The binary sampling architecture of, for example but without limitation, the embodiments of Figure 2 provides an efficient implementation for this two-basis selection, requiring minimal entropy consumption per sample. In some embodiments, the basis selection probabilities are equal (pa = pb = 0.5). In some embodiments, asymmetric basis probabilities are employed to optimize protocol efficiency.
[0101] In some embodiments, the hierarchical architecture of, for example but without limitation, the embodiments of Figure 7 is applied to sample from the complete set of four BB84 states (horizontal, vertical, diagonal-plus, diagonal-minus). The tree structure enables this four-way selection using only two binary sampling stages, thereby minimizing entropy consumption while maintaining uniform or weighted selection probabilities as required by the protocol.In some embodiments, the device or system is configured to sample intensity levels for decoystate QKD protocols. Decoy-state protocols typically employ three or four intensity levels (vacuum, weak decoy, and signal; or multiple decoy intensities). The sampling architectures described herein enable efficient selection among these intensity levels according to optimized probability distributions that maximize secure key rate. In some embodiments, the intensity probabilities are dynamically adjusted based on channel conditions or security requirements.
[0102] In some embodiments, the device or system is configured to sample discrete phase values for phase-randomized QKD protocols. Continuous-variable or discrete-modulated QKD systems may require selection from a finite set of phase values. The multinomial sampling architectures described herein provide efficient selection from these discrete phase alphabets.
[0103] In QKD applications, the randomness source 103 of Figure 1 preferably comprises a Quantum Random Number Generator (QRNG) to ensure that the unpredictability of sampling selections is guaranteed by fundamental quantum mechanical principles, provided by modules adapted to provide random numbers through quantum mechanical processes, rather than computational assumptions. The entropy efficiency of the disclosed sampling methods is particularly advantageous in this context, as QRNG-generated bits represent a finite resource that must be consumed judiciously. In some embodiments, the QRNG is integrated within the same photonic system that implements the QKD protocol, enabling compact and secure implementations.
[0104] Embodiments of methods, devices, and systems described in this disclosure provide multinomial distribution sampling with a hierarchical combination of binary sampling, Gumbel-based transformation, and parallel processing techniques that enables efficient implementation across a wide range of applications and hardware platforms.
[0105] The disclosed architectures address several critical challenges in modern sampling applications. First, they enable direct handling of unnormalized probabilities, eliminating the computational overhead of normalization. Second, they provide efficient hardware implementations through binary logarithm optimizations and parallel processing structures. Third, they maintain sampling accuracy across a wide range of probability distributions through specialized techniques such as tail sampling.
[0106] Disclosed methods, devices and systems are particularly relevant for contemporary applications such as machine learning, where sampling operations often represent significant computational bottlenecks. The ability to perform efficient sampling in hardware, with reducedlatency and improved numerical stability, enables new applications and enhances the performance of existing systems.
[0107] The modular nature of the described architectures allows for flexible implementation across different scales, from small embedded systems to large-scale computational platforms. This flexibility, combined with the fundamental efficiency improvements provided by the binary logarithm optimization and parallel processing capabilities, represents a significant advance in the field of multinomial sampling.
[0108] It should be noted that the above-described examples of the present solution are for the purpose of illustration. Although the solution has been described in conjunction with specific examples thereof, numerous modifications are possible without materially departing from the teachings of the subject matter described herein. Other substitutions, modifications and changes may be made without departing from the spirit of the present solution.
[0109] All of the features and applications disclosed in the present disclosure (including any accompanying claims, abstract and drawings), and / or all of the parts of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and / or parts are mutually exclusive.
Claims
CLAIMS1. A device for multinomial distribution sampling, comprising:a. at least one processing unit; andb. a transformation module configured to:i. transform random values inputted into the device into transformed random values;ii. combine the transformed random values with probability values inputted into the device to generate modified probability values; and iii. select an output based on comparing the modified probability values.
2. The device of claim 1, further comprising a random data input configured to receive the random values, and a probability input configured to receive the probability values.
3. The device of any one of the preceding claims, wherein the probability values are unnormalized probability values.
4. The device of any one of the preceding claims, wherein the transformation module is configured to transform the random values using cascaded binary logarithm operations that introduce a constant scaling factor that does not affect the selection of the output.
5. The device of any one of the preceding claims, wherein the transformation module comprises:a prefactor computation unit configured to generate scaling factors;a multiplication unit configured to apply the scaling factors to the transformed random values; andan output unit configured to produce scaled transformed values suitable for comparison.
6. The device of any one of the preceding claims, further comprising a Gumbel tail sampling module configured to:detect when at least one of the probability values falls below a predetermined threshold;apply sampling operations to the detected probability values, thereby providing results; andcombine the results with the output of the transformation module.
7. The device of any one of the preceding claims, further comprising an accumulator configured to maintain running statistics of the selected outputs providing the multinomial distribution sampling.
8. The device of any one of the preceding claims, wherein the at least one processing unit comprises parallel processing elements configured to:process multiple probability pairs simultaneously;reduce dimensionality of the multinomial sampling progressively; and / or combine multinomial sampling results through a pipelined architecture.
9. The device of any one of the preceding claims, wherein the device is configured to receive random values from a Quantum Random Number Generator, wherein unpredictability of the random values is guaranteed through quantum mechanical processes.
10. The device of claim 9, wherein the device is configured to perform sampling operations for a Quantum Key Distribution system.
11. The device of claim 10, wherein the sampling operations comprise at least one of:basis selection for a prepare-and-measure Quantum Key Distribution protocol; state selection within a selected basis; intensity level selection for a decoy-state protocol; or discrete phase value selection for a phase-encoded protocol.
12. The device of any one of the preceding claims, wherein the multinomial distribution comprises four or fewer categories, and wherein the transformation module is configured to implement a binary tree sampling architecture optimized for a reduced category count.
13. A system for high-dimensional multinomial sampling, comprising:a. multiple binary sampling modules, according to any one of the preceding claims, arranged in a tree structure;b. a Gumbel distribution sampling module connected to outputs of the binary sampling modules;c. an accumulator configured to track probability sums across the tree structure;andd. a selection module configured to produce final choices based on accumulated probabilities.
14. The system of claim 13, further comprising:parallel processing units implementing the binary sampling modules; synchronization mechanisms between the parallel processing units; and / or pipelined data paths connecting successive sampling stages.
15. The system of any one of claims 13-14, wherein the system is configured to:adapt a sampling thereof based on input probability distributions;switch between standard and tail sampling modes dynamically; and maintain sampling accuracy across varying probability scales.
16. The system of any one of claims 13-15, further comprising a Quantum Random Number Generator.
17. A method for multinomial distribution sampling, comprising:a. receiving random values and probability values;b. transforming the random values through cascaded binary logarithm operations; c. combining the transformed random values with the probability values; and d. selecting an output based on comparing the combined values;wherein the binary logarithm operations introduce a constant scaling factor that does not affect the output selection.
18. The method of claim 17, further comprising:detecting probability values below a predetermined threshold;applying tail sampling operations to the detected values; andcombining tail sampling results with sampling results to produce the final output.
19. The method of any one of claims 17-18, wherein selecting an output comprises:generating a first binary choice between a first pair of probability values; generating a second binary choice between a second pair of probability values; andcombining the first and second binary choices hierarchically to produce a final selection.
20. The method of any one of claims 17-19, further comprising:receiving selection values corresponding to each probability value; associating transformed probabilities with their corresponding selection values; andoutputting a selected value from the selection values based on the probability comparison.
21. The method of any one of claims 17-20, wherein transforming the random values comprises:applying a first binary logarithm operation to the random values; applying a second binary logarithm operation to results of the first binary logarithm operation; andscaling the twice-transformed values using a predetermined factor.
22. The method of any one of claims 17-21, wherein the method further comprises producing statistically correct sampling results using binary logarithms.
23. The method of any one of claims 17-22, wherein receiving the random values comprises receiving the random values from a Quantum Random Number Generator, and wherein selecting an output comprises selecting a cryptographic parameter for a Quantum Key Distribution protocol.