Hardware implementation method and device of sigmoid function, electronic equipment and computer readable storage medium

Abstract

This invention proposes a hardware implementation method, apparatus, electronic device, and computer-readable storage medium for the Sigmoid function, relating to the field of data processing technology. The method constructs a function mapping table for the Sigmoid function, containing discrete non-negative floating-point values ​​within the unsaturated interval, based on the symmetry and saturation of the Sigmoid function, thereby substantially reducing the scope of the lookup table. On this basis, the absolute value of the floating-point values ​​belonging to the unsaturated interval is taken to obtain the value to be processed. This value is then converted to the target fixed-point format, and integer operations are used for table lookup, interpolation, and symmetry restoration to obtain the output value of the Sigmoid function based on the target floating-point value, improving computational efficiency. While ensuring the function output capability across the entire range, the size and storage overhead of the function mapping table are effectively reduced, while linear interpolation under the fixed-point format improves output accuracy.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a hardware implementation method, apparatus, electronic device, and computer-readable storage medium for the Sigmoid function. Background Technology

[0002] With the increasing popularity of embedded intelligent devices and dedicated acceleration hardware, the sigmoid function, as one of the most commonly used non-linear activation functions in neural network inference, directly affects the real-time performance, energy efficiency, and integration of embedded AI chips.

[0003] Current mainstream hardware implementations have significant limitations: floating-point operations rely on the FPU (Flexible Processing Unit), resulting in high resource consumption, latency, and power consumption, and they cannot run on low-end MCUs without an FPU. Traditional full-range lookup table methods, while avoiding floating-point operations, require substantial storage to ensure accuracy, far exceeding the ROM capacity of most embedded chips. Polynomial approximation methods are limited by poor range adaptability and a sharp increase in boundary errors, requiring additional logic for compensation, increasing area and latency.

[0004] On hardware chips with extremely limited resources, the above solutions struggle to simultaneously meet the comprehensive constraints of low latency, low resource consumption, low power consumption, and high precision. Therefore, there is an urgent need for a hardware implementation method for the sigmoid function that balances low storage, low resource consumption, low power consumption, and high precision to meet the core requirements of embedded hardware and dedicated acceleration chips. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a hardware implementation method, apparatus, electronic device and computer-readable storage medium for the Sigmoid function, which can balance the reduction of hardware resource consumption and the improvement of output accuracy.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a hardware implementation method for the Sigmoid function, the method comprising: Extract the sign bit of the received target floating-point value, and take the absolute value of the target floating-point value to obtain the value to be processed; If the target floating-point value is within a preset non-saturation range, the value to be processed is converted into a target fixed-point format to obtain the target fixed-point value; the non-saturation range is determined by a positive threshold and the opposite of the positive threshold; The mapping interval is determined based on the target fixed-point format, the positive threshold, and the number of mapping segments; the number of mapping segments is the number of Sigmoid function values ​​in the function mapping table minus 1, and the Sigmoid function values ​​are generated based on the non-negative numbers of the unsaturated interval. The target index value and offset are determined based on the target fixed-point value and the mapping interval; Interpolation is performed based on the mapping interval, the target index value, the offset, and the function mapping table to obtain a preliminary output value; The initial output value is symmetrically restored based on the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

[0007] In an optional implementation, converting the value to be processed into a target fixed-point format to obtain the target fixed-point value includes: The value to be processed is shifted left by the number of decimal places in the target fixed-point format and then rounded to obtain the target fixed-point value.

[0008] In an optional implementation, determining the mapping interval based on the target fixed-point format, the positive threshold, and the number of mapping segments includes: The positive number threshold is shifted left by the number of decimal places in the target fixed-point format to obtain the fixed-point number threshold; The number of mapping bits is determined based on the number of mapping segments; The mapping interval is obtained by right-shifting the fixed-point threshold by the number of mapping bits.

[0009] In an optional implementation, determining the target index value and offset based on the target fixed-point value and the mapping interval includes: The target fixed-point value is shifted to the right according to the mapping interval to obtain the target index value; The low-order bits of the target fixed-point value are extracted by bitmasking, and the low-order bits are determined as the offset.

[0010] In an optional implementation, the step of performing interpolation based on the mapping interval, the target index value, the offset, and the function mapping table to obtain a preliminary output value includes: The Sigmoid function value corresponding to the target index value in the function mapping table is determined as the target function value; The Sigmoid function value corresponding to the next index value of the target index value in the function mapping table is determined as the reference function value; Interpolation adjustments are performed based on the target function value, the reference function value, the offset, and the mapping interval to obtain a preliminary output value.

[0011] In an optional implementation, the step of interpolating and adjusting based on the objective function value, the reference function value, the offset, and the mapping interval to obtain a preliminary output value includes: The difference between the reference function value and the target function value is determined as the adjustment value; Add half of the mapping interval to the product of the adjustment value and the offset to obtain the interpolation compensation amount; The interpolation compensation amount is shifted to the right according to the mapping interval to obtain the interpolation compensation value; The sum of the objective function value and the interpolation compensation value is determined as the initial output value.

[0012] In an optional implementation, the step of performing symmetrical restoration processing on the initial output value based on the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value includes: If the sign bit indicates that the target floating-point value is positive, the initial output value is determined as the output value of the Sigmoid function based on the target floating-point value; If the sign bit indicates that the target floating-point value is non-positive, the difference between 1 and the initial output value is determined as the output value of the Sigmoid function based on the target floating-point value.

[0013] In an optional implementation, after taking the absolute value of the target floating-point value to obtain the value to be processed, the method further includes: Determine whether the value to be processed is greater than the positive threshold; If the value to be processed is greater than the positive threshold, then the preset saturation value is used as the initial output value, and the initial output value is symmetrically restored according to the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

[0014] Secondly, the present invention provides a hardware implementation apparatus for the Sigmoid function, the apparatus comprising: The processing module is used to extract the sign bit of the received target floating-point value and take the absolute value of the target floating-point value to obtain the value to be processed; if the target floating-point value is within a preset non-saturation range, the value to be processed is converted into a target fixed-point format to obtain a target fixed-point value; the non-saturation range is determined by a positive threshold and the opposite of the positive threshold. The determination module is used to determine the mapping interval based on the target fixed-point format, the positive threshold, and the number of mapping segments; the number of mapping segments is the number of Sigmoid function values ​​in the function mapping table minus 1, and the Sigmoid function values ​​are generated based on the non-negative numbers of the unsaturated interval; the module also determines the target index value and offset based on the target fixed-point value and the mapping interval. The mapping module is used to perform interpolation processing based on the mapping interval, the target index value, the offset, and the function mapping table to obtain a preliminary output value; and to perform symmetric restoration processing on the preliminary output value based on the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

[0015] Thirdly, the present invention provides an electronic device including a hardware chip, the hardware chip being capable of executing a computer program to implement the hardware implementation method of the Sigmoid function as described in any of the foregoing embodiments.

[0016] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a hardware chip, implements a hardware implementation method of the Sigmoid function as described in any of the foregoing embodiments.

[0017] Compared to existing technologies, the hardware implementation method, apparatus, electronic device, and computer-readable storage medium for the Sigmoid function provided in this invention construct a function mapping table for the Sigmoid function, containing discrete non-negative floating-point values ​​within the unsaturated interval, based on the symmetry and saturation of the Sigmoid function. This substantially reduces the scope of the lookup table. On this basis, the absolute value of the floating-point values ​​belonging to the unsaturated interval is taken to obtain the value to be processed. This value is then converted to the target fixed-point format, and lookup and interpolation are performed using integer operations to obtain a preliminary output value. Finally, the preliminary output value is symmetrically restored using the sign bit of the target floating-point value to obtain the output value of the Sigmoid function based on the target floating-point value, thus improving computational efficiency. While ensuring the ability to output functions across the entire range, the size and storage overhead of the function mapping table are effectively reduced, while linear interpolation under the fixed-point format improves output accuracy.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This diagram illustrates a flowchart of a hardware implementation method for the Sigmoid function provided in an embodiment of the present invention.

[0021] Figure 2 This diagram illustrates another flowchart of the hardware implementation method of the Sigmoid function provided in an embodiment of the present invention.

[0022] Figure 3 A block diagram of a hardware implementation device for the Sigmoid function provided in an embodiment of the present invention is shown.

[0023] Figure 4 A block diagram of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0026] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] The Sigmoid function, as a core nonlinear activation function in fields such as artificial intelligence, signal processing, and automatic control, has the following mathematical expression: The Sigmoid function is widely used in key scenarios such as neural network inference, nonlinear signal fitting, and probabilistic model construction.

[0028] With the rapid popularization of embedded intelligent devices (such as edge computing terminals, industrial controllers, and smart home chips) and dedicated acceleration hardware (such as field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs), the hardware implementation of the Sigmoid function has been subjected to coordinated requirements of low latency, low power consumption, low resource consumption, and high precision.

[0029] It is particularly important to note that the sigmoid function primarily serves as the activation and gating function in the neural network inference stage of hardware chips. It is frequently called after convolution operations or matrix multiplication, making it one of the most frequently invoked and densely executed fundamental nonlinear operations in current AI inference chips. Its hardware implementation efficiency directly constrains the overall power efficiency and real-time performance of the hardware chip. Therefore, it possesses significant engineering necessity and industrial urgency.

[0030] Currently, there are three main technical solutions for hardware implementation of the Sigmoid function in the industry: The first type is a pure floating-point arithmetic implementation scheme, which directly relies on the hardware's floating-point unit (FPU) and strictly follows the mathematical definition: first, the floating-point exponentiation unit calculates... Then calculate using a floating-point adder. Finally, it is obtained through a floating-point flip-flop. This solution relies on a mature floating-point architecture, requiring no external algorithm optimization; it only needs to call the basic instructions of the hardware FPU to implement the Sigmoid function.

[0031] The second type is the traditional lookup table implementation scheme. To avoid the high hardware overhead caused by floating-point exponentiation and division operations, this scheme adopts a pre-computation + lookup table strategy: the Sigmoid function value is pre-calculated within the discrete input range, and the result is stored in a hardware storage unit (such as read-only memory (ROM), block random access memory (BRAM), etc.). At runtime, the result is obtained by looking up the table based on the input value.

[0032] A typical implementation involves three steps: 1) Covering the effective dynamic input range of the Sigmoid function (i.e., the entire input range, such as [-18, 18]). 2) Uniformly discretizing this range with a fixed step size (e.g., 0.01), pre-calculating and storing the Sigmoid function values ​​point by point. 3) After receiving the input, the hardware uses copper drum quantization mapping to obtain a lookup table index and directly outputs the value from the table. This scheme does not rely on an FPU and is widely used on hardware platforms such as embedded microcontroller units (MCUs) and FPGAs that lack floating-point capabilities. It is a technical alternative to pure floating-point arithmetic implementations.

[0033] The third category is polynomial approximation schemes. Based on mathematical approximation theories (such as Taylor expansion, Chebyshev polynomials, etc.), the Sigmoid function is approximated as a low-order polynomial (such as 3rd to 5th order). The hardware only needs to perform multiple multiplication and addition operations to complete the calculation. For example, in the interval [-5, 5], the Sigmoid function can be approximated as... This approach balances computational complexity and accuracy, and is often used in ASIC designs that are sensitive to storage resources.

[0034] Through in-depth analysis of the three solutions, the inventors discovered that although these three solutions have been implemented in actual products, they all exhibit fundamental shortcomings when facing next-generation embedded AI and edge intelligence scenarios, failing to simultaneously achieve the four objectives of low latency, low power consumption, low resource consumption, and high precision. Specifically, these shortcomings are as follows: The main drawbacks of the pure floating-point arithmetic implementation scheme are reflected in the following three aspects: 1) High hardware overhead: Floating-point exponent units and division units consume a large amount of hardware resources, requiring several times the resources of integer logic units (LUTs) and digital signal processing units (DSPs) in FPGAs. In ASICs, taking 40nm process as an example, the area of ​​a single-precision floating-point exponent unit is about 0.5 mm², which is more than 50 times that of an integer multiplier of the same bit width, seriously squeezing the chip area budget.

[0035] 2) High latency and power consumption: Floating-point exponentiation operations typically require 10–20 clock cycles, and division operations require 5–10 cycles, with an overall latency exceeding 20 cycles. Furthermore, the switching power consumption of floating-point units is 3–5 times that of integer units, making it difficult to meet low-power requirements in battery-powered embedded devices.

[0036] 3) Poor versatility: Low-end MCUs without hardware FPUs cannot implement this directly and must rely on software to simulate floating-point operations, which further reduces efficiency, resulting in a single Sigmoid calculation taking more than 1,000 CPU cycles.

[0037] The main drawbacks of the traditional table lookup method are reflected in the following three aspects: 1) There is a sharp contradiction between storage and precision: To ensure precision, the lookup step size needs to be reduced and the lookup range expanded, leading to an exponential increase in storage requirements. For example, a lookup covering the range [-18, 18] with a step size of 0.01 requires storing 3600 32-bit floating-point values, occupying 14.4KB of storage. If 16-bit fixed-point storage is used, although the storage can be reduced to 7.2KB, it still far exceeds the on-chip ROM capacity of the embedded MCU (some MCUs have only a few KB of on-chip ROM). If storage is compressed (the step size is increased), the precision drops sharply, with the maximum error exceeding 0.005 when the step size is 0.1, which cannot meet the precision requirements of neural network inference.

[0038] 2) Severe redundancy in intervals: The saturation property of the Sigmoid function is not utilized, i.e., when... hour, .when hour, Traditional full range lookup tables store a large number of nearly constant redundant values ​​in the ranges [-18,8] and [8,18], which wastes storage space and increases invalid addressing time.

[0039] 3) Lack of interpolation compensation: Directly outputting discrete table values ​​does not compensate for the error between adjacent table entries, resulting in large fluctuations in local accuracy. The maximum measured error can reach more than 10% of the step size, and the accuracy stability is poor.

[0040] The main drawbacks of the polynomial approximation scheme are reflected in the following three aspects: 1) Poor controllability of approximation error: Polynomial approximation has high accuracy only within a specific interval (such as [-5,5]). Beyond the specific interval, the error increases exponentially, requiring additional judgment and compensation logic, which increases hardware complexity.

[0041] 2) High computational complexity: 3rd to 5th order polynomials require multiple multiplication and addition operations. For example, a 5th order polynomial requires 5 multiplications and 4 additions. The hardware latency is still 5-8 clock cycles. Furthermore, the increased number of multipliers leads to increased power consumption and area.

[0042] 3) Poor parameter adaptability: Different application scenarios have different requirements for accuracy and range, requiring the polynomial coefficients to be re-derived and the hardware logic to be modified, which cannot be flexibly adapted.

[0043] In summary, the shortcomings of existing hardware implementations of the Sigmoid function have become industry bottlenecks in core application scenarios such as embedded AI inference, industrial real-time control, and edge computing: First, there is a performance bottleneck: pure floating-point operations and polynomial approximation implementations have excessively high latency, failing to meet the requirements of real-time controllers with millisecond-level response times. Second, there is a resource bottleneck: traditional lookup table schemes have large storage requirements, making them unsuitable for low-end MCUs and resource-constrained FPGA / ASIC designs. Third, there is a precision bottleneck: compressed lookup tables or low-order polynomial approximations lack sufficient precision, leading to a decrease in the accuracy of neural network inference on hardware (the model accuracy drops by 1-3 percentage points when the error exceeds 0.001). Fourth, there is a power consumption bottleneck: the high power consumption of pure floating-point operations and polynomial operations cannot meet the battery life requirements of mobile devices and IoT devices.

[0044] Based on this, embodiments of the present invention provide a hardware implementation method, apparatus, electronic device, and computer-readable storage medium for the Sigmoid function. This method constructs a function mapping table for the Sigmoid function, containing discrete non-negative floating-point values ​​within the unsaturated interval, based on the symmetry and saturation of the Sigmoid function, thereby substantially reducing the scope of the lookup table. Furthermore, floating-point values ​​belonging to the unsaturated interval are converted to the target fixed-point format, and lookup and interpolation are performed using integer arithmetic, improving computational efficiency. While ensuring the ability to output functions across the entire range, the size and storage overhead of the function mapping table are effectively reduced, while linear interpolation under the fixed-point format improves output accuracy.

[0045] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0046] Please refer to Figure 1 , Figure 1 This diagram illustrates a flowchart of a hardware implementation method for the Sigmoid function provided in an embodiment of the present invention. The method includes the following steps: Step S10: Extract the sign bit of the received target floating-point value and take the absolute value of the target floating-point value to obtain the value to be processed.

[0047] In this invention embodiment, the Sigmoid function is calculated in real time in hardware chips such as ASICs, embedded MCUs, and FPAGs to suit inference scenarios in artificial intelligence. The inventors have discovered that the Sigmoid function possesses mathematical symmetry and saturation.

[0048] Saturation refers to the fact that when the absolute value of the input value exceeds a certain range, the output value of the Sigmoid function almost stops changing. That is, when... hour, .when hour, Symmetry means that the sum of the output value of the Sigmoid function for a positive number and the output value of the Sigmoid function for the opposite of the positive number is 1. .

[0049] Before calling the Sigmoid function, the unsaturated interval (e.g., [-8, 8]) is pre-determined based on the symmetry and saturation of the Sigmoid function. The unsaturated interval can be set with the required precision for the actual application scenario, and this invention does not limit this setting.

[0050] When the Sigmoid function is called, in order to accommodate the symmetry of the Sigmoid function (i.e., The Sigmoid function first extracts the sign bit from the received target floating-point value and then takes the absolute value of the target floating-point value to obtain the value to be processed within the non-negative range. For example, if the target floating-point value is -7, the sign bit will be negative (e.g., marked as 1), and the value to be processed will be 7. If the target floating-point value is 7, the sign bit will be positive (e.g., marked as 0), and the value to be processed will be 7.

[0051] Step S20: If the received target floating-point value is within the preset non-saturation range, the value to be processed is converted into the target fixed-point format to obtain the target fixed-point value; the non-saturation range is determined by the positive threshold and the opposite of the positive threshold.

[0052] Next, it is determined whether the received target floating-point value falls within a preset unsaturated range. This unsaturated range is defined by a positive threshold and its opposite, forming a finite input range symmetric about zero. Only when the target floating-point value is within the unsaturated range is the output value of the Sigmoid function generated based on the target floating-point value through a lookup table (i.e., a function mapping table). This reduces redundant calculations on floating-point input values ​​that are clearly approaching saturation, saving processing time and hardware power consumption.

[0053] To eliminate the dependency on floating-point operations, the values ​​to be processed that require table lookups are converted into integers according to the target fixed-point format, resulting in target fixed-point values. This facilitates rapid location and calculation via bitwise operations while retaining sufficient numerical resolution to support precision requirements. The target fixed-point formats include, but are not limited to, Q12, Q15, and Q16. For example, the Q17.16 format has 17 bits for the integer part and 16 bits for the fractional part.

[0054] Step S30: Determine the mapping interval based on the target fixed-point format, positive threshold, and number of mapping segments; the number of mapping segments is the number of Sigmoid function values ​​in the function mapping table minus 1, and the Sigmoid function values ​​are generated based on non-negative numbers in the unsaturated interval.

[0055] In this embodiment of the invention, the mapping interval, or mapping step size, of adjacent function values ​​is determined by combining the number of decimal places in the target fixed-point format, the size of the positive number threshold, and the number of Sigmoid function values ​​pre-stored in the function mapping table (i.e., lookup table, LUT). That is, for each increase in mapping step size, a corresponding discrete function value is stored in the function mapping table.

[0056] It should be noted that the function mapping table predetermines the fixed-point value corresponding to each index based on the number of decimal places, the number of mapping segments, and the positive number threshold of the target fixed-point format. The fixed-point value corresponding to each index is then input into the actual Sigmoid function to obtain the Sigmoid function value stored in the function mapping table for each index.

[0057] It should be understood that the larger the number of mapping segments, the smaller the mapping interval, the shorter the interpolation interval, the lower the function curvature error, and the better the accuracy of the sigmoid function approximation. For example, when the number of mapping segments is 128, the fitting error is relatively large. When the number of mapping segments is 256, it is considered a baseline configuration. When the number of mapping segments is 1024, the fitting error is significantly reduced.

[0058] In high-precision scenarios, setting the number of mapping segments to 1024 occupies approximately 2KB of storage space, while improving precision to approximately [missing information]. In low-resource scenarios, setting the number of mapped segments to 128 occupies approximately 256 bytes of storage space, with precision controlled within approximately... By simply adjusting the number of mapping segments and the function mapping table, it can be adapted to different application scenarios, solving the problem of "hardware reconstruction required for parameter adaptation" in traditional solutions.

[0059] Step S40: Determine the target index value and offset based on the target fixed point value and the mapping interval.

[0060] In this embodiment of the invention, the target index value for accessing the function mapping table is determined based on the proportional relationship between the target positioning value and the mapping interval, and the relative position, i.e., the offset, between the target fixed-point value and the Sigmoid function value corresponding to the target index value is obtained simultaneously. The offset provides more accurate positional information for linear interpolation.

[0061] Step S50: Perform interpolation based on the mapping interval, target index value, offset, and function mapping table to obtain the preliminary output value.

[0062] In this embodiment of the invention, interpolation is performed based on the mapping interval, target index value, offset, and Sigmoid function value stored in the function mapping table, and the interpolated value is determined as the initial output value.

[0063] Step S60: Perform symmetrical restoration processing on the initial output value according to the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

[0064] In this embodiment of the invention, a symmetrical restoration operation is performed on the initial output value based on the sign bit of the target floating-point value to obtain the output result of the Sigmoid function that conforms to the mathematical definition, that is, the output value of the Sigmoid function based on the target floating-point value.

[0065] In summary, the hardware implementation method for the Sigmoid function provided in this invention constructs a function mapping table for the Sigmoid function, containing discrete non-negative floating-point values ​​within the unsaturated interval, based on the symmetry and saturation of the Sigmoid function. This substantially reduces the scope of the lookup table. On this basis, the absolute value of the floating-point values ​​belonging to the unsaturated interval is taken to obtain the value to be processed. This value is then converted into the target fixed-point format, and lookup and interpolation are performed using integer operations to obtain the initial output value. Finally, the initial output value is symmetrically restored using the sign bit of the target floating-point value to obtain the output value of the Sigmoid function based on the target floating-point value, thus improving computational efficiency. While ensuring the function output capability across the entire range, the size and storage overhead of the function mapping table are effectively reduced, while linear interpolation under the fixed-point format improves output accuracy.

[0066] Alternatively, one possible implementation for converting floating-point numbers to a target fixed-point format is provided below. Figure 1 The sub-steps of step S20 may include: The value to be processed is shifted left by the number of decimal places in the target fixed-point format and rounded to obtain the target fixed-point value.

[0067] In this embodiment of the invention, to reduce the reliance on floating-point units, the function mapping table pre-stores the values ​​of the Sigmoid function at discrete sampling points (in the target fixed-point format). Therefore, when calling the Sigmoid function and looking up the function mapping table, the function to be processed (i.e., the target floating-point value after taking the absolute value) must first be converted into the target fixed-point format, and the converted value is rounded to obtain the target fixed-point value that matches the data format of the function mapping table.

[0068] Assuming the target fixed-point format is Q16 (17 bits for the integer part and 16 bits for the fractional part), convert the floating-point value to be processed into a Q16 format integer. The conversion formula is defined as follows:

[0069] in, It is a target fixed-point value in Q16 format. These are the values ​​to be processed; It is a rounding function, which can be implemented in hardware by adding half of the least significant bit (LSB). This means shifting the value to be processed 16 bits to the left.

[0070] It should be understood that Q16 fixed-point numbers amplify the floating-point values ​​to be processed. This multiplies the value so that the floating-point number 1.0 strictly corresponds to the integer 65536 in Q16 format, achieving the minimum resolution. This is sufficient to meet the error tolerance of neural network inference for activation functions, and can also improve hardware computing efficiency. For example, if the target floating-point value is 1.5, shifting it left by 16 bits yields the value 98304, and rounding it off gives the target fixed-point value 98304.

[0071] As can be seen, the embodiments of the present invention utilize the target fixed-point format to convert floating-point values ​​into integer target fixed-point values, and use the target fixed-point values ​​to perform index calculations and table lookup operations, ensuring that all operations in the Sigmoid function are completed based on the integer Arithmetic Logic Unit (ALU), without the need for floating-point operation support.

[0072] Alternatively, the following is a possible implementation for determining the step size (i.e., mapping interval) of each discrete Sigmoid function value in the function mapping table. Figure 1 The sub-steps of step S30 may include: Step S300: Shift the positive threshold to the left by the number of decimal places in the target fixed-point format to obtain the fixed-point threshold.

[0073] In this embodiment of the invention, it is assumed that the target fixed-point format is Q16, and the number of decimal places in the target fixed-point format is 16. For example, the function mapping table is constructed based on the non-negative interval [0,8]. Negative numbers are restored through symmetry properties, reducing the table lookup interval by 50%. The threshold for positive numbers is 8.

[0074] Shifting the positive number threshold 8 to the left by 16 bits yields the fixed-point number threshold 524288. This converts the lookup area corresponding to the floating-point type to the lookup area [0, 524288] corresponding to the target fixed-point format.

[0075] Step S310: Determine the number of mapping bits based on the number of mapping segments.

[0076] Step S320: Shift the fixed-point threshold to the right by the number of mapping bits to obtain the mapping interval.

[0077] In this embodiment of the invention, assuming the number of mapping segments is 256, the lookup table interval is divided into 256 equal segments according to the number of mapping segments. When the lookup table interval is divided into 256 segments, the function mapping table stores 257 discrete Sigmoid function values. Assuming each Sigmoid function value is stored using the uint16_t type, the 257 Sigmoid function values ​​occupy 514 bytes of storage space. Since the number of mapping segments is 256, it is 2... 8 Therefore, the number of bits in the mapping is 8.

[0078] Finally, the fixed-point threshold of 524288 is shifted to the right by 8 bits, resulting in a mapping interval of 2048. In other words, the mapping interval of 2048 refers to the step size of dividing the lookup area [0, 524288] corresponding to the target fixed-point format into 256 segments. The corresponding floating-point field step size is 8 / 256 = 0.03125, which is completely consistent with dividing the non-negative interval [0, 8] into 256 segments.

[0079] It should be understood that the number of bits corresponding to the mapping interval (i.e., the exponent with base 2) is determined based on the positive threshold, the number of decimal places in the target fixed-point format, and the number of mapping segments. Specifically, it is the sum of the number of bits corresponding to the positive threshold and the number of decimal places in the target fixed-point format, minus the number of bits corresponding to the number of mapping segments. For example, if the positive threshold is 8 and the number of mapping segments is 256, the mapping interval has 11 bits in the Q16 format (i.e., a mapping interval of 2048), 10 bits in the Q15 format (i.e., a mapping interval of 1024), and 7 bits in the Q12 format (i.e., a mapping interval of 128).

[0080] It should be noted that the displacement strategy for the mapping interval is scalable and universal, and can be adjusted according to the fixed-point accuracy. More bits result in higher linear interpolation accuracy, but this increases the hardware multiplication bit width. This embodiment makes a trade-off between accuracy and hardware complexity; for example, the mapping interval corresponds to 11 bits in the Q16 format.

[0081] Alternatively, the following is a possible implementation for determining the target index and offset of the target fixed-point value in the function mapping table. Figure 1 The sub-steps of step S40 may include: Step S400: Shift the target fixed-point value to the right according to the mapping interval to obtain the target index value.

[0082] In this embodiment of the invention, the number of bits corresponding to the mapping interval (i.e., the exponent with base 2) is obtained, and the target index value is obtained by right-shifting the target fixed-point value by the number of bits corresponding to the mapping interval. For example, if the target fixed-point value is 4196 and the mapping interval is 2048, then the number of bits corresponding to the mapping interval is 11. Shifting the target fixed-point value 4196 right by 11 bits yields a target index value of 2.

[0083] It should be understood that the target index value is defined as the integer quotient obtained by dividing the target fixed-point value by the mapping interval. Its maximum value is the number of entries contained in the function mapping table (for example, the function mapping table is divided into 256 mapping segments, that is, 257 Sigmoid function values ​​are configured, with index values ​​from 0 to 256), thereby ensuring that accessing the function mapping table based on the index value will not go out of bounds.

[0084] It should be noted that the security of index calculation depends on whether the target floating-point value is within the preset non-saturated range. If the target floating-point value is not within the preset non-saturated range, the output value of the Sigmoid function based on the target floating-point value will not be generated by looking up the table.

[0085] Step S410: Extract the low-order data of the target fixed-point value through the bit mask, and determine the low-order data as the offset.

[0086] In this embodiment of the invention, the bitmask uses a mask value of the form 0x7FF (i.e., 11 consecutive binary 1s). A bitwise AND operation is performed on the target fixed-point number to retain the low-order data (e.g., the lower 11 bits), thus obtaining the offset. The offset is used to characterize the precise offset position of the target fixed-point value within the current mapping area. The offset ranges from 0 to the mapping interval minus 1. Assuming the mapping interval is 2048, the offset range is 0 to 2047.

[0087] Alternatively, the following is a possible implementation for generating the initial output value through table lookup and interpolation. Figure 1 The sub-steps of step S50 may include: Step S500: Determine the Sigmoid function value corresponding to the target index value in the function mapping table as the target function value.

[0088] Step S510: Determine the Sigmoid function value corresponding to the next index value of the target index value in the function mapping table as the reference function value.

[0089] In this embodiment of the invention, the Sigmoid function value corresponding to the target index value is obtained from the function mapping table to obtain the target function value. Then, the Sigmoid function value corresponding to the next index value of the target index value is obtained from the function mapping table to obtain the parameter function value.

[0090] Step S520: Perform interpolation adjustment based on the objective function value, reference function value, offset, and mapping interval to obtain the preliminary output value.

[0091] In this embodiment of the invention, the numerical span between two adjacent Sigmoid function values ​​is determined based on the target function value and the reference function value. Then, linear interpolation is performed using the offset to obtain a preliminary output value. The preliminary output value transitions from the target function value towards the parameter function value as the offset increases.

[0092] As can be seen, the embodiments of the present invention use the Sigmoid function value corresponding to the target index value and the next Sigmoid function value as boundary constraints, and combine the offset for interpolation processing. While ensuring the calculation speed, it effectively alleviates the problem of accuracy loss caused by discretization storage.

[0093] Optionally, regarding how to generate the output value of the Sigmoid function corresponding to the target floating-point value, the following is a possible implementation method. Please refer to... Figure 2 The sub-steps of step S520 may include: Step S521: The difference between the reference function value and the target function value is determined as the adjustment value.

[0094] In this embodiment of the invention, the difference between the parameter function value and the target function value is calculated to obtain the adjustment value. The adjustment value is the numerical difference between the Sigmoid function value corresponding to the target index value in the function mapping table and the next Sigmoid function; its magnitude reflects the adjustable range of the local region where the target floating-point value is located.

[0095] Step S522: Add half of the mapping interval to the product of the adjustment value and the offset to obtain the interpolation compensation amount.

[0096] Step S523: Shift the interpolation compensation amount to the right according to the mapping interval to obtain the interpolation compensation value.

[0097] In this embodiment of the invention, the formula for calculating the interpolation compensation value is as follows:

[0098] in, It is the interpolation compensation value; It is a reference function value; It is the value of the objective function; It is the offset; It is the mapping interval, and half of the mapping interval is used to round integers. It represents the number of bits corresponding to the mapping interval.

[0099] It should be noted that the interpolation compensation values ​​are determined in the hardware using standard linear interpolation calculation, the mathematical form of which is:

[0100] in, yes The corresponding function value; yes The corresponding function value; It is the step size; It is the function value corresponding to x obtained by interpolation.

[0101] Step S524: The sum of the objective function value and the interpolation compensation value is determined as the initial output value.

[0102] In this embodiment of the invention, the sum of the objective function value and the interpolation compensation value is calculated to obtain the preliminary output value.

[0103] Alternatively, regarding how to determine the output value of the Sigmoid function based on the sign bit of the target floating-point value, the following is a possible implementation method. Figure 1 The sub-steps of step S60 may include: If the sign bit indicates that the target floating-point value is positive, the initial output value is determined as the output value of the Sigmoid function based on the target floating-point value; if the sign bit indicates that the target floating-point value is non-positive, the difference between 1 and the initial output value is determined as the output value of the Sigmoid function based on the target floating-point value.

[0104] In this embodiment of the invention, if the target floating-point value is positive, the initial output value is directly used as the output value of the Sigmoid function based on the target floating-point value. If the target floating-point value is negative or 0, based on the symmetry of the Sigmoid function, 1 is subtracted from the initial output value to obtain the output value of the Sigmoid function based on the target floating-point value.

[0105] It should be understood that, in this embodiment of the invention, the Sigmoid function value corresponding to the target floating-point value can be determined by storing the Sigmoid function values ​​in the non-negative interval and combining them with the sign of the target floating-point value. This improves the output accuracy of the Sigmoid function without increasing the size of the lookup table.

[0106] Optionally, regarding how to generate the output value of the Sigmoid function corresponding to the target floating-point value outside the non-saturation range, the following is a possible implementation method. Figure 1 After step S10, the method may further include the following steps: Step S70: Determine whether the value to be processed is greater than the positive threshold.

[0107] Step S60: If the value to be processed is greater than the positive threshold, the preset saturation value is used as the initial output value, and the initial output value is symmetrically restored according to the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

[0108] In this embodiment of the invention, based on the saturation characteristic of the Sigmoid function, the preset saturation value is the stable output value of the Sigmoid function when the input value is positive. For example, when the positive threshold is 8, The preset saturation values ​​include, but are not limited to, 0.999664 and 1.

[0109] When the target floating-point value input to the Sigmoid function is greater than a positive threshold, a preset saturation value is determined as the output value of the Sigmoid function based on the target floating-point value. Furthermore, based on the symmetry of the Sigmoid function, when the target floating-point value input to the Sigmoid function is less than the negative of the positive threshold, the difference between 1 and the preset saturation value is determined as the output value of the Sigmoid function based on the target floating-point value. For example, when the positive threshold is 8, .

[0110] As can be seen, the embodiments of the present invention solve the problem of increased power consumption and latency caused by performing redundant table lookups on inputs whose function output values ​​tend to be stable, and can effectively improve the efficiency of hardware implementation of the Sigmoid function.

[0111] It should be noted that, in terms of hardware implementation, the function mapping table can be stored in read-only memory (ROM) or block random access memory (BRAM), shift operations can be implemented by shift registers, branch judgment can be completed by multiplexers (MUX), and interpolation operations can be implemented by fixed-point multipliers and adders working together. The overall structure does not depend on complex sequential logic or floating-point arithmetic units.

[0112] In this embodiment of the invention, the processing logic of the hardware implementation of the Sigmoid function, including sign determination, absolute value calculation, saturation region determination, target index value calculation, offset calculation, table lookup to obtain the target function value and the next function value (i.e., parameter function value), interpolation operation and result restoration, is decomposed into multi-level pipeline logic (each pipeline executes at least one processing logic, without floating-point operation, without complex exponentiation or division operation, and the hardware latency is as low as 1 to 5 clock cycles).

[0113] Based on the same inventive concept, the basic principle and technical effects of the hardware implementation device for the Sigmoid function provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments.

[0114] Please refer to Figure 3 , Figure 3 This is a block diagram of a hardware implementation device 400 for the Sigmoid function provided in an embodiment of the present invention. The hardware implementation device 400 for the Sigmoid function includes a processing module 410, a determination module 420, and a mapping module 430.

[0115] The processing module 410 is used to extract the sign bit of the received target floating-point value and take the absolute value of the target floating-point value to obtain the value to be processed; if the target floating-point value is within the preset non-saturation range, the value to be processed is converted into the target fixed-point format to obtain the target fixed-point value; the non-saturation range is determined by the positive threshold and the opposite of the positive threshold.

[0116] The determination module 420 is used to determine the mapping interval based on the target fixed-point format, positive threshold, and number of mapping segments; the number of mapping segments is the number of Sigmoid function values ​​in the function mapping table minus 1, and the Sigmoid function values ​​are generated based on non-negative numbers in the unsaturated interval; the target index value and offset are determined based on the target fixed-point value and the mapping interval.

[0117] The mapping module 430 is used to perform interpolation processing based on the mapping interval, target index value, offset and function mapping table to obtain the initial output value; and to perform symmetric restoration processing on the initial output value based on the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

[0118] In summary, the hardware implementation device for the Sigmoid function provided in this embodiment of the invention constructs a function mapping table for the Sigmoid function, containing discrete non-negative floating-point values ​​within the unsaturated interval, based on the symmetry and saturation of the Sigmoid function, thereby substantially reducing the scope of the lookup table. On this basis, the absolute value of the floating-point values ​​belonging to the unsaturated interval is taken to obtain the value to be processed. This value is then converted into the target fixed-point format, and lookup and interpolation are performed using integer operations to obtain the initial output value. Finally, the initial output value is symmetrically restored using the sign bit of the target floating-point value to obtain the output value of the Sigmoid function based on the target floating-point value, thus improving computational efficiency. While ensuring the function output capability across the entire range, the size and storage overhead of the function mapping table are effectively reduced, while output accuracy is improved through linear interpolation in the fixed-point format.

[0119] Optionally, the processing module 410 is used to shift the value to be processed to the left by the number of decimal places in the target fixed-point format and round it to obtain the target fixed-point value.

[0120] Optionally, the determining module 420 is used to shift the positive number threshold to the left by the number of decimal places in the target fixed-point format to obtain the fixed-point number threshold; determine the number of mapping bits according to the number of mapping segments; and shift the fixed-point number threshold to the right by the number of mapping bits to obtain the mapping interval.

[0121] Optionally, the determining module 420 is used to right-shift the target fixed-point value according to the mapping interval to obtain the target index value; and to extract the low-order data of the target fixed-point value through a bit mask, and determine the low-order data as the offset.

[0122] Optionally, the mapping module 430 is used to determine the Sigmoid function value corresponding to the target index value in the function mapping table as the target function value; determine the Sigmoid function value corresponding to the next index value of the target index value in the function mapping table as the reference function value; and perform interpolation adjustment based on the target function value, the reference function value, the offset, and the mapping interval to obtain the preliminary output value.

[0123] Optionally, the mapping module 430 is used to determine the difference between the reference function value and the target function value as the adjustment value; add half of the mapping interval to the product of the adjustment value and the offset to obtain the interpolation compensation amount; shift the interpolation compensation amount to the right according to the mapping interval to obtain the interpolation compensation value; and determine the sum of the target function value and the interpolation compensation value as the initial output value.

[0124] Optionally, the mapping module 430 is used to determine the initial output value as the output value of the Sigmoid function based on the target floating-point value if the sign bit indicates that the target floating-point value is positive; and to determine the difference between 1 and the initial output value as the output value of the Sigmoid function based on the target floating-point value if the sign bit indicates that the target floating-point value is non-positive.

[0125] Optionally, the mapping module 430 is used to determine whether the value to be processed is greater than a positive threshold; if the value to be processed is greater than the positive threshold, the preset saturation value is used as the initial output value, and the initial output value is symmetrically restored according to the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

[0126] Taking FPGA as an example, the hardware implementation of the Sigmoid function in FPGA includes a processing module 410, a determination module 420, and a mapping module 430. The processing module 410 includes a comparator, a multiplexer, a shift register, and a LUT. It uses the comparator and multiplexer to determine whether the target floating-point value is within a preset non-saturated range, uses the LUT to determine the sign of the target floating-point value, and uses the shift register to convert the target floating-point value located in the non-saturated range into a target fixed-point value.

[0127] The determination module 420 includes a BRAM, a shift register, and a bitmask circuit. It stores the function mapping table in the BRAM (i.e., writes the SIGMOID_LUT_ROM into the FPGA's Block RAM, configured in read-only mode), and uses the shift register and bitmask circuit to determine the mapping interval, target index value, and offset. The mapping module 430 includes an adder, a subtractor, a DSPSlice multiplier, a shift register, and a multiplexer. It performs interpolation and result restoration (i.e., restoration using symmetry properties) using the adder, subtractor, DSP Slice multiplier, shift register, and multiplexer, ultimately completing the hardware implementation of the Sigmoid function.

[0128] In the hardware implementation of the Sigmoid function, the sigmoid_q16_lut function implementation is broken down into a 5-stage pipeline, including sign determination and absolute value calculation, saturation region determination, target index and offset calculation, table lookup to obtain the target function value and the next function value (i.e., the parameter function value), interpolation operation, and result restoration. The overall system occupies 1.2K LUTs, 8 BRAMs, and 16 multipliers, with resource utilization not exceeding 30%. Compared to traditional floating-point schemes (requiring dedicated FPU IP), resource consumption is reduced by approximately 60%.

[0129] To improve efficiency, parallel processing (i.e., multi-channel table lookup) can be implemented. For example, 16 Sigmoid operation cores can be instantiated to process 16 tensor elements (i.e., 16 floating-point values) in parallel. With a clock frequency set to 100MHz, the throughput can reach [a certain value]. The operation is performed at a rate of approximately 1.2 KLUTs, 8 BRAMs (approximately 16 Kb), and 16 multipliers, with a resource utilization rate of no more than 30%.

[0130] It's important to note that the 16-way parallelism is achieved by replicating a single pipeline unit 16 times, with each unit processing one data element independently. Each unit contains 5 pipeline stages, each requiring only one clock cycle to complete the computation, making the entire process highly optimized. Data distribution is handled globally, distributing input data to the 16 parallel units, each performing independent computation, and finally summing the results in the output buffer. In this way, the hardware can process multiple data points in parallel within the same clock cycle, significantly improving computational throughput. This ensures both the efficiency of single-element operations and meets the requirements of batch processing tensors in deep learning.

[0131] Taking an embedded MCU as an example, the processing module 410 includes comparison instructions and an integer ALU. The comparison instructions determine whether the target floating-point value is within a preset non-saturated range, and the integer ALU converts the target floating-point value into a target fixed-point value. The determination module 420 includes ROM, shift instructions, and subtraction instructions. The ROM stores a function mapping table, and the shift and subtraction instructions determine the mapping interval, target index value, and offset. The mapping module 430 includes a hardware multiplier, shift instructions, an integer addition unit, and subtraction instructions to perform interpolation and result restoration, obtaining the output value of the Sigmoid function based on the target floating-point value.

[0132] It should be noted that the SIGMOID_LUT_ROM array can be placed in the read-only data segment of Flash using the __attribute__((section(".rodata"))) attribute to permanently store the function mapping table. A static array is used to store the tensor data, avoiding dynamic allocation. Testing was conducted using the test data [-5, -1, 0, 1, 2, 5, 8, -8], and the output Sigmoid result was compared with the standard floating-point implementation; the error did not exceed [the specified value]. .

[0133] Taking ASIC as an example, the SIGMOD_LUT_ROM is designed as a mask ROM with an area of ​​approximately 0.01 mm² to store the function mapping table. (Customization) A 32-bit fixed-point multiplier and a 32-bit adder are implemented, with power consumption optimized to approximately 0.1mW / MHz. An 8-stage pipeline is used to implement the Sigmoid function, increasing the clock frequency to approximately 500MHz and reducing latency to approximately 8ns. The overall area is approximately 0.1mm², with power consumption of approximately 5mW (500MHz), meeting the low-power requirements of mobile chips. Compared to traditional solutions (0.5mm² area, 50mW power consumption), the area is reduced by approximately 80%, and power consumption is reduced by approximately 90%.

[0134] By comparing and analyzing the traditional floating-point lookup table method, the traditional fixed-point lookup table method, the polynomial approximation method, and the Sigmoid function implemented in this invention embodiment, the Sigmoid function implemented in this invention embodiment has the following beneficial effects: 1. Reduced storage space usage The traditional floating-point lookup table method covers the range [-18, 18] with a step size of 0.01, occupies 14.4KB of storage space, and has a maximum error of 0.0005. The traditional fixed-point lookup table method covers the range [-18, 18] with a step size of 0.1, occupies 7.2KB of storage space, and has a maximum error of 0.005. The polynomial approximation method covers the range [-5, 5] with 0KB of storage space and a maximum error of 0.001 within the range. The Sigmoid function implemented in this embodiment covers the range [-18, 18] with a step size of 0.03125, occupies 0.5KB of storage space, accounting for only about 7% of the traditional lookup table method, and covers the entire range with a maximum error of [missing value]. .

[0135] 2. Significantly improved computational efficiency The embodiments of this invention utilize multi-stage pipelines to achieve a latency of 1-5 clock cycles, which is 4-20 times faster than traditional floating-point schemes (20+ clock cycles) and 1-2 times faster than polynomial schemes (5-8 cycles). The embodiments of this invention achieve a throughput of approximately [missing information - likely a specific throughput value] on an FPGA through 16-way parallel processing. The throughput reaches approximately [number] times per second, implemented with an ASIC at 500MHz. It achieves real-time inference requirements at times per second.

[0136] 3. Precision is controllable and there are no interval limitations. The maximum error in the embodiments of the present invention does not exceed [a certain value] across the entire range. The average error does not exceed The maximum percentage error does not exceed 0.05%. Compared to the polynomial scheme which can only cover the [-5,5] interval, the embodiments of the present invention cover the entire interval [-18,18], and the error does not grow exponentially.

[0137] 4. Advantages in hardware resources and power consumption When implemented on an FPGA, this invention occupies approximately 1.2 KLUTs, 8 BRAMs, and 16 multipliers, with a resource utilization rate of no more than 30%, representing a reduction of approximately 60% in resource usage compared to floating-point solutions. When implemented on an ASIC, the overall area is approximately 0.1 mm², with a power consumption of approximately 5 mW (500 MHz), representing an approximately 80% reduction in area and a approximately 90% reduction in power consumption compared to traditional solutions (0.5 mm² area, 50 mW power consumption).

[0138] 5. Strong adaptability and scalability It is compatible with various MCU models (such as ARM Cortex-M / R, RISC-V), FPGAs, and ASICs (such as 40nm and 20nm processes), supports parallel processing of NCHW format tensors (i.e., parallel processing of each floating-point value in the NCHW format floating-point set, storing the output value of the Sigmoid function corresponding to each floating-point value in the output tensor, and returning the output values ​​of the Sigmoid function corresponding to all floating-point values ​​in the output tensor after processing), and can be directly integrated into the activation function unit of deep learning accelerators.

[0139] Please refer to Figure 4 This is a block diagram of an electronic device 500 provided in an embodiment of the present invention. The electronic device 500 includes, but is not limited to, devices equipped with hardware chips such as embedded MCUs, FPGAs, and ASICs that need to perform AI inference. The electronic device 500 includes a hardware chip 510 and a communication module 520. The components of the hardware chip 510 and the communication module 520 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0140] The hardware chip 510 is used to store computer programs or data, and to read the computer programs and execute corresponding functions. For example, when the hardware chip 510 executes the stored computer program, it can implement the hardware implementation method of the Sigmoid function disclosed in the above embodiments.

[0141] The hardware chip 510 can be, but is not limited to, an embedded microcontroller unit (MCU), a field-programmable gate array (FPAG), or an application-specific integrated circuit (ASIC).

[0142] The communication module 520 is used to establish a communication connection between the electronic device 500 and other communication terminals via a network, and to send and receive data via the network.

[0143] It should be understood that, Figure 4 The structure shown is only a schematic diagram of the electronic device 500. The electronic device 500 may also include components that are larger than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof.

[0144] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a hardware chip 510, implements the hardware implementation method of the Sigmoid function disclosed in the above embodiments.

[0145] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0146] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0147] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A hardware implementation method for the Sigmoid function, characterized in that, The method includes: Extract the sign bit of the received target floating-point value, and take the absolute value of the target floating-point value to obtain the value to be processed; If the target floating-point value is within a preset non-saturation range, the value to be processed is converted into a target fixed-point format to obtain the target fixed-point value; the non-saturation range is determined by a positive threshold and the opposite of the positive threshold; The mapping interval is determined based on the target fixed-point format, the positive threshold, and the number of mapping segments; the number of mapping segments is the number of Sigmoid function values ​​in the function mapping table minus 1, and the Sigmoid function values ​​are generated based on the non-negative numbers of the unsaturated interval. The target index value and offset are determined based on the target fixed-point value and the mapping interval; Interpolation is performed based on the mapping interval, the target index value, the offset, and the function mapping table to obtain a preliminary output value; The initial output value is symmetrically restored based on the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

2. The hardware implementation method of the Sigmoid function according to claim 1, characterized in that, The step of converting the value to be processed into a target fixed-point format to obtain the target fixed-point value includes: The value to be processed is shifted left by the number of decimal places in the target fixed-point format and then rounded to obtain the target fixed-point value.

3. The hardware implementation method of the Sigmoid function according to claim 1, characterized in that, The step of determining the mapping interval based on the target fixed-point format, the positive threshold, and the number of mapping segments includes: The positive number threshold is shifted left by the number of decimal places in the target fixed-point format to obtain the fixed-point number threshold; The number of mapping bits is determined based on the number of mapping segments; The mapping interval is obtained by right-shifting the fixed-point threshold by the number of mapping bits.

4. The hardware implementation method of the Sigmoid function according to claim 1, characterized in that, The step of determining the target index value and offset based on the target fixed-point value and the mapping interval includes: The target fixed-point value is shifted to the right according to the mapping interval to obtain the target index value; The low-order bits of the target fixed-point value are extracted by bitmasking, and the low-order bits are determined as the offset.

5. The hardware implementation method of the Sigmoid function according to claim 1, characterized in that, The step of interpolating based on the mapping interval, the target index value, the offset, and the function mapping table to obtain a preliminary output value includes: The Sigmoid function value corresponding to the target index value in the function mapping table is determined as the target function value; The Sigmoid function value corresponding to the next index value of the target index value in the function mapping table is determined as the reference function value; Interpolation adjustments are performed based on the target function value, the reference function value, the offset, and the mapping interval to obtain a preliminary output value.

6. The hardware implementation method of the Sigmoid function according to claim 5, characterized in that, The step of interpolating and adjusting based on the objective function value, the reference function value, the offset, and the mapping interval to obtain the preliminary output value includes: The difference between the reference function value and the target function value is determined as the adjustment value; Add half of the mapping interval to the product of the adjustment value and the offset to obtain the interpolation compensation amount; The interpolation compensation amount is shifted to the right according to the mapping interval to obtain the interpolation compensation value; The sum of the objective function value and the interpolation compensation value is determined as the initial output value.

7. The hardware implementation method of the Sigmoid function according to claim 1, characterized in that, The step of performing symmetrical restoration processing on the initial output value based on the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value includes: If the sign bit indicates that the target floating-point value is positive, the initial output value is determined as the output value of the Sigmoid function based on the target floating-point value; If the sign bit indicates that the target floating-point value is non-positive, the difference between 1 and the initial output value is determined as the output value of the Sigmoid function based on the target floating-point value.

8. The hardware implementation method of the Sigmoid function according to claim 1, characterized in that, After taking the absolute value of the target floating-point value to obtain the value to be processed, the method further includes: Determine whether the value to be processed is greater than the positive threshold; If the value to be processed is greater than the positive threshold, then the preset saturation value is used as the initial output value, and the initial output value is symmetrically restored according to the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

9. A hardware implementation device for the Sigmoid function, characterized in that, The device includes: The processing module is used to extract the sign bit of the received target floating-point value and take the absolute value of the target floating-point value to obtain the value to be processed; if the target floating-point value is within a preset non-saturation range, the value to be processed is converted into a target fixed-point format to obtain a target fixed-point value; the non-saturation range is determined by a positive threshold and the opposite of the positive threshold. The determination module is used to determine the mapping interval based on the target fixed-point format, the positive threshold, and the number of mapping segments; the number of mapping segments is the number of Sigmoid function values ​​in the function mapping table minus 1, and the Sigmoid function values ​​are generated based on the non-negative numbers of the unsaturated interval; the module also determines the target index value and offset based on the target fixed-point value and the mapping interval. The mapping module is used to perform interpolation processing based on the mapping interval, the target index value, the offset, and the function mapping table to obtain a preliminary output value; and to perform symmetric restoration processing on the preliminary output value based on the sign bit to obtain the output value of the Sigmoid function based on the target floating-point value.

10. An electronic device, characterized in that, It includes a hardware chip that can execute a computer program to implement the hardware implementation method of the Sigmoid function according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the hardware chip, it implements the hardware implementation method of the Sigmoid function as described in any one of claims 1-8.

Images