A model data access method, apparatus, device, and computer storage medium

By determining the target execution layer, correcting the access address, and prefetching weighted data into the preset level cache, the problem of limited cache resources on terminal devices is solved, achieving efficient task response and stability under resource-constrained conditions.

CN123068075APending Publication Date: 2026-11-10CHINA MOBILE COMM GRP TERMINAL +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611152994.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-31
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Terminal devices have limited on-chip cache resources. Long-term reservation of fixed cache space will consume system resources, resulting in slow task response speed and poor stability. Existing technologies cannot effectively solve the problems of response latency and cache missing under resource-constrained conditions.

Method used

By determining the target execution layer of the target model, correcting the access address of the weight data, dividing the sub-intervals and generating a prefetch instruction sequence, prefetching the weight data into the preset level cache, and locking the cache line to ensure data reliability, the fast access is achieved by utilizing the characteristics of the preset level cache.

Benefits of technology

Under resource-constrained conditions, it improves task response speed and stability, reduces memory access latency, and ensures the continuity and efficiency of model computation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068075A_ABST
    Figure CN123068075A_ABST
Patent Text Reader

Abstract

This application belongs to the field of artificial intelligence, and particularly relates to a model data access method, apparatus, device, and computer storage medium, comprising: determining the target execution layer of the target model; correcting the theoretical access address based on the offset between the theoretical access address of the weight data in memory and the actual access address in historical inference trajectory data, to obtain the access address range of the weight data in memory; dividing the access address range of the weight data into multiple sub-ranges, and generating a weight prefetch instruction sequence based on the line length of a preset-level cache; prefetching weight data from multiple sub-ranges into the preset-level cache using the weight prefetch instruction sequence; and locking the cache line containing the weight data after the weight data is written to the preset-level cache. This method predicts the position of the weights using a resource-constrained cache, and then locks and unlocks the weights, thereby ensuring task response speed and execution stability under resource constraints.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence, and in particular relates to a model data access method, apparatus, device and computer storage medium. Background Technology

[0002] With the rapid development of the Internet of Things (IoT), real-time inference and feature matching on the edge have become core technologies for realizing human-computer interaction, environmental perception, and instant response. Current mainstream solutions generally deploy deep learning models on terminal devices, relying on the Neural Processing Unit (NPU) to perform real-time computation locally, thereby achieving various recognition, detection, and matching functions under low-power constraints.

[0003] However, existing technologies have obvious technical bottlenecks in practical applications: due to the limited on-chip cache resources of terminal devices, reserving fixed cache space for weights for a long time will crowd out system resources, further exacerbating the system resource shortage; if the weights are not cached for protection, they are easily replaced during the calculation process, resulting in large fluctuations in response latency and reduced task response speed.

[0004] Therefore, there is an urgent need in this technical field for a method that can adapt to resource-constrained conditions while ensuring task response speed, in order to solve the above-mentioned technical problems. Summary of the Invention

[0005] This application provides a model data access method, apparatus, device, and computer storage medium that can adapt to resource-constrained conditions while ensuring task response speed.

[0006] On one hand, embodiments of this application provide a model data access method, the method including: Determine the target execution layer of the target model; Based on the offset between the theoretical access address of the weight data in memory and the actual access address in the historical inference trajectory data, the theoretical access address is corrected to obtain the access address range of the weight data in memory. The access address range of weight data is divided into multiple sub-ranges, and a weight prefetch instruction sequence is generated based on the line length of the preset level cache. Weighted data is prefetched from multiple sub-intervals into a preset level cache using a weighted prefetch instruction sequence. After the weight data is written to the preset level cache, the cache line containing the weight data is locked so that the target execution layer can read the weight data from the locked cache line for calculation.

[0007] On the other hand, embodiments of this application provide a model data access device, the device comprising: The determination module is used to determine the target execution layer of the target model; The correction module is used to correct the theoretical access address based on the offset between the theoretical access address of the weight data in memory and the actual access address in the historical inference trajectory data of the target execution layer. Based on the corrected theoretical access address, the access address range of the weight data in memory is obtained. The generation module is used to divide the access address range of weight data into multiple sub-ranges and generate a weight prefetch instruction sequence based on the line length of the preset level cache. The prefetch module is used to prefetch weight data from multiple sub-intervals into a preset level cache using a weight prefetch instruction sequence; The processing module is used to lock the cache line containing the weight data after the weight data is written to the preset level cache, so that the target execution layer can read the weight data from the locked cache line for calculation.

[0008] In another aspect, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions; When the processor executes computer program instructions, it implements the model data access method of any one of the first aspects.

[0009] In another aspect, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the model data access method of any one of the first aspects is implemented.

[0010] In another aspect, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the model data access method of any one of the first aspects.

[0011] The model data access method, apparatus, device, computer storage medium, and computer program product of this application embodiment determine the target execution layer of the target model, then determine the theoretical access address in memory of the weight data required for the target model inference through the target execution layer, then correct the theoretical access address based on the offset between the actual access addresses in the historical inference trajectory, and finally obtain the access address range of the weight data in memory. Determining the target execution layer is the preliminary decision basis for prefetching weight data in subsequent steps, and determines the calculation logic of subsequent memory access operations. Since the weight data is stored in Double Data Rate (DDR) memory, it is necessary to determine the specific access address range of the weight data in DDR based on the specific target execution layer, so that the weight data can be prefetched into a preset level cache according to the access address range, and the cache line where the weight data is located can be locked. The preset-level cache is a first-level cache that can be directly written to by Direct Memory Access (DMA) or the cache controller. Other levels of cache cannot be directly operated on. Based on the characteristics of the preset-level cache, it is possible to read weight data from the locked cache line and then perform calculations on the target model based on the weight data, thereby ensuring task response speed under resource-constrained conditions. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the flow of a model data access method provided in one embodiment of this application; Figure 2 This is a schematic diagram of the flow of a model data access method provided in another embodiment of this application; Figure 3 This is a schematic diagram of the acoustic scene complexity assessment and hierarchy selection process in this application; Figure 4 This is a schematic diagram of the physical address prediction process based on historical reasoning trajectories in this application; Figure 5 This is a schematic diagram of the cache prefetch instruction generation and data locking process of this application; Figure 6 This is a schematic diagram of the zero-miss reasoning process under cache locking in this application; Figure 7 This is a schematic diagram of the cache resource release and performance recovery process in this application; Figure 8This is a schematic diagram of the hardware structure of the model data access device provided in this application; Figure 9 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0014] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0015] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0016] With the widespread adoption of smart terminal devices and the rapid development of IoT technology, voice interaction and facial recognition have become important methods of human-computer communication. As a triggering element in interpersonal communication, the wake-up technology of smart terminal devices needs to continuously receive trigger information from the environment while maintaining extremely low power consumption. Trigger information is data that enables the smart terminal device to transition from standby or other non-working states to working states. For example, when the primary wake-up method for a smart terminal device is voice interaction, the trigger information is the wake word; when the primary wake-up method is image recognition, the trigger information is the image features of the screen. Other devices will not be discussed further. Therefore, the performance of a smart terminal device directly determines the user's initial experience. This is because the performance of a smart terminal device directly affects the speed and stability of its trigger information recognition. If the recognition speed is too slow, users will have to wait too long, affecting the user experience. If the stability is poor, users will need to repeatedly provide trigger information, further deteriorating the user experience.

[0017] Deep learning models are commonly used in related technologies. By deploying an NPU on the edge, power consumption and performance are balanced, enabling deep learning models to run in real time on resource-constrained embedded devices. In one specific example, this technology receives audio data from a smart terminal device and caches audio for a preset duration in a buffer. When it initially determines that the received audio data contains a specified trigger information, i.e., a wake word, the cached audio and subsequent audio are sent to a cloud server for secondary confirmation. The contextual information of the wake word is combined to determine the user's intent and reduce false wake-ups. While this technology has made progress in improving wake-up accuracy, its reliance on cloud interaction inevitably introduces network transmission latency. Furthermore, the wake-up decision-making process spans both the edge and cloud, resulting in a relatively long overall response time, making it difficult to meet the low-latency requirements of completely offline scenarios. In another specific example, this technology sets up a common buffer pool to simultaneously send data streams to the wake-up engine and the cloud caching module, and synchronizes the timing position after a wake-up is triggered to initiate the upload. This design optimizes the data flow path, but mainly focuses on flow control management for cloud interaction. It does not address the specific computational efficiency and memory access latency during model execution on the edge. When the model weight data is large, there is still a bottleneck caused by frequent waiting for data to be loaded from the main memory when running on the processor of a resource-constrained embedded device.

[0018] A significant issue in related technologies is the limited on-chip cache resources of terminal devices. Long-term reservation of fixed cache space for weights severely squeezes resources allocated to other system tasks. Consequently, the weight data required for model inference needs to be frequently loaded from main memory, and memory access latency restricts the terminal's response speed. When the execution path and weight data dynamically switch with the input scenario, the cross-task reuse rate of weights is low, and the long-term reserved cache space quickly expires with weight switching, offering no reuse benefits and continuously occupying scarce on-chip cache resources. In this scenario, conventional long-term cache residency or locking solutions are not feasible. However, without cache residency protection for weights, the weights loaded into the cache are subject to the system's global replacement mechanism, and may be randomly replaced during computation. This leads to uncontrollable timing and frequency of cache misses in a single round of computation, ultimately causing large fluctuations in response latency and poor execution stability.

[0019] Therefore, with the increasing complexity of model structures and the diversification of application scenarios, this technical field urgently needs a method that can adapt to resource-constrained conditions while ensuring task response speed, in order to solve the above-mentioned technical problems.

[0020] To address the problems of the prior art, embodiments of this application provide a model data access method, apparatus, device, and computer storage medium. The model data access method provided in this application embodiment will be described first.

[0021] In one specific embodiment of this application, a model data access method is provided, such as... Figure 1 As shown, Figure 1 A flowchart illustrating the model data access method provided in this embodiment is shown. The specific process is as follows: Step S101: Determine the target execution layer of the target model.

[0022] Step S102: Based on the offset between the theoretical access address of the weight data in memory and the actual access address in the historical inference trajectory data, the theoretical access address is corrected to obtain the access address range of the weight data in memory.

[0023] Specifically, the target model is an AI inference model deployed on a terminal device, possessing multiple execution layers with different computing power specifications. For example, an intelligent voice wake-up model can dynamically select the corresponding execution layer to complete inference operations based on different scenarios. Since the target model contains several execution layers, each layer generally differs in structure and computation. These execution layers are the prerequisites for subsequent weight prefetching, cache locking, and inference operations, directly determining the target scope and computational logic of subsequent memory access operations. Therefore, the terminal device's scheduling and control unit determines a target execution layer from among these layers. The target execution layer is the network inference layer selected based on the task scenario complexity and adapted to the current load computing power specifications. It is the corresponding working layer where the system loads the corresponding weights and performs inference operations, such as an execution layer containing loop structures or an execution layer containing convolutional structures. The terminal device's scheduling and control unit selects the target execution layer based on the specific task: whether it's used to enhance the modeling capability for long-term dependencies (execution layers containing loop structures) or to reduce computational path depth and shorten inference latency (execution layers containing convolutional structures), providing a hierarchical foundation for subsequent operations.

[0024] Because each execution layer stores pre-defined weight data at a specific location, this weight data consists of the training parameters of the target model for each execution layer. Stored in main memory, it is the core computational data that the execution layer needs to load and call during inference operations. This specific location is the theoretical access address; ideally, the device should be able to retrieve the weight data from this address. However, in practice, the theoretical access address is affected by factors such as memory page alignment, data block loading start offset, and scheduling logic. The starting cache line accessed during actual inference may differ from the theoretical access address. Directly prefetching from the theoretical address results in data that is not actually used from the cache line. When the cache expires, it must be read back from main memory, causing latency instability. Historical inference trajectory data is the statistical record of weight data and actual memory access address information from multiple rounds of inference in the model. It is used to calculate the offset between the theoretical memory access address and the actual memory access address to correct the prefetch address. Therefore, calculating the offset between the theoretical access address of the target execution layer's weight data in memory and the actual access address in the historical inference trajectory data is to use the statistical deviation of the historical memory access trajectory to correct the theoretical access address, so that the prefetched cache line hits the actual access position exactly, and finally the access address range can be obtained.

[0025] At the hardware level, the scheduling control unit reads the access history table corresponding to the target execution layer. This access history table is a pre-stored historical runtime statistics table that corresponds one-to-one with each model execution layer. It records the historical inference memory access trajectory and average memory bandwidth usage of the corresponding execution layer, used to correct the access address range and schedule prefetch bandwidth. Since the table structure of the access history table and the physical address mapping of the target execution layer is known, the consistency of the data structure can be guaranteed. Furthermore, since the total byte length of the corresponding weight data for each model layer has been determined during the compilation stage, and the weight data exhibits a continuous segmented linear distribution in Dynamic Random Access Memory (DRAM), the address range of the weight data in DRAM can be inferred by confirming the target execution layer.

[0026] To more intuitively and accurately determine in which segment of main memory to retrieve all the weight data of the target execution layer, and to avoid the problem of poor retrieval accuracy caused by the deviation between the theoretical access address and the actual access address, in a specific embodiment of this application, the theoretical access address of the weight data of the target execution layer in memory can be calculated based on the target execution layer provided in step S101 in the following manner: Where i is used to identify the i-th execution layer, and i is a positive integer, therefore It is the hierarchical number of the target execution layer. It is the theoretical memory access address of the weight data of the target execution layer. express The starting address of the weight data for the corresponding target execution layer originates from the static mapping table established during the model loading phase. yes The total byte length of the weight data of the target execution layer is determined by the target execution layer itself.

[0027] To further improve prediction accuracy, the system also stores historical inference trajectories from the most recent N (N is a positive integer) inference processes in the access history table. By performing offset statistics on these historical inference trajectories, the prediction accuracy can be calculated. The offset between the theoretical access address of the corresponding target execution layer's weight data in memory and the actual access address in the historical inference trajectory data. Then, the address range is corrected to obtain the memory access address range of the weight data. The memory access address range of the weight data is calculated using the following formula: Where i, as above, is the hierarchy number of the execution layer of the target model. yes The corresponding target execution layer's weight data's memory access address range represents the precise physical address range that needs to be prefetched for this inference; This represents the average offset between the first cache line hit address and the starting address of the address range in the historical inference trajectory. It is used to eliminate address drift errors caused by layout during compilation and page mapping during runtime. N represents the number of historical inference trajectories used for statistics. N is selected by the system while maintaining a scenario complexity consistent with the current task scenario. This ensures that the historical inference trajectories are consistent with the complexity level of the current task scenario. The reason is that not all historical inference trajectories from the most recent N inference processes can be arbitrarily retrieved. The system first performs scenario classification, selecting only past inference records from scenarios with the same complexity as the current task scenario. Since the main memory access patterns of the corresponding level of weight data differ in scenarios with different noise and fluctuation levels, calculating the average actual memory access starting offset using only historical inference trajectories from the same scenario results in more accurate memory address predictions. This avoids prediction errors caused by mixed samples and improves prediction stability.

[0028] Step S103: Divide the access address range of the weight data into multiple sub-ranges, and generate a weight prefetch instruction sequence based on the line length of the preset level cache.

[0029] Step S104: Prefetch weight data from multiple sub-intervals into a preset level cache using a weight prefetch instruction sequence.

[0030] Specifically, the weight prefetch instruction sequence is a set of instructions generated based on the line length of the preset level cache, the divided weight address sub-ranges, and the bandwidth scheduling timing. It is used to control the orderly prefetching of corresponding weight data from main memory to the preset level cache. The preset level cache is a specific cache level pre-specified in the scheme that supports direct writing by DMA or the cache controller and can lock the cache line through registers. The preset level cache is usually the last level cache, such as a multi-core shared L3 cache or an embedded dynamic random access memory (eDRAM) integrated on the wafer. The reason for transferring weight data within the address range from main memory to the last level cache is that the last level cache is a level cache that can be directly written to by DMA or the cache controller. Other levels of cache usually cannot be directly written to. Therefore, the last level cache is a prerequisite for the system to lock the cache step through the cache control register. After all weighted data is prefetched into the last-level cache, the cache line containing the cached data is directly locked via the cache control register. This allows the Least Recently Used (LRU) replacement algorithm to skip locked cache lines, preventing the accidental replacement of prefetched critical weighted data, ensuring that inference data resides in the cache, and reducing access latency. The line length of the default-level cache is the smallest data management and read / write unit specified by the hardware for that cache level, measured in bytes. It determines the alignment rules of the cached data, the prefetch granularity, and the minimum range of locking operations. It is the core hardware benchmark for dividing the weighted address sub-ranges and calculating the number of cache lines to be prefetched. For example, when a multi-core processor's shared L3 cache is used as the default-level cache, its line length is typically 64 bytes. The weighted access address sub-ranges must be aligned to 64 bytes, and each aligned sub-range corresponds to a line of cache prefetch write and locking operations. It should be noted that the default-level cache is not limited to the last-level cache. Any L1 cache that can be directly written to by DMA or the cache controller can be a default-level cache. Any level cache that meets the above conditions is a default-level cache.

[0031] Step S105: After the weight data is written to the preset level cache, the cache line containing the weight data is locked so that the target execution layer can read the weight data from the locked cache line for calculation.

[0032] Specifically, in order to prevent the prefetched key weight data from being accidentally replaced by the LRU replacement algorithm, ensure that inference data resides in the cache and reduce access latency, the NPU directly reads the weight data from the locked cache line to participate in multiply-accumulate operations. This avoids waiting to load data from main memory due to cache missing data and shortens the memory access latency during model execution.

[0033] In the embodiments of this application, the target execution layer of the target model is determined, and then the theoretical access address of the weight data required for the target model inference in memory is determined through the target execution layer. The theoretical access address is then corrected based on the offset between the actual access addresses in the historical inference trajectory, finally obtaining the access address range of the weight data in memory. Determining the target execution layer is the preliminary decision basis for prefetching weight data in subsequent steps, determining the computational logic of subsequent memory access operations. Since the weight data is stored in DDR, it is necessary to determine the specific access address range of the weight data in DDR based on the specific target execution layer. This allows the weight data to be prefetched into a preset-level cache based on the access address range, locking the cache line where the weight data is located. The preset-level cache is a level-one cache that can be directly written to by direct memory access DMA or a cache controller; other levels of cache cannot be directly operated on. Based on the characteristics of the preset-level cache, it is possible to read weight data from the locked cache line and then perform target model calculations based on the weight data, thereby ensuring task response speed under resource-constrained conditions.

[0034] In a specific embodiment of this application, since the target execution layer determines the various data used in subsequent calculations and is the foundation of all calculations, a more accurate way is needed to determine the target execution layer of the target model to improve the accuracy of the overall inference process, such as... Figure 2 As shown, step S101 further includes the following steps: Step S3011: Based on the received trigger information from the target execution layer, determine the fluctuation level information and the dispersion level information of the trigger information respectively.

[0035] Step S3012: Determine the complexity of the task scenario based on the fluctuation information, dispersion information, preset fluctuation coefficient, and preset dispersion coefficient of the trigger information.

[0036] Specifically, in determining the complexity of a task scenario, the volatility of the trigger information represents the drasticness of its temporal changes, while the dispersion of its numerical distribution represents the dispersion of its values. These two values ​​are multiplied by their respective preset coefficients and then summed with weights to obtain a unified task scenario complexity value. This quantifies the load intensity and input instability of the current inference task, supporting the subsequent selection of the target execution layer and the control of prefetch bandwidth. The preset volatility coefficient and preset dispersion coefficient are pre-configured parameters used for weighting, corresponding to the temporal volatility amplitude and numerical dispersion of the trigger information, respectively. Multiplying these two values ​​by their corresponding dimension's calculation results and summing them yields the task scenario complexity value, used to adjust the influence of these two dimensions on the load intensity determination result. For example, in a voice wake-up scenario, the preset volatility coefficient can be set to 0.6 and the preset dispersion coefficient to 0.4, and the weighted summation of the temporal change amplitude and numerical dispersion of the audio trigger signal quantifies the scenario complexity value of the current task. Based on this, the corresponding trigger information is first obtained through the terminal device. In a specific example, the type of trigger information corresponds to the terminal device. If the trigger information is voice wake-up information, it is received through the audio receiver of the terminal device, and in this case, the voice wake-up information is the corresponding trigger information. If the trigger information is facial information, it is obtained through the terminal device. In this embodiment, any information that can be input to the terminal device can be considered trigger information, which will not be elaborated further here.

[0037] After obtaining the trigger information, the system performs preliminary verification through the front-end processing module. Then, it outputs the feature tensor, already calculated in the front-end module, to a shared memory buffer. Based on its physical start address and length information, it enters the access history table corresponding to the target execution layer. The shared memory buffer serves as a data intermediary for the feature tensor output by the front-end module. The access history table retains data address information, ensuring a consistent address index structure and providing a basis for subsequent model parameter physical address prediction and cache prefetching.

[0038] Based on the received trigger information from the target execution layer, the fluctuation level and dispersion level of the trigger information are determined respectively. The specific formula for determining the task scenario complexity using preset fluctuation and dispersion coefficients is as follows: in, This metric measures the non-stationarity of the current input trigger information in the time and frequency domains, thus determining whether to use a convolutional layer or a recurrent layer. For example, in a voice wake-up scenario, when environmental noise is high and the audio input signal fluctuates frequently, the calculated task complexity is higher, and the system will correspondingly select an execution layer with more powerful computing capabilities. The access history table contains information on the degree of fluctuation. , This value measures the degree of drastic change in input features in the temporal or spatial dimensions. A higher value indicates a more unstable signal or feature, making it more difficult to identify. It also includes information on the degree of discreteness. , This value measures the dispersion of input features in the feature space; a larger value indicates more dispersed features, lower density of effective information, and higher recognition difficulty. It also includes... and , which are preset fluctuation coefficients and preset dispersion coefficients, respectively, used to balance the contribution ratio of the above two dimensions to the complexity determination, and are preset empirical coefficients.

[0039] Step S3013: When the complexity of the task scenario is greater than a preset threshold, the target execution layer is determined to be a high-complexity model execution layer; when the complexity of the task scenario is less than the preset threshold, the target execution layer is determined to be a low-complexity model execution layer.

[0040] Specifically, after calculating the scene complexity, when the task scene complexity exceeds a preset threshold, a high-complexity model execution layer containing recurrent layers can be selected. The total byte length of the weight data is determined by the path of the recurrent layer because the recurrent unit corresponding to the recurrent layer path has a larger hidden dimension, a larger weight scale, and more data volume, which is used to handle complex acoustic scenarios and support long-term feature processing. When the task scene complexity is less than the preset threshold, a low-complexity model execution layer containing convolutional layers can be selected. The total byte length of the weight data is determined by the path of the convolutional layer because the convolutional kernel size corresponding to the convolutional layer path is smaller, resulting in a smaller overall weight data volume and lower dimension, which is suitable for the task scene. The above thresholds are empirical values ​​for the task scene complexity set according to the specific task, model, or terminal device. This threshold, combined with the performance boundaries of the two sets of execution layers with low and high complexity, is a dividing value for balancing recognition accuracy, inference latency, and operating power consumption. The dimensions of the above thresholds are consistent with the output of the scene complexity formula, which is used to achieve binary classification scheduling of high and low complexity.

[0041] In a specific example, if applied to a device-side voice wake-up scenario, one can first... Normalize each value to the range [0,1] to eliminate dimensional differences, and then set weights α and β to improve the final task scenario complexity. The output range is between [0,1].

[0042] For example, threshold calibration for voice wake-up scenarios on the device side can be performed by statistically analyzing different thresholds on a test set covering typical scenarios such as quiet indoor environments, noisy streets, and mixed voices from multiple people. The wake-up accuracy and inference latency of the two execution layers can be used as constraints, with the low-complexity model achieving a satisfactory wake-up rate and the high-complexity model having acceptable latency. Finding the dividing point allows for the determination of a threshold, such as a threshold value. It can be set to 0.38. The threshold can be flexibly determined according to the actual situation. As long as it can achieve the goal of determining whether to use a high-complexity model or a low-complexity model based on whether the task scenario complexity is greater than or less than the preset threshold, then the threshold is within the scope of this application.

[0043] Therefore, after obtaining the trigger information and determining the corresponding access history table, the task scenario complexity can be calculated based on the information in the access history table. For trigger information with high scenario complexity, the scheduling control unit can select a relatively complex model to enhance the modeling capability for long-term dependencies; for trigger information with low scenario complexity, the scheduling control unit can select a relatively low complex model to reduce the computation path depth and shorten the inference latency, while also reducing the time consumed by the inference process.

[0044] In the embodiments of this application, to improve the stability of task execution and reduce latency, the target execution layer is selected based on the complexity of the task scenario. When the complexity of the task scenario exceeds a preset threshold, a high-complexity model execution layer is used to enhance the modeling capability for long-term temporal dependencies; when the complexity of the task scenario is below the threshold, a low-complexity model execution layer is used to reduce the computation path depth and shorten inference latency. The application of weight index numbers marks the selected execution layer, providing a hierarchical basis for subsequent retrieval of the corresponding weight data, access address range, and execution cache prefetching. This ensures that the selection result of the execution layer is completely aligned with the subsequent model parameter address prediction steps in terms of data structure. It is crucial data for realizing the transition from scenario recognition to cache prefetching and is an indispensable component of the continuous hardware scheduling link.

[0045] In a specific embodiment of this application, to avoid bandwidth contention caused by prefetching to the currently running task, the embodiment of this application transmits data within the sub-interval to the cache at a pace limited by bandwidth ratio. This ensures that the transmission process does not affect the storage of trigger information in the cache, thereby guaranteeing the stability of task execution. Therefore, based on step S103, the following steps are also included: Step S5031: Determine the number of cache lines to be prefetched based on the start address of the access address range, the end address of the access address range, and the line length of the preset level cache.

[0046] Specifically, the start and end addresses of the access address range are the beginning and end addresses of the access address range, respectively. These are two boundary addresses that define the contiguous storage range of the weight data to be prefetched in main memory. The former marks the memory start point of the weight storage area, and the latter marks the memory end point. Together, they determine the complete memory read / write range of the weight prefetch operation. For example, if the weight data of a certain execution layer corresponds to the address range 0x80000000 to 0x80007FFF in main memory, then 0x80000000 is the start address of the access address range, and 0x80007FFF is the end address. The prefetch operation will use this address range to move the corresponding weight data. Determining the number of cache lines to be prefetched is to align the access address range to the smallest management unit of the cache hardware, providing a benchmark for subsequent sub-range partitioning, prefetch instruction generation, and cache line locking to match hardware rules. This process is repeated when obtaining the access address range. Then, the scheduling and control unit uses the access address range obtained in the previous stage. The starting address of the access address range can be determined based on its starting and ending addresses. End address of the access address range The access address range is split into one or more contiguous sub-ranges aligned with the cache line size, and the sub-ranges are determined according to the line length of the preset cache level. Once the prefetch instruction sequence is generated, the number of cache lines that need to be prefetched can be determined using the following formula: in, This indicates the number of cache lines that need to be prefetched. Indicates the line length of the preset level cache. Determined by the System on Chip (SoC) hardware specifications, and These are the starting address and ending address of the access address range, respectively.

[0047] Step S5032: Based on the number of cache lines to be prefetched and the access address range, the access address range is divided into multiple sub-ranges.

[0048] Step S5033: Determine the allowed bandwidth ratio for this prefetch based on the average memory bandwidth usage of the target execution layer and the complexity of the task scenario.

[0049] Specifically, to avoid bandwidth contention for currently running tasks due to prefetched weight data, the system uses the average memory bandwidth usage from the historical inference trajectory of the target execution layer and the complexity of the task scenario within the target execution layer, taking values ​​from the same statistical window. This ensures that the bandwidth data perfectly matches the currently running target execution layer and task scenario, accurately reflecting the current memory bandwidth usage of the system. It's important to note that the bandwidth ratio is the ratio of the allowed memory bandwidth for weight prefetching to the total available memory bandwidth of the system. It is dynamically adjusted by the task scenario complexity to constrain the weight prefetching rate, ensuring timely loading of inference weights while avoiding excessive encroachment on the memory bus resources of other system tasks. For example, if the total available bandwidth of the terminal's main memory is 16GB / s, in a low-complexity, quiet environment with voice wake-up scenario, the bandwidth ratio can be set to 5%, corresponding to a prefetch bandwidth limit of 0.8GB / s; in a high-complexity, noisy environment scenario, the bandwidth ratio can be increased to 15%, corresponding to a prefetch bandwidth limit of 2.4GB / s, thus matching the weight prefetching timeliness requirements of different loads.

[0050] Average memory bandwidth usage is a quantitative indicator used in the weight prefetching phase to measure the level of memory bus resource usage. It refers to the average memory bandwidth occupied by the weight prefetching operation per unit time. It is a core control parameter for regulating the prefetching rate and preventing excessive prefetching from crowding out the memory resources of other system tasks. For example, if the total available bandwidth of the terminal device's main memory is 16GB / s, the upper limit of the average memory bandwidth usage for weight prefetching can be set to 1.5GB / s. This ensures that the weights are loaded into the cache on time while reserving sufficient bandwidth for other system tasks to use normally. Using the same statistical window specifically means that when calculating input feature indicators such as volatility and dispersion, a statistical interval with consistent time length, sampling frequency, and data volume is selected as the calculation benchmark. This ensures that each dimension of the indicator is calculated based on the same batch of sampled data and the same time range, guaranteeing a unified calculation benchmark for the complexity of the task scenario and making the results comparable. For example, in the voice wake-up scenario, when calculating the volatility and dispersion of the audio signal, a statistical window with a duration of 200ms and containing 160 sampling points is used to ensure that the calculation of the two dimensions is based on the same audio sample data, avoiding deviations in the determination of scenario complexity due to inconsistent statistical ranges.

[0051] Because memory bandwidth consumption varies greatly across tasks of different complexities and model levels, using outdated or incompatible bandwidth data to adjust prefetching can lead to either excessively fast prefetching that consumes computational bandwidth or excessively slow prefetching that fails to mask memory access latency. The purpose of using the same window value is to eliminate this bias and adjust the prefetch rate accordingly, accurately matching the system's bandwidth margin and preventing weight prefetching from competing for memory bandwidth with normal inference operations. The allowable bandwidth ratio for this weight prefetching is calculated using the following formula: Where i is the same as above, representing the hierarchy number of the execution layer of the target model. This represents the allowed bandwidth percentage for this prefetch. Average memory bandwidth usage is used to regulate the prefetching pace, representing the target execution layer. Average memory bandwidth requirements during historical inference processes Data is taken from hardware performance counters within the same statistical period as the current operation. This indicates the peak bandwidth of the main memory controller. For the complexity of the task scenario, It is the upper limit of task scenario complexity recorded during the device or chip manufacturing and firmware initialization phase, and is used for reference. Specifically, it is used to normalize the complexity of task scenarios.

[0052] S5034 schedules the data transmission timing in each sub-interval according to the bandwidth ratio, and generates a weighted prefetch instruction sequence based on the transmission timing and the line length of the preset level buffer.

[0053] Specifically, to avoid prefetching weighted data from crowding out the bandwidth allowed for this prefetch, the data block transmission timing needs to be scheduled according to the system's allowed bandwidth for this prefetch, and a prefetch instruction sequence needs to be generated according to the line length of the preset level cache. The prefetch instruction sequence can be a single prefetch instruction or a set of several prefetch instructions. The line length of the preset level cache is the smallest data management and read / write unit specified by the hardware for that cache level, measured in bytes. It determines the alignment rules of the cached data, the prefetch granularity, and the minimum range of locking operations, and is the core hardware benchmark for dividing the weighted address sub-intervals and calculating the number of cache lines to be prefetched. For example, when a multi-core processor's shared L3 cache is used as the preset level cache, its line length is typically 64 bytes. The weighted access address sub-intervals need to be aligned to 64 bytes, and each aligned sub-interval corresponds to a prefetch write and locking operation for one line of cache.

[0054] pass After controlling the pace of prefetching weighted data, the cache controller generates a prefetch instruction sequence according to a linearly increasing address, where the k-th (0≤k≤ The physical address corresponding to each prefetch instruction satisfy ,in, This represents the starting physical address of the k-th cache line, where k is a positive integer; The starting address of the access address range, This is the line length of the preset cache level. Each prefetch instruction is written to the SoC's prefetch register queue and... The defined rhythm triggers main memory read operations, transferring data from the sub-range to the preset-level cache. Because... Based on the average starting offset of actual memory access The correction was made, and the target execution layer Total byte length of weighted data This has been confirmed. Therefore, the above generation formula can completely cover the contiguous physical space of the target execution layer's weight data in main memory, thus stably loading all weight data into the low-latency cache before the NPU officially initiates the multiplication and addition operation.

[0055] In the embodiments of this application, the data of the sub-intervals is transmitted to the cache at a pace limited by the bandwidth ratio, thereby ensuring that the transmission process does not affect the storage of other information in the cache, thus ensuring the stability of the execution of this task and the stability of the execution of other tasks.

[0056] In a specific embodiment of this application, in order to further ensure that the prefetched weight data can remain stably resident throughout the entire inference window, that is, during or before the multiplication-addition operation of the read weight data, other applications or background services cannot perform replacement operations on these locked cache lines even if new memory access requests are generated, the basis of step S104 in this embodiment of the application further includes the following steps: Step S7041: Determine the cache group index and cache tag entry of the corresponding data block in the cache group based on the prefetch start address of each weight prefetch instruction.

[0057] Step S7042: Locate the corresponding cache group according to the cache group index, and set the cache tag entry in the tag register corresponding to the cache group from the default state to the lock flag bit.

[0058] Specifically, the weighted prefetch instruction is any instruction in the weighted prefetch instruction sequence. The cache controller follows... Complete all After prefetching instructions and writing the corresponding data to the last-level cache, the scheduling control unit immediately reads the cache directory table and matches each instruction with the corresponding data. The corresponding cache tag entry is determined, and its cache group index and tag entry within the cache group are calculated. The number of cache groups that can be addressed by the cache group is also calculated. Satisfy the following formula: Among them, the degree of group interconnection This is a core configuration parameter for caching, indicating the number of cache lines contained in each cache group. Since the cache is first divided into multiple groups, the group associativity... The number of independent cache lines that can be stored in a group directly affects the cache addressing rules and the probability of conflict. It is also an important basis for allocating addresses and managing cache resources when performing cache prefetching and cache line locking in this embodiment. Indicates the number of cache sets that can be addressed; It is the total capacity of the last-level cache. This refers to the line length of the default level cache. The cache set index is an address code extracted from memory in the default level cache of a set-associative architecture, used to locate the cache set to which the target data belongs. The cache tag entry is a storage entry that corresponds one-to-one with each cache line, used to store the high-order tag field of the memory address. Together, they complete cache addressing, hit detection, and line locking operations, serving as the core addressing basis for the cache controller to perform weight prefetching and locking. For example, a terminal's level 3 cache (as the default level cache) is 8-way set-associative with a line length of 64 bytes. The memory address is split bitwise: the lower 6 bits are the line offset, bits 6 to 14 are the cache set index, and the remaining high-order address bits are the tag value. When prefetching weighted data, the system first locates the corresponding cache set using the cache set index, and then stores the high-order tag of the weight address into the cache tag entry of the corresponding cache line. When performing a hit query or cache line locking, the system similarly first locates the target cache set using the cache set index, and then accurately matches the target cache line by comparing the content of the cache tag entry.

[0059] Based on the calculated number of addressable cache sets, the cache set index can be determined. The formula for calculating the cache set index is as follows: in, It is the cache set index of the cache set containing the k-th prefetch instruction. This indicates the cache line number corresponding to the physical address. This indicates the number of addressable cache groups. The scheduling and control unit determines this based on... The specific associated path within the corresponding group is located, and the tag register of that path is read, thereby writing the lock flag bit into the lock flag bit in the buffer control register. This allows the LRU replacement algorithm to skip all cases that satisfy the Least Recently Used (LRU) algorithm when executing the Least Recently Used (LRU) algorithm. Cache lines.

[0060] The reason is that after completing the preliminary steps of target execution layer determination, weight memory access address prediction, and cache prefetching process generation, a corresponding lock flag is generated for the cache lines that have been prefetched and are ready to participate in inference operations. After writing the lock flag, the LRU replacement algorithm can skip the locked cache lines, preventing critical weight data that needs to be prefetched from being accidentally replaced, thus ensuring that inference data resides in the cache and reducing access latency. Furthermore, because all... All originate from the access address range Access address range and the level number of the target execution layer Average offset and task scenario complexity To maintain consistency, the locked cache group completely overlaps with the cache line where the weight data actually needs to be stored for this inference. During the NPU's reading of weight data and execution of multiply-accumulate operations, even if other applications or background services generate new memory access requests, they cannot perform replacement operations on these locked cache lines, ensuring that the prefetched data remains stably resident throughout the entire inference window.

[0061] To prevent cache space from being occupied for extended periods, which could lead to a decrease in subsequent memory access hit rates and system performance degradation, and to free up the corresponding limited cache resources for subsequent task scheduling, the cache line lock is released and the system is restored to its default state after inference, until the corresponding cache lines are uniformly cleared in subsequent steps. Therefore, after step S7042 above, the following is also included: Step S7043: Based on the cache group index corresponding to all locked cache lines in this inference, locate the corresponding cache group and release the resources of the corresponding cache group.

[0062] Step S7044: Set the cache tag entry in the corresponding tag register of the corresponding cache group to the default state from the lock flag bit.

[0063] In a specific example, the NPU completes the access address range and the hierarchical number of the target execution layer. After performing all the corresponding multiply-accumulate operations, the scheduling and control unit reads the set of locked cache lines recorded during this inference period. This set consists of all lines that satisfy the following conditions. The cache lines consist of [number] cache lines, and their quantity is [number]. This is consistent with the number of cache lines. The control unit accesses the corresponding cache group index line by line. Clear the lock flag in the cache control register, so that The system reverts to the default state from 1, thus returning it to the replacement policy management. It should be noted that... If you want to set it as a lock flag, it's not limited to... The value is assigned to 1, and restoring it to the default state is not limited to... Assigning a value of 0, or any value that serves a distinguishing purpose, falls within the scope of this embodiment.

[0064] In the embodiments of this application, the locking flag is reset to the default state so that the cache and bandwidth resources that were originally compressed in the locking phase can quickly return to the normal allocation state, ensuring that the cache resources are released immediately after the wake-up model inference window ends, and maintaining the dynamic balance between cache utilization and system throughput in a multi-tasking environment.

[0065] In an optional embodiment of this application, after reading weight data from the locked cache line and performing calculations on the target model based on the weight data, the calculation results are applied according to the specific use case. Therefore, the method of this application further includes the following steps: Step a: Perform calculations on the target model based on the weight data and determine the calculation results.

[0066] Step b: Based on the calculation result, output a confirmation signal.

[0067] Specifically, the weighted data and trigger information are multiplied and added together, and a decision is made on whether to output a confirmation signal based on the specific calculation result. The method for deciding whether to output a confirmation signal can be by setting a threshold or setting conditions; any method that can output a confirmation signal based on the calculation result is within the scope of this embodiment and will not be elaborated upon here.

[0068] In an optional embodiment provided in this application, when the trigger information is voice wake-up information, the model data access method is applied to wake-up scenarios related to smart voice terminals, which is more in line with the application requirements of specific scenarios. This is because, while related technologies have made beneficial explorations in areas such as false wake-up suppression, cloud-based collaborative scheduling, user interaction strategies, and model structure optimization, they generally overlook a core issue: the latency overhead caused by the frequent access of the neural network accelerator to read weight data from off-chip main memory during the actual inference process of the wake-up model. Existing solutions fail to dynamically adjust the model execution path according to the real-time changing acoustic scene, and also cannot implement precise cache prefetching and resident locking for weight data at specific levels. This results in a situation where, even if the front-end acoustic processing module has quickly completed voice detection, the subsequent model inference stage still experiences a large number of waiting cycles due to frequent cache misses, and the processor computing array is often in a state of starvation, making memory access latency the main bottleneck restricting the wake-up response speed. Especially when a large-scale model layer needs to be called in a complex acoustic environment, this latency effect is further amplified, seriously affecting the user's interactive experience in noisy environments or scenarios with high real-time requirements. Therefore, this embodiment provides a specific adaptive scenario voice wake-up optimization method, including the following steps: Step 1: When the user's voice wake-up event is detected and initially verified by the front-end acoustic processing module, the specific level of the wake-up model to be executed on the edge NPU is immediately identified. This level includes convolutional layers or recurrent layers that are dynamically selected based on the acoustic scene complexity.

[0069] Step 2: Based on the fixed memory access patterns of the identified wake-up model level's weight parameters and intermediate activation values, and combined with the data address access patterns of this level during historical execution, predict the precise physical address range of the model parameters to be loaded in the dynamic random access memory for this inference.

[0070] Step 3: Send a dedicated prefetch instruction to the SoC's cache controller to transfer the model parameter data within the prediction address range from the main memory to the last-level cache in advance. This last-level cache is either a shared L3 cache for multi-core processors or an embedded dynamic random access memory integrated on the wafer.

[0071] Step 4: Temporarily lock the cache tag corresponding to the cache line containing the model parameter data. By setting the lock flag in the cache control register, prevent other running applications or system background services from flushing or replacing the data content in these cache lines.

[0072] Step 5: During the wake-up model inference process, the NPU directly reads the model parameters from the locked cache line to participate in the multiplication and addition operations, without having to wait to load data from main memory due to cache missing data, thus significantly shortening the memory access latency during model execution.

[0073] Step Six: After all hierarchical inference calculations of the wake-up model are completed and a wake-up confirmation signal is output, immediately clear the previously set cache line lock flag, release the occupied cache resources for other concurrent tasks to use normally, and ensure efficient utilization of system resources and overall performance balance.

[0074] Specifically, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the acoustic scene complexity assessment and hierarchy selection process in this embodiment. In step one, after the voice wake-up event is detected in step S31 and the preliminary verification by the front-end acoustic processing module is passed in step S32, the system will first perform step S33 to read the acoustic feature tensor in the shared memory. The acoustic feature tensor is the acoustic feature tensor that has been calculated in the front-end module for the read speech segment. It comes from the Mel spectrum energy sequence under the fixed frame length and frame shift configuration and is output by the front-end module to the shared memory buffer. Its physical start address and length information have been established in the access history table in step two. Therefore, in this step, the same address index structure is directly reused to maintain consistency with the physical address prediction logic in the subsequent cache prefetch stage.

[0075] Subsequently, the acoustic scene complexity is calculated by executing step S34 through the end-side scheduling and control unit. This is used to measure the non-stationarity of the current input signal in the time and frequency domains, thereby determining whether to call a convolutional layer or a recurrent layer. The acoustic scene complexity is calculated using the following formula: , in It represents the variance of short-time energy within consecutive speech frames, reflecting the degree of fluctuation in speech intensity over time. The time mean of the spectral entropy is used to characterize the dispersion of the frequency domain distribution. This is the energy fluctuation weighting coefficient, used to control the proportion of influence of time-domain perturbations on complexity assessment. These are the frequency domain discrete weighting coefficients, used to balance the contribution of the spectral structure to the complexity assessment.

[0076] The feature buffer's physical address is directly calculated and cached by the front-end acoustic processing module during the endpoint detection phase. This address is recorded in the same mapping table as the weight access history, thus eliminating the need for additional memory access paths during complexity calculations. The calculation is performed frame-by-frame based on the stored Mel-frequency spectrum vectors and completed in the preprocessing unit of the edge NPU, thus ensuring a consistent data path with the subsequent inference execution hardware. Then, step S35, complexity threshold judgment, is executed. When the preset threshold is exceeded, the scheduling control unit executes step S351 to select a cyclic layer path to enhance the modeling capability for long-term time-series dependencies; when When the value is below the threshold, step S352 is executed to select a convolutional layer path to reduce the computation path depth and shorten inference latency. Then, the weight index number of the corresponding level is written to the inference scheduling register, and step S36 is executed to determine the current execution level index number. This level index number shares the same numbering system as the historical execution record table in step two, allowing direct access to the weight physical address range record after identifying the specific level. This ensures that the model level selection result and the subsequent model parameter address prediction steps are completely aligned in data structure, achieving a continuous hardware scheduling link from scene recognition to cache prefetching.

[0077] In the steps proposed in this embodiment, after the voice wake-up event is initially verified, the non-stationarity of the acoustic scene is immediately quantified by calculating the short-time energy variance and frequency domain features of the current speech segment. Based on this quantification index, a dynamic decision is made on whether to call the convolutional layer or the recurrent layer for this wake-up inference. This achieves a precise match between the model's computational load and real-time environmental noise and speech fluctuation characteristics, providing a differentiated basis for the weighted data scale for the subsequent cache optimization stage while ensuring wake-up accuracy.

[0078] In step two, such as Figure 4 The diagram shown is a schematic of the physical address prediction process based on historical inference trajectories in this embodiment. This process is implemented after completing the acoustic scene complexity... After calculating and obtaining the current execution level index number in step S41 and determining the specific execution level, the scheduling control unit executes step S42 to read the historical execution record table. This historical execution record table corresponds to the level index number and is used to store data from the previous stage. and The physical address mapping table structure is consistent, thus ensuring data structure uniformity. The size and arrangement of the weight tensors for each model level are determined during the compilation phase, so their storage in dynamic random access memory satisfies the characteristics of a continuous piecewise linear distribution.

[0079] Step S43 extracts the starting address of the weights in the static mapping table, assuming the currently selected level number is... The starting address of its weight tensor in physical memory is The total length of the weight in bytes is Then the theoretical continuous address range can be represented as: in This indicates the physical address range in main memory where all weights of this level reside. This indicates the starting address of the weight at this level, which comes from the static mapping table established during the model loading phase. This represents the total number of bytes in the weight tensors of this level, determined by the kernel size or the hidden dimension of the recurrent unit. This dimension is related to the complexity metric of the previous stage. The triggered level selection corresponds directly.

[0080] To further improve prediction accuracy, the system stores the most recent data in the historical execution record table. The actual memory access trajectory during the next inference process is used. Therefore, step S44 is executed to calculate the average offset of historical memory access trajectories, that is, to perform offset statistics on these trajectories and calculate the average starting offset of the actual memory access. Then, step S45 corrects the theoretical address range, and step S46 generates the predicted physical address range, which is the final access address range: in This indicates the precise range of physical addresses that need to be prefetched for this inference. This represents the average offset between the address of the first cache line hit in the historical memory access trajectory and the static starting address. It is used to eliminate address drift errors caused by compile-time layout and runtime page mapping. This represents the number of historical inferences used for statistical purposes, which is determined by the system while maintaining a consistency with complexity metrics. The samples were selected under consistent acoustic scene classification conditions, thus ensuring that historical samples are consistent with the current acoustic complexity level. The loop structure corresponding to high values ​​has a greater presence in the historical data table. With more stable The distribution is such that the accessed address range exhibits a continuous large block characteristic; when When the value is in the low range and the convolutional layer path is triggered, Smaller and more concentrated address offsets ensure that subsequent cache prefetching stages can rely on... A complete prefetch instruction sequence is generated in one go, enabling locked loading of a precise physical address range in main memory. Following this, step S47 outputs the predicted start and end addresses.

[0081] In the steps proposed in this embodiment, the above steps, for the specific model level determined in the previous step, read the pre-compiled static starting address and byte length of its weight tensor. Simultaneously, combining this with the most recent inference memory access offset statistics stored in the historical execution record table that matches the level and the complexity of the current acoustic scene, the static address is corrected, thereby accurately predicting the physical address range of the model parameters to be loaded in main memory for this inference. By introducing historical behavior feedback, the address drift error caused by compile-time layout and runtime page mapping is effectively eliminated, enabling subsequent prefetching operations to accurately lock onto the target data and avoiding invalid prefetching and bandwidth waste due to inaccurate address prediction.

[0082] In step three, such as Figure 5 The diagram shown is a schematic of the cache prefetch instruction generation and data locking process in this embodiment. In step S47, the predicted physical address range is obtained. Next, in step S51, the predicted physical address range is input. Then, the scheduling control unit executes step S52 to split the address according to the cache line size, dividing the range into contiguous blocks aligned with the cache line size, and based on the preset line length of the last level cache. Generate the prefetch instruction sequence. Let the starting address of the access address range be... The end address is ,in and Derived from the previous stage Then, step S53 is executed to calculate the number of cache lines that need to be prefetched. The calculation is as follows: in This indicates the number of instructions that need to be written to the cache controller's prefetch queue. This indicates the single-line byte capacity of the last-level cache; this parameter is determined by the SoC hardware specifications. and These are the start and end addresses of the predicted physical interval, respectively.

[0083] To avoid bandwidth contention for currently running tasks due to prefetching, the system uses the average memory bandwidth usage from historical inference records. and the complexity of current acoustic scenarios Step S54 calculates the prefetching rhythm based on historical loan usage. This prefetching rhythm is controlled by the bandwidth ratio, therefore it is necessary to calculate the allowed bandwidth ratio for this prefetching. The calculation method is as follows: in Indicates hierarchy The average memory bandwidth requirement during historical reasoning stems from... Performance counter data within the same statistics window, This indicates the peak bandwidth of the main memory controller. This refers to the acoustic scene complexity index calculated in the preceding steps. This is a reference complexity upper limit recorded during system calibration, used to normalize the complexity.

[0084] pass After controlling the prefetch rhythm, step S55 is executed through the cache controller to generate a linearly increasing prefetch instruction sequence. The first prefetch instruction sequence... The physical address corresponding to each prefetch instruction satisfy ,in The range of values ​​is to , Indicates the first The starting physical address of each cache line. Step S56: The cache controller performs prefetching; each prefetch instruction is written to the SoC's prefetch register queue, and... The defined rhythm triggers main memory read operations, followed by step S57 where data is written to the last cache level. This last cache level is either a multi-core shared L3 cache or an embedded dynamic random access memory integrated on the wafer. Because... Based on historical offset Correction, and hierarchy Weighting scale It has been determined that the above linear address generation formula can completely cover the contiguous physical space of the target weight tensor in main memory. Therefore, before the NPU officially initiates the multiplication-accumulation operation, all weight data is stably loaded into the low-latency cache level, constructing a system consistent with the preceding complexity determination. and level selection Consistent, continuous hardware scheduling path.

[0085] In step four, the cache controller follows... Complete all After prefetching instructions and writing the corresponding data to the last-level cache, step S58 is executed to calculate the index and tag of each cache group. The scheduling control unit immediately reads the cache directory table and each The corresponding cache tag entry is determined, and its cache group index and tag entry in the cache group are calculated. Let the total capacity of the last-level cache be... The degree of group connectivity is The cache line size is Then the number of cache groups satisfy: in Indicates the number of addressable cache sets. This indicates the total size of the last-level cache in bytes. This indicates the number of cache lines that each group can hold. This is the single-line byte capacity, a parameter used in the previous stage for calculation. For each physical address Its cache group index The calculation is as follows: in Indicates the first The cache group number where the prefetched data is located. This indicates the cache line number corresponding to the physical address. This refers to the number of cache groups. The scheduling control unit determines this based on... After locating the specific associated path within the corresponding group and reading the tag register of that path, step S59 is executed to set the lock flag bit of the cache control register, locking the prefetched data. Specifically, the lock flag bit is written into the lock bit field of the cache control register. This allows the replacement strategy logic to skip all cases that satisfy the least recently used algorithm when executing it. Cache lines.

[0086] Because of all All originate from the access address range This interval is related to the level number. Historical offset and complexity metrics To maintain consistency, the locked set completely overlaps with the weight set actually needed for this inference. Therefore, while the NPU is reading weights and performing multiply-accumulate operations, other applications or background services, even if they generate new memory access requests, cannot replace these locked cache lines. This ensures that the prefetched data remains stably resident throughout the entire inference window until subsequent steps uniformly clear the corresponding data. Until the flag is reached.

[0087] In this embodiment, the predicted precise physical address range is divided into consecutive prefetch instruction sequences according to the cache line size through the above steps. The prefetching rhythm is controlled based on a ratio calculated from historical bandwidth usage and current scenario complexity. After the data is transferred from main memory to the last-level cache, the cache tags corresponding to all cache lines containing model parameter data are immediately locked by setting the lock flag in the cache control register, preventing other running applications or system background services from accessing and flushing or replacing this data. During the entire wake-up model inference window, the weight data required by the neural network accelerator can stably reside in the low-latency cache level, fundamentally eliminating memory access wait cycles triggered by cache misses.

[0088] In step five, such as Figure 6 The diagram shown is a schematic of the zero-miss inference process under cache locking in this embodiment. After completing the process for all... Lock and confirm the corresponding cache line Next, in step S61, the neural network accelerator initiates a read request, and the NPU initiates a continuous read request to the last level cache through the on-chip interconnect bus; in step S62, the read address is generated according to the predicted sequence, and the read address sequence is still based on... Incremental generation maintains a completely consistent physical address mapping with the prefetch and locking phases, after which step S63 is executed to access the last-level cache. Because all All have been based on Preloaded and resident in the cache, and with the replacement policy masked by the lock bit, step S64 checks the cache line lock flag, and the cache hit rate during inference is... satisfy: in This indicates the number of cache lines actually hit. The total number of cached lines to be prefetched; this parameter is derived from the aforementioned... Calculation results. A cache hit rate of 1 means that all key accesses are completed in the last-level cache, without triggering main memory access transactions. Therefore, step S642 theoretically does not occur. If the cache is locked, the following steps are executed sequentially: S6411 cache hit, S6412 data directly participates in multiplication and addition operations, S6413 continuous data stream fills the computation array, S6414 determines the main memory access latency, S6415 completes all levels of computation, and S6416 outputs a wake-up confirmation signal. Based on this, the main memory access latency during the inference phase is... It can be represented as: in This represents the average latency for reading a single cache line from the last-level cache. This represents the average latency for reading the same data block from main memory; this value is typically much larger than... .because The above equation degenerates into Therefore, there is no main memory access latency, thus stably limiting the memory access path to the low-latency cache level. Combined with the aforementioned level numbering... Corresponding weight scale and complexity metrics The determined execution path length, the total memory access time for the entire wake-up model inference is equal to the number of cache reads multiplied by . This avoids triggering errors due to cache misses. Level latency accumulation enables the NPU to fill the computation array with a continuous data stream when performing convolution or loop layer multiply-accumulate operations, eliminating bubble cycles that wait for data to return from main memory, and ultimately achieving a smoother model execution cycle compared to the prefetching phase. The contiguous address space is fully aligned, forming a complexity-based decision. A closed-loop hardware acceleration chain from cache locking to low-latency inference.

[0089] In step six, as Figure 7 The diagram shown is a schematic of the cache resource release and performance recovery process in this embodiment, at the NPU completion level. After performing all corresponding multiply-accumulate operations and executing step S71 to output the wake-up confirmation signal, the scheduling control unit executes step S72 to read the currently locked cache line set, which consists of all lines satisfying the specified conditions. The cache lines consist of [number] cache lines, and their quantity is [number]. According to the prefetching stage The calculated number of cache lines remains consistent. The corresponding cache group index is accessed line by line by the control unit. Step S73 clears the lock flag bits in the cache control register one by one, making... The value is restored from 1 to 0, thus returning it to the replacement policy management. To assess the impact of lock release on system resources, a cache usage ratio is introduced. Step S74 calculates the cache occupancy ratio, which is expressed as: in This indicates the number of cache lines locked during this inference period. For the capacity of a single cache line in bytes, This is the total capacity of the last-level cache; this parameter was used in the preceding steps to calculate the number of cache groups. . This indicates the proportion of cached resources occupied by locked data. After inference is complete, all cached data will be cleared. This ratio immediately returns to 0, thus executing step S75 to release all cached resources. To ensure overall system performance balance, the scheduling and control unit also considers the average memory access latency of this inference. Compared with historical statistics bandwidth requirements Calculate the performance recovery factor after cache resource recovery Step S76 calculates the performance recovery factor, whose expression is: in Indicates hierarchy Historical average memory bandwidth usage Main memory peak bandwidth, For the last-level cache access latency, This represents the actual average memory access latency during this inference phase. Because the locking phase satisfies... ,therefore Step S77 adjusts the memory access priority of other tasks and releases the cache, based on the system bandwidth scheduling logic. Adjust the memory access priority of other concurrent tasks; step S78 restores the system to its normal scheduling state, allowing the cache and bandwidth resources that were compressed during the locked phase to quickly return to their normal allocation state; finally, step S79 maintains the balance between cache utilization and throughput. Through the above-mentioned lock bit clearing and performance recovery adjustment mechanism, it is ensured that cache resources are released immediately after the wake-up model inference window ends, and a dynamic balance between cache utilization and system throughput is maintained in a multi-task environment.

[0090] As an optional embodiment, in order to reduce memory access latency during the wake-up model inference process, the core of this embodiment lies in avoiding the prediction method of historical trajectory correction of physical addresses with specific level weights and the cache residency method based on locking flags.

[0091] One possible workaround is to replace the prediction mechanism itself. For example, instead of relying on historical memory access offsets to correct static addresses, a lightweight neural network based on input features and model layers could be used to predict the logical address range where weight data might be loaded in real time. Alternatively, tagging instructions inserted during model compilation could be used to allow hardware to directly capture the logical address of weight accesses and trigger prefetching at runtime, thus avoiding explicit correction and precise calculation of physical addresses. Another workaround is to change the implementation of cache locking and data residency. For example, instead of setting cache line locking flags, the weight parameters of the cache replacement policy could be adjusted to assign extremely high priority to data related to waking up the model, making it almost impossible for it to be evicted in the LRU algorithm. Alternatively, a portion of the last-level cache could be dynamically divided into a dedicated area, and the predicted weight data could be directly moved to this dedicated area during wake-up inference, prohibiting other cores from accessing it. This would achieve stable data residency while avoiding locking operations on specific cache tags.

[0092] Another, more macro-level approach to avoid this issue involves adjusting the architecture of the entire scheduling process. The core of this embodiment lies in tightly coupling level selection, address prediction, and cache locking. Avoidance schemes can attempt to decouple this chain. For example, instead of immediately performing complex scenario complexity calculations and level selection after front-end acoustic processing verification, a unified, memory-access-friendly model structure can be adopted. This ensures that the weight accesses executed by the model exhibit highly regular and predictable streaming characteristics regardless of the acoustic scenario, simplifying the address prediction logic and allowing a simple streaming prefetching mechanism to meet the requirements. Alternatively, the wake-up model, originally executed on the edge accelerator, can be split up. The memory-intensive but computationally simple parts can be moved to near-data computation units integrated near memory. This allows weight data to bypass the cache layer, completing some calculations directly at the storage end, only sending back intermediate results. This eliminates the latency of long-distance data transmission at the physical architecture level, completely avoiding the necessity of cache prefetching and locking, thus achieving the same low-latency goal while employing a completely different technical path.

[0093] In the steps proposed in this embodiment, the above steps first dynamically select the execution level of the wake-up model based on real-time acoustic characteristics, then accurately predict the physical address of the weight data of that level based on static compilation information and dynamic historical trajectories, and then initiate a locking cache prefetch to preload the data and reside it in the last-level cache. This allows the neural network accelerator to directly read parameters from the cache with a 100% hit rate during inference to participate in multiplication and addition operations. After inference is completed, the locking flag is immediately cleared to release cache resources, and the system bandwidth scheduling is adjusted according to the performance recovery factor. A complete hardware acceleration closed loop from scene recognition to underlying memory access optimization is constructed, which significantly shortens the memory access latency during model execution, achieves a significant improvement in voice wake-up response speed, and ensures the dynamic balance and efficient utilization of system resources in a multi-tasking environment.

[0094] In an optional embodiment, this application also provides an apparatus capable of applying the above-described model data access method, such as... Figure 8 As shown, Figure 8 The hardware structure diagram of the model data access device provided in the embodiments of this application specifically includes: Module 801 is used to determine the target execution layer of the target model; The correction module 802 is used to correct the theoretical access address based on the offset between the theoretical access address of the weight data of the target execution layer in memory and the actual access address in the historical inference trajectory data, and to obtain the access address range of the weight data in memory based on the corrected theoretical access address. The generation module 803 is used to divide the access address range of weight data into multiple sub-ranges and generate a weight prefetch instruction sequence based on the line length of the preset level cache. The prefetch module 804 is used to prefetch weight data from multiple sub-intervals into a preset level cache through a weight prefetch instruction sequence; The processing module 805 is used to read weight data from the locked cache line and perform calculations on the target model based on the weight data.

[0095] In an optional embodiment, the correction module 802 specifically obtains the memory access address range of the weight data through the following algorithm: Where i is used to identify the i-th execution layer, and i is a positive integer. It is the hierarchical number of the target execution layer. yes The corresponding memory address range for the weight data of the target execution layer. yes The starting address of the weight data of the corresponding target execution layer. yes The offset between the theoretical access address of the corresponding target execution layer's weight data in memory and the actual access address in the historical inference trajectory data. yes The total byte length of the weight data for the corresponding target execution layer.

[0096] In an optional embodiment, the determining module 801 is specifically used to determine the fluctuation degree information and the discrete degree information of the trigger information according to the received trigger information of the target execution layer; and to determine the task scenario complexity based on the fluctuation degree information, the discrete degree information, the preset fluctuation degree coefficient and the preset discrete degree coefficient of the trigger information. When the complexity of the task scenario exceeds a preset threshold, the target execution layer is determined to be a high-complexity model execution layer; when the complexity of the task scenario is less than the preset threshold, the target execution layer is determined to be a low-complexity model execution layer.

[0097] In an optional embodiment, the determining module 801 specifically determines the task scenario complexity using the following algorithm: in, It is the complexity of the task scenario. It is information about the degree of fluctuation. It is information about the degree of dispersion. It is a preset fluctuation coefficient. It is a preset dispersion coefficient.

[0098] In one optional embodiment, the generation module 803 further includes: determining sub-modules and dividing sub-modules; The first determining submodule is used to determine the number of cache lines to be prefetched based on the start address of the access address range, the end address of the access address range, and the line length of the preset level cache. The sub-module is used to divide the access address range into multiple sub-ranges based on the number of cache lines to be prefetched and the access address range; The first determining submodule is also used to determine the bandwidth ratio allowed for this prefetch based on the average memory bandwidth usage of the target execution layer and the complexity of the task scenario; The generation module 803 is specifically used to schedule the transmission timing of data in each sub-interval according to the bandwidth ratio, and generate a weighted prefetch instruction sequence based on the transmission timing and the row length of the preset level buffer.

[0099] In an optional embodiment, the determining module 801 specifically determines the allowed bandwidth ratio for this prefetch using the following algorithm: in, It is the hierarchical number of the target execution layer. This is the percentage of bandwidth allowed for this prefetch. yes The corresponding average memory bandwidth usage of the target execution layer, yes The corresponding maximum peak bandwidth of the target execution layer, It is the complexity of the task scenario. It serves as a reference upper limit for the complexity of the task scenario.

[0100] In one optional embodiment, the processing module 805 specifically includes: a second determining submodule, a setting module, and a processing submodule; The second determination submodule is used to determine the cache group index and cache tag item of the corresponding sub-interval in the cache group according to the prefetch start address of each weight prefetch instruction. The weight prefetch instruction is any instruction in the weight prefetch instruction sequence. The configuration module is used to locate the corresponding cache group based on the cache group index and set the cache tag entry in the tag register corresponding to that cache group from the default state to the lock flag bit. The processing submodule is used to locate the corresponding cache group based on the cache group index of the cache tag entry in all tag registers in this inference, which is the lock flag bit, and release the resources of the corresponding cache group. The configuration module is also used to set the cache tag entry in the tag register corresponding to the cache group from the locked flag to the default state.

[0101] In the embodiments of this application, the device determines the target execution layer of the target model, then determines the theoretical access address in memory of the weight data required for the target model inference through the target execution layer, and then corrects the theoretical access address based on the offset between the actual access addresses in the historical inference trajectory, finally obtaining the access address range of the weight data in memory. Determining the target execution layer is the preliminary decision basis for prefetching weight data in subsequent steps, determining the calculation logic of subsequent memory access operations. Since the weight data is stored in DDR, it is necessary to determine the specific access address range of the weight data in DDR based on the specific target execution layer. This allows the weight data to be prefetched into a preset-level cache based on the access address range, locking the cache line where the weight data is located. The preset-level cache is a level 1 cache that can be directly written to by direct memory access DMA or the cache controller. Other levels of cache cannot be directly operated on. Based on the characteristics of the preset-level cache, it is possible to read weight data from the locked cache line and then perform target model calculations based on the weight data, thereby ensuring task response speed under resource-constrained conditions.

[0102] The electronic device may include a processor 901 and a memory 902 storing computer program instructions.

[0103] Specifically, the processor 901 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0104] Memory 902 may include mass storage for data or instructions. For example, and not limitingly, memory 902 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 902 may include removable or non-removable (or fixed) media. Where appropriate, memory 902 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 902 is non-volatile solid-state memory.

[0105] In a particular embodiment, memory 902 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0106] The processor 901 implements any of the model data access methods in the above embodiments by reading and executing computer program instructions stored in the memory 902.

[0107] In one example, the electronic device may also include a communication interface 903 and a bus 910. Wherein, as... Figure 9 As shown, the processor 901, memory 902, and communication interface 903 are connected through bus 910 and complete communication with each other.

[0108] The communication interface 903 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0109] Bus 910 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 910 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0110] Furthermore, in conjunction with the model data access methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the model data access methods in the above embodiments.

[0111] This application also provides a computer program product, including a computer program, which, when executed, implements any of the model data access methods described in the above embodiments.

[0112] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0113] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0114] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0115] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0116] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for accessing model data, characterized in that, include: Determine the target execution layer of the target model; Based on the offset between the theoretical access address of the weight data of the target execution layer in memory and the actual access address in the historical inference trajectory data, the theoretical access address is corrected to obtain the access address range of the weight data in memory. The access address range of the weight data is divided into multiple sub-ranges, and a weight prefetch instruction sequence is generated according to the line length of the preset level cache. The weight data is prefetched from the multiple sub-intervals into the preset level cache using the weight prefetch instruction sequence. After the weight data is written to the preset level cache, the cache line containing the weight data is locked so that the target execution layer can read the weight data from the locked cache line for calculation.

2. The method according to claim 1, characterized in that, The memory access address range of the weight data is calculated using the following formula: Where i is used to identify the i-th execution layer, and i is a positive integer. It is the hierarchy number of the target execution layer. It is the aforementioned The corresponding memory access address range for the weight data of the target execution layer. It is the aforementioned The starting address of the weight data corresponding to the target execution layer. It is the aforementioned The offset between the theoretical access address of the weight data of the target execution layer in memory and the actual access address in the historical inference trajectory data. It is the aforementioned The total byte length of the weight data corresponding to the target execution layer.

3. The method according to claim 1, characterized in that, The target execution layer for determining the target model includes: Based on the received trigger information from the target execution layer, determine the fluctuation level information and the dispersion level information of the trigger information respectively; The complexity of the task scenario is determined based on the fluctuation information of the triggering information, the dispersion information, the preset fluctuation coefficient, and the preset dispersion coefficient. When the complexity of the task scenario is greater than a preset threshold, the target execution layer is determined to be a high-complexity model execution layer; when the complexity of the task scenario is less than the preset threshold, the target execution layer is determined to be a low-complexity model execution layer.

4. The method according to claim 3, characterized in that, The complexity of the task scenario is calculated using the following formula: in, This refers to the complexity of the task scenario. This refers to the fluctuation level information. This refers to the degree of dispersion information. It is the preset fluctuation coefficient. It is the preset dispersion coefficient.

5. The method according to claim 4, characterized in that, The step of dividing the access address range of the weight data into multiple sub-ranges and generating a weight prefetch instruction sequence based on the line length of a preset level cache specifically includes: The number of cache lines to be prefetched is determined based on the start address of the access address range, the end address of the access address range, and the line length of the preset level cache. Based on the number of cache lines to be prefetched and the access address range, the access address range is divided into multiple sub-ranges; The allowable bandwidth ratio for this prefetch is determined based on the average memory bandwidth usage of the target execution layer and the complexity of the task scenario. The transmission timing of data within each of the sub-intervals is scheduled according to the bandwidth ratio, and a weighted prefetch instruction sequence is generated based on the transmission timing and the row length of the preset level cache.

6. The method according to claim 5, characterized in that, The bandwidth ratio allowed for this prefetch is calculated using the following formula: Where i is used to identify the i-th execution layer, and i is a positive integer. It is the hierarchy number of the target execution layer. This refers to the allowed bandwidth ratio for this prefetch. It is the aforementioned The corresponding average memory bandwidth usage of the target execution layer, It is the aforementioned The corresponding maximum peak bandwidth of the target execution layer, the This refers to the complexity of the task scenario. This is a reference upper limit for the complexity of the task scenario.

7. The method according to claim 5, characterized in that, The step of locking the cache line containing the weight data after the weight data is written to the preset level cache includes: Based on the prefetch start address of each weight prefetch instruction, determine the cache group index and cache tag entry of the corresponding sub-interval in the cache group, wherein the weight prefetch instruction is any instruction in the weight prefetch instruction sequence; Locate the corresponding cache group based on the cache group index, and set the cache tag entry in the tag register corresponding to the cache group from the default state to the lock flag bit; Based on the cache group index of the cache tag entry in all the tag registers in this reasoning, locate the corresponding cache group and release the resources of the corresponding cache group; Set the cache tag entry in the tag register corresponding to the cache group to the default state from the lock flag bit.

8. The method according to any one of claims 3-7, characterized in that, The triggering information is voice wake-up information, the method is applied to a smart voice terminal, and the calculation result of the target execution layer is used to generate a voice wake-up judgment result.

9. A model data access device, characterized in that, The device includes: The determination module is used to determine the target execution layer of the target model; The correction module is used to correct the theoretical access address based on the offset between the theoretical access address of the weight data of the target execution layer in memory and the actual access address in the historical inference trajectory data, and to obtain the access address range of the weight data in memory based on the corrected theoretical access address. The generation module is used to divide the access address range of the weight data into multiple sub-ranges and generate a weight prefetch instruction sequence based on the line length of the preset level cache. The prefetch module is used to prefetch the weight data from the plurality of sub-intervals to the preset level cache using the weight prefetch instruction sequence; The processing module is configured to lock the cache line containing the weight data after the weight data is written to the preset level cache, so that the target execution layer can read the weight data from the locked cache line for calculation.

10. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the model data access method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the model data access method as described in any one of claims 1-8.