Artificial intelligence reinforcement learning service platform

Abstract

The invention discloses an artificial intelligence reinforcement learning service platform, particularly relates to the field of computer data processing, and is used for solving the problem that training sample semantics drifts and cannot be rechecked due to data center telemetering backfilling disorder and caliber version change superposition. The method comprises the following steps: generating an original record set by collecting telemetering, establishing double indexes, generating an action bill according to a fixed period, locking a backfill time window, extracting observation to form an aligned observation packet, and labeling missing section entries and late section entries; according to the aperture version identification execution state construction expression and the reward calculation expression, packaging the training sample packet and generating a re-calculation abstract, calculating a backfill link completeness index and an aperture shadow difference risk value, obtaining a release judgment value through a biaxial risk matrix, and writing the training sample packet into a sample library or an isolation area. And generating a strategy release packet, completing admission judgment according to the morphological interval table, triggering the freezing strategy release packet to output a bottom guarantee control instruction, and registering a repair queue.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer data processing, and more particularly, to an artificial intelligence reinforcement learning service platform. BACKGROUND

[0002] In a production environment such as a data center, an enterprise often wants to build a reinforcement learning service platform from a one-time trial project that can provide continuous capabilities to the outside, so that different business systems can complete data access, state construction, strategy training, effect evaluation, online reasoning, and online release and monitoring when they need to do resource scheduling, load management, capacity control, or operation parameter optimization. In order to keep the strategy up to date with production changes, the platform needs to access long-term multi-source operation information, which comes from different collection links and different system perspectives. It needs to support offline training and playback, as well as online decision-making and gray release, and still needs to be usable and controllable when the system evolves, the collection rules are adjusted, and the meaning of the indicators changes.

[0003] In existing practices, such a platform often uses multi-source data as a general data processing flow to splice, directly combines the states reported by different systems, and then uses the results of post-statistics to calculate rewards and backfill training records. However, in the data center scenario, there is often a delay, a lack, and a recalculation between the state and the result, and the meaning and calculation perspective of the same indicator will change with the system upgrade, which will lead to the fact that the state seen when the action occurs and the result obtained subsequently do not belong to the same real causal chain. The root of the problem lies in the fact that the time semantics of each link is not consistent, and the platform lacks mechanisms to align the time, leave traces of the perspective, bind the rewards, and review the playback as original capabilities of the platform, which leads to the fact that the training phase learns from data that does not occur at the same time, or even data whose meaning has changed, as the basis at the same time, and then attributes the wrong results to the actions that are not responsible. Such incorrect attribution can be hidden in the seemingly normal training convergence, and only when the strategy enters online operation does it show that the strategy output is unstable, the same strategy performs very differently at different times, offline evaluation and online effect do not match, and in severe cases, the strategy release and rollback cannot be explained, audited, and reproduced, so that the reinforcement learning service platform is difficult to be stably landed in the data center for a long time.

[0004] To solve the above problems, a technical solution is provided. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide an artificial intelligence reinforcement learning service platform, which generates an original record set by collecting telemetry and establishing double indexes, generates an action ticket at a fixed period and locks a backfill time window, extracts observations to form an aligned observation package and labels missing segment items and late segment items, constructs an expression and a reward calculation expression according to the state of the caliber version identification, encapsulates a training sample package and generates a recalculation summary, calculates a backfill link completeness index and a caliber shadow difference risk value, and obtains a release judgment value through a double-axis risk matrix, writes the training sample package into a sample library or an isolation area, generates a strategy release package and completes the access determination with a form interval table, triggers the frozen strategy release package to output a bottom control instruction and registers a repair queue, so as to solve the problems raised in the above background art.

[0006] To achieve the above object, the present application provides the following technical solutions: The telemetry collection module: collects event time identifiers and core index values in the data center running telemetry, generates access time identifiers, writes source identifiers and record serial numbers, performs time semantics verification and index value legality verification, writes verification result identifiers, stores in the original stream storage area, forms an original record set and establishes indexes; The ticket arrangement module: extracts state summaries based on the original record set at a fixed period and registers trigger point serial numbers, generates action tickets when outputting control actions, writes the action tickets into an action ledger, and binds the action tickets with trigger point serial numbers, backfill time windows, caliber version identifiers, execution domain identifiers and state summary values; The observation alignment module: reads the action tickets, extracts observation records from the original record set according to the backfill time window, sorts the aligned observation packages according to the event time identifiers, labels the missing segments and the late segments of the aligned observation packages, writes the missing segments into a completion request queue, retains the record serial numbers of the late segments and writes a recalculation flag, and stores the aligned observation packages into an observation storage area; The sample access module: executes state construction and reward calculation on the aligned observation packages according to the caliber version identifiers, generates a training sample package, synchronously executes playback consistency verification on the training sample package, outputs a release judgment value, and determines whether the training sample package is written into a sample library or an isolation area according to the release judgment value, and writes a judgment reason identifier; The strategy escort module: generates a strategy release package when the release judgment value is in a release state, binds the strategy release package with caliber version identifiers and aligned observation package summaries, reads the strategy release package and the latest aligned observation package online, checks the backfill integrity and caliber version consistency, freezes the strategy release package and outputs a bottom control instruction when inconsistency is triggered, and writes into a repair queue.

[0007] Further, the telemetry collection module comprises: reading event time identifiers and core indicator values from the telemetry access end, generating access time identifiers, writing source identifiers and record serial numbers, performing time semantics verification and indicator value legality verification and generating verification result identifiers, appending records to the original stream storage area, establishing a main index according to the source identifiers and the record serial numbers, establishing an auxiliary index according to the event time identifiers, and forming an original record set.

[0008] Further, the ticket arrangement module comprises: intercepting a fixed period window according to the event time identifiers based on the auxiliary index of the original record set, merging window records into an execution domain window set according to the execution domain mapping table, screening an effective record set according to the verification result identifiers, selecting a latest effective record to obtain a core indicator latest effective value according to the event time identifiers, obtaining a core indicator median effective value by sorting and taking a median effective value according to the core indicator, writing a core indicator change direction identifier by comparing the two, determining a missing placeholder count by counting a verification result exception count, and generating a state digest value.

[0009] Further, the ticket arrangement module further comprises: calculating arrival time delay values and sorting them in the historical original record set according to the source identifiers, determining a source time delay upper bound according to a maximum break position of adjacent difference values, obtaining an execution domain time delay upper bound by taking a maximum value of the source time delay upper bounds of the same execution domain according to the execution domain mapping table, determining a backfill time window start event time identifier and a backfill time window end event time identifier, counting a late record placeholder count by counting the arrival time delay values that exceed the source time delay upper bound and the event time identifiers that are within the window range, generating a trigger point serial number and writing it into an action ticket when outputting a control action, and writing the action ticket into an action ledger.

[0010] Further, the observation alignment module comprises: reading the action ticket from the action ledger, obtaining an execution domain identifier, a trigger point serial number, a backfill time window start event time identifier, and a backfill time window end event time identifier, extracting observation records from the original record set according to the execution domain allowed set and the verification result identifier, generating observation site identifiers according to the source identifiers and the indicator names, sorting them according to the event time identifiers and stabilizing the same order with the record serial numbers, and assembling an aligned observation package.

[0011] Further, the observation alignment module further comprises: dividing the backfill time window by a fixed period length, constructing a trigger point placeholder string and identifying a continuous missing interval to form a missing segment entry, writing the missing segment entry into the aligned observation package and into a fill request queue; maintaining a prefix maximum record serial number on an event time sorting sequence, recording a record serial number backoff as a late record and merging it into a late segment entry, writing a record covered by the late segment into a recalculation flag, generating an observation package identifier and writing it into an observation storage area.

[0012] Further, the sample admission module comprises: constructing an expression and a reward calculation expression from the caliber library loading state according to the caliber version identifier of the action ticket, and parsing into a table expression tree, completing field reference node and operator node checking, and calculating a state vector and a reward value on the aligned observation package, performing normalized serialization and generating a state digest recalculation value and a reward digest recalculation value, and encapsulating a training sample package, including an action ticket identifier, an observation package identifier, and a caliber version identifier.

[0013] Further, the sample admission module further comprises: the platform extracts observation records from the original record set and reconstructs a playback aligned observation package according to the backfill time window and the execution domain allowed set of the action ticket, recalculation is performed to obtain a playback state digest and a playback reward digest and generate a recalculation digest, a completeness profile is generated according to the must-achieve observation list and a backfill link completeness index is mapped, a shadow difference feature set is generated according to the last stable caliber version identifier and a caliber shadow difference risk value is calculated, and a backfill trust level and a caliber offset level are mapped and a release decision value is generated through a two-axis risk matrix, and the training sample package is written into the sample library or the isolation area according to the release decision value and a judgment reason identifier is written.

[0014] Further, the sample admission module comprises: reading a training sample package from the sample library, screening the training sample package with a release decision value of release and controlled release, grouping according to the caliber version identifier and checking the must-achieve observation list version identifier and the two-axis risk matrix version identifier in the group, and writing the version conflict training sample package into the isolation area; sorting the training sample package according to the execution domain identifier and the trigger point serial number, obtaining the control action content by reading the action ledger, assembling the training segment and updating the action value estimation table, generating a strategy release package and writing the caliber version identifier, the must-achieve observation list version identifier, the two-axis risk matrix version identifier and the training set digest.

[0015] Further, the strategy escort module further comprises: performing normalized serialization on the strategy release package and generating a strategy release package identifier written into the release ledger, the online inference entrance reads the strategy release package and the latest aligned observation package, checks the caliber version identifier, the must-achieve observation list version identifier and the two-axis risk matrix version identifier, and outputs a control instruction after checking the missing segment item and the late segment item form compliance according to the form interval table; when the admission decision fails, the strategy release package is frozen and registered in the freeze ledger, a bottom control instruction is output, and the admission failure reason, the observation site identifier and the recalculation digest positioning information are written into the repair queue.

[0016] The application provides an artificial intelligence reinforcement learning service platform, which relates to reinforcement learning and deep learning technology, and has the following technical effects and advantages: 1. Through locking action tickets in action ledger and extracting with backfill time window constraint original record set, combined with observation site identification, missing segment item, late segment item and recalculation flag of aligned observation package, training sample package only absorbs observation facts consistent with the same trigger point sequence number of the same execution domain identification, and the difference is located to the observation site identification and expression tree node in the recalculation summary, so that the playback consistency verification and repair queue registration have the same evidence chain, and the link jitter and out-of-order backfill mislead the strategy learning are reduced.

[0017] 2. Through the expression tree of caliber library, the state construction expression and reward calculation expression are solidified under the execution of caliber version identification, while the shadow difference feature set is generated and the caliber shadow difference risk value is calculated by introducing the last stable caliber version identification in parallel, the backfill link completeness index is generated by cooperating with the complete observation list and the completeness profile, and the release decision value is output by using the double-axis risk matrix and written into the decision reason identification, so that the separation judgment and joint constraint of time direction and caliber direction are realized, so that the sample library maintains semantic consistency and avoids semantic drift caused by caliber change in training.

[0018] 3. Through the strategy release package carrying caliber version identification, complete observation list version identification and double-axis risk matrix version identification at the same time, the online reasoning entrance performs version consistent check and form interval table check on the latest aligned observation package, and outputs the bottom control instruction when the access decision fails, and the repair queue item references the observation site identification and the recalculation summary positioning information, so that the homologous constraint and closed loop tracing of training side release decision and reasoning side access decision are realized, and the controllability and auditability of control instruction in data center scene during observation exception and caliber switching are guaranteed. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 It is a structural schematic diagram of an artificial intelligence reinforcement learning service platform. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] Embodiment 1: Figure 1 A kind of artificial intelligence reinforcement learning service platform of the present application is given, comprising: Telemetry collection module: Collect event time identifiers and core indicator values in the telemetry running data center, generate access time identifiers, write source identifiers and record serial numbers, perform time semantics verification and indicator value legality verification, write verification result identifiers, store in the original stream storage area, form the original record set and establish an index.

[0022] Ticket arrangement module: Based on the original record set, extract state summaries and register trigger point serial numbers at fixed periods, generate action tickets when outputting control actions, write action tickets into action ledgers, bind trigger point serial numbers, bind backfill time windows, bind dimension version identifiers, execution domain identifiers, and state summary values.

[0023] Observation alignment module: Read action tickets, extract observation records from the original record set according to the backfill time window, sort and generate aligned observation packages according to event time identifiers, mark missing segments and late segments in the aligned observation packages, write missing segments into the completion request queue, and record serial numbers and write recalibration flags into the late segments., and store the aligned observation packages in the observation storage area.

[0024] Sample admission module: According to the dimension version identifier, perform state construction and reward calculation on the aligned observation package, generate a training sample package, and simultaneously perform playback consistency verification on the training sample package. Output release decision value, release decision value determines whether to write training sample package into sample library or isolation area, and write decision reason identifier.

[0025] Strategy escort module: When the release decision value is in the release state, generate a strategy release package, bind the dimension version identifier and the aligned observation package summary, and read the strategy release package and the latest aligned observation package online. Reasoning, check the backfill integrity and dimension version consistency, freeze the strategy release package and output the bottom control instruction when inconsistency is triggered, and write to the repair queue.

[0026] Before entering the action ticket generation and backfill extraction, the data center reinforcement learning service platform first encounters problems from the disorder and delay of multi-source telemetry. Event time identifier describes the time when the business occurs, access time identifier describes the time when the record arrives, and when the two types of time semantics are inconsistent, window extraction and backfill extraction will splice unrelated observations into the same period, causing sample semantic drift. However, the ticket arrangement module relies on the arrival order structure and event time structure in the original record set to generate state summaries and backfill time windows, and the telemetry collection module must ensure that the original record set has traceable order markers and interpretable time verification conclusions.

[0027] S101. Three record generation and field filing.

[0028] Multi-source telemetry often does not conform to the time when the link is reported and the time when the platform is arrived. The ternary record separates the occurrence time from the arrival time, and ensures that the out-of-order scene can still explain the timing relationship. The platform reads the event time identifier and the core indicator value from the telemetry access end, generates an access time identifier at the receiving moment, and writes the three into the same record. The record is written into the append-only log segment of the original stream storage area, and a disk landing mark is generated after writing is completed. The disk landing mark is written at the tail of the log segment and is associated with the log offset position. The log offset position becomes a unified positioning point for subsequent indexing, and the record content and the positioning point correspond one by one, which facilitates subsequent backtracking to complete records according to any index.

[0029] S102. Allocation rules of source identifier and record sequence number.

[0030] Arrival order is the basis for judging late segments and sequence reversal segments. The record sequence number is monotonically increasing within the source identifier range, which can preserve the arrival sequence without relying on external time. The platform writes the source identifier according to the collection link mark of the telemetry access end, which is used to bind the parsing rules and unit attribution. A sequence number cursor is maintained for the same source identifier, and the sequence number cursor is incremented and written into the record sequence number when a new record is received. At the same time, a sequence number check bit is generated, which is obtained by the record sequence number through a ring operation. The ring operation maps the sequence number to a finite set, which is used to detect repeated writing and number skipping. The source identifier and the record sequence number are written as the primary index key field in the record header. The primary index key field is bound to the log offset position, which ensures that the record can be stably positioned according to the arrival order.

[0031] S103. Time semantic verification and time offset envelope boundary calculation.

[0032] There is a stable delay interval in different links of data center telemetry. Time semantic verification describes the deviation of arrival time from occurrence time using time offset, and then uses envelope to constrain the deviation range, which can eliminate obviously abnormal timestamp drift. The platform calculates the time offset for each record, which is calculated by subtracting the event time identifier from the access time identifier. The time offset maintains the time dimension. A reference set of time offsets is maintained for each source identifier, which only includes recent time offsets with a pass verification result. The reference set is sorted by size, and a segment close to the low end of the sorting result is taken as the lower bound of the time offset envelope, and a segment close to the high end of the sorting result is taken as the upper bound of the time offset envelope. The low end and the high end are defined by a fixed rule of the platform, which ensures consistent output of different instances.

[0033] The main rhythm interval is calculated, the main rhythm interval is the frequency of the difference between adjacent event time identifiers under the same source identifier, the highest frequency difference is selected as the main rhythm interval, and the main rhythm interval maintains the time dimension. The segment reverse tolerance segment is calculated, the segment reverse tolerance segment is the length of the upper bound of the short segment reverse in the recent record sequence number and event time identifier through the record, and the upper bound is determined by the segment close to the high end after sorting. The time semantic judgment is marked by time, and the legality of the event time identifier format and the consistency of the time zone are checked at the same time. The time offset is within the lower bound and the upper bound of the time offset envelope, and the distance from the boundary is not more than the buffer range determined by the main rhythm interval and the segment reverse tolerance segment, which is the passing condition. The time semantic passing mark is written into the time semantic bit of the check result identifier, and the time semantic bit clearly records that the check conclusion comes from the offset envelope and the main rhythm interval, which is convenient for review and explanation.

[0034] S104. Index value legal verification and unit attribution consistency determination.

[0035] The core index value comes from different collection links, and the inconsistent analysis method and unit attribution will lead to the mixing of the same index with different dimensions, and the subsequent state construction expression and reward calculation expression will produce unexplainable results. The platform positions the index analysis rule table according to the source identifier, and the index analysis rule table gives the analysis type and expected unit attribution of the core index value. When performing index value legal verification, first convert the core index value from the original representation to the calculable representation according to the analysis rule, and write the abnormal reason code and mark the index legal bit as not passing when the conversion fails. After successful conversion, check whether the calculable representation is in the limited expressible range, and mark the index legal bit as not passing and write the abnormal reason code when the non-limited form appears.

[0036] When performing unit attribution consistency determination, read the unit attribution mark carried by the record and compare it with the expected unit attribution in the index analysis rule table, mark the unit consistent bit as passing when they are consistent, and mark the unit consistent bit as not passing and write the abnormal reason code when they are inconsistent. Write the index legal bit and the unit consistent bit into the check result identifier, and the check result identifier fully reflects the specific reasons for analysis failure, out-of-bound form and unit conflict, which is convenient for positioning the missing segment cause in the aligned observation package stage.

[0037] S105. Additional storage and double index construction consistency verification.

[0038] The original record set supports both playback in arrival order and windowing by event time. The dual index structure fixes both access paths at log offset positions, avoiding inconsistency of the same record under two paths. The platform constructs a primary index with source identifier and record sequence number after writing a record into the append-only log segment of the original stream storage area. The primary index maps the primary index key field to the log offset position. The secondary index is constructed with event time identifier. The secondary index maps the event time identifier to a list of log offset positions, which are arranged in event time order and keep the corresponding record sequence number, ensuring that out-of-order visibility.

[0039] When performing index readback verification, a log offset position is taken out from the primary index and the record is read back. Then the list of log offset positions corresponding to the same event time identifier is located from the secondary index and the record is read back. If the source identifier and record sequence number of the two readback records are consistent with the verification result identifier, it is determined that the index is consistent. When inconsistency is found, an index exception identifier is written and the write channel corresponding to the source identifier is suspended. The index exception identifier is written together with the log segment position, ensuring that the exception is traceable and does not pollute the original record set.

[0040] After the telemetry collection module is completed, the original stream storage area forms the original record set, which includes the append-only log segment, the primary index, the secondary index, and the verification result identifier of each record. The primary index provides the arrival order perspective of the source identifier and the record sequence number, and the secondary index provides the time window perspective of the event time identifier. The verification result identifier fixes the time semantic bit, the indicator legality bit, the unit consistency bit, and the exception reason code in the record. The ticket arrangement module can directly reference the event time identifier and the verification result identifier when windowing and constructing the state digest. The observation alignment module can directly reference the record sequence number and the order information in the secondary index list when extracting observation records and labeling the late segment, ensuring that the action ticket and the backfill time window are established on the same original record set semantics.

[0041] The original record set has fixed the event time identifier, the access time identifier, the source identifier, the record sequence number, and the verification result identifier to the append-only log and the dual index view. The platform has both the path of windowing by event time identifier and the path of playback in arrival order by source identifier and record sequence number. However, after the reinforcement learning action is issued, the effect backfill often lags behind the next time window. The action ticket needs to lock down the state digest value and the backfill time window at the moment of action issuance, ensuring that the backfill extraction boundary remains consistent under different data out-of-order patterns.

[0042] S201. Window alignment and windowing.

[0043] Fixed period trigger point needs to be established on event time identifier, access time identifier only reflects arrival, and is not suitable for defining service occurrence window. The platform advances window end event time identifier from early to late with event time identifier on auxiliary index, and pushes back window start event time identifier with fixed period length, extracts records with event time identifier between start and end as window record set, and retains log offset position, source identifier, record serial number, and check result identifier in window record set. Formation of window record set makes observations in the same time slice fall into the same set, avoiding changing window boundary interpretation due to out-of-order arrival.

[0044] S202. Perform domain merging and valid record screening.

[0045] State summary and action ticket need to fall within the same control boundary, and execution domain identifier assumes the responsibility of boundary definition. The platform maps the source identifier of each record in the window record set to the execution domain identifier according to the execution domain mapping table, and merges records with the same execution domain identifier into the execution domain window set. The valid record set is screened in the execution domain window set, and the screening conditions of the valid record set require that the time semantic bit of the check result identifier passes, the index legal bit passes, and the unit consistent bit passes. The valid record set excludes noise records caused by timestamp drift, parsing failure, and unit conflict, and the source of the state summary field is consistent, so that subsequent review does not need to introduce additional interpretation conditions.

[0046] S203. State summary generation.

[0047] The reinforcement learning action issuing moment needs to compress the expression of the execution domain state, and the state summary needs to retain not only the core index level, but also the abnormality and observation gap pattern. The platform selects the latest valid record in the valid record set according to the event time identifier, and writes the core index value of the latest valid record into the core index latest valid value. The core index values in the valid record set are sorted from small to large, and the core index value corresponding to the central position after sorting is taken as the core index median valid value. The core index latest valid value and the core index median valid value are compared, and if the latest valid value is greater than the median valid value, the uplink direction identifier is written, if the latest valid value is less than the median valid value, the downlink direction identifier is written, and if the two are equal, the stable direction identifier is written.

[0048] The core index value is used to reflect the key running state of the execution domain at the time corresponding to the event time identifier, for example, when the source identifier is a rack switch and the index name is port packet loss rate, the core index value records the numerical value of the port packet loss rate, or when the source identifier is a computing node and the index name is processor utilization, the core index value records the numerical value of the processor utilization.

[0049] In the execution domain window set, the number of record entries that do not pass the check is identified, and the check result abnormal count is written. When calculating the missing placeholder count, a plurality of historical windows with the same execution domain and the same fixed cycle length are selected from the historical original record set, the check result abnormal count of each historical window is calculated, the historical window with the minimum abnormal count is taken as the reference window, the number of occurrences of the valid record entries in the reference window is counted, the valid record entry with the maximum number of occurrences is taken as the expected valid entry, and the missing placeholder count is obtained by subtracting the current window valid record entry from the expected valid entry. When the difference is negative, the missing placeholder count is written as zero. The state summary writes the level, direction, abnormality and missing in the structured field, and the action ticket carries the state summary value, which can directly record the observation quality and state position when the action is issued.

[0050] S204. Link latency image generation and backfill time window determination.

[0051] The backfill time window needs to cover the observations of the lag backfill, and the link latency image needs to extract the stable boundary from the original record set to avoid the extreme delay from lengthening the boundary. The platform collects a group of records that pass the time semantic check in the historical original record set according to the source identifier, calculates the arrival latency value of each record, and the arrival latency value is equal to the access time identifier minus the event time identifier. The arrival latency values are sorted from small to large, the difference between the arrival latency values of the adjacent two items after sorting is calculated, the maximum difference position is found, the arrival latency value before the breaking position is taken as the source latency upper bound, and the source latency upper bound represents the upper edge of the normal delay range. The execution domain mapping table is mapped to the maximum value of the plurality of source latency upper bounds belonging to the same execution domain to obtain the execution domain latency upper bound. The backfill time window start event time identifier is determined as the window end event time identifier, the execution domain latency upper bound is added to the fixed cycle length, and then added to the window end event time identifier to obtain the backfill time window end event time identifier.

[0052] When calculating the late record placeholder count, the arrival latency value is calculated for each record in the execution domain window set, and if the arrival latency value exceeds the corresponding source latency upper bound and the event time identifier is within the window range, the record is counted into the late record placeholder count. The backfill time window and the late record placeholder count share the same source latency upper bound, the boundary explanation is unified, and different late conclusions for the same record in different links are avoided.

[0053] S205. Trigger point sequence number generation and action ticket registration.

[0054] The action ticket needs to fix the state summary at the moment of action issuance and the backfill time window as an auditable voucher, and the trigger point serial number is used for sorting and positioning. The platform maintains a trigger point serial number cursor for each execution domain. When the window end event time identifier arrives at the trigger condition, the trigger point serial number cursor is incremented and written into the trigger point serial number. The reading aperture version identifier in the aperture library is read. The aperture version identifier is mapped to the corresponding reinforcement learning service aperture according to the execution domain. The action ticket is generated when the output control action is generated. The action ticket is written into the action ledger. The action ticket field includes the execution domain identifier, the trigger point serial number, the aperture version identifier, the state summary value, the backfill time window start event time identifier, and the backfill time window end event time identifier. The action ledger is indexed according to the execution domain identifier and the trigger point serial number. The action ticket forms a stable order in the action ledger. The backfill extraction does not depend on the access disorder and can accurately position the backfill time window.

[0055] The ticket arrangement module performs window alignment interception, execution domain merging, effective record screening, state summary generation, link delay image calculation, backfill time window determination, trigger point serial number registration, and action ticket settlement. The action ledger obtains an action ticket set containing the execution domain identifier, the trigger point serial number, the aperture version identifier, the state summary value, and the backfill time window. The window record set and the execution domain window set convert the original record set into a time slice view organized according to the control boundary. The late record placeholder count and the missing placeholder count directly write the observation quality into the state summary. The backfill time window fixes the extraction boundary of the effect backfill as an event time identifier range. The observation alignment module can directly extract observation records and generate aligned observation packages according to the backfill time window when reading the action ledger.

[0056] The original record set has fallen into the original flow storage area with the event time identifier and the record serial number. The action ledger has fallen into the action ticket with the execution domain identifier, the trigger point serial number, the aperture version identifier, the state summary value, and the backfill time window. The aligned observation package needs to extract observation records from the original record set and complete the alignment sorting according to the backfill time window defined by the action ticket. At the same time, the missing section and the late section form a reviewable label. However, once the extraction and labeling lack unified judgment aperture, subsequent playback consistency verification will have the problem that the same observation is given different conclusions at different stages.

[0057] S301. Action ticket reading and observation extraction predicate generation.

[0058] After the backfill time window is fixed in the action ticket, the observation extraction predicate must be completely derived from the action ticket field to avoid differences in extraction boundaries at different execution nodes. The platform reads the action ticket from the action ledger, reads the execution domain identifier, trigger point sequence number, backfill time window start event time identifier, backfill time window end event time identifier, and extracts the execution domain allowed set corresponding to the execution domain identifier from the execution domain mapping table. The execution domain allowed set contains the source identifier set allowed to enter the execution domain. The observation extraction predicate is generated, which requires that the event time identifier fall within the backfill time window start event time identifier and the backfill time window end event time identifier, that the source identifier belong to the execution domain allowed set, and that the time semantic bit of the check result identifier pass, the index legal bit pass, and the unit consistent bit pass. After the observation extraction predicate is uniquely determined by the action ticket field, the same boundary interpretation is obtained for any playback extraction.

[0059] S302. Observation record extraction and observation site generation.

[0060] The same name index has different meanings or different unit attributions in different collection links, and the observation site needs to use the source identifier and the index name as identifiers at the same time to avoid mixing records with the same name into the same sequence. The platform locates the log offset position that meets the conditions on the double index of the original record set according to the observation extraction predicate, and reads the records corresponding to the log offset position to form an observation record set. For each observation record, read the source identifier and the index name, and combine the source identifier and the index name to generate an observation site identifier, which is used as an archiving key. The observation record set is divided into multiple observation record lists according to the observation site identifier, and each observation record list retains the event time identifier, the access time identifier, the record sequence number, the core index value, and the check result identifier. After the observation site identifier is fixed, subsequent difference positioning can directly refer to the source identifier and the index name.

[0061] S303. Event time sorting and alignment observation package assembly.

[0062] The alignment observation package requires different observation sites to share a unified time axis, and the event time identifier assumes the role of the main axis, and the record sequence number assumes the role of stable sorting to avoid sequence drift caused by repeated reporting. The platform sorts each observation record list according to the event time identifier from early to late, and when the event time identifier of the records is the same, the record sequence number is arranged from small to large. The sorted observation record list is written into the alignment observation package, and the execution domain identifier, trigger point sequence number, backfill time window start event time identifier, backfill time window end event time identifier are written in the header of the alignment observation package, and the observation site identifier and the number of record entries are written for each observation site. After the sorting rule is unique, the alignment observation package maintains consistent record arrangement during recalculation, and the recalculation summary positioning will not drift due to list order differences.

[0063] S304. Missing segment identification and request generation.

[0064] Missing segment identification must rely on trigger point rhythm, which is defined by fixed cycle length consistent with trigger point sequence number in action ticket system. Missing segment can be aligned and reviewed with action ticket only. The platform reads the fixed cycle length, takes the backfill time window start event time identifier as the reference, divides the backfill time window into continuous trigger point time slices, each of which has a starting boundary and an ending boundary. The starting boundary is the backfill time window start event time identifier plus the fixed cycle length multiplied by the time slice sequence number, and the ending boundary is the starting boundary plus the fixed cycle length.

[0065] For each observation site, construct a trigger point placeholder string, which is arranged according to the time slice sequence number. In each time slice, search for the observation record list corresponding to the observation site. If there is at least one record whose event time identifier falls within the time slice boundary, it is recorded as arrival; if not, it is recorded as missing.

[0066] Scan the trigger point placeholder string, and register the starting time slice sequence number and the ending time slice sequence number of continuous missing as missing segment entries. Write the missing segment entries into the aligned observation package. Convert the missing segment entries into a request for completion and write them into the request queue. The request for completion contains source identifier, missing interval and observation site set. The source identifier is taken from the source identifier in the observation site identifier. After the missing segment entries and the request for completion fields are consistent, the completion process can directly locate the gap according to the missing interval and review the completion effect.

[0067] S305. Late segment identification and recalculation flag writing.

[0068] The late segment identification needs to use the record sequence number to reserve the arrival order, the record sequence number is monotonically increasing in the source identification range, the record sequence number back after the event time sorting represents the late backfill or out-of-order arrival, and the late segment needs to form a paragraph-level label for review. The platform performs scanning on the event time sorting sequence of each observation site, and maintains the prefix maximum record sequence number during the scanning process. The prefix maximum record sequence number represents the maximum record sequence number that has occurred before the current position. The record sequence number of the next record is read and compared with the prefix maximum record sequence number. When the record sequence number is less than the prefix maximum record sequence number, the record is marked as a late record. The continuous late records are merged into a late segment entry. The late segment entry records the late segment start event time identification, the late segment end event time identification, the minimum record sequence number in the late segment, and the maximum record sequence number in the late segment. The late segment entry is written into the aligned observation package. The observation records covered by the late segment are written with a recalculation flag. The recalculation flag is written into the record-level metadata of the aligned observation package and the index metadata of the observation storage area, to ensure that the playback consistency verification can accurately locate the observation subset that needs to be recalculated. After the prefix maximum record sequence number scanning rule is fixed, the late segment boundary determination method is unique, and the late segment entry can be read back verified using the original record set master index.

[0069] S306. Observation package identification generation and storage.

[0070] The storage of the aligned observation package needs a stable identification. The stable identification must be generated from the determination field in the action ticket to avoid duplication or mismatch in concurrent writing. The platform normalizes the execution domain identification, trigger point sequence number, backfill time window start event time identification, and backfill time window end event time identification in a fixed field order. The normalization unifies the time format to the same time zone and the same precision. The encoding result is used as the observation package identification. The aligned observation package is written into the observation storage area and the observation package identification index. The observation package identification index points to the storage location of the aligned observation package. After the observation package identification is stable, the training sample package can form a unique reference with the action ticket identification and the observation package identification, and the calculation summary backtracking path is clear.

[0071] After the observation alignment module is completed, the aligned observation package already contains the observation site identification, the event time sorting sequence, the missing segment entry, the late segment entry, the recalculation flag, and the observation package identification. The aligned observation package forms a stable storage object in the observation storage area. The missing segment entry compresses the observation gap into a reviewable trigger point interval. The late segment entry compresses the out-of-order and late into a reviewable event time interval and record sequence number interval. The recalculation flag explicitly labels the observation subset that needs to be reviewed. When the sample admission module executes the state construction expression and the reward calculation expression, it can directly perform consistency verification on the aligned observation package and locate the difference causes.

[0072] The alignment observation package has solidified the observation site identification, missing segment item, late segment item, recalculation flag, and observation package identification into the observation storage area, and the action ticket has solidified the execution domain identification, trigger point sequence number, caliber version identification, state summary value, and backfill time window into the action ledger. Once the reinforcement learning training sample package mistakenly treats missing segments and late segments as real feedback, the strategy will learn the link jitter as a regularity, and once the caliber version switching is mistakenly treated as an environmental change, the strategy will learn the semantic drift as a regularity. However, the sample admission module bears the dual responsibilities of training sample package generation and release judgment value generation, and the sample admission module must aggregate the consistency verification, backfill link completeness index, and caliber shadow difference risk value into the judgment reason identification to ensure consistent explanation of sample storage.

[0073] S401. Expression tree loading and expression tree verification.

[0074] The caliber version identification determines the observation site identification and operator semantics referenced by the state construction expression and reward calculation expression. Expression tree verification before expression loading can separate expression errors and data errors. The platform reads the caliber version identification from the action ticket, reads the state construction expression and reward calculation expression from the caliber library, parses the two sets of expressions into expression trees, and the expression tree node list includes field reference nodes, time alignment operator nodes, missing processing operator nodes, discretization operator nodes, and boundary clipping operator nodes. Verify whether the field reference node exists corresponding observation site identification, verify whether the operator node belongs to the caliber library allowed operator table, and verify whether the segment boundary of the boundary clipping node exists and the boundary order is legal. Write the failed nodes to the expression exception field of the recalculation summary and terminate the sample generation. After the expression tree verification is passed, the expression tree structure is stable, the expression tree calculation path is unique, and the calculation result is consistent in explanation.

[0075] S402. State vector generation and state summary recalculation value generation.

[0076] The state vector is directly used for training without abstracting and solidifying, which can cause consistency verification to compare field by field and amplify small format differences, and the state abstract recalculation value compresses the state vector into a stable abstract for easy review. The platform reads the aligned observation package in the observation storage area, calculates the state construction expression by recursively calculating the expression tree root node downward, the time alignment operator node positions the observation record to the corresponding trigger point time slice according to the event time identifier, the missing processing operator node triggers the missing branch according to the missing segment item and writes the missing placeholder value, the discretization operator node maps the continuous value to the interval mark according to the discretization rule of the gauge library, and the boundary clipping operator node clips the out-of-bound value to the boundary mark according to the boundary rule. The state vector is output by the summary expression tree root node, and the state vector is written in a fixed field order by executing the normalization serialization, the unit attribution mark is written as the unified code of the gauge library, the empty field is written as the fixed missing mark, and the numerical field is written as the fixed precision format. The state abstract recalculation value is obtained by performing the fixed length hash abstract operation on the normalized serialization result, and the state abstract recalculation value is completely consistent when the field order is consistent, and the state abstract recalculation value is necessarily changed when the field content is changed, so that the consistency verification can stably locate the difference.

[0077] S403. Reward value generation and reward abstract recalculation value generation and packaging of training sample package.

[0078] The reward value comes from the reward calculation expression, and the reward value and the state vector come from the same aligned observation package and the same gauge version identifier, so as to ensure the semantic uniformity of the sample. The platform executes the expression tree calculation of the reward calculation expression on the same aligned observation package, and the time alignment operator node and the missing processing operator node reuse the alignment and missing rules of the state vector generation, to obtain the reward value and write it into the training sample package. The reward value is written as a fixed precision format and written into a unit attribution mark by performing the normalization serialization on the reward value, and the reward abstract recalculation value is obtained by performing the fixed length hash abstract operation on the normalized serialization result. The action ticket identifier, observation package identifier, gauge version identifier, state vector, reward value, state abstract recalculation value, reward abstract recalculation value are packaged into a training sample package, the training sample package has a fixed field order, and the training sample package has a uniform semantic that can be audited and reviewed.

[0079] S404. Original record set playback extraction and playback aligned observation package packaging.

[0080] The observation storage area alignment observation package has completed extraction and sorting, and the playback consistency verification needs to return to the original record set to reconstruct the observation process. The reconstruction process reuses the action ticket boundary and the execution domain allowed set to ensure the consistency of the calculation boundary. The platform reads the backfill time window start event time identifier and the backfill time window end event time identifier according to the action ticket, reads the execution domain allowed set according to the execution domain identifier, extracts observation records in the original record set, and the extraction conditions meet the following conditions: the event time identifier is within the backfill time window, the source identifier belongs to the execution domain allowed set, the verification result identifier time semantic bit passes, the index legal bit passes, and the unit consistent bit passes. The extracted observation records are combined to generate observation site identifiers according to the source identifier and the index name, and are archived to form an observation record list according to the observation site identifier. Each observation record list is sorted according to the event time identifier and is stabilized in the same order with a record serial number. The playback alignment observation package is assembled and written into the same package header field as the alignment observation package. The playback alignment observation package reconstruction process can be verified by reading the main index.

[0081] S405. Playback calculation and calculation summary generation.

[0082] The calculation summary needs to cover the state difference and the reward difference, and also needs to give the difference source link. The difference source link is associated with the observation site identifier, the missing segment entry, and the late segment entry to guide the repair. The platform executes the state construction expression and the reward calculation expression on the playback alignment observation package to generate the playback state vector and the playback reward value. The playback state summary and the playback reward summary are obtained by normalizing serialization and hash summary operation consistent with the training sample package. The state summary recalculated value is compared with the playback state summary, and the reward summary recalculated value is compared with the playback reward summary. When any comparison is inconsistent, node-level calculation is started. The node-level calculation normalizes the serialization and hash summary operation of each node output of the expression tree, locates the first difference node set, and writes the difference node list. The difference node list records the observation site identifier, the missing segment entry number, the late segment entry number, and the recalculated flag position. The consistent mark, the difference node list, and the difference propagation path are summarized to generate the calculation summary, which has a traceable link to the original record set log offset position.

[0083] S406. Mandatory observation list maintenance and criticality level derivation.

[0084] The backfill link perfect index needs to determine the execution domain within the mandatory observation site, the mandatory observation site comes from the stable operation interval rather than the arbitrary historical interval, and the criticality level needs to come from the control surface dependency rather than manual experience. The platform filters the sample library to release the decision value, and the re-computed summary is consistent with the label of all passed training sample packages. The corresponding alignment observation package is read, and the trigger point placeholder string is generated for each observation site identifier. The trigger point placeholder string records the arrival, absence, and delay states according to the trigger point time slice.

[0085] The dominant form of the trigger point placeholder string is merged into the rhythm label, which includes continuous mandatory, intermittent mandatory, and event mandatory. Continuous mandatory corresponds to the absence and delay of long-term placeholder string, intermittent mandatory corresponds to the absence of placeholder string, and event mandatory corresponds to the stable appearance of action occurrence adjacent time slice.

[0086] Read the control surface dependency table, which records the observation site identifier set referenced by the control action generation rule and the reward calculation expression. The observation site identifiers that appear in both the control action generation rule and the reward calculation expression reference set are marked as high criticality level, the observation site identifiers that only appear in the control action generation rule reference set are marked as medium criticality level, and the observation site identifiers that only appear in the state construction expression reference set are marked as low criticality level.

[0087] Write the observation site identifier, rhythm label, and criticality level into the mandatory observation list. The mandatory observation list version is written into the access library, and the mandatory observation list generation process has an auditable basis.

[0088] S407. Perfect profile generation and backfill link perfect index calculation.

[0089] The backfill link perfect index needs to convert the absence segment entry and the delay segment entry into a unified profile representation. The profile representation preserves the factual details and can be compressed into an index. The double-layer representation is convenient for consistency determination and review. The platform reads the mandatory observation list and locks the mandatory observation list version. The observation site identifier in the mandatory observation list is processed one by one on the current alignment observation package. The arrival form code is generated according to the trigger point time slice. The arrival form code records the arrival, absence, and delay states in each trigger point time slice.

[0090] The paragraph compression is performed on the arrival form code, the paragraph compression combines the continuous missing fragments into a continuous missing segment list, combines the continuous late fragments into a late segment list, and extracts the event time identifier from the late segment entry, reverses the order of the fragments, and generates a sequence reversal segment list according to the degree of overlap between the record serial number reverse fragments. The relative position level of the missing segment near the action occurrence point is calculated, and the relative position level is obtained by dividing the time distance between the event time identifier of the start point of the missing segment and the event time identifier of the end point of the window by the fixed period length and taking the integer part.

[0091] The continuous missing segment list, the sequence reversal segment list, and the relative position level are summarized into a completeness profile according to the observation site identifier, and the maximum penalty segment and the second maximum penalty segment are retained in the completeness profile according to the criticality level. The backfill index gear table is read, and the backfill index gear table maps the criticality level, the missing segment length level, the relative position level, and the sequence reversal overlap level to the backfill link completeness index.

[0092] The backfill link completeness index is obtained by first converting the completeness profile into a set of enumerable level labels, and then using the backfill index gear table to obtain the result: the platform first reads the continuous missing segment list and the sequence reversal segment list at each observation site covered by the must-arrive observation list, takes the longest missing segment in the continuous missing segment to obtain the missing segment length level, takes the segment closest to the action occurrence point from the longest missing segment to obtain the relative position level, takes the strongest overlapping fragment of the event time identifier reverse fragment and the record serial number reverse fragment in the sequence reversal segment list to obtain the sequence reversal overlap level, and reads the criticality level in the must-arrive observation list at the same time; the platform then concatenates the criticality level and the three levels in a fixed order to form a gear key, and then uses the gear key to locate the corresponding index gear in the backfill index gear table and outputs it as the backfill link completeness index. The hit item in the table lookup is written into the judgment reason identifier for review.

[0093] Accordingly, the backfill link completeness index is output, and the mapping hit item is written into the judgment reason identifier, and the backfill link completeness index is calculated to be reviewable and interpretable.

[0094] S408. Shadow difference feature set generation and caliber shadow difference risk value calculation.

[0095] The caliber shadow difference risk value needs to isolate the semantic drift caused by caliber change, and the shadow difference feature set is obtained from the same aligned observation package, which avoids the mutual confusion of environmental change and caliber change. The platform reads the last stable caliber version identifier according to the caliber library stability mark field, and the platform performs the expression tree corresponding to the caliber version identifier and the expression tree corresponding to the last stable caliber version identifier on the same aligned observation package to obtain two sets of state summaries and two sets of reward summaries.

[0096] The platform generates a symbol flip list, an order flip list, and an interval crossing list. The symbol flip list records the reward direction flip and the key state field direction flip. The order flip list encodes the relative order change of the key state field in the execution domain as a ranking consistency fragment. The interval crossing list records the number of times the key state field crosses the boundary clipping segment boundary and the trigger point position. The platform encodes the three types of lists into a shadow difference feature set. Each feature in the shadow difference feature set is first binned. The binning boundary comes from the feature binning table stored in the caliber library. The feature binning table is generated from historical treatment records and is versioned. The caliber risk model is loaded in the form of model parameters of a logistic regression model. The model parameters are saved according to the feature binning items. The hit binning item parameters are accumulated item by item to obtain a risk score. The risk score is converted into a caliber shadow difference risk value through a monotonic probability mapping function. The hit binning item and the risk score are written into the judgment reason identifier. The caliber shadow difference risk value source is transparent. The pseudo-law caused by caliber changes is suppressed before entering the sample library.

[0097] S409. Grade mapping biaxial risk matrix judgment and sample drop-off.

[0098] The release judgment value needs to be discretized in the time direction and caliber direction respectively and then combined. The discretization boundary version is fixed to ensure that the same input obtains the same judgment. The platform reads the backfill credible grade boundary table. The backfill credible grade boundary table segments the backfill link completeness index into three segments: complete, suspicious, and damaged. The boundary table is generated after the historical release samples are sorted according to the fixed segmentation position and is written into the caliber library version. The caliber offset grade boundary table is read. The caliber offset grade boundary table segments the caliber shadow difference risk value into three segments: stable, offset, and high risk. The boundary table is also versioned and saved.

[0099] The backfill credible grade boundary table is disclosed as follows: The platform summarizes the historical training sample packages according to the execution domain identifier and the must-arrive observation list version identifier. Two types of samples are extracted respectively. One type comes from the training sample package in the sample library with a release judgment value of release and no backfill from the repair queue. The other type comes from the training sample package in the isolation area or the repair queue marked for repair. The platform takes the maximum value of the backfill link completeness index in the repair sample and the minimum value of the backfill link completeness index in the stable sample and writes it into the boundary table. When the maximum value is less than the minimum value, values less than or equal to the maximum value are mapped to damaged, values greater than or equal to the minimum value are mapped to complete, and values between the two are mapped to suspicious. When the maximum value is greater than or equal to the minimum value, values less than or equal to the minimum value are mapped to damaged, values greater than or equal to the maximum value are mapped to complete, and values between the two are mapped to suspicious.

[0100] The caliber deviation level boundary table is disclosed as follows: the platform aggregates the historical training sample packages according to the execution domain identification and caliber version identification, extracts two types of samples respectively, one type is from the training sample package in the sample library with the release judgment value being release and the repair queue not generating backfill, and the other type is from the training sample package in the isolation area or the repair queue marked for repair; the platform takes the maximum value of the caliber shadow difference risk value in the stable sample and the minimum value of the caliber shadow difference risk value in the repair sample, and writes them into the boundary table, when the maximum value is less than the minimum value, the value less than or equal to the maximum value is mapped as stable, the value greater than or equal to the minimum value is mapped as high risk, and the value between the two is mapped as deviation, when the maximum value is greater than or equal to the minimum value, the value less than or equal to the minimum value is mapped as stable, the value greater than or equal to the maximum value is mapped as high risk, and the value between the two is mapped as deviation.

[0101] According to the two boundary tables, the backfill link completeness index is mapped to the backfill trust level, and the caliber shadow difference risk value is mapped to the caliber deviation level, and the release judgment value is output according to the double-axis risk matrix, and the matrix rule is fixed as isolation corresponding to the backfill trust level being damaged, isolation corresponding to the caliber deviation level being high risk and the backfill trust level being suspicious, controlled release corresponding to the caliber deviation level being high risk and the backfill trust level being complete, release corresponding to the caliber deviation level being stable and the backfill trust level being complete, and controlled release corresponding to the rest combinations. The platform writes the recalculation summary, backfill link completeness index, caliber shadow difference risk value, backfill trust level, caliber deviation level and release judgment value into the judgment reason identification. The platform writes the training sample package with the release judgment value being release and controlled release into the sample library and writes the controlled mark, and writes the training sample package with the release judgment value being isolation into the isolation area and writes the isolation reason field, and the sample shunt result has stable basis and traceable evidence chain.

[0102] After the sample access module is completed, the training sample package is formed under the caliber version identification constraint, the recalculation summary solidifies the consistency verification fact, the backfill link completeness index compresses the missing segment item and the late segment item into a determinable index, the caliber shadow difference risk value compresses the caliber change semantic deviation into a determinable index, and the double-axis risk matrix combines the two types of indexes into the release judgment value and writes it into the judgment reason identification. The sample library and the isolation area each form a stable set indexed by the action ticket identification and the observation package identification, and the training and online checking can review the sample source with the same evidence chain.

[0103] The aligned observation package is divided into sample library after passing through the release judgment value, the training sample package has bound action ticket identification, observation package identification, caliber version identification, and judgment reason identification, and the sample semantic boundary has been fixed in the caliber library and action ledger record. However, policy training and online reasoning still face two types of mixed risks, one from semantic drift caused by inconsistency between caliber version identification and mandatory observation list version identification, and the other from observation distortion caused by missing segment entries and late segment entry form out of bounds, and the strategy escort module needs to convert consistency verification conclusions into hard rules for training access and reasoning access.

[0104] S501. Sample library screening and training set grouping.

[0105] Caliber version identification determines the interpretation method of state vector and reward value, mandatory observation list version identification and double-axis risk matrix version identification determine the observation quality judgment caliber, and the mixed use of three types of version identification will put incompatible samples into the same training set and weaken the stability of the strategy. Training set grouping can directly use the built-in fields in the training sample package to complete hard screening.

[0106] The platform reads the training sample package from the sample library, only keeps the training sample package with release judgment value as release and controlled release, reads the caliber version identification, mandatory observation list version identification, double-axis risk matrix version identification and consistent flag carried by each training sample package. First, group by caliber version identification, then do consistency screening by mandatory observation list version identification and double-axis risk matrix version identification within each group, and write version conflict source in the judgment reason identification. After screening, the training set, the version boundary is clear within each group, and the training data range can be directly reproduced by the version field when auditing and reviewing.

[0107] S502. Strategy training data assembly and strategy update.

[0108] The training sample package saves the state vector and reward value, the action ticket identification points to the action ledger record, and the action ledger record contains the control action content. The lack of control action content will cause the strategy update to lack the action dimension and cannot form the action value estimate. The action field can be read from the action ledger and aligned with the training sample package.

[0109] The platform identifies and sorts the training sample packages by execution domain and trigger point sequence number within each training set, splices two training sample packages corresponding to adjacent trigger point sequence numbers into a training segment, and the training segment contains the current state vector, control action content, reward value, and next state vector. The action value estimation table is maintained, and the action value estimation table is indexed to an estimated value by the joint state vector and control action content. For each training segment, calculate the time difference residual, which is equal to the reward value plus the maximum action value estimation of the next state vector on all candidate control action contents, minus the action value estimation corresponding to the current state vector and control action content.

[0110] For each training segment, first calculate the time difference residual, and the calculation process is completed in a fixed order. First, take the reward value, then take the maximum action value estimation of the next state vector on the candidate control action content, and subtract the action value estimation corresponding to the current state vector and control action content after adding the two. The platform splits the time difference residual into sign and amplitude, and the sign only distinguishes between positive and negative, and the amplitude takes the absolute value. The platform uses the amplitude to find the hit interval in the segmented interval table, which is arranged in ascending order according to the amplitude boundary, and each interval is bound to an update step value and written into the access library with the access version identifier versioned. The platform obtains the update step value according to the hit interval, determines the increment direction according to the sign, and adds the update step value to the estimated value of the corresponding item in the action value estimation table according to the direction, while limiting the updated estimated value not to exceed the boundary with the estimated value boundary table. The update record is written into the strategy training log and associated with the action ticket identifier and observation package identifier. When the amplitude falls into a lower interval, the step value is smaller, thereby weakening the fluctuations introduced by the controlled release training sample package, and when the amplitude continuously falls into a higher interval, the step value is larger, thereby speeding up the response to stable reward trends.

[0111] S503. Strategy release package generation and strategy release package identifier solidification.

[0112] Online reasoning requires stable reference to strategies and version metadata. The lack of fixed identifiers in strategy release packages can cause release rollback and audit positioning difficulties. Strategy release packages can generate stable identifiers through standardized serialization and hash digest operations.

[0113] After completing the strategy training for each training set, a strategy release package is generated, which writes the entry path version identifier, the must-achieve observation list version identifier, and the two-axis risk matrix version identifier, and writes the training set summary, which contains the covered execution domain identifier range, the trigger point sequence number range, the observation site identifier summary, and the form interval table version number. The strategy release package is subjected to standardized serialization, which fixes the field order, the time format and time zone expression, the empty field writing method, the numerical value field precision expression, and the list field sorting rule. The standardized serialization result is subjected to a hash digest operation to obtain the strategy release package identifier, which is written into the release storage area together with the strategy release package and registered in the release ledger. After the strategy release package identifier is fixed, the same identifier can be used to locate the same release content for online reasoning and offline review, and the version drift risk is significantly reduced.

[0114] S504. Online reasoning entry verification and access determination.

[0115] When the online reasoning reads the latest aligned observation package, the missing segment entry and the late segment entry form a structural gap in the state vector structure, and the inconsistent caliber version identifier will cause the policy meaning to mismatch the input meaning. The reasoning entry can solidify version consistency and form compliance as access determination.

[0116] At the reasoning entry, the strategy release package and the latest aligned observation package are read simultaneously, and the caliber version identifier of the aligned observation package is verified to be equal to the caliber version identifier of the strategy release package, the must-achieve observation list version identifier referenced by the aligned observation package is verified to be equal to the must-achieve observation list version identifier of the strategy release package, and the two-axis risk matrix version identifier referenced by the aligned observation package is verified to be equal to the two-axis risk matrix version identifier of the strategy release package. The missing segment entry and the late segment entry of the aligned observation package are read, the form interval table is read from the caliber library and the form interval table version number is locked, the missing segment length level and the relative position level of each missing segment entry are compared with the form interval table item by item, and the reverse order segment level and the recalculation flag coverage proportion level of each late segment entry are compared with the form interval table item by item. The three version consistency conclusions and the form comparison conclusions are combined to form the access determination, and when the access determination passes, the strategy reasoning is entered and the control instruction is output. After the access determination is formed, the online reasoning input boundary and the offline training boundary remain consistent, and the probability of abnormal observation entering the strategy reasoning path is significantly reduced.

[0117] S505. Freeze strategy release package and repair queue registration.

[0118] The access decision failure represents version inconsistency or morphology out-of-bound, and directly using the policy release package to output control instructions can amplify observation exceptions into control exceptions. The platform can use frozen and bottom control instructions to isolate risks and structure repair information. When the access decision fails, the platform freezes the policy release package, the frozen record is written into the frozen account book, and the frozen record includes the policy release package identifier, the execution domain identifier, the trigger point serial number, and the frozen time identifier. Output the bottom control instruction, the bottom control instruction comes from the control face preset safety action template, the safety action template is bound with the execution domain identifier and written into the control instruction channel. Write the access failure reason into the repair queue, the repair queue entry includes the execution domain identifier, the observation site identifier, the missing segment entry abstract, the late segment entry abstract, the caliber offset level, the recalculation abstract positioning information, the caliber version identifier check result, the must-arrive observation list version identifier check result, the double-axis risk matrix version identifier check result, and the morphology interval table comparison hit item. After the frozen account book and the repair queue are formed, the online control output enters the safety boundary, and the repair personnel can quickly locate the root cause along the observation site identifier and the recalculation abstract positioning information and restore the release.

[0119] After the policy escort module is revised, the sample library screening stage fixes the caliber version identifier, the must-arrive observation list version identifier, and the double-axis risk matrix version identifier as the training set boundary, the policy training stage locks the control action content from the action account book and updates the action value estimation table with time series difference residual, the policy release stage fixes the policy release package identifier with standardization serialization and hash abstract operation, the online reasoning stage completes the access decision with version consistency and morphology interval table, the access failure triggers the frozen account book, the bottom control instruction, and the repair queue registration, and the training and reasoning behavior boundary forms a unified auditable link.

[0120] Specifically, the above is only a preferred embodiment of the present application and is not used to limit the present application.

[0121] In the description of the present specification, the description referring to the terms "one embodiment", "an example", "a specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are contained in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0122] The preferred embodiments of the application disclosed above are only to facilitate the elucidation of the application. The preferred embodiments do not describe all the details of the application and limit the application to the specific embodiments. Obviously, many modifications and variations can be made in light of the teachings above. The description is chosen and described in order to provide the best illustration of the application and its practical application to those skilled in the art and to enable those skilled in the art to best utilize the application. The application is limited only by the claims and their full scope and equivalents.

Claims

1. An artificial intelligence reinforcement learning service platform, characterized in that, include: Telemetry Acquisition Module: Acquires event time identifiers and core indicator values ​​during telemetry operations in the data center, generates access time identifiers, writes source identifiers and record sequence numbers, performs time semantic verification and indicator value validity verification, writes verification result identifiers, stores them in the raw stream storage area, forms the raw record set, and establishes an index; Ticket Arrangement Module: Based on the original record set, extract the status summary at fixed intervals and register the trigger point sequence number. When outputting control actions, generate action tickets. Write the action tickets to the action ledger. Bind the action tickets to the trigger point sequence number, backfill time window, caliber version identifier, execution domain identifier, and status summary value. The observation alignment module reads the action ticket, extracts the observation records from the original record set according to the backfill time window, sorts them according to the event time identifier to generate an alignment observation package, marks the missing segments and late segments in the alignment observation package, writes the missing segments into the completion request queue, retains the record sequence number of the late segments and writes the recalculation flag, and stores the alignment observation package in the observation storage area. Sample admission module: Based on the caliber version identifier, the module performs state construction and reward calculation on the aligned observation package, generates training sample package, performs playback consistency verification on the training sample package, outputs release judgment value, and determines whether the training sample package is written to the sample library or the isolation area, and writes the judgment reason identifier. The strategy protection module generates a strategy release package when the release judgment value is in the release state. The strategy release package is bound to the caliber version identifier and the alignment observation package summary. Online inference reads the strategy release package and the latest alignment observation package, checks the backfill integrity and caliber version consistency. If an inconsistency is triggered, the strategy release package is frozen and a backup control command is output and written to the repair queue.

2. The artificial intelligence reinforcement learning service platform according to claim 1, characterized in that: The telemetry acquisition module includes: reading event time identifiers and core indicator values ​​from the telemetry access terminal, generating access time identifiers, writing source identifiers and record sequence numbers, performing time semantic verification and indicator value validity verification and generating verification result identifiers, appending records to the raw stream storage area, establishing a main index according to the source identifier and record sequence number, establishing an auxiliary index according to the event time identifier, and forming a raw record set.

3. The artificial intelligence reinforcement learning service platform according to claim 2, characterized in that: The document arrangement module includes: extracting fixed-period windows based on the event time identifier using the auxiliary index of the original record set; merging window records into execution domain window sets according to the execution domain mapping table; filtering the set of valid records based on the verification result identifier; selecting the most recent valid record by the event time identifier to obtain the most recent valid value of the core indicator; sorting by the core indicator and taking the median valid value to obtain the median valid value of the core indicator; comparing the two and writing the core indicator change direction identifier; counting the abnormal count of the verification result and determining the missing placeholder count; and generating the status summary value.

4. The artificial intelligence reinforcement learning service platform according to claim 3, characterized in that: The ticket arrangement module also includes: calculating and sorting arrival delay values ​​according to source identifiers in the historical original record set; determining the upper limit of source delay based on the location of the largest break between adjacent differences; obtaining the upper limit of execution domain delay by taking the maximum value of the upper limit of source delay in the same execution domain according to the execution domain mapping table; determining the event time identifier of the start of the backfill time window and the event time identifier of the end of the backfill time window; counting late arrival records based on arrival delay values ​​exceeding the upper limit of source delay and event time identifiers being within the window range; generating trigger point sequence numbers and writing them into action tickets when outputting control actions; and writing action tickets into the action ledger.

5. The artificial intelligence reinforcement learning service platform according to claim 4, characterized in that: The observation alignment module includes: reading action tickets from the action ledger, obtaining the execution domain identifier, trigger point sequence number, backfill time window start event time identifier, and backfill time window end event time identifier; extracting observation records from the original record set based on the execution domain allowed set and verification result identifier; generating observation site identifiers according to the source identifier and indicator name; sorting by event time identifier and stabilizing the order of the records according to the record sequence number; and assembling the alignment observation package.

6. The artificial intelligence reinforcement learning service platform according to claim 5, characterized in that: The observation alignment module also includes: dividing the backfill time window with a fixed period length, constructing a trigger point placeholder string and identifying continuous missing intervals to form missing segment entries, writing the missing segment entries into the alignment observation package and writing them into the completion request queue; maintaining the prefix maximum record number on the event time sorting sequence, recording the record number back as a late record and merging it into a late segment entry, writing the late segment overwritten record into a recalculation flag, generating an observation package identifier and writing it into the observation storage area.

7. The artificial intelligence reinforcement learning service platform according to claim 6, characterized in that: The sample admission module includes: loading state construction expressions and reward calculation expressions from the caliber library based on the caliber version identifier of the action ticket and parsing them into an expression tree; calculating the state vector and reward value on the aligned observation package after completing the field reference node and operator node verification; performing normalization serialization and generating state summary recalculation value and reward summary recalculation value; and encapsulating the training sample package, which includes the action ticket identifier, observation package identifier, and caliber version identifier.

8. The artificial intelligence reinforcement learning service platform according to claim 7, characterized in that: The sample admission module also includes: the platform extracts observation records from the original record set based on the backfill time window and execution domain allowed set of the action ticket, reconstructs the replay aligned observation package, recalculates the replay status summary and replay reward summary and generates a recalculation summary, generates a completeness profile based on the must-achieve observation list and maps it to obtain the backfill link completeness index, generates a shadow difference feature set based on the previous stable caliber version identifier and calculates the caliber shadow difference risk value, maps it to the backfill confidence level and caliber offset level and generates a release judgment value through a dual-axis risk matrix, and writes the training sample package into the sample library or isolation area according to the release judgment value and writes the judgment reason identifier.

9. The artificial intelligence reinforcement learning service platform according to claim 8, characterized in that: The sample admission module includes: reading training sample packages from the sample library, filtering training sample packages with release judgment values ​​of release and controlled release, grouping them by caliber version identifier and verifying within each group that the version identifier of the must-reach observation list is consistent with the version identifier of the dual-axis risk matrix, and writing training sample packages with version conflicts into the isolation area; sorting training sample packages by execution domain identifier and trigger point sequence number, reading back the action ledger to obtain control action content, assembling training fragments and updating the action value estimation table, generating a policy release package and writing in the caliber version identifier, the version identifier of the must-reach observation list, the version identifier of the dual-axis risk matrix, and the training set summary.

10. The artificial intelligence reinforcement learning service platform according to claim 9, characterized in that: The strategy escort module also includes: performing standardized serialization on the strategy release package and generating a strategy release package identifier to write into the release ledger; the online inference entry reads the strategy release package and the latest aligned observation package, verifies the consistency of the caliber version identifier, the required observation list version identifier, and the dual-axis risk matrix version identifier; and outputs control instructions after verifying the morphological compliance of missing segment entries and late segment entries according to the morphological interval table. When the admission judgment fails, the strategy release package is frozen and the freeze ledger is registered. The minimum control instruction is output and the admission failure reason, observation site identifier, and recalculation summary location information are written into the repair queue.

Images