A model release version isolation running method and system

Abstract

The application relates to the technical field of online reasoning services, and discloses a model release version isolated operation method and system, wherein the method comprises the following steps: acquiring feature processing link description files of a newly released version and a baseline reference version, comparing operator type identifiers and parameter content abstracts position by position, and generating an operator differential list; classifying unchanged operators into a shared operator pool, instantiating changed operators into version-specific operator instances, and generating a mixed execution path graph; calling each node operator instance to execute feature processing in topological order; performing interval verification at the output end of the changed operator, and falling back to a complete isolated link when an exception occurs; and transmitting a feature vector into a reasoning instance and outputting a result.

Description

Technical Field

[0001] This invention relates to the field of online inference service technology, specifically to a method and system for isolating and running model release versions. Background Technology

[0002] In online services that continuously iterate and release models, the system needs to run multiple released model versions simultaneously to support scenarios such as canary releases, version rollback retention, and multi-version comparisons.

[0003] The existing approach binds a complete feature processing chain to each release and instantiates all the processing operators for that release in an independent runtime environment.

[0004] However, the feature processing chains between adjacent releases often differ only locally, with the vast majority of operators being identical across versions. Existing solutions fail to identify the commonalities and differences at the operator level between versions, and each version independently holds a complete set of operator chain instances, causing system memory usage to expand linearly with the number of versions, resulting in significant redundant resource consumption. Summary of the Invention

[0005] This invention provides a method and system for isolating and running model release versions, which solves the technical problems in related technologies such as the inability to effectively isolate feature processing links when multiple versions of models are run concurrently, the repeated occupation of memory resources by operator instances, and abnormal inference results caused by data interference between versions.

[0006] This invention discloses a method for isolating and running model release versions, comprising the following steps: Obtain the feature processing link description file of the newly released version and the feature processing link description file of the baseline reference version. Extract the operator type identifier and parameter content summary of the operators at the same position in the two files and perform item-by-item comparison to generate an operator level difference list. Mark the operator at each position as an invariant operator or a changed operator. Based on the operator-level difference list, execution instances of invariant operators are assigned to the shared operator pool, and modified operators are instantiated as version-specific operator instances according to the current release version identifier. A hybrid execution path graph is generated with operator position as node and processing dependency relationship as directed edge. Invariant nodes store references to the corresponding instances in the shared operator pool, and modified nodes store references to the version-specific operator instances. Receive inference requests, parse the target release version identifier, load the corresponding hybrid execution path graph, and sequentially call the operator instances referenced by each node in the node topology order to perform feature processing on the original request data. Shared operator instances adopt a stateless concurrent execution mode, while version-specific operator instances are executed in an isolated context. In the hybrid execution path diagram, a checkpoint is inserted at the output end of each change operator node. The hash value of the output feature data is calculated and interval check is performed with the expected output range. When the output value exceeds the expected range, the execution is interrupted and the request is rolled back to the complete isolated link for reprocessing. The feature vectors output from the hybrid execution path graph are passed to the model inference instance corresponding to the current release version to perform forward computation and output the inference results.

[0007] Furthermore, the parameter content digest is the hash digest value calculated after serializing the operator configuration parameters; if and only if the operator type identifiers of two operators with the same position are consistent and the parameter content digests are the same, they are determined to be immutable operators; otherwise, they are determined to be change operators.

[0008] Furthermore, the feature processing link description file is expressed in the form of a directed acyclic graph, and the operator position is identified by the node number; when generating the operator-level difference list, the node sets of the two description files are combined, and the nodes that only appear in the newly released version are directly marked as changed operators, and the nodes that only appear in the baseline reference version are marked as removed operators and recorded in the operator-level difference list.

[0009] Furthermore, each operator instance in the shared operator pool is indexed using a combination of the operator type identifier and the parameter content summary as a unique key; when multiple release versions of invariant operators have the same unique key, only one instance is retained in the shared operator pool, and the invariant nodes in the mixed execution path graph of each version all point to the same instance.

[0010] Furthermore, the shared operator pool maintains a reference counter for each instance, recording the number of release versions currently referencing that instance; when a release version is taken offline, the reference counters corresponding to all invariant nodes in its hybrid execution path graph are decremented by one; when the reference counter reaches zero, the shared operator pool removes the instance from memory.

[0011] Furthermore, the hybrid execution path graph completes topological sorting during the generation phase and persistently stores the execution order sequence of nodes; when receiving an inference request, it directly loads the sorted node sequence and does not recalculate the topological order during runtime.

[0012] Furthermore, the expected output range is the numerical range obtained by statistically analyzing the output feature data of each change operator position during the offline verification phase before the version release, and is stored together with the link description file; the judgment criterion for interval verification is: if the value of each dimension of the output feature data falls within the expected range of the corresponding dimension, the verification passes; if the value of any dimension exceeds the expected range, a rollback is triggered.

[0013] Furthermore, the hash calculation and interval determination of the verification point are performed asynchronously and in a bypass manner, and the verification calculation and the data transmission of the downstream node are carried out in parallel; only when the verification result returns an exception will the execution of the subsequent node be interrupted and a rollback be triggered.

[0014] Furthermore, it also includes: recording the number of times the shared operator instance is reused and the number of times the version-specific operator instance is executed in this request, and writing the count data into the resource scheduling optimization log; the resource scheduling optimization log aggregates the count data according to two dimensions: release version identifier and operator unique key, and periodically generates access popularity distribution reports for shared operator instances; for shared operator instances with continuously zero access popularity, a reference count check is triggered based on the access popularity distribution report to identify reclaimable instances and release memory.

[0015] This invention discloses a model release version isolation operation system, comprising: The differential list generation module is used to obtain the feature processing link description files of the newly released version and the baseline reference version, extract the operator type identifier and parameter content summary of the operators at the same position in the two files and perform item-by-item comparison to generate an operator-level differential list, and mark the operator at each position as an invariant operator or a changed operator. The path graph generation module is used to classify invariant operator instances into the shared operator pool and instantiate changed operators into version-specific operator instances based on the operator-level difference list. It generates a hybrid execution path graph with operator positions as nodes and processing dependencies as directed edges. Invariant nodes store references to the corresponding instances in the shared operator pool, and changed nodes store references to version-specific operator instances. The feature processing module is used to receive inference requests, parse the target release version identifier, load the corresponding hybrid execution path graph, and sequentially call the operator instances referenced by each node in the order of node topology to perform feature processing on the original request data. Shared operator instances adopt a stateless concurrent execution mode, while version-specific operator instances are executed in an isolated context. The verification rollback module is used to insert verification points at the output of each change operator node in the hybrid execution path graph, calculate the hash value of the output feature data and perform interval verification with the expected output range. When the output value exceeds the expected range, the execution is interrupted and the request is rolled back to the complete isolated link for reprocessing. The inference output module is used to transmit the feature vectors output by the hybrid execution path graph to the model inference instance corresponding to the current release version for forward computation, output the inference results, and write the number of times the shared operator instance is reused and the number of times the version-specific operator instance is executed into the resource scheduling optimization log.

[0016] The beneficial effects of this invention are as follows: This invention identifies invariant and variable operators between versions through an operator-level difference list. Invariant operators are grouped into a shared operator pool and maintained with only one instance. This transforms memory usage from a linear correlation with the total number of versions to a correlation with the cumulative differences between versions, solving the technical problem of linear redundant expansion of operator instances with the number of versions and effectively suppressing resource redundancy during concurrent multi-version runtime. Simultaneously, a checkpoint is inserted at the output of the variable operator to perform interval verification. In case of anomalies, a complete isolation link rollback is triggered, ensuring the correctness of feature processing results and consistency of version isolation semantics for each release version while achieving operator reuse. Attached Figure Description

[0017] Figure 1 This is a flowchart of the model release version isolation operation method provided in the embodiments of the present invention. Detailed Implementation

[0018] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0019] At least one embodiment of the present invention discloses a method for isolating and running model release versions, see Figure 1 This includes the following steps: Step 1: Generate the operator level difference list.

[0020] Obtain the feature processing link description file of the newly released version and its baseline reference version. Extract the operator type identifier and parameter content summary for each operator at the same location in both description files, calculate the item-by-item comparison results for each operator, and generate an operator-level difference list. In the operator-level difference list, each operator is marked as either "invariant operator" or "changed operator".

[0021] It should be noted that the parameter content digest refers to the hash digest value calculated after serializing the operator configuration parameters. Two operators with the same position are determined to be immutable operators if and only if their operator type identifiers are the same and their parameter content digests are identical; otherwise, they are determined to be modified operators.

[0022] In this embodiment, to support nonlinear link structures (e.g., processing links with parallel feature branches), the description file structure is expressed as a directed acyclic graph, and operator positions are identified by node numbers. The generation process of the difference list involves taking the union of the node sets of the two description files, directly marking nodes that only appear in the newly released version as changed operators, and marking nodes that only appear in the baseline reference version as removed operators and recording them in the operator-level difference list for subsequent path graph trimming.

[0023] Step 2: Generate the hybrid execution path graph.

[0024] Based on the operator-level difference list, all operator execution instances marked as invariant operators are grouped into a cross-version shared operator pool, while all operators marked as change operators are instantiated into version-specific operator instances according to the current release version identifier. A hybrid execution path graph for the current release version is generated, using operator positions as nodes and processing dependencies as directed edges. In the hybrid execution path graph, invariant nodes store references to the corresponding instances in the shared operator pool, while change nodes store references to version-specific operator instances.

[0025] It should be noted that each operator instance in the shared operator pool is indexed using a combination of the operator type identifier and the parameter content summary as a unique key. When multiple release versions of an invariant operator have the same unique key, only one instance is retained in the shared operator pool, and the invariant nodes in the mixed execution path graph of each version all point to the same instance.

[0026] In this embodiment, to avoid premature release of shared instances due to cross-version reference counting errors, the shared operator pool maintains a reference counter for each instance, recording the number of release versions currently referencing that instance. When a release version is taken offline, the reference counters corresponding to all invariant nodes in its hybrid execution path graph are decremented by one; the shared operator pool only removes the instance from memory when the reference counter reaches zero.

[0027] Step 3: Feature processing based on the hybrid execution path graph.

[0028] Upon receiving an inference request, the system parses the target release version identifier carried in the request and loads the hybrid execution path graph corresponding to that version. It then sequentially calls the operator instances referenced by each node in the topological order of the hybrid execution path graph to perform feature processing on the original request data. Instances in the shared operator pool use a stateless concurrent execution mode to process data from requests from different versions; version-specific operator instances execute within the isolation context of that version, and data is not shared between isolation contexts.

[0029] It should be noted that the stateless concurrent execution mode means that when the shared operator instance processes each request data, it neither reads nor writes any internal mutable state across requests. All inputs come from the output of the upstream node of the current node, and all outputs are only passed to the downstream node of the current node, thereby ensuring the thread safety of concurrent calls.

[0030] In this embodiment, to reduce the overhead of redundant calculations in topology sorting, the hybrid execution path graph completes the topology sorting during the generation phase and persistently stores the execution order sequence of nodes. When a reasoning request is received, the sorted node sequence is directly loaded, eliminating the need to recalculate the topology order at runtime.

[0031] Step 4: Change operator output verification and rollback.

[0032] A checkpoint is inserted at the output of each node corresponding to a change operator in the hybrid execution path graph. The hash value of the node's output feature data is calculated and compared with the expected output range for that position recorded in the current release's link description file. If the output value exceeds the expected range, the execution of the current hybrid execution path graph is interrupted, and the request is rolled back to the complete isolated link of the current release for reprocessing.

[0033] It should be noted that the expected output range refers to the numerical range obtained by statistically analyzing the output feature data of each changed operator position in the link description file of the current release version during the offline verification phase before the version release, and is stored along with the link description file. The criterion for range verification is: if the value of each dimension of the output feature data falls within the expected range of the corresponding dimension, the verification passes; if the value of any dimension exceeds the expected range, a rollback is triggered.

[0034] In this embodiment, to reduce the impact of verification points on the latency of normal request processing, the hash calculation and interval determination of the verification points are performed asynchronously and in a bypass manner, that is, the verification calculation and the data transmission of downstream nodes are performed in parallel. The execution of subsequent nodes is interrupted and a rollback is triggered only when the verification result returns an error; if the verification result is normal, the downstream node has already completed data reception during the verification calculation and does not need to wait for the verification to be completed before continuing execution.

[0035] Step 5: Model inference and resource scheduling log recording.

[0036] The feature vectors output from the hybrid execution path graph are transmitted to the model inference instance corresponding to the current release version, forward computation is performed, the inference result is obtained and output. Simultaneously, the number of times the shared operator instance is reused and the number of times the version-specific operator instance is executed in this request are recorded, and the count data is written to the resource scheduling optimization log.

[0037] In this embodiment, to support dynamic adjustments to the shared operator pool, the resource scheduling optimization log aggregates the counting data according to two dimensions: release version identifier and operator unique key, and periodically generates access popularity distribution reports for shared operator instances. For shared operator instances with consistently zero access popularity, the system can trigger reference count checks in advance based on the access popularity distribution report to identify reclaimable instances and release memory.

[0038] On the other hand, the present invention also proposes a model release version isolation operation system, including: The differential list generation module is used to obtain the feature processing link description files of the newly released version and the baseline reference version, extract the operator type identifier and parameter content summary of the operators at the same position in the two files and perform item-by-item comparison to generate an operator-level differential list, and mark the operator at each position as an invariant operator or a changed operator. The path graph generation module is used to classify invariant operator instances into the shared operator pool based on the operator level difference list, instantiate changed operators into version-specific operator instances, and generate a hybrid execution path graph with operator positions as nodes and processing dependencies as directed edges. Invariant nodes store references to the corresponding instances in the shared operator pool, and changed nodes store references to version-specific operator instances. The feature processing module is used to receive inference requests, parse the target release version identifier, load the corresponding hybrid execution path graph, and sequentially call the operator instances referenced by each node in the order of node topology to perform feature processing on the original request data. Shared operator instances adopt a stateless concurrent execution mode, while version-specific operator instances are executed in an isolated context. The verification rollback module is used to insert verification points at the output of each change operator node in the hybrid execution path graph, calculate the hash value of the output feature data and perform interval verification with the expected output range. When the output value exceeds the expected range, the execution is interrupted and the request is rolled back to the complete isolated link for reprocessing. The inference output module is used to transmit the feature vectors output by the hybrid execution path graph to the model inference instance corresponding to the current release version for forward computation, output the inference results, and write the number of times the shared operator instance is reused and the number of times the version-specific operator instance is executed into the resource scheduling optimization log.

[0039] By way of example, the system proposed in this invention may also include various features and combinations thereof in the method embodiments, which will not be elaborated here.

[0040] This invention generates an operator-level difference list during version release, grouping identical operators across versions into a shared operator pool and maintaining only one instance. This transforms memory usage from being linearly correlated with the total number of versions to being correlated with the cumulative differences between versions. As the number of versions continues to grow while the differences between versions are concentrated in a small number of operators, the size of the shared operator pool tends to converge, thus suppressing the problem of resource consumption expanding linearly with the number of versions. Changing the interval verification at the operator output specifically addresses the potential error propagation risk introduced by shared reuse: since the upstream and downstream connections of shared operators may have slight differences in different versions, the verification point intercepts abnormal output at each difference boundary and triggers a complete isolation link rollback, ensuring that the feature processing results of each released version are consistent with the complete isolation scheme. This maintains the correct semantics of version isolation while achieving resource reuse.

[0041] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A method for isolating and running model release versions, characterized in that, Includes the following steps: Obtain the feature processing link description file of the newly released version and the feature processing link description file of the baseline reference version. Extract the operator type identifier and parameter content summary of the operators at the same position in the two files and perform item-by-item comparison to generate an operator level difference list. Mark the operator at each position as an invariant operator or a changed operator. Based on the operator-level difference list, execution instances of invariant operators are assigned to the shared operator pool, and modified operators are instantiated as version-specific operator instances according to the current release version identifier. A hybrid execution path graph is generated with operator position as node and processing dependency relationship as directed edge. Invariant nodes store references to the corresponding instances in the shared operator pool, and modified nodes store references to the version-specific operator instances. Receive inference requests, parse the target release version identifier, load the corresponding hybrid execution path graph, and sequentially call the operator instances referenced by each node in the node topology order to perform feature processing on the original request data. Shared operator instances adopt a stateless concurrent execution mode, while version-specific operator instances are executed in an isolated context. In the hybrid execution path diagram, a checkpoint is inserted at the output end of each change operator node. The hash value of the output feature data is calculated and interval check is performed with the expected output range. When the output value exceeds the expected range, the execution is interrupted and the request is rolled back to the complete isolated link for reprocessing. The feature vectors output from the hybrid execution path graph are passed to the model inference instance corresponding to the current release version to perform forward computation and output the inference results.

2. The model release version isolation and operation method according to claim 1, wherein, The parameter content digest is the hash digest value calculated after serializing the operator configuration parameters; if and only if the operator type identifiers of two operators with the same position are consistent and the parameter content digests are the same, they are determined to be immutable operators; otherwise, they are determined to be change operators.

3. The model release version isolation operation method according to claim 1, wherein, The feature processing link description file is expressed in the form of a directed acyclic graph, and the operator position is identified by the node number. When generating the operator-level difference list, the node sets of the two description files are combined. Nodes that only appear in the newly released version are directly marked as changed operators, and nodes that only appear in the baseline reference version are marked as removed operators and recorded in the operator-level difference list.

4. The model release version isolation operation method according to claim 1, wherein, Each operator instance in the shared operator pool is indexed by a unique key consisting of a combination of the operator type identifier and the parameter content summary. When multiple release versions of an invariant operator have the same unique key, only one instance is retained in the shared operator pool, and the invariant nodes in the mixed execution path graph of each version all point to the same instance.

5. The model release version isolation operation method according to claim 4, wherein, The shared operator pool maintains a reference counter for each instance, recording the number of release versions currently referencing that instance. When a release version is taken offline, the reference counters corresponding to all invariant nodes in its hybrid execution path graph are decremented by one. When the reference counter reaches zero, the shared operator pool removes the instance from memory.

6. The model release version isolation operation method according to claim 1, wherein, The hybrid execution path graph completes topological sorting during the generation phase and persistently stores the execution order sequence of nodes; when receiving an inference request, it directly loads the sorted node sequence and does not recalculate the topological order during runtime.

7. The model release version isolation operation method according to claim 1, wherein, The expected output range is the numerical range obtained by statistically analyzing the output feature data of each changed operator position during the offline verification phase before the version release, and it is stored together with the link description file; the judgment criterion for interval verification is: if the value of each dimension of the output feature data falls within the expected range of the corresponding dimension, the verification passes; if the value of any dimension exceeds the expected range, a rollback is triggered.

8. The model release version isolation operation method according to claim 7, wherein, The hash calculation and interval determination of the verification point are performed asynchronously and in a bypass manner. The verification calculation and the data transmission of the downstream node are carried out in parallel. The execution of the subsequent node is interrupted and a rollback is triggered only when the verification result returns an exception.

9. The model release version isolation operation method according to claim 1, wherein, Also includes: Record the number of times the shared operator instance is reused and the number of times the version-specific operator instance is executed in this request, and write the count data into the resource scheduling optimization log; Resource scheduling optimization logs aggregate count data according to two dimensions: release version identifier and operator unique key, and periodically generate access popularity distribution reports for shared operator instances; for shared operator instances with consistently zero access popularity, reference count checks are triggered based on the access popularity distribution reports to identify reclaimable instances and release memory.

10. A model release version isolation system, comprising: The differential list generation module is used to obtain the feature processing link description files of the newly released version and the baseline reference version, extract the operator type identifier and parameter content summary of the operators at the same position in the two files and perform item-by-item comparison to generate an operator-level differential list, and mark the operator at each position as an invariant operator or a changed operator. The path graph generation module is used to classify invariant operator instances into the shared operator pool and instantiate changed operators into version-specific operator instances based on the operator-level difference list. It generates a hybrid execution path graph with operator positions as nodes and processing dependencies as directed edges. Invariant nodes store references to the corresponding instances in the shared operator pool, and changed nodes store references to version-specific operator instances. The feature processing module is used to receive inference requests, parse the target release version identifier, load the corresponding hybrid execution path graph, and sequentially call the operator instances referenced by each node in the order of node topology to perform feature processing on the original request data. Shared operator instances adopt a stateless concurrent execution mode, while version-specific operator instances are executed in an isolated context. The verification rollback module is used to insert verification points at the output of each change operator node in the hybrid execution path graph, calculate the hash value of the output feature data and perform interval verification with the expected output range. When the output value exceeds the expected range, the execution is interrupted and the request is rolled back to the complete isolated link for reprocessing. The inference output module is used to transmit the feature vectors output by the hybrid execution path graph to the model inference instance corresponding to the current release version for forward computation, output the inference results, and write the number of times the shared operator instance is reused and the number of times the version-specific operator instance is executed into the resource scheduling optimization log.

Images