A visual model continuous release method and system based on high-risk scenario slice regression gating

Abstract

The application belongs to the technical field of computer vision model publishing, and discloses a visual model continuous publishing method and system based on a high-risk scene slice regression gate. The method compares the performance of a candidate publishing model and a baseline model in the network at the slice level by constructing a high-risk scene slice set, generates a slice regression result, and determines passing slices and blocking slices based on hard gating and soft gating. The passing slices are subjected to conditional volume increase and online routing, and local rollback or global rollback is performed according to online monitoring results, and the baseline model in the network is updated after stabilization. The scheme can reduce the publishing risk caused by performance degradation in key scenes, and improve the safety and fine control ability of model continuous publishing.

Description

Technical Field

[0001] This invention belongs to the field of computer vision model publishing technology, and discloses a method and system for continuous publishing of visual models based on high-risk scene slice regression gating. Background Technology

[0002] With the continuous application of computer vision models in scenarios such as gas station safety monitoring, pipeline network on-site identification, municipal video surveillance, security inspection, and drone inspection, the version iteration and release control after model deployment have gradually become crucial factors affecting system stability and recognition performance. In existing technologies, after training, visual models are typically replaced entirely after offline verification to meet preset metrics, or a gradual rollout using a gray-scale approach is adopted to mitigate the risks of deploying a new version entirely. However, video data in real-world business environments exhibit significant scene heterogeneity. Different devices, locations, time periods, and weather, lighting, and occlusion conditions result in substantial differences in image distribution and target presentation. This means that even if a model performs well on overall evaluation metrics, it may experience increased false alarms, higher false negatives, or decreased temporal stability in a few high-risk critical scenarios, thus impacting business security and operational efficiency.

[0003] For example, Chinese patent application publication number CN109725928A discloses a gray-scale release method, apparatus, device, and readable storage medium. This method establishes a gray-scale release mapping table by receiving dimensional feature data input by the user. Upon receiving a version update request, it selects the corresponding gray-scale resource package based on individual dimensional features and distributes the resource package to the user terminal. Simultaneously, this solution can calculate the percentage of abnormal accesses based on the number of abnormal accesses in business access compared to the total number of accesses, and adjust the current user traffic distribution accordingly, thereby improving the convenience and flexibility of gray-scale release. This solution has certain reference value in version updates and traffic distribution based on dimensional features.

[0004] However, existing technologies primarily control the canary release process of applications or resource packages, focusing on version distribution based on user or terminal dimensions. They lack specific mechanisms for identifying high-risk scenarios, constructing slices, verifying slice-level performance regression, and implementing slice-level blocking and rollback during continuous visual model releases. Especially in visual recognition applications, relying solely on overall metrics or general dimensions for canary releases makes it difficult to effectively guarantee that the performance of new and old models does not degrade in high-risk scenarios such as nighttime, backlighting, rain and fog, key equipment locations, and key target categories.

[0005] Therefore, how to establish a segmented regression verification and release gating mechanism for high-risk scenarios, and realize the release of data based on segment conditions, local blocking, and local rollback during continuous release, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the problems of existing technologies, this invention provides a method and system for continuous release of visual models based on high-risk scene slice regression gating. This solves the problem that existing visual model release processes rely solely on overall metrics for version replacement, making it difficult to detect and suppress model performance degradation in critical high-risk scenarios.

[0007] To achieve the above-mentioned technical objectives, the present invention provides the following technical solution.

[0008] In a first aspect, this invention discloses a method for continuous deployment of visual models based on regression gating of high-risk scenario slices, comprising: S1, acquiring candidate deployment models, a current baseline model, and verification samples with scene metadata and annotation results, and generating a set of high-risk scenario slices based on business risk level, historical false negative and false negative losses, and scene sample coverage; S2, calculating the slice performance indicators of the candidate deployment model and the current baseline model on each high-risk scenario slice, and generating slice regression results relative to the current baseline model; S3, performing deployment gating based on the risk weight, slice performance indicators, and slice regression results of each high-risk scenario slice, obtaining a set of passed slices and a set of blocked slices; S4, importing the online inference traffic corresponding to the set of passed slices into the candidate deployment model according to the slice matching rules, and keeping the online inference traffic corresponding to the set of blocked slices in the current baseline model; S5, collecting online monitoring results during continuous deployment, calculating the online regression results of each passed slice, and performing a rollback on the corresponding slice or candidate deployment model when the online regression result of any passed slice triggers a rollback condition; S6, updating the current baseline model when the candidate deployment model completes slice deployment without triggering a rollback condition.

[0009] Specifically, the process of generating a high-risk scene slice set based on business risk level, historical false negative / missed report losses, and scene sample coverage includes: extracting device type, location identifier, time period label, weather label, illumination label, occlusion level, image quality level, and target category label; generating candidate scene slices based on single-factor or multi-factor combinations; and calculating a slice risk score for each candidate scene slice.

[0010] in, The candidate scene slice number; For the first Slice risk score for each candidate scene slice; This is a normalized value for the business risk level. Normalized value for losses caused by historical false alarms and / or missed alarms; For scene sample coverage; The boundary difficulty coefficient; , , , Let be the weight coefficient, and satisfy... ; Filter to meet And sample size The candidate scene slices are used as a set of high-risk scene slices; among them... This is the risk screening threshold; For the first The sample size of each candidate scene slice; This is the minimum sample size threshold.

[0011] Specifically, the step of calculating the slice performance metrics of the candidate release model and the current network baseline model on each high-risk scenario slice, and generating slice regression results relative to the current network baseline model, includes: calculating the precision, recall, mean precision, false positive rate, false negative rate, adjacent frame prediction consistency rate, and latency of the candidate release model and the current network baseline model on each high-risk scenario slice; and constructing a direction-consistent slice regression score based on each slice performance metric. in, For the first Slice regression scores for high-risk scenario slices; The number of performance metrics; For performance indicator item numbers; For the first The weighting coefficients of the performance indicators, and satisfying ; For direction coefficients, when the first... When a performance indicator is considered to be better the larger the value, When the first When a performance indicator is considered to be better the smaller the value, ; For candidate release models in the first The first high-risk scenario slice Performance index values; For the current network baseline model in the first The first high-risk scenario slice The performance index values ​​are used to construct the slice regression results, which are composed of the slice regression scores and the differences between the various performance indices.

[0012] Specifically, the release gating based on the risk weights, performance metrics, and regression results of each high-risk scenario slice includes: performing a hard gating judgment on each high-risk scenario slice, where the false negative rate of the candidate release model on that slice is determined relative to the current network baseline model. Greater than the false negative threshold Or the increment of the false alarm rate relative to the existing baseline model. Greater than the false alarm threshold Or the increment of latency relative to the existing network baseline model Greater than the delay threshold or slice regression score Below the slice gate threshold At that time, the first High-risk scenario slices are included in the blocking slice set; a weighted release score is calculated for high-risk scenario slices that are not included in the blocking slice set. in, The scores are released as a weighted average. This is a set of high-risk scenario slices that have not been included in the blocking slice set; For the first Risk weights for each high-risk scenario slice, and satisfying ;when And when the global performance metrics meet the preset global release threshold, the collection will be... It is determined to be through a slice set; where, This is the weighted publishing threshold.

[0013] Specifically, the step of importing online inference traffic corresponding to the slice set into the candidate release model according to the slice matching rules includes: extracting scene metadata consistent with the definition of high-risk scene slices from the online inference request; determining the high-risk scene slice to which the online inference request belongs according to the slice matching rules; and determining the proportion of traffic released in the current release phase when the high-risk scene slice to which the online inference request belongs belongs to the slice set. If the stage limit is not reached, the online inference request is routed to the candidate release model; otherwise, it is routed to the existing baseline model. For the first The high-risk scenario slice in the first The volume ratio at each release stage; This is the release phase number.

[0014] Specifically, the process of collecting online monitoring results during continuous release, calculating the online regression results for each passed slice, and performing a rollback on the corresponding slice or candidate release model when the online regression result of any passed slice triggers the rollback condition includes: calculating the online precision, online recall, mean online accuracy, online false positive rate, online false negative rate, online adjacent frame prediction consistency rate, and online latency for each passed slice at each release stage; and calculating the online deviation score based on the performance indicators of the corresponding slice during the offline verification period. in, For the first The online deviation score is obtained through the slice; For the first The online deviation weighting coefficient of the item, and satisfying the following conditions. ; For candidate release models in the first The first through slice Online performance index values; For candidate release models in the first The first through slice Offline verification performance metrics; when If any key performance indicator triggers the online hard rollback threshold, the first step is revoked. Traffic is imported into candidate release models via slices; when the number of slices triggering rollback or the corresponding traffic percentage exceeds the overall rollback threshold, all online traffic imports to candidate release models are revoked; among which... This is the online deviation threshold.

[0015] Specifically, when the candidate release model completes the slicing volume and no rollback condition is triggered, updating the live network baseline model includes: registering the candidate release model as the new live network baseline model; solidifying the slicing performance indicators and risk weights of the new live network baseline model on each high-risk scenario slicing to form the regression benchmark for the next release cycle; and writing the samples corresponding to the blocking slicing set and the rollback slicing set into the sample pool to be optimized to trigger subsequent sample supplementation, data augmentation, fine-tuning training, or manual review processes.

[0016] Secondly, this invention also discloses a continuous visual model deployment system based on high-risk scene slice regression gating, used to implement the method described in the first aspect, comprising: a data access module, a slice construction module, a performance comparison module, a deployment gating module, a slice routing module, an online rollback module, and a baseline update module; the data access module is used to acquire candidate deployment models, live baseline models, validation samples, and online inference requests; the slice construction module is used to generate a set of high-risk scene slices based on the scene metadata of the validation samples; the performance comparison module is used to calculate the slice performance indicators of the candidate deployment model and the live baseline model on each high-risk scene slice, and generate slice regression results; the deployment... The gating module is used to perform release gating based on the risk weight, slice performance indicators, and slice regression results of each high-risk scenario slice, resulting in a set of passable slices and a set of blocked slices. The slice routing module is used to import the online inference traffic corresponding to the passable slice set into the candidate release model according to the slice matching rules, and to keep the online inference traffic corresponding to the blocked slice set in the current network baseline model. The online rollback module is used to collect online monitoring results during continuous release, calculate the online regression results of each passable slice, and perform rollback on the corresponding slice or candidate release model when the rollback condition is triggered. The baseline update module is used to update the current network baseline model when the candidate release model has completed slice release and the rollback condition has not been triggered.

[0017] Specifically, the release gating module includes a risk scoring unit, a slice regression unit, a hard gating unit, and a soft gating unit; the slice routing module includes a scene metadata parsing unit, a slice matching unit, and a phased rollout unit; and the online rollback module includes an online indicator statistics unit, a deviation assessment unit, a local rollback unit, and a global rollback unit.

[0018] Thirdly, the present invention also discloses a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the method described in the first aspect.

[0019] Compared with existing technologies, this invention first constructs a set of high-risk scenario slices based on business risk level, historical false negative and false negative losses, and scenario sample coverage. Then, using the existing network baseline model as a reference, it performs slice-level performance regression analysis on candidate release models. It determines releaseable slices by combining hard gating and weighted soft gating, thereby avoiding the new model from masking performance degradation in key scenarios such as nighttime, backlight, rain and fog, occlusion, key locations, or key target categories, even when the overall indicators meet the standards. At the same time, this invention does not uniformly release all traffic, but instead directs the traffic corresponding to the slice to the candidate release model according to the slice matching results, keeping the blocked slices in the existing network baseline model. Combined with online monitoring results, it achieves slice-level local or global rollback. Therefore, it can significantly improve the security, fine-grained control, and business adaptability of continuous visual model release, reduce the business risks caused by erroneous releases, and improve the stability and operability of model version iteration. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall structure of the visual model continuous publishing system of the present invention;

[0021] Figure 2 This is a flowchart illustrating the overall process of the visual model continuous publishing method of the present invention.

[0022] Figure 3 A schematic diagram illustrating the construction process of a high-risk scenario slice set;

[0023] Figure 4 A schematic diagram of the slice performance comparison, slice regression result generation, and two-layer release gating decision process;

[0024] Figure 5 This is a schematic diagram of the process of scaling up traffic based on scene metadata matching and online traffic routing for slice conditions.

[0025] Figure 6 A schematic diagram illustrating the online monitoring, online regression result calculation, and local and global rollback processes;

[0026] Figure 7A schematic diagram of the closed-loop process for blocking slices, reverting slices, corresponding sample backflow, and baseline model update;

[0027] Figure 8 This is a schematic diagram comparing the performance of high-risk scene slicing in the offline verification stage between the example and the comparative example. Detailed Implementation

[0028] The technical solutions in the embodiments of the present invention will be clearly and completely described below. It should be noted that the present invention is not intended to limit a specific visual model structure, but rather to provide a regression-gated continuous publishing mechanism for high-risk scene slices. Therefore, the candidate publishing model and the existing baseline model can be an object detection model, an instance segmentation model, a key point detection model, a behavior recognition model, or a combination thereof. To enable those skilled in the art to implement the present invention without creative effort, the following description uses the continuous publishing of an object detection model in a security / gas safety scenario as an example, but the present invention is not limited to this example.

[0029] I. System Composition and Deployment Environment

[0030] like Figure 1 As shown, one implementation system of the present invention includes a data access module, a slice construction module, a performance comparison module, a release gating module, a slice routing module, an online monitoring module, a rollback execution module, and a baseline update module.

[0031] The system comprises the following modules: a data access module for accessing candidate release models, existing baseline models, offline verification samples, online inference streams, and manual review results; a slice construction module for generating a set of high-risk scenario slices based on scenario metadata; a performance comparison module for calculating the performance metrics of the candidate release model and the existing baseline model on each high-risk scenario slice; a release gating module for making slice-level release decisions based on slice risk weights, performance regression results, and preset hard and soft thresholds; a slice routing module for implementing conditional releases according to slice matching rules during the online inference phase; an online monitoring module for continuously calculating the online false positive rate, false negative rate, timing stability, and inference latency for each slice; a rollback execution module for performing traffic switching when local or global rollback conditions are triggered; and a baseline update module for solidifying the candidate release model into a new existing baseline model after it has been running stably for a period of time, and synchronously updating the slice performance benchmarks required for the next round of releases.

[0032] The above modules can be deployed in a microservice architecture within a cloud-edge collaborative training and push platform. Both the candidate release model and the existing baseline model are encapsulated as independent inference services; slice matching and traffic routing logic are deployed in the release control service; online monitoring and rollback logic are deployed in the monitoring service; and offline verification, slice construction, and release determination are deployed in the model management service. All modules communicate with each other through a unified message bus and metadata service.

[0033] II. Model Description Related to This Invention

[0034] 2.1 Model Structure

[0035] Both the current baseline model and the candidate release model employ a single-stage object detection network. This network consists of three parts: a backbone feature extraction network, a feature fusion network, and a decoupled detection head. The backbone feature extraction network uses a convolutional backbone containing residual blocks and cross-stage local connectivity structures to extract semantic features of images at different scales. The feature fusion network uses a top-down and bottom-up pyramid feature fusion structure to handle both small and large object detection. The decoupled detection head outputs a category branch, a bounding box regression branch, and an object confidence branch, respectively.

[0036] Input image size is The output categories include eight targets: flames, smoke, not wearing a safety helmet, intrusion into boundaries, unauthorized hot work, and construction disturbance. The current baseline model and the candidate release model use the same network structure, differing only in training data and fine-tuning strategies.

[0037] 2.2 Training Methods

[0038] The baseline model for the current network is trained on the initial training set. The candidate release model is incrementally fine-tuned based on the weights of the baseline model, incorporating newly added backflow samples. Training employs an adaptive moment estimation optimizer with weight decay for parameter updates, with an initial learning rate of... The total number of training rounds was 120, with the first 10 rounds set as a learning rate warm-up phase. The batch size was 32. Data augmentation included random cropping, scaling, color perturbation, motion blur simulation, fog simulation, and random occlusion.

[0039] The focus of this invention is not on the model itself; therefore, the model training part can adopt methods known in the art.

[0040] 2.3 Dataset and Metadata Structure

[0041] The dataset is derived from videos of fixed cameras, perimeter cameras, mobile inspection terminals, and drone inspections at gas stations. A total of 240,000 images were extracted and labeled, including 192,000 images in the training set, 24,000 images in the validation set, and 24,000 images in the test set. Each image, in addition to bounding box annotations, also includes scene metadata, which at least includes: equipment type, location level, acquisition time period, weather type, lighting type, occlusion level, image quality level, target category, risk area identifier, and manual review label.

[0042] Equipment types include fixed bullet cameras, gimbal cameras, and drones; location levels include ordinary locations, important locations, and key locations; data collection periods include daytime, dusk, and nighttime; weather types include sunny, rainy, and foggy days; lighting types include normal light, backlight, and low illumination; occlusion levels range from 0 to 3; and image quality levels include clear, slightly blurred, and heavily blurred.

[0043] To avoid fragmentation of data slices, this embodiment adopts a slice construction strategy of "single-factor priority, two-factor expansion, and three-factor amplitude limitation." First, single-factor slices are generated from the metadata of each scenario. Second, single-factor slices with significantly increased risk and sufficient sample size are combined into two-factor slices. Third, only combined slices with significant business significance are expanded into three-factor slices. The highest order of the combined slices is preferably no more than three to balance risk representation ability and sample size stability.

[0044] III. Detailed Implementation Process of the Method of the Invention

[0045] like Figure 2 As shown, the method of this invention can be briefly summarized as follows: First, acquire candidate release models, existing baseline models, and validation samples; then, construct a set of high-risk scenario slices based on scenario metadata; next, calculate the performance indicators of the candidate release models and existing baseline models on each high-risk scenario slice, generate slice regression results, and execute release gating; for slices that pass the gating, implement online traffic ramping and routing according to slice conditions; for slices that fail the gating, continue to be processed by the existing baseline model; during continuous release, monitor the online operation results and determine whether to perform partial or global rollback based on the monitoring results; when the candidate release model completes continuous release without triggering rollback conditions, update it to a new existing baseline model, thereby forming a closed loop for continuous release of the visual model. The following details each step.

[0046] 3.1 Step S1: Construction of the high-risk scene slice set

[0047] Compared to existing technologies that only evaluate based on the overall sample or conduct canary releases based on simple user dimensions, this invention first constructs a slice set for high-risk scenarios, so that subsequent release decisions are no longer solely based on overall average metrics. For example... Figure 3 As shown, the construction of high-risk scene slices includes scene metadata extraction, candidate slice generation, risk score calculation, and high-risk slice screening.

[0048] First, the offline validation samples are standardized according to scene metadata. For continuous attributes, such as brightness, blur, and occlusion ratio, a bucketing approach is used for discretization. For discrete attributes, such as device type, weather type, and target category, enumerated values ​​are used directly. Then, candidate slice expressions are generated, for example:

[0049] at night;

[0050] Backlighting;

[0051] Drones ∧ Rain and Fog (the symbol "∧" represents a logical AND relationship, meaning that adjacent conditions must be true simultaneously, the same below);

[0052] Key points: Flames / smoke;

[0053] Low light intensity ∧ shading level ≥ 2;

[0054] Nighttime ∧Key locations ∧Flame / Smoke

[0055] For each candidate slice, calculate its risk score:

[0056]

[0057] in, For the first Risk score of each candidate slice; This is a normalized value for the business risk level. Normalized value for losses caused by historical misreporting and underreporting; For sample coverage; The boundary difficulty coefficient; , , , These are the weighting coefficients.

[0058] Preferred selection , , , Among them, the business risk level is pre-set by the business department. For example, target categories involving flames, smoke, and unauthorized hot work that may escalate the accident have a higher risk level than general behavioral targets. Historical false alarm and missed reporting losses are obtained by statistically analyzing past manual work orders and alarm handling records. Sample coverage reflects the representativeness of the scenario in the validation set. The lower the coverage, the scarcer the scenario is and the easier it is to be masked by the overall index. The boundary difficulty coefficient is jointly determined by the proportion of low-confidence samples, manually reviewed disputed samples, and samples with inconsistent predictions in adjacent frames.

[0059] when If the number of candidate slice samples is not less than 600, it will be included in the high-risk scenario slice set. If two slices have a high degree of overlap in sample members and the difference in risk scores is less than the preset merging threshold, then slice merging will be performed, retaining the slice with clearer semantics and stronger business meaning.

[0060] Through this step, the present invention can make explicit the scenarios that have the greatest impact on business security in advance, so that subsequent release decisions are no longer affected by the overall sample averaging effect.

[0061] 3.2 Step S2: Slice-level performance comparison and generation of slice regression results

[0062] Step S2 is used to calculate the slice performance metrics of the candidate release model and the existing network baseline model on each high-risk scenario slice, and to generate slice regression results relative to the existing network baseline model; such as Figure 4 As shown, after obtaining the set of high-risk scenario slices, the slice performance indicators of the candidate release model and the existing network baseline model are calculated respectively, and the slice regression results are generated.

[0063] For each high-risk scenario slice, the following metrics are calculated for both the candidate release model and the existing baseline model: precision, recall, mean average precision (mAP@0.5), false positive rate, false negative rate, prediction consistency rate between adjacent frames, and average inference latency. Since the optimization directions for each metric differ, this embodiment introduces direction consistency processing to generate slice regression scores:

[0064]

[0065] in, For the first Slice regression scores for high-risk slices; The number of indicator items; For the first The weighting coefficient of each indicator; This is the direction coefficient; a larger value for this index is generally better. When the smaller the value of this indicator, the better. ; For candidate release models in the first The first slice Item indicator value; For the current network baseline model in the first The first slice Item indicator value.

[0066] Recall and false negative rate are given high weights, and are preferably set as follows: recall 0.25, false negative rate 0.25, mean precision 0.20, precision 0.10, false positive rate 0.10, adjacent frame prediction consistency rate 0.05, and inference latency 0.05. This setting is because the present invention primarily targets high-risk business scenarios, where the business losses caused by missed detections are usually greater than those caused by false positives. Therefore, the importance of recall and false negative rate needs to be emphasized in the slice regression score.

[0067] In addition, to make slice gating more suitable for high-risk scenarios, it does not rely solely on the overall score. It also retains the differences between key indicators so that hard gating can be implemented in the next step.

[0068] 3.3 Step S3: Dual-layer gating judgment

[0069] Unlike existing releases that are based solely on overall metrics or simple grayscale ratios, such as Figure 4 As shown, this invention employs a two-layer mechanism of slice hard gating and weighted soft gating.

[0070] First, hard gating is implemented. For each high-risk slice, if any of the following conditions are met, the slice is directly determined to be a blocking slice:

[0071] Increment of the false negative rate of the candidate release model on this slice More than 0.8 percentage points;

[0072] Increment of false positive rate of candidate release model on this slice More than 1.2 percentage points;

[0073] The average inference latency increment of the candidate release model on this slice More than 8ms;

[0074] Slice regression score of candidate release model on this slice Below this slice threshold .

[0075] In this embodiment, the slice threshold Dynamically set according to risk score:

[0076]

[0077] in, For the first The minimum acceptable regression threshold for each slice; The basic threshold is set to 0 in this embodiment; The risk amplification factor is set to 0.06 in this embodiment; This represents the risk score for that slice. It is evident that slices with higher risk require more stringent regression requirements from the candidate release model.

[0078] For slices that did not trigger hard gating, weighted soft gating is then applied, and the weighted release score is calculated:

[0079]

[0080] in, The set of slices that were not blocked by hard gating; For the first Risk weights for each slice, and all The sum is 1; The overall weighted score is released.

[0081] when Furthermore, a candidate model can only proceed to the next step if its overall precision, recall, and latency on the complete validation set all reach the preset global thresholds. Otherwise, it will not be released.

[0082] 3.4 Step S4: Increase volume and route traffic according to slice conditions

[0083] Step S4 is used to direct the online inference traffic corresponding to the slice set into the candidate release model, and to keep the online inference traffic corresponding to the blocking slice set in the existing network baseline model. Figure 5 As shown, after extracting scenario metadata from online inference requests, slice matching is performed. Traffic belonging to the slice set is imported into the candidate release model in stages, while other traffic remains in the existing network baseline model.

[0084] During the online inference phase, each inference request includes scene metadata. For fixed camera locations, device registration information, location level, and installation area information can be directly read; for dynamic video streams, illumination level, blur, and occlusion level can be calculated in real time by the edge side; for slices related to target categories, candidate categories can be output by a pre-filter or a front-end lightweight model and then used for slice matching.

[0085] The matching strategy prioritizes "exact match first, longest expression first, and highest risk score first." Specifically, if an online inference request hits multiple slices simultaneously, the slice with more condition combination dimensions is prioritized; if the number of dimensions is the same, the slice with the higher risk score is prioritized.

[0086] For online traffic belonging to the slice set, it is not all switched to the candidate release model at once, but rather released in stages. The release ratio is 20% in stage 1, 50% in stage 2, 80% in stage 3, and 100% in stage 4. Each stage lasts for 24 hours, and online monitoring indicators need to be re-verified before entering the next stage. For traffic belonging to the blocking slice set, it is always handled by the existing network baseline model.

[0087] 3.5 Step S5: Online Regression Monitoring and Local / Global Backtracking

[0088] Existing rollback techniques mostly rely on global anomaly rate triggers, resulting in coarse granularity and slow response. This invention, however, de-emphasizes rollback granularity down to the high-risk slice level, such as... Figure 6 As shown, during continuous release, online monitoring is performed on the slices. If a local rollback condition is triggered, a local rollback is performed on the corresponding slice; if a global rollback condition is triggered, a global rollback is performed on the candidate release model.

[0089] The online monitoring module performs statistical analysis on each slice at each release stage, including online precision, online recall, mean online precision, online false positive rate, online false negative rate, online adjacent frame prediction consistency rate, and online average inference latency. For false positive / false negative results that rely on manual verification, a combination of manual sample review and high-confidence automatic labeling can be used for hysteresis statistics.

[0090] To compare the degree of deviation between the offline release judgment results and the actual online effect, this embodiment defines an online deviation score:

[0091]

[0092] in, For the first The online deviation score is obtained through the slice; This refers to the online deviation weighting coefficient. For the first The first one obtained through online phase statistics via slices Item indicator value; For the same slice in the offline verification phase Item indicator value.

[0093] when If the online false alarm rate of the slice exceeds the offline value by 1.0 percentage point for two consecutive monitoring windows, or the online false alarm rate exceeds the offline value by 1.5 percentage points for two consecutive monitoring windows, or the consistency rate of adjacent frames decreases by more than 3 percentage points, a partial rollback will be immediately performed on the slice, that is, the import of the slice traffic into the candidate release model will be cancelled, and the processing will be restored to the existing network baseline model.

[0094] When the number of slices triggering a partial rollback exceeds 40% of the total number of slices, or the corresponding traffic exceeds 35%, or a major business risk event occurs, a global rollback is executed, which cancels the online traffic import of the candidate release model on all slices.

[0095] Thus, this invention achieves closed-loop release control of "offline slice regression gating - online slice regression monitoring - local priority rollback - global rollback when necessary".

[0096] 3.6 Step S6: Baseline Update and Closed-Loop Feedback

[0097] like Figure 7 As shown, when a candidate release model completes phased scaling on all passes and does not trigger local or global rollback conditions, it is upgraded to the new live baseline model. At this point, its offline performance metrics and online monitoring metrics on each high-risk slice are retained as regression benchmarks for the next round of candidate release models. Simultaneously, samples corresponding to the blocked slice set and the rolled-back slice set are written into the optimization sample pool. Samples in the optimization sample pool are prioritized for the next round of data cleaning, data augmentation, manual review, and fine-tuning training to achieve a closed-loop continuous evolution of "slice problem discovery—release gating blocking problem—sample backflow repair problem—next round of re-validation".

[0098] IV. Examples

[0099] 4.1 Example 1: Continuous Release of Safety Identification Model for Gas Stations

[0100] 1. Experimental Setup

[0101] This embodiment selects gas station and perimeter monitoring services as the experimental scenario, and the existing network baseline model is denoted as... The candidate release model is denoted as Both are single-stage target detection models. The candidate release model is based on the existing baseline model, with 18,200 manually reviewed samples returned in the last 30 days added for fine-tuning.

[0102] The offline verification set consists of 24,000 images, of which 6 high-risk scene slices were obtained according to step S1, as detailed in Table 1.

[0103] Table 1 High-risk scene slices

[0104]

[0105] 2. Offline slice evaluation results

[0106] Table 2 shows the offline results of the candidate release model and the existing baseline model on high-risk scene slices.

[0107] Table 2 shows the offline slice comparison results between the candidate release model and the existing baseline model.

[0108]

[0109] like Figure 8 As shown, although the candidate release model outperforms the existing baseline model on the overall validation set, it exhibits significant regression in some high-risk scenario slices. Therefore, release determination needs to be based on the slice regression gating mechanism of this invention.

[0110] Table 2 shows the candidate release models. The mean precision on the overall validation set improved from 89.6% to 91.1%, and the overall recall improved from 88.7% to 90.0%. Based solely on these overall metrics, the model would be considered suitable for deployment. However, on key high-risk slices such as Slice-1, Slice-3, and Slice-4, the candidate deployment model showed significant regression. Specifically, the recall rate of Slice-1 decreased by 4.6 percentage points, while the false negative rate increased by 2.8 percentage points; the consistency rate of Slice-4 decreased by 3.2 percentage points, indicating a deterioration in the stability of predictions between adjacent frames. These results demonstrate that relying solely on overall metrics for continuous deployment carries significant risks.

[0111] 3. Two-layer gating results

[0112] According to the hard gating condition in step S3 of the present invention, Slice-1, Slice-3, and Slice-4 are identified as blocking slices, while Slice-2, Slice-5, and Slice-6 enter the pass-through slice set. After weighted soft gating calculation, the weighted release score is... It is greater than the threshold of 0.012, thus allowing conditional publishing to be implemented on the slice set.

[0113] At this point, the slice set covers approximately 82.7% of the online traffic, while the blocking slice set covers approximately 17.3% of the online traffic. In other words, this invention does not negate the entire value of the candidate release model due to the regression of a small number of key scenarios, but only blocks its release on high-risk degraded slices, thus achieving more granular and secure continuous release.

[0114] 4.2 Comparative Example 1: Direct Release Method Based on Overall Indicators

[0115] Comparative Example 1 uses a standard overall performance indicator release method: a full replacement is performed when the average precision of the candidate release model on the complete validation set improves by more than 1 percentage point, and the average latency increases by no more than 10%. According to this rule, the candidate release model... It will be directly uploaded to all traffic.

[0116] 4.3 Comparative Example 2: Publishing Method Based on Uniform Gray Scale

[0117] Comparative Example 2 uses a conventional, uniform canary deployment method: first, 20% of global traffic is imported into the candidate deployment model, and then the scale-up to 50%, 80%, and 100% is determined based on the global false alarm rate and global latency. This method does not distinguish between high-risk scenario slices, nor does it support local blocking or local rollback.

[0118] V. Online Operation Experiment

[0119] The system was run continuously for 14 days in a real business system, and comparisons were made between the following: the existing baseline model scheme, the Comparative Example 1 scheme, the Comparative Example 2 scheme, and the scheme of this invention embodiment. Online evaluation metrics included: overall false alarm rate, critical high-risk slice false alarm rate, manual review time, number of major alarm failures, number of partial rollbacks, and number of global rollbacks. The results are shown in Table 3.

[0120] Table 3. 14-day online operation results of different release schemes

[0121]

[0122] As shown in Table 3, although the overall capability of the candidate release model in Comparative Example 1 was improved, the performance degradation on high-risk scenario slices was masked by the overall metrics, resulting in a false negative rate of 3.3% for key high-risk slices after deployment, 11 major alarm failures, and 2 global rollbacks. Although Comparative Example 2 reduced some risks through unified canary releases, it still did not differentiate between different scenario slices, so the false negative rate on key high-risk slices still rose to 2.8%, and 1 global rollback was still triggered.

[0123] In comparison, the embodiments of this invention, through slice construction, slice-level regression gating, conditional scaling, and local rollback mechanisms, outperform the existing baseline model and the two comparative models in terms of overall false positive rate, false negative rate of key high-risk slices, and manual review time. Specifically, the overall false positive rate decreased from 4.2% to 3.9%, the false negative rate of key high-risk slices decreased from 2.1% to 1.8%, and the manual review time decreased from 6.4... It dropped to 5.7 Furthermore, no global rollback was triggered throughout the process; only a local rollback was performed once for a single slice. This fully demonstrates that the present invention can more fully release the gains of the candidate release model while ensuring the safety of the online deployment.

[0124] Further analysis of the local rollback slice revealed that the slice, Slice-6, experienced a 1.6 percentage point increase in online false alarm rate compared to the offline value during the second phase of deployment, exceeding the local rollback threshold. Therefore, the system automatically revoked the import of this slice's traffic into the candidate deployment model, while the remaining slices continued normal deployment. This demonstrates that the present invention possesses excellent anomaly isolation capabilities, preventing fluctuations in the deployment of a single slice from escalating into a global deployment incident.

[0125] VI. Secondary Release After Sample Return

[0126] After the execution of Example 1, the high-value samples corresponding to the blocking slices Slice-1, Slice-3, Slice-4, and the local rollback slice Slice-6 were fed back to the sample pool to be optimized, adding a total of 5600 samples, of which 4180 samples were manually reviewed and confirmed, and 1420 samples were scene enhancement samples. A second round of fine-tuning was performed on the candidate release model based on these samples to obtain a new candidate model. .

[0127] The method of this invention was re-executed on the same validation set. Results showed that the recall rate of Slice-1 increased from 82.6% to 88.9%, the mAP@0.5 of Slice-3 increased from 78.7% to 81.8%, and the consistency rate of Slice-4 increased from 88.1% to 91.7%. Hard gating was no longer triggered in any of these cases, and all six high-risk slices passed the gating process. New candidate model. In the same 14-day online validation, the overall false positive rate was 3.8%, the false negative rate for key high-risk slices was 1.6%, and no local or global rollback was triggered. These results demonstrate that this invention can not only block high-risk regressions before release but also drive continuous model evolution through closed-loop feedback, thereby gradually expanding the scope of safe release.

[0128] Based on the above embodiments and comparative examples, it can be seen that:

[0129] First, by constructing a set of high-risk scenario slices, the decision-making units continuously released by the model are moved down from the overall model to the high-risk slices. This allows the performance degradation in key scenarios to be identified even when the overall indicators improve, thus preventing high-risk regression from being masked by the overall average effect.

[0130] Second, the present invention uses a two-layer release determination mechanism of hard gating of slices and weighted soft gating, which can not only strongly block obvious degradation on key slices, but also take into account the overall benefits of candidate release models in other scenarios, thus avoiding unnecessary overall verification failures.

[0131] Third, by increasing the volume according to slice conditions rather than according to a globally uniform ratio, this invention enables candidate deployment models to prioritize the traffic of scenarios that have been proven safe and effective, while high-risk degraded slices are still handled by the existing network baseline model, thereby achieving more refined online control.

[0132] Fourth, this invention, through online slice-level monitoring and a local priority rollback mechanism, reduces the rollback granularity from the overall model level to the slice level, which not only shortens the anomaly response time but also significantly reduces the probability of triggering global rollback, thereby improving the stability of continuous system operation.

[0133] Fifth, this invention automatically feeds back the samples corresponding to the blocking slices and rollback slices to the sample pool to be optimized, forming a closed-loop continuous evolution link of "high-risk slice identification - gating release - online monitoring - partial rollback - sample return - next round of repair", which can continuously enhance the model's adaptability to complex business scenarios.

[0134] The foregoing description of embodiments of the present invention, through which those skilled in the art are able to implement or use the present invention, will be readily apparent to those skilled in the art. Various modifications to these embodiments will be readily apparent to those skilled in the art. The general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novelty disclosed herein.

Claims

1. A method for continuous deployment of visual models based on high-risk scene slice regression gating, characterized in that, include: S1. Obtain candidate release models, live baseline models, and verification samples with scene metadata and annotation results. Generate a set of high-risk scene slices based on business risk level, historical false negative and false negative loss, and scene sample coverage. S2. Calculate the slice performance index of the candidate release model and the existing network baseline model on each high-risk scenario slice, and generate slice regression results relative to the existing network baseline model. S3. Based on the risk weights, slice performance indicators, and slice regression results of each high-risk scenario slice, execute release gating to obtain the set of approved slices and the set of blocked slices. S4. Extract scene metadata that matches the definition of a high-risk scene slice from the online inference request; determine the high-risk scene slice to which the online inference request belongs according to the slice matching rules; When the high-risk scenario slice to which the online inference request belongs belongs to the pass slice set, and the release ratio of the slice in the current release phase has not reached the phase limit, the online inference request is imported into the candidate release model; when the high-risk scenario slice to which the online inference request belongs belongs to the blocking slice set, the online inference request is kept in the existing network baseline model; S5, during the continuous release process, online monitoring results are collected, online regression results of each pass slice are calculated, and when the online regression result of any pass slice triggers the rollback condition, rollback is performed on the corresponding slice or candidate release model; S6, when the candidate release model completes the slice release and does not trigger the rollback condition, the existing network baseline model is updated.

2. The method for continuous publishing of visual models as described in claim 1, characterized in that, The process involves generating a set of high-risk scene slices based on business risk level, historical false alarm / missed alarm losses, and scene sample coverage. This includes extracting device type, location identifier, time period label, weather label, illumination label, occlusion level, image quality level, and target category label. Candidate scene slices are generated based on single-factor or multi-factor combinations. A slice risk score is then calculated for each candidate scene slice. ; in, The candidate scene slice number; For the first Slice risk score for each candidate scene slice; This is a normalized value for the business risk level. Normalized value for losses caused by historical false alarms and / or missed alarms; For scene sample coverage; The boundary difficulty coefficient; , , , Let be the weight coefficient, and satisfy... ; Filter to meet And sample size The candidate scene slices are used as a set of high-risk scene slices; among them... This is the risk screening threshold; For the first The sample size of each candidate scene slice; This is the minimum sample size threshold.

3. The method for continuous publishing of visual models as described in claim 1, characterized in that, The step of calculating the slice performance metrics of the candidate release model and the current network baseline model on each high-risk scenario slice, and generating slice regression results relative to the current network baseline model, includes: calculating the precision, recall, mean precision, false positive rate, false negative rate, adjacent frame prediction consistency rate, and latency of the candidate release model and the current network baseline model on each high-risk scenario slice; and constructing a direction-consistent slice regression score based on each slice performance metric. in, For the first Slice regression scores for high-risk scenario slices; The number of performance metrics; For performance indicator item numbers; For the first The weighting coefficients of the performance indicators, and satisfying ; For direction coefficients, when the first... When a performance indicator is considered to be better the larger the value, When the first When a performance indicator is considered to be better the smaller the value, ; For candidate release models in the first The first high-risk scenario slice Performance index values; For the current network baseline model in the first The first high-risk scenario slice The performance index values ​​are used to construct the slice regression results, which are composed of the slice regression scores and the differences between the various performance indices.

4. The method for continuous publishing of visual models as described in claim 1, characterized in that, The release gating based on the risk weights, performance metrics, and regression results of each high-risk scenario slice includes: performing hard gating for each high-risk scenario slice; and determining the release model's false negative rate relative to the existing baseline model on that slice. Greater than the false negative threshold Or the increment of the false alarm rate relative to the existing baseline model. Greater than the false alarm threshold Or the increment of latency relative to the existing network baseline model Greater than the delay threshold or slice regression score Below the slice gate threshold At that time, the first High-risk scenario slices are included in the blocking slice set; a weighted release score is calculated for high-risk scenario slices that are not included in the blocking slice set. in, The scores are weighted and published accordingly. This is a set of high-risk scenario slices that have not been included in the blocking slice set; For the first Risk weights for each high-risk scenario slice, and satisfying ;when And when the global performance metrics meet the preset global release threshold, the collection will be... It is determined to be through a slice set; where, This is the weighted publishing threshold.

5. The method for continuous publishing of visual models as described in claim 1, characterized in that, The process of collecting online monitoring results during continuous release, calculating the online regression results for each passed slice, and performing a rollback on the corresponding slice or candidate release model when the online regression result of any passed slice triggers the rollback condition includes: calculating the online precision, online recall, online average precision, online false positive rate, online false negative rate, online adjacent frame prediction consistency rate, and online latency for each passed slice at each release stage; and calculating the online deviation score based on the performance indicators of the corresponding slice during the offline validation period. in, For the first The online deviation score is obtained through the slice; For the first The online deviation weighting coefficient of the item, and satisfying ; For candidate release models in the first The first through slice Online performance index values; For candidate release models in the first The first through slice Offline verification performance metric values; when If any key performance indicator triggers the online hard rollback threshold, the first step is revoked. Traffic is imported into candidate release models via slices; when the number of slices triggering rollback or the corresponding traffic percentage exceeds the overall rollback threshold, all online traffic imports to candidate release models are revoked; among which... This is the online deviation threshold.

6. The method for continuous publishing of visual models as described in claim 1, characterized in that, When the candidate release model completes the slicing and no rollback condition is triggered, the existing network baseline model is updated, including: registering the candidate release model as the new existing network baseline model; solidifying the slicing performance indicators and risk weights of the new existing network baseline model on each high-risk scenario slicing to form the regression benchmark for the next release cycle; and writing the samples corresponding to the blocking slicing set and the rollback slicing set into the sample pool to be optimized to trigger subsequent sample supplementation, data augmentation, fine-tuning training, or manual review processes.

7. A continuous publishing system for visual models based on high-risk scene slice regression gating, used to implement the method according to any one of claims 1 to 6, characterized in that, include: The module includes a data access module, a slice construction module, a performance comparison module, a release gating module, a slice routing module, an online rollback module, and a baseline update module. The data access module is used to acquire candidate release models, live baseline models, verification samples, and online inference requests; The slice construction module is used to generate a set of high-risk scene slices based on the scene metadata of the verification sample; The performance comparison module is used to calculate the slice performance indicators of the candidate release model and the existing network baseline model on each high-risk scenario slice, and generate slice regression results. The release gating module is used to perform release gating based on the risk weight, slice performance index and slice regression result of each high-risk scenario slice to obtain the set of passes and the set of blocks. The slice routing module is used to import the online inference traffic corresponding to the slice set into the candidate release model according to the slice matching rules, and to keep the online inference traffic corresponding to the blocked slice set in the existing network baseline model. The online rollback module is used to collect online monitoring results during continuous release, calculate the online regression results of each slice, and perform rollback on the corresponding slice or candidate release model when the rollback condition is triggered; the baseline update module is used to update the existing network baseline model when the candidate release model has completed the slice release and the rollback condition has not been triggered.

8. The visual model continuous publishing system as described in claim 7, characterized in that, The release gating module includes a risk scoring unit, a slice regression unit, a hard gating unit, and a soft gating unit; the slice routing module includes a scene metadata parsing unit, a slice matching unit, and a phased rollout unit; the online rollback module includes an online indicator statistics unit, a deviation assessment unit, a local rollback unit, and a global rollback unit.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

Images