Training data construction and model update methods, equipment, media and products

By constructing a task graph to record node operation evidence, tracing back and replacing the output of the perception operator, identifying the target perception operator and constructing training samples, the problem of training data matching and verification when the performance of the perception model deteriorates is solved, and the efficiency and effectiveness of model updates are improved.

CN123115804APending Publication Date: 2026-11-17SHANGHAI JINGZHI IND CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611620169.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In industrial vision tasks, when the performance of a perception model degrades, it becomes difficult to determine which perception operator needs to be updated, and it is also difficult to construct matching training samples and verify the effectiveness of model updates.

Method used

By constructing a task graph to record node operation evidence, backtracking candidate perception operators, replacing their outputs and replaying downstream business logic, determining the target perception operator based on the business results, constructing training samples according to their input patterns for incremental training, and finally validating the candidate model in the task graph.

Benefits of technology

It improves the matching degree between training data and target perception tasks, reduces irrelevant data processing and model training, and reduces the impact of model updates on historical categories or operating conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115804A_ABST
    Figure CN123115804A_ABST
Patent Text Reader

Abstract

This application discloses a method, device, medium, and product for training data construction and model updating. The method includes: constructing a task graph based on industrial vision task configuration and recording node operation evidence; responding to a model update trigger event, tracing candidate perception operators backward along dependency edges from abnormal business logic nodes; keeping the model version of the candidate perception operator unchanged, replacing its output with a reference output and replaying the affected downstream business logic, and determining the target perception operator based on the comparison between the replay result and the reference business result; constructing training samples using production data associated with the operation evidence according to the input pattern of the target perception operator and incrementally training to obtain a candidate model; placing the candidate model in the task graph for historical business replay, and deploying the candidate model when the business indicators and historical performance indicators meet the verification threshold, thereby enabling targeted construction of training samples and verification of the impact of model updates on business results and historical performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method, device, medium and product for constructing training data and updating models based on industrial vision. Background Technology

[0002] Industrial vision systems are widely used in scenarios such as defect detection, personnel safety monitoring, operation procedure recognition, equipment status recognition, and production counting. Industrial vision tasks typically include perception operators such as object detection, image segmentation, object tracking, and action recognition, and generate business results through business logic such as region judgment, temporal judgment, and state transition.

[0003] As product appearance, lighting conditions, camera position, equipment operating conditions, or personnel operation methods change, the performance of some perception models may degrade. Because there are dependencies between each perception operator and business logic, users typically only detect anomalies in the final business results, making it difficult to determine which perception operators need updating and the production data related to the anomalies. Supplementing data and training multiple models uniformly can easily increase the resources required for data processing and model training.

[0004] Furthermore, different perception operators require training data with different structures, and the trained models also need to be validated in complete industrial vision tasks to assess their operational effectiveness and historical performance. Therefore, how to determine the perception operators that need to be updated based on abnormal business results, construct matching training samples for them, and verify the effectiveness of model updates are technical problems that need to be solved. Summary of the Invention

[0005] One objective of this application is to provide a method, device, medium, and product for constructing training data and updating models based on industrial vision, at least to address the technical problems of difficulty in determining which perception operators need to be updated, difficulty in constructing matching training samples for the perception operators, and difficulty in verifying the model update effect by combining the complete business process and historical performance when the business results of industrial vision tasks are abnormal.

[0006] To achieve the above objectives, some embodiments of this application provide the following aspects: In a first aspect, some embodiments of this application provide a method for constructing training data and updating a model, the method comprising: A task graph is constructed based on the industrial vision task configuration. The task graph includes perception operator nodes, business logic nodes, and dependency edges between nodes. Evidence of node operation is recorded when the task graph is run. In response to a model update trigger event, candidate perception operators are traced back along the dependency edges from the corresponding abnormal business logic node. While keeping the model version of the candidate sensing operator unchanged, replace the output of the candidate sensing operator with the reference output and replay the downstream business logic affected by the output replacement. Based on the comparison between the replayed business results and the reference business results, the target perception operator is determined; According to the input pattern of the target perception operator, training samples are constructed using the production data associated with the operational evidence, and the model of the target perception operator is incrementally trained using the training samples to obtain a candidate model. In the task graph, the model used by the target perception operator node is replaced with the candidate model, business playback is performed on historical production data, and when the business indicators and historical performance indicators meet the corresponding verification thresholds, the candidate model is deployed as the running model of the target perception operator.

[0007] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0008] Thirdly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the method described above.

[0009] Fourthly, some embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0010] Compared with related technologies, the solution provided in this application, by constructing a task graph representing the execution dependencies of industrial vision tasks and recording node operation evidence, enables the reverse tracing of candidate perception operators from abnormal business logic nodes. By replacing the perception output and replaying the affected downstream business logic while keeping the model version unchanged, the target perception operator is determined based on the changes in business results, linking the selection of training objects with the correction effect of business errors. Based on this, constructing training samples according to the input pattern of the target perception operator helps improve the matching degree between training data and the target perception task, reducing irrelevant data processing and unnecessary model training. By placing the candidate model in the original task graph for historical business replay and combining business indicators and historical performance to maintain indicator control deployment, it is beneficial to verify the impact of model updates on the complete business process and reduce the risk of performance degradation in historical categories or historical operating conditions after the update. Attached Figure Description

[0011] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0012] Figure 1 A flowchart illustrating a training data construction and model update method provided in some embodiments of this application; Figure 2 A flowchart illustrating another method for constructing training data and updating a model, provided in some embodiments of this application; Figure 3 A flowchart for candidate sensing operator tracing and target sensing operator determination is provided for some embodiments of this application; Figure 4 A flowchart of candidate model service playback and verification provided for some embodiments of this application; Figure 5 This is an exemplary structural diagram of an electronic device provided for some embodiments of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] Figure 1 and Figure 2 This is an exemplary flowchart illustrating a training data construction and model update method provided in some embodiments of this application. The method can be executed by a program deployed on a server, a local factory computing device, or an edge computing device.

[0015] S101 constructs a task graph based on the industrial vision task configuration. The task graph includes perception operator nodes, business logic nodes, and dependency edges between nodes, and records node running evidence when running the task graph.

[0016] Specifically, the industrial vision task configuration describes the processing nodes included in the industrial vision task, the inputs and outputs of each processing node, and the conditions for generating the business results. The system parses the industrial vision task configuration, divides the processing nodes into perception operator nodes and business logic nodes according to the functions performed by each processing node, and establishes dependency edges based on the data usage relationships between processing nodes to form a task graph.

[0017] Perception operator nodes are nodes that run visual models or visual processing components to process images, videos, or upstream perception data and output perception results. These perception results can be bounding boxes, segmented regions, target trajectories, pose keypoints, text recognition results, action categories, or visual states. For example, a tool segmentation node is used to determine the image region corresponding to a tool, and a target tracking node is used to determine the positional relationship of the same target in consecutive video frames.

[0018] A business logic node is a node that uses the output of one or more upstream nodes to generate intermediate business states or final business results according to the processing rules configured for the industrial vision task. These processing rules can include region relationship judgment, target binding, contact relationship judgment, duration calculation, counting, state transition, or condition combination. For example, an operation stage node can determine whether an operation step is completed based on the tool, workpiece, and action result; an alarm node can generate an alarm state based on the upstream judgment result.

[0019] When the output of one node is used by another node for subsequent processing, a dependency edge is established between the two nodes, pointing from the output node to the node using the output. The dependency edge represents the direction of transmission of the perception result or business state in the task graph. Through these dependency edges, the actual perception operator nodes and business logic nodes traversed when forming a certain business result can be determined.

[0020] When running the task graph, on-site images or videos are processed by perception operator nodes to generate perception results. These perception results are then passed along dependency edges to business logic nodes. The business logic nodes process the received results and generate business results. During this processing, node operation evidence is generated for each node. Node operation evidence records the inputs and outputs of the corresponding node during this operation, as well as its correlation with the operation evidence of upstream and downstream nodes. It also establishes a connection with the images, videos, or other production data that generated the inputs, enabling the tracing of nodes and production data involved in forming the business results based on the business results.

[0021] S102 responds to the model update trigger event by tracing back the candidate perception operator from the corresponding abnormal business logic node along the dependency edge.

[0022] Specifically, the model update trigger event is used to indicate that the business results generated by the task graph need to enter attribution processing. The model update trigger event can be generated when the business result is confirmed to have an error, the business result is corrected, or the business result is inconsistent with the trusted reference result. The model update trigger event records its corresponding industrial vision task, business result, reference business result, and event occurrence time, and can also record the target object or production location associated with the business result.

[0023] An abnormal business logic node is a business logic node that outputs abnormal business results in the task graph, or whose output is used to form abnormal business results and is indicated as abnormal by the model update trigger event. Different industrial vision tasks can correspond to different types of abnormal business logic nodes. For example, when alarm results show false alarms or missed alarms, the alarm node that generates the alarm result can be used as an abnormal business logic node; when operation steps are judged incorrectly or timed out, the node that generates the operation stage or step status can be used as an abnormal business logic node; when equipment status is judged incorrectly, the status fusion node that generates the equipment status can be used as an abnormal business logic node; when production counting is incorrect, the counting node that generates the counting result can be used as an abnormal business logic node.

[0024] In one embodiment, the model update trigger event directly records the identifier of the business logic node that generates the abnormal business result, and the system determines the abnormal business logic node based on the business logic node identifier. In another embodiment, the system determines the business logic node that generates the business result in the task graph based on the task identifier, business result type, and event occurrence time corresponding to the model update trigger event, and identifies the business logic node as the abnormal business logic node.

[0025] After identifying the abnormal business logic node, the upstream nodes that provide the sensing results or intermediate business states to it are searched sequentially in the reverse direction of the dependency edges, starting from the abnormal business logic node. When a business logic node is encountered, the upstream tracing continues along the dependency edge corresponding to its input; when a sensing operator node is encountered, the operational evidence of the sensing operator node is obtained, and it is determined whether its output participates in the formation of this abnormal business result. Sensing operators whose output influence needs further verification are identified as candidate sensing operators. After obtaining the operational evidence, if the sensing operator node still has upstream nodes that participate in the formation of this business result, the search continues in reverse along the corresponding dependency edge.

[0026] Therefore, abnormal business logic nodes are used to determine the starting point of reverse tracing, dependency edges are used to determine the formation path of business results, and candidate perception operators are the perception operators in the formation path that need to be further verified through output replacement and downstream business logic replay.

[0027] S103 While keeping the model version of the candidate sensing operator unchanged, replace the output of the candidate sensing operator with the reference output and replay the downstream business logic affected by the output replacement.

[0028] Specifically, the reference output is data that represents the correct or reliable output of the candidate sensing operator under the corresponding production data. The reference output can be obtained based on confirmed sensing results, sensing results directly expressible from independent production signals, historical correct results, or model outputs that meet consistency requirements. The reference output corresponds to the output object and output format of the candidate sensing operator so that it can be input into downstream business logic.

[0029] When performing output replacement, the model version used by the candidate sensing operator is not changed. Instead, a reference output is injected into the output position of the candidate sensing operator, replacing the original output recorded in the node's running evidence. Thus, output replacement is used to verify the impact of changes in sensing results on business results without introducing interference from model version changes to the verification results.

[0030] After the output replacement is completed, the reference output is passed to the downstream nodes using the output according to the dependencies in the task graph. The downstream processing affected by the output replacement, from the candidate sensing operator to the abnormal business logic node, is then re-executed according to these dependencies to obtain the replayed business results. Replaying the downstream business logic affected by the output replacement means propagating the replaced output along the business result path in the task graph and re-executing the downstream nodes in that path that need to be updated due to the output change. Business paths not affected by the output replacement do not need to be re-executed.

[0031] When there are other perception operator nodes between the candidate perception operator and the abnormal business logic node, the replaced output is passed to the other perception operator nodes along the dependency edge. While keeping the model version of the other perception operator nodes unchanged, the corresponding perception processing is re-executed, and the obtained output is passed to the subsequent affected nodes along the dependency edge until the abnormal business logic node re-forms the business result.

[0032] S104 determines the target perception operator based on the comparison between the replayed service result and the reference service result.

[0033] Specifically, the reference operational result is used to represent the operational outcome that should be achieved for this industrial vision task under the corresponding production data. The reference operational result can be a confirmed alarm result, operation stage, equipment status, or count result, or it can be an operational status provided by a reliable production signal.

[0034] The replayed service result is compared with the reference service result to determine whether the abnormal service result is corrected after the candidate perception operator's output is replaced by the reference output. When the comparison result shows that the replayed service result meets the service requirements corresponding to the reference service result, it is determined that the original output of the candidate perception operator has an impact on the abnormal service result, and the candidate perception operator is determined as the target perception operator. Therefore, the relationship between the perception operator output and the abnormal service result is verified before model training, avoiding the direct determination of the training object based solely on the final service anomaly.

[0035] S105 According to the input mode of the target perception operator, the production data associated with the running evidence is used to construct training samples, and the model of the target perception operator is incrementally trained using the training samples to obtain candidate models.

[0036] Specifically, after determining the target perception operator, the input pattern of the target perception operator is read. The input pattern represents the data type, data structure, and data organization method received by the target perception operator. Based on the node operation evidence of the target perception operator, production data associated with the node output that caused the abnormal business results is found, and the data required for training is extracted and organized from the production data according to the input pattern.

[0037] For example, when the target-aware operator processes a single-frame image, it can extract corresponding image data from associated scene images or videos; when the target-aware operator processes continuous frames or temporal information, it can extract corresponding continuous data from associated videos. The extracted data is organized according to the input requirements of the target-aware operator and combined with labels that can be used for training to form training samples.

[0038] The training samples are used to incrementally train the model currently used by the target perception operator, enabling the model to learn production data features related to the current abnormal business outcome. After incremental training is completed, the trained model and its training data association information are saved, and the trained model is identified as a candidate model to be validated.

[0039] S106 In the task graph, the candidate model is used to replace the model used by the target perception operator node. Business replay is performed on historical production data. When the business indicators and historical performance indicators meet the corresponding verification thresholds, the candidate model is deployed as the running model of the target perception operator.

[0040] Specifically, after the candidate model is trained, it is loaded into the verification runtime environment corresponding to the target perception operator node, enabling the target perception operator node to use the candidate model to process historical production data. The business logic in the task graph is re-executed based on the perception results generated by the candidate model, resulting in the business playback results corresponding to the candidate model.

[0041] Based on the business playback results, business metrics corresponding to the industrial vision task are calculated to evaluate the impact of candidate models on the business results. Simultaneously, data corresponding to historical categories or historical operating conditions are used to evaluate the performance changes of candidate models, obtaining historical performance retention metrics. The business metrics and historical performance retention metrics are then compared with corresponding validation thresholds.

[0042] When both business metrics and historical performance retention metrics meet the corresponding validation thresholds, the candidate model is determined to have passed business playback validation and is deployed as the running model for the target-aware operator. If the candidate model fails validation, the target-aware operator continues to use the model before incremental training, and the candidate model does not participate in the generation of actual business results.

[0043] In this embodiment, the correlation between business results and the output of the perception operator is established through task graphs and node operation evidence. The target perception operator is determined by replacing the output of candidate perception operators and replaying downstream business logic, enabling the model to be trained to select based on the actual impact of its output on business results. Constructing training samples according to the input pattern of the target perception operator improves the matching degree between training data and model processing requirements. After training, the candidate model is placed in the original task graph for historical production data playback. Combining business metrics and historical performance indicators to maintain metric control over model deployment helps reduce data processing and model training unrelated to business anomalies and minimizes the impact of model updates on historical categories or historical operating conditions.

[0044] In one embodiment, constructing the task graph based on the industrial vision task configuration includes: Based on the configuration source, functional fields, and input / output modes of each node in the industrial vision task configuration, each node is identified as a perception operator node, a business logic node, or an external production signal node. When the output of an upstream node is used by a downstream node, the dependency edge is generated based on the data transmission relationship, logical reference relationship, temporal relationship or spatial relationship between the upstream node and the downstream node. Based on the data transmission requirements between nodes, record the field mapping relationship, coordinate transformation relationship or time synchronization relationship corresponding to the dependent edge, and record the validity information of the results output by the node.

[0045] Specifically, the task graph can be represented as a directed graph G=(V,E), where V represents the set of nodes and E represents the set of dependent edges. The set of nodes V can be represented as: in, Represents the set of perception operator nodes. Represents a set of business logic nodes. This represents the set of external production signal nodes.

[0046] The system parses the industrial vision task configuration to obtain the configuration source, function fields, input mode, and output mode of each node. The configuration source indicates the configuration object or data source corresponding to the node; the function fields indicate the processing function performed by the node; and the input / output mode indicates the data types and data structures received and output by the node. The system determines the node type based on this information.

[0047] A node is identified as a perception operator node when it contains an executable visual model or visual processing component and receives images, videos, or upstream perception results, and outputs target bounding boxes, segmented regions, target trajectories, pose keypoints, text recognition results, action categories, or visual states. For example, processing nodes corresponding to person detection, part detection, image segmentation, target tracking, pose estimation, action recognition, and state recognition can all be identified as perception operator nodes.

[0048] When a node does not directly perform visual model inference, but instead performs tasks such as region inclusion judgment, target binding, contact relationship judgment, duration calculation, counting, threshold judgment, state transition, or condition combination based on the output of upstream nodes, the node is identified as a business logic node. For example, a node that generates region entry results based on personnel location and dangerous area range, and a node that generates operation stages based on action recognition results and state transition conditions, can both be identified as business logic nodes.

[0049] When a node's data source is control equipment, production management equipment, quality inspection equipment, information acquisition equipment, or confirmation information from the production site, and it provides equipment status, production status, quality results, or confirmation results to the task graph, the node is identified as an external production signal node. External production signal nodes can integrate non-visual information from the industrial site into the task graph for business logic nodes to perform status fusion or result comparison.

[0050] After node identification is completed, the system determines whether a dependency edge is established between two nodes based on whether the output of one node is actually used by another node. The direction of the dependency edge is from the upstream node providing the output to the downstream node using the output.

[0051] When the output field of an upstream node is used to satisfy the input requirements of a downstream node, a dependency edge is generated based on the data transfer relationship between the two nodes. For example, when the personnel location output by the personnel detection node is used by the personnel tracking node, a dependency edge is generated between the personnel detection node and the personnel tracking node.

[0052] When a downstream node's business processing rules reference the output of an upstream node, a dependency edge is generated based on the logical reference relationship. For example, when a danger zone judgment node references the personnel location output by a personnel tracking node, a dependency edge is generated between the personnel tracking node and the danger zone judgment node.

[0053] When the processing of a downstream node depends on the occurrence order, duration, time window, or state order of the output of an upstream node, a dependency edge is generated based on the temporal relationship. When the processing of a downstream node depends on the image region, containment relationship, intersection relationship, distance relationship, or target binding relationship corresponding to the output of an upstream node, a dependency edge is generated based on the spatial relationship. A dependency edge can simultaneously carry one or more of the following: data transfer relationship, logical reference relationship, temporal relationship, and spatial relationship.

[0054] After generating dependency edges, the system records the processing relationships corresponding to the dependency edges based on the data transfer requirements between connected nodes. Field mapping relationships are used to record the correspondence between upstream node output fields and downstream node input fields, as well as the data type conversions, category encoding conversions, or data structure conversions required for field transfer. When a downstream node runs, it retrieves the required data from the upstream node's output based on the field mapping relationships.

[0055] When upstream and downstream nodes use different image coordinate representations, the coordinate transformation relationship corresponding to the dependent edge is recorded. This coordinate transformation relationship can describe scaling, offsetting, rotating, or projecting transformations between the original image, the task processing region, the cropped image, and the model input image. When node output is passed along the dependent edge, the coordinates of the target bounding box, segmented region, or pose keypoints are transformed according to the coordinate transformation relationship, ensuring that upstream and downstream nodes use the same spatial information of the target in their corresponding coordinate spaces.

[0056] When upstream and downstream nodes process different video frames, different video streams, or production signals from different sources, they record the time synchronization relationships corresponding to the dependent edges. These time synchronization relationships may include video frame identifiers, data acquisition times, node processing times, allowed time differences, and matching rules between video data and production signals. During task graph execution, data belonging to the same business process is selected based on these time synchronization relationships for downstream nodes to process.

[0057] For dependent edges that do not involve coordinate transformation or multi-source time synchronization, it is sufficient to record only the field mapping relationship or other data transfer relationship that is actually needed, without setting transformation information that is irrelevant to the processing of the dependent edge.

[0058] The system also records the validity information of the node output. This validity information indicates the time when the node output became valid, the duration it can continue to be used, whether it is allowed to continue using it in subsequent video frames, the output quality, and the reason for output failure. When a downstream node reads the output of an upstream node, it determines whether the output can participate in the current business processing based on the validity information.

[0059] During task graph execution, node execution evidence stores the output of the corresponding node, the mapping or synchronization results used, and the reference relationships of upstream node execution evidence. Thus, the system can reconstruct the data transmission path used when a business result was formed based on dependent edges, and provide consistent node relationships and data interpretation for subsequent reverse tracing, output replacement, sample construction, and business replay.

[0060] In this embodiment, different types of nodes are identified by configuring sources, functional fields, and input / output patterns. The generation of dependency edges is based on whether the node output is actually used, enabling the task graph to reflect the actual execution relationships of industrial vision tasks. By recording field mappings, coordinate transformations, or time synchronization relationships on dependency edges, and recording the validity information of node output results, consistency in data transmission between nodes can be maintained, providing a basis for reverse tracing of business results and replaying subsequent business logic.

[0061] In one embodiment, the step of tracing candidate awareness operators backward from the corresponding abnormal business logic node along the dependency edge in response to a model update trigger event includes: Starting from the abnormal business logic node corresponding to the model update trigger event, search for the upstream node along the dependency edge; Based on at least one of the time range, spatial range, and target object associated with the model update trigger event, the operational evidence corresponding to the upstream node is filtered; Determine the degree of impact of the output of the perception operator corresponding to the screened operational evidence on the abnormal business results and the degree of conflict with the reference evidence. Calculate candidate scores based on the degree of impact and the degree of conflict, and screen candidate perception operators for output replacement and downstream business logic replay based on the candidate scores.

[0062] Specifically, such as Figure 3 As shown, the model update trigger event corresponds to the abnormal business logic node in the task graph, and records the event occurrence time, abnormal business result, and reference business result. The model update trigger event can also record the target object, image region, workstation, or other production location associated with the abnormal business result to limit the operational evidence related to this abnormality.

[0063] The system starts with an abnormal business logic node and searches for upstream nodes along the dependency edges pointing to that node. If the found upstream node is a business logic node, the search continues along the dependency edges corresponding to its input. If the found upstream node is a perception operator node, the system obtains the operational evidence of that perception operator node during the current task execution. If the perception operator node still has upstream nodes participating in the formation of the current business result, the system continues searching backwards along the corresponding dependency edges. For example, when a business judgment depends on a tracking result, and the tracking result depends on a detection result, after obtaining the operational evidence of the tracking node, the system continues to search for the detection node and its corresponding operational evidence to obtain the upstream nodes participating in the formation of the abnormal business result and their dependencies.

[0064] In a specific implementation, the reverse depth can be set for reverse tracing. When the reverse tracing reaches the reverse depth If the search continues along the corresponding path, it stops upon reaching the original data source, an external production signal node, or a node where no upstream node exists. Reverse depth The scope of reverse tracing can be controlled by setting the scale of the task graph or the type of abnormal business logic nodes.

[0065] After obtaining the operational evidence corresponding to the upstream node, the operational evidence is filtered based on at least one of the time range, spatial range, and target object associated with the model update trigger event.

[0066] When filtering by time range, the occurrence time of the trigger event can be updated according to the model. Determine the length of time before the event occurs. and the length of time after the event Limited time range Preserve operational evidence where the data acquisition time, node processing time, or result validity time overlaps with the aforementioned time range. For tasks with continuous actions or business states, and The validity period can be determined based on the context length required for the action or the validity period of the node output.

[0067] When using spatial range for filtering, based on the target area, workstation, or business area associated with the model update trigger event, it is determined whether the target bounding box, segmented region, target trajectory, or attitude key points in the operational evidence correspond to the spatial range. Operational evidence whose spatial location is within the spatial range, intersects with the spatial range, or can be mapped to the spatial range through spatial relationships in the task graph is retained.

[0068] When using target objects for filtering, the target object identifier is updated according to the model to record the trigger event, retaining operational evidence belonging to the same target object. When different nodes use different target identifiers, the outputs of different nodes can be determined to correspond to the same target object based on the target tracking relationship or target binding relationship recorded in the task graph. For example, when a personnel target is bound to the tool it uses, the operational evidence corresponding to the tool can be used as operational evidence related to personnel operation anomalies.

[0069] Time range, spatial range, and target object can be used individually or in combination for filtering. After filtering, operational evidence that does not conform to the selected time, spatial, or target object filtering conditions is excluded, while the perception operator outputs and their evidence chains related to the results of this abnormal business operation are retained.

[0070] For the sensing operator v corresponding to the screened operational evidence, determine the degree of influence I(v) of the sensing operator output on the abnormal business results. The degree of influence I(v) represents the magnitude of the role of the sensing operator output in the formation process of abnormal business results. The degree of influence can be determined based on whether the sensing operator is in the valid formation path of the abnormal business results, whether the sensing operator output participates in the key judgment conditions in the business logic, and the impact of changes in the sensing operator output on the business results in historical tasks. In a specific implementation, the degree of influence I(v) can be determined by using the proportion of the path where the sensing operator is located to the relevant business paths, the sensitivity of the business logic to the sensing operator output, or the business influence weight configured in the industrial vision task.

[0071] When determining the degree of conflict C(v) between the output of the sensing operator and the reference evidence, the output of the sensing operator is compared with the reference evidence corresponding to time, target object, and spatial location. For discrete results such as category, switch state, or action stage, the degree of conflict can be determined based on whether the results are consistent; for continuous quantities, the degree of conflict can be determined based on the normalized difference between the output of the sensing operator and the reference evidence. The reference evidence can come from manual verification results, PLC signals, MES feedback, or other independent reference information.

[0072] Candidate scores can be calculated based on the degree of influence I(v) and the degree of conflict C(v).

[0073] In a specific implementation, the candidate score can also be calculated by combining time relevance, historical error frequency, model divergence, and evidence reliability, and its expression is as follows: Where A(v) represents the candidate score of the perception operator v; I(v) represents the degree of influence of the perception operator output on abnormal business results; C(v) represents the degree of conflict between the perception operator output and the reference evidence; T(v) represents the time correlation between the perception operator's running evidence and the model update trigger event; H(v) represents the historical error frequency of the same product, workstation, shift, or error mode; M(v) represents the degree of output divergence between the current model and the historical model or teacher model; Q(v) represents the reliability of the running evidence; α, β, γ, δ, ε, and ζ are the corresponding non-negative weights.

[0074] The time correlation degree T(v) can be determined based on the time distance between the time corresponding to the running evidence of the sensing operator and the time of the event. The smaller the time distance, the higher the time correlation degree. Alternatively, it can be segmented based on whether the running evidence falls within the valid time range of abnormal business results. The historical error frequency H(v) can be determined based on the proportion of the sensing operator that conflicts with or is identified as the target sensing operator in historically confirmed anomalies for the same product, workstation, shift, or error mode.

[0075] The model divergence level M(v) can be determined based on the differences in output from the current model, historically stable models, or teacher models on the same production data. For classification results, it can be determined based on whether the categories are consistent or the differences between category probability distributions; for detection or segmentation results, it can be determined based on target matching results, the degree of region overlap, or differences in confidence. Evidence reliability Q(v) can be determined based on image sharpness, data integrity, time synchronization quality, chain of evidence integrity, reliability level of the reference signal, and the presence of dropped or black frames.

[0076] The system sorts the perception operators based on candidate scores and identifies those whose scores meet the selection criteria as candidate perception operators. These selection criteria can be determined based on the distribution of candidate scores for historical normal and abnormal tasks, the calibration results of verification data, or the configuration requirements of industrial vision tasks. Alternatively, a certain number of perception operators can be selected based on the ranking of candidate scores.

[0077] Candidate scores are used to narrow down the range of perception operators that require output replacement and downstream business logic replay. The highest candidate score, or meeting the screening criteria, only indicates that the corresponding perception operator needs to proceed to subsequent verification; it cannot be determined as the target perception operator solely based on the candidate score. The system will provide the node identifier, corresponding operational evidence, and associated abnormal business logic nodes of the selected candidate perception operators to the subsequent output replacement and downstream business logic replay process.

[0078] In this embodiment, by tracing upstream nodes involved in forming abnormal business results along the dependency edges and filtering operational evidence using time, space, or target object information associated with model update trigger events, the number of nodes and evidence unrelated to the current anomaly can be reduced. Furthermore, by calculating candidate scores based on the degree of influence of the perception operator output on the abnormal business results and its conflict with reference evidence, the range of perception operators requiring output replacement and business logic replay can be narrowed, reducing the processing load of subsequent attribution verification.

[0079] In one embodiment, determining the target perception operator based on the comparison between the replayed service result and the reference service result includes: Calculate the initial deviation between the business result before output replacement and the reference business result, and the post-replacement deviation between the business result after output replacement and replay of downstream business logic and the reference business result; When the difference between the initial deviation and the replacement deviation exceeds the correction threshold, and the replacement deviation is within the allowable range, the corresponding candidate sensing operator is determined as the target sensing operator. When the output of each candidate sensing operator is replaced individually, the resulting post-replacement deviation exceeds the allowable range. Based on the candidate score, multiple candidate sensing operators are selected for joint output replacement and the affected downstream business logic is replayed. The post-replacement deviation between the joint replacement business result and the reference business result is calculated. When the difference between the initial deviation and the deviation after joint replacement exceeds the correction threshold, and the deviation after joint replacement is within the allowable range, the candidate sensing operator participating in the joint output replacement is determined as the target sensing operator.

[0080] Specifically, the original business result, the reference business result, and the node execution evidence of the candidate perception operator corresponding to the model update trigger event are obtained. The original business result is the business result formed by the actual output of the perception operator in the task graph when the model update trigger event occurs; the reference business result is the reference result corresponding to this event and used to determine whether the business result is correct. Both correspond to the same business process, so that the comparison of subsequent output replacements has a consistent benchmark.

[0081] For each candidate sensing operator, a reference output that expresses the corresponding sensing result is obtained. The reference output can originate from direct correction of the current sample, an independent production signal that expresses the sensing operator's output, historically confirmed correct results, or consensus results from multiple teacher models that satisfy consistency conditions. The reference output is used to replace the sensing operator's output, and the reference business result is used to evaluate the replay results of downstream business logic; these correspond to the sensing processing layer and the business result layer, respectively.

[0082] The obtained reference output is adapted to the output requirements of the candidate perception operators, ensuring that the fields, data types, coordinates, and time representations used in the reference output correspond to the outputs of the candidate perception operators. For example, for a perception operator that outputs the target location, the reference output should be able to express the location of the corresponding target; for a perception operator that outputs the visual state, the reference output should be able to express the visual state at the corresponding time. If the reference information can only describe the final business state and cannot express the specific output of the candidate perception operator, then the output of the perception operator will not be directly replaced by the reference information.

[0083] When performing individual output replacements, the model version of the candidate perceptual operator remains unchanged, and its inference service, dependent libraries, and running hardware remain unchanged within a specific implementation. The upstream evidence of the candidate perceptual operator, as well as the results of parallel nodes that have no dependency on the candidate perceptual operator, are fixed based on the node running evidence, and the original output of the candidate perceptual operator is replaced with the reference output.

[0084] After the replacement is completed, the downstream business subgraph affected by the output replacement is determined based on the dependency edges, from the candidate sensing operator to the abnormal business logic node. The reference output is then passed to the downstream business subgraph, and the business logic is re-executed to obtain the replaced business result. When verifying different candidate sensing operators separately, each verification is carried out based on the original node's running evidence to avoid the result of the previous output replacement becoming the input of the next separate replacement.

[0085] Calculate the initial deviation between the original service result and the reference service result, and the post-replacement deviation between the replaced service result and the reference service result. The same comparison method is used for both deviation calculations. Subtract the post-replacement deviation from the initial deviation to obtain the service correction amount of the candidate sensing operator, expressed as: Where v represents a candidate sensing operator; Indicates the original business result; Indicates reference to business results; This represents the business result obtained after replacing the output of the candidate sensing operator v and replaying the downstream business logic; L represents the function used to measure the deviation between the business result and the reference business result. This represents the business correction amount of the candidate perception operator v.

[0086] Corresponding to the initial deviation, The corresponding deviation after replacement. A positive business correction amount indicates that the deviation between the output business result after replacement and the reference business result has decreased; a zero business correction amount indicates that the deviation has not changed; a negative business correction amount indicates that the deviation has increased. The specific comparison method of the deviation corresponds to the business result type. For example, comparing whether the results of alarm status or equipment status are consistent, or comparing the difference between the count or action start and end time and the reference result.

[0087] Business correction volume With correction threshold The comparison is performed to determine whether the deviation after replacement is within the allowable range. The correction threshold is used to limit the degree of deviation reduction that the output replacement should achieve, and the allowable range is used to limit the acceptable deviation between the replaced business result and the reference business result. When the business correction amount exceeds the correction threshold, and the deviation after replacement is within the allowable range, the corresponding candidate sensing operator is determined as the target sensing operator.

[0088] For example, in a hazardous area helmet detection task, the task graph includes personnel detection, personnel tracking, helmet detection, hazardous area judgment, duration judgment, and alarm nodes. When an alarm result is confirmed as a false alarm, and personnel detection, personnel tracking, and area relationships are all correct, a reference output corresponding to the helmet detection node can be obtained. After replacing the reference output with the helmet detection node, the downstream judgment and alarm logic related to the helmet detection result are re-executed, and the replayed alarm result is compared with the confirmed reference alarm result. Based on this, the operational correction amount is calculated, and it is determined whether the confirmation conditions of the target perception operator are met.

[0089] When the output replacements performed individually by each candidate sensing operator result in deviations exceeding the allowable range, joint output replacement is executed. Candidate sensing operators are ranked according to their scores, and nodes participating in joint verification are selected from the top-scoring candidates. In a specific implementation, two nodes can be selected from the top K ranked candidate sensing operators to form a verification combination, or multiple nodes can be selected to form a verification combination according to a set number of joint nodes. K and the number of joint nodes control the processing scope of the joint verification.

[0090] For each selected combination, the reference output corresponding to each candidate sensing operator is obtained. In the same verification, the output of the selected candidate sensing operator is replaced, and the downstream business logic affected by the joint output replacement is replayed. The joint verification determines the evidence retention range based on the node group participating in the replacement. Nodes participating in the joint replacement use their respective reference outputs, while nodes that did not participate in the replacement and are not affected by the joint replacement retain their original operating results, so as to ensure that the joint verification can reflect the business results after the selected nodes' outputs have changed together.

[0091] The deviation between the business result after joint replacement and the reference business result is calculated to obtain the deviation after joint replacement. Using the same initial deviation as the individual replacement verification, the deviation after joint replacement is subtracted to obtain the business correction amount of joint replacement. When the business correction amount of joint replacement exceeds the correction threshold, and the deviation after joint replacement is within the allowable range, the candidate sensing operators participating in joint output replacement are determined as target sensing operators, and the corresponding joint correction effect is recorded. When joint output replacement corresponds to multiple target sensing operators, training samples are constructed according to the input mode of each target sensing operator, and corresponding candidate models are obtained through incremental training; each incremental training task can be executed according to priority. After obtaining each candidate model, the candidate models are placed together in the corresponding target sensing operator node, and joint business playback is performed on historical production data. When the business indicators obtained from the joint business playback and the historical performance retention indicators of each candidate model meet the corresponding verification thresholds, the corresponding models are updated; otherwise, the models before incremental training of the corresponding target sensing operators are retained.

[0092] Each individual or joint output replacement generates a verification record, which includes the nodes involved in the replacement, the original output, the reference output, the source of the reference output, operational evidence that remained unchanged, the affected downstream business subgraph, the reference business result, and the replay result. This verification record is associated with the confirmation result of the target perception operator, used to trace the basis for confirmation, and to provide relevant production data and reference evidence for the subsequent training sample construction process.

[0093] In this embodiment, by replacing the output of candidate sensing operators while keeping the model version unchanged, and comparing the deviation between the business results before and after the replacement and the reference business results, it is possible to simultaneously verify the degree of business improvement resulting from output correction and whether the corrected business results meet the requirements. For cases where a single replacement cannot restore the business results, jointly replacing the output to examine the combined impact of multiple sensing operators helps to identify the sensing operators that need to be updated collaboratively, and provides traceable verification evidence for subsequent targeted sample construction and model training.

[0094] Furthermore, in one embodiment, if the target perception operator is still not determined after completing the applicable individual output replacement or joint output replacement verification, the model incremental training task is not generated based on the current attribution result. When the deviation after individual output replacement is within the allowable range, but the business correction amount does not exceed the correction threshold, the corresponding candidate perception operator is not determined as the target perception operator; whether to perform joint output replacement is still determined according to the aforementioned triggering conditions for joint output replacement.

[0095] For cases where the target perception operator is undetermined, the business rule configuration, hardware, data link, and external production signals are examined in conjunction with operational evidence. For example, if the perception result is confirmed to be correct by a trusted reference but the business result is still abnormal, the business logic expression, threshold, state transition, or field mapping are checked; if there are image anomalies, missing frames or messages, time asynchrony, or persistent external signal conflicts, the corresponding anomaly information is recorded. Invalid operational evidence and associated data unsuitable for model training are marked as unusable for training and excluded when constructing the automatic training set.

[0096] In one embodiment, constructing training samples using production data associated with the operational evidence according to the input pattern of the target perception operator includes: The sample extraction time interval is determined based on the occurrence time of the model update trigger event, the validity period of the results output by the nodes in the task graph, and the context length required by the target perception operator. Based on the sample extraction time interval, the input pattern of the target perception operator, and the target object and spatial range associated with the model update trigger event, target production data is extracted from the production data associated with the running evidence. Based on the coordinate transformation relationship recorded in the task map, coordinate mapping is performed on the target production data and the reference evidence with spatial coordinates; Organize the target production data after coordinate mapping according to the input pattern of the target perception operator, and generate corresponding labels according to the label pattern of the target perception operator based on the reference evidence after coordinate mapping, to form training samples.

[0097] Specifically, after determining the target perception operator, the operator type, input mode, time window, coordinate system, and label mode of the target perception operator are read. These are then combined with the target object, time range, spatial range, and reference evidence associated with the model update trigger event to determine the data extraction requirements for the training samples. The input mode specifies the type and organization of the input data, such as a single-frame image, consecutive image frames, or a skeleton sequence. The label mode specifies the annotation content and representation corresponding to the input data, such as bounding boxes, segmentation masks, target identifiers, keypoints, or action start and end times.

[0098] When determining the sample extraction time interval, the relevant production process is located based on the occurrence time of the model update trigger event. Then, the node outputs that actually participated in the calculation of abnormal business results are found based on the validity period of the node output results. The validity period of the results represents the time range within which the node output can be used by downstream business logic. Therefore, the outputs that participated in the calculation of abnormal business results may have been generated before the trigger event occurred. Based on the context length required by the target perception operator, the data time period associated with the corresponding output is extended forward or backward so that the extracted data covers the process required for the target perception operator to complete the recognition.

[0099] For detection or segmentation operators, keyframes are located based on the time of anomaly occurrence and operational evidence, and representative neighboring image frames are extracted. For tracking, action recognition, or state recognition operators, continuous segments are extracted based on the configured time window or the start and end times of related actions. For example, when a business logic triggers an anomaly due to an action not ending on time, the time from the start time of the candidate action to the timeout determination time can be extracted, and context segments before the action starts and after the timeout determination can be retained.

[0100] When extracting target production data, the production data is read based on the image or video storage location, frame identifier, and time information recorded in the operational evidence. Then, objects associated with abnormal business results are selected based on target object identifiers, target trajectories, or object binding relationships. When extracting local images, the clipping range is determined based on the target bounding box, target trajectory, or region of interest in the task configuration, and expanded and clipped according to the input requirements of the target perception operator. For example, for an abnormal alarm caused by a safety helmet failure, image frames before and after a person enters the hazardous area can be extracted, and local images related to the safety helmet can be determined based on the upper region of the person's bounding box.

[0101] When the target production data or reference evidence comes from multiple data streams, the time correspondence of each data stream is first determined according to a unified clock. Then, within the allowed time difference, the image frame, node output and reference evidence are associated using nearest neighbor matching or a predetermined deterministic matching method, so that the subsequently generated labels correspond to the same target object and the same production period as the input data.

[0102] During coordinate mapping, the scaling, offset, cropping, image scaling and filling, or homography transformation relationships recorded in the task graph are read to determine the correspondence between the original image coordinates, local region coordinates, model input coordinates, and training label coordinates. When performing cropping, scaling, or other processing on the target production data, the target bounding boxes, masks, polygons, or keypoints in the reference evidence are processed using the corresponding coordinate transformation relationships to ensure that the spatial labels maintain positional correspondence with the processed input data. For categories, target identifiers, device status, or time information that do not have spatial coordinates, no spatial coordinate transformation is performed; their original representation is retained and associated with the corresponding target production data. The coordinate-mapped reference evidence includes reference evidence that has undergone the corresponding spatial coordinate transformation, as well as reference evidence that retains its original representation during coordinate mapping processing because it does not have spatial coordinates.

[0103] The input data and labels are organized according to the type of object-aware operator. For detection operators, the image or local image is associated with the bounding box and category label, and the bounding box is represented as the coordinates of two diagonal points, or position coordinates and width and height, according to the label pattern. For segmentation operators, the image or local image is associated with the corresponding binary mask or polygon label, and the segmentation category information is preserved.

[0104] For the tracking operator, consecutive image frames are organized chronologically, and the bounding boxes in each frame are associated with target identifiers to form cross-frame trajectory labels. Visibility or occlusion markers are preserved based on the label patterns. For the pose recognition operator, images or consecutive image frames are organized with corresponding two-dimensional or three-dimensional keypoints, and the visibility or confidence information of the keypoints is preserved.

[0105] For action recognition operators, action video clips are formed based on the input pattern, or skeletal sequences are organized according to time, key points, and feature channels, and labels are represented by action category and start and end times. For state recognition operators, multiple frames of input are organized, state labels are associated with corresponding valid times, and state transition information is recorded to represent the changes in state over time during the production process.

[0106] Before generating corresponding labels, it is checked whether the content expressed by the reference evidence meets the label requirements of the target perception operator. Reference evidence with target boxes, masks, or key points can be used to generate corresponding spatial labels after coordinate mapping; reference evidence that only indicates the operation of equipment or the completion status of business cannot be directly converted into target boxes, masks, or key point labels that lack evidence. After the labels are generated, the input data is paired with the labels according to the input mode and label mode of the target perception operator to form training samples for incremental training.

[0107] In the above embodiments, it is possible to extract input content that matches the target perception operator from the production data associated with abnormal business results, retain the temporal context required for recognition, and make the reference evidence correspond to the input data in terms of time, object and spatial location, thereby reducing the impact of sample mismatch on incremental training of the model.

[0108] In one embodiment, generating corresponding labels according to the labeling pattern of the target perception operator based on the coordinate-mapped reference evidence includes: Check whether the reference evidence can directly express the label required by the target perception operator. If it can be directly expressed, generate candidate labels based on the reference evidence, and determine whether to use the candidate labels as hard labels based on the reliability of the reference evidence. When conflicting labels are generated from different reference sources, they are aligned according to the time, target object, and spatial location of the corresponding reference evidence. If the conflict still exists after alignment, a training disabling flag is set for the conflicting labels, and hard labels with the training disabling flag are excluded during incremental training. When the reference evidence cannot directly express the label required by the target perception operator, multiple teacher models are used to generate candidate labels. When the consistency of the candidate labels reaches a consistency threshold, consensus pseudo-labels are generated, and training weights are determined based on the consistency level.

[0109] Specifically, based on the labeling pattern of the target perception operator, the annotation content required for training is determined, and it is checked whether the reference evidence contains information capable of forming that annotation content. "Capable of direct expression" means that the reference evidence can provide the annotation information required by the target perception operator, such as category, location, shape, or time interval. For example, confirmed target bounding boxes, segmentation masks, keypoints, and action start and end intervals can be used to generate corresponding detection, segmentation, pose, or action labels, respectively; external production signals that only indicate the device is in operation cannot be directly used to generate target bounding boxes or color labels for indicator lights.

[0110] For reference evidence that can directly express the required label, candidate labels are generated according to the labeling pattern of the target perception operator, and the label content, source type, source identifier, confidence level, timestamp, reliability level, and corresponding evidence citation are recorded. Among them, the source identifier is used to distinguish different reference sources, and the evidence citation is used to associate the operation records or production data on which the label is based, so as to perform reliability judgment and conflict checking.

[0111] For reference evidence that can directly express the labels required by the target perception operator, after aligning time, target object, and spatial location, it is checked whether there are any unresolved label conflicts between different sources. If no such conflicts exist, the reliability of the reference evidence is determined based on the source reliability level, multi-source consistency, and alignment quality, and accordingly, it is determined whether to adopt the corresponding candidate label as a hard label. For example, confirmed direct correction results and independently generated signals that have been verified and are semantically consistent with the target label can be assigned a higher reliability level; historically confirmed labels can be assigned a corresponding reliability level. Candidate labels designated as hard labels are used as determined supervisory labels for incremental training. The confidence level of a candidate label does not independently determine its usability; in the absence of independent evidence, the current running model's own prediction is not used as the sole true value for correcting the model's errors.

[0112] When different reference sources provide inconsistent labels for corresponding production data, the timestamps, target object identifiers, and spatial locations of each reference are first checked to determine if the conflict is caused by time offset, incorrect object association, or inconsistent coordinates. Based on the check results, references from the same time period, the same object, and the corresponding spatial location are re-associated, and then the corresponding labels are compared. This alignment process is used to verify the comparability between labels and avoid misjudging normal differences between different objects or different times as label conflicts.

[0113] If conflicts persist after alignment, a training disable flag is set in the corresponding label record. During incremental training, this flag is checked when reading labels, and labels with this flag are excluded from hard label supervision to avoid forcibly selecting labels that have not yet resolved conflicts simply because a certain source has a higher reliability level. Conflicting labels and their evidence references can be retained for recording the reasons for disabling and tracing the label's source.

[0114] When the reference evidence cannot directly express the desired label, the corresponding image, video clip, or other data conforming to the input pattern is input into multiple teacher models, and each teacher model generates candidate labels. In one implementation, at least two teacher models with different structures are used to reduce the impact of prediction bias from a single model on label generation.

[0115] Based on the label type of the target perception operator, the consistency of categories, spatial overlap of bounding boxes or masks, and temporal consistency in consecutive frames or action segments are checked among the outputs of multiple teacher models. For detection or segmentation labels, the categories and corresponding bounding boxes or masks of the same target are compared; for labels with temporal requirements, the association results or action time intervals of the target in consecutive frames are further compared. Based on the corresponding comparison results, it is determined whether the consistency of candidate labels reaches the consistency threshold. Common prediction results that meet the consistency requirements are organized into consensus pseudo-labels according to the label pattern; if the consistency threshold is not reached, the corresponding prediction results are not used as consensus pseudo-labels for this incremental training.

[0116] After generating consensus pseudo-labels, the consistency level of candidate labels is associated with the corresponding training samples, and training weights are determined accordingly. During training, these weights are used to adjust the contribution of the corresponding samples to the training loss, allowing pseudo-labels with different consensus levels to participate in model updates with different weights. For reference evidence that can only express business status, its business-level semantics can be retained as auxiliary supervision information, but precise visual labels are not created based on it.

[0117] In this embodiment, by sequentially judging the expressiveness, reliability, and consistency between sources of the reference evidence labels, it is possible to reduce the number of semantically mismatched or conflicting labels entering hard label training; through multi-teacher model consensus and corresponding training weights, auxiliary supervision is provided when direct labels are lacking, reducing the risk of a single model error being repeatedly reinforced in incremental training.

[0118] Furthermore, in one embodiment, before generating incremental training tasks, a task effectiveness score is calculated for the training samples, and a training priority is determined based on the task effectiveness score. The task effectiveness score is expressed as: Where i represents a sample, P(i) represents the task effectiveness score; A(i) represents the sample's contribution to the attribution of business anomalies, U(i) represents model uncertainty, N(i) represents the sample's novelty relative to the historical training set, B(i) represents business risk, F(i) represents the frequency of similar errors, M(i) represents the degree of prediction divergence among multiple models, and C(i) represents the labeling or training cost; w1 to w7 represent the weights of the corresponding scoring items. A(i) is the attribution contribution at the sample level, determined based on the attribution results of the target perception operator associated with the sample and the correspondence between the sample and the business anomaly.

[0119] Based on the attribution results of sample associations, model predictions, historical training data, and business configurations, corresponding scoring information is obtained. Task effectiveness scores are calculated, and samples are ranked, prioritizing those closely related to business errors and with higher business risks for inclusion in the training queue. Business risk can also be used to adjust the sample retention ratio, training task priority, and candidate model validation thresholds.

[0120] Before training, the samples are further evaluated for quality, taking into account the consistency between the model and the label, the consistency among multiple models, image clarity, repetition, class balance, the geometric rationality of the bounding boxes or masks, and feature consistency with historical samples of the same type. Low-quality samples are removed based on the quality score, or the training weights of suspected low-quality samples are reduced. The task effectiveness score is used to determine training priorities, while the quality score is used to filter or adjust the samples participating in training to reduce the impact of irrelevant and low-quality samples on incremental training.

[0121] Further, in one embodiment, an incremental training task is generated based on the target-aware operator and training samples, recording the model to be updated, target category, sample composition, data augmentation method, and training parameters. The sample composition includes the ratio of new samples to historical samples, as well as the composition of positive / negative samples and hard samples; the training parameters are used to determine the trainable network layers, learning rate, loss function weights, and training termination conditions. The incremental training task is executed to update the model parameters of the target-aware operator, obtaining a candidate model.

[0122] The data augmentation method is determined based on the error pattern of the target perception operator. For recognition errors caused by nighttime lighting or color temperature changes, low-light or color temperature enhancement is used; for tool segmentation loss caused by hand occlusion, occlusion enhancement is used; for image blurring caused by rapid movement, motion blur enhancement is used, so that the training samples cover image changes related to the actual errors.

[0123] Establish a historical knowledge repository to store representative samples for each category, historically difficult samples, easily confused category samples, high-business-risk samples, category feature prototypes, and historical model outputs. During incremental training, new samples are mixed with historical samples from the historical knowledge repository, ensuring that the training process simultaneously includes data related to the current anomaly and data from historical categories or operating conditions.

[0124] Knowledge distillation is performed on the soft labels or features generated from historical samples using the model before incremental training. The corresponding historical outputs or features are used as training constraints to reduce the output or feature shift of the candidate model on historical samples. Combined with class balance and class-level historical performance preservation constraints, the model retains its historical recognition ability while learning new samples. After training, the candidate model, corresponding training data, and training task information are saved, and the candidate model is provided for subsequent historical business playback for validation.

[0125] In one embodiment, the step of replacing the model used by the target perception operator node with the candidate model in the task graph, performing business replay on historical production data, and deploying the candidate model as the running model of the target perception operator when the business indicators and historical performance retention indicators meet the corresponding verification thresholds includes: While keeping the configuration of other nodes in the task graph, the model versions of other perception operators, and their dependencies unchanged, the same historical production data is used to perform business replays using the model before incremental training and the candidate model, respectively, to obtain baseline business results and candidate business results; for task graphs that use external production signals, the same external production signal is used for both business replays. The business metrics corresponding to the baseline business results and the candidate business results are compared, and the historical performance retention metrics are determined based on the performance changes of the candidate model relative to the model before incremental training in historical categories or historical operating conditions. When the business metrics and the historical performance retention metrics respectively meet the corresponding verification thresholds, the candidate model is deployed as the running model of the target perception operator; otherwise, the model before incremental training is retained.

[0126] Specifically, such as Figure 4 As shown, after the candidate model training is completed, historical production images, videos, and corresponding reference service results are obtained for verification. The task graph version used for playback, the original model version of the target perception operator, service configuration, object binding relationships, and coordinate transformation relationships are recorded. For task graphs using external production signals, historical production signals and their time information are included in the playback data, ensuring that the two service playbacks have the same data source and configuration basis.

[0127] The first business replay uses the model before incremental training as the model used by the target-aware operator node. The task graph is run according to the original chronological order of historical production data to obtain the baseline business results. The second business replay replaces the model used by this node with a candidate model and runs the task graph again using the same historical production data to obtain candidate business results. In both replays, the business rules, model versions, field mappings, and coordinate and time processing rules for other nodes remain consistent, ensuring that the comparison of business results reflects the impact of changes in the target-aware operator model.

[0128] For business logic with external production signal inputs, the same equipment status, production stage, or counting signal is provided for both playbacks based on the time correspondence in historical records. For example, when the equipment status fusion logic uses both visual recognition results and equipment control signals, the same equipment control signal at the corresponding time is read for both playbacks to avoid external signal changes affecting the comparison between the old and new models.

[0129] During business playback, the output of the target-aware operator is passed to downstream nodes along the dependency edges of the task graph, where the downstream business logic recalculates the business results. For tasks involving continuous frame processing or state transitions, tracking, action judgment, or stage transitions are performed in chronological order of historical data, ensuring that the impact of model output changes on subsequent business processes is reflected in the playback results.

[0130] Based on the task type, business metrics corresponding to the baseline business results and candidate business results are statistically analyzed. Business metrics may include false alarm rate, missed alarm rate, accuracy rate of standard operating procedure completion judgment, action start and end time error, and accuracy rate of equipment status judgment or counting error. Using the same set of reference business results as the evaluation basis, the results obtained from two replays are evaluated separately, and then the candidate models are compared to see if they meet the required level of business improvement, or whether they maintain a performance no lower than the baseline model in key business operations.

[0131] Historical performance retention metrics are used to evaluate how well candidate models retain their original recognition capabilities. Based on category or working condition information from historical production data, the performance of the model before incremental training and the candidate model on historical categories or working conditions is statistically analyzed, and changes in both are used to determine whether performance degradation exceeds acceptable limits. For example, after incremental training to address changes in nighttime lighting, in addition to evaluating the improvement on nighttime data, the recognition performance of the candidate model on historical daytime data is also evaluated to determine whether the model update affects performance under the original working conditions.

[0132] For historical evaluation metrics where larger values ​​indicate better performance, the historical performance retention metric is considered to meet the validation threshold when the historical performance of the candidate model is not lower than the historical performance of the model before incremental training minus the allowable performance degradation. This is expressed as: in, This indicates the performance of the candidate model on historical categories or historical conditions. This indicates the performance of the model before incremental training on the same historical categories or historical scenarios. This indicates the permissible performance degradation. For different historical categories or operating conditions, corresponding evaluation indicators and permissible degradation amounts can be used to avoid the overall indicators masking performance degradation in specific categories or operating conditions.

[0133] For example, in scenarios where false alarms are caused by missed helmet detection, the same historical videos are played back using both the old and new models to compare the false alarm situation in hazardous areas, while also examining changes in the historical helmet detection recall rate. The effectiveness of the candidate model in reducing false alarms and its retention of historical helmet detection capabilities correspond to business metric verification and historical performance retention verification, respectively.

[0134] The validation thresholds are determined based on business requirements and historical benchmarks. When the business metrics and historical performance retention metrics of the candidate model meet their respective validation thresholds, the candidate model is deployed as the running model for the target-aware operator. If either threshold is not met, the model before incremental training is retained, and the failed metrics and corresponding historical data are recorded. The recorded reasons for validation failures can be fed back to the sample construction or incremental training stages for subsequent adjustments.

[0135] In this embodiment, by comparing the complete business playback results of the old and new models under the same historical data and task configuration, the impact of model updates on actual business processes can be evaluated. Combining the performance retention of historical categories or working conditions to decide whether to deploy candidate models helps to reduce model updates that improve local recognition results but do not improve business results, or reduce the original recognition capabilities.

[0136] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0137] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as cellular phones, smartphones, wearable devices, and other similar computing devices.

[0138] The electronic device includes: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments. Figure 5An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0139] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103 and output device 1104 may be connected by a bus or other means, as shown in the figure, which is connected by a bus 1105.

[0140] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0141] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback); and input from the user can be received in any form (e.g., voice input or tactile input).

[0142] In this embodiment, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0143] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0144] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0145] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0146] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, read-only optical discs, digital versatile optical discs or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0147] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0149] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0150] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0151] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0152] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for constructing training data and updating a model, characterized in that, The method includes: A task graph is constructed based on the industrial vision task configuration. The task graph includes perception operator nodes, business logic nodes, and dependency edges between nodes. Evidence of node operation is recorded when the task graph is run. In response to a model update trigger event, candidate perception operators are traced back along the dependency edges from the corresponding abnormal business logic node. While keeping the model version of the candidate sensing operator unchanged, replace the output of the candidate sensing operator with the reference output and replay the downstream business logic affected by the output replacement. Based on the comparison between the replayed business results and the reference business results, the target perception operator is determined; According to the input pattern of the target perception operator, training samples are constructed using the production data associated with the operational evidence, and the model of the target perception operator is incrementally trained using the training samples to obtain a candidate model. In the task graph, the model used by the target perception operator node is replaced with the candidate model, business playback is performed on historical production data, and when the business indicators and historical performance indicators meet the corresponding verification thresholds, the candidate model is deployed as the running model of the target perception operator.

2. The training data construction and model update method according to claim 1, characterized in that, The process of constructing a task graph based on industrial vision task configuration includes: Based on the configuration source, functional fields, and input / output modes of each node in the industrial vision task configuration, each node is identified as a perception operator node, a business logic node, or an external production signal node. When the output of an upstream node is used by a downstream node, the dependency edge is generated based on the data transmission relationship, logical reference relationship, temporal relationship or spatial relationship between the upstream node and the downstream node. Based on the data transmission requirements between nodes, record the field mapping relationship, coordinate transformation relationship or time synchronization relationship corresponding to the dependent edge, and record the validity information of the results output by the node.

3. The training data construction and model update method according to claim 1, characterized in that, The step of responding to a model update trigger event by tracing candidate perception operators backward along the dependency edge from the corresponding abnormal business logic node includes: Starting from the abnormal business logic node corresponding to the model update trigger event, search for the upstream node along the dependency edge; Based on at least one of the time range, spatial range, and target object associated with the model update trigger event, the operational evidence corresponding to the upstream node is filtered; Determine the degree of impact of the output of the perception operator corresponding to the screened operational evidence on the abnormal business results and the degree of conflict with the reference evidence. Calculate the candidate score based on the degree of impact and the degree of conflict, and screen the candidate perception operators that will be included in the output replacement and downstream business logic replay based on the candidate score.

4. The training data construction and model update method according to claim 3, characterized in that, The step of determining the target perception operator based on the comparison between the replayed service result and the reference service result includes: Calculate the initial deviation between the business result before output replacement and the reference business result, and the post-replacement deviation between the business result after output replacement and replay of downstream business logic and the reference business result; When the difference between the initial deviation and the replacement deviation exceeds the correction threshold, and the replacement deviation is within the allowable range, the corresponding candidate sensing operator is determined as the target sensing operator. When the output of each candidate sensing operator is replaced individually, the resulting post-replacement deviation exceeds the allowable range. Based on the candidate score, multiple candidate sensing operators are selected for joint output replacement and the affected downstream business logic is replayed. The post-replacement deviation between the joint replacement business result and the reference business result is calculated. When the difference between the initial deviation and the deviation after joint replacement exceeds the correction threshold, and the deviation after joint replacement is within the allowable range, the candidate sensing operator participating in the joint output replacement is determined as the target sensing operator.

5. The training data construction and model update method according to claim 1, characterized in that, The step of constructing training samples using production data associated with the operational evidence according to the input pattern of the target perception operator includes: The sample extraction time interval is determined based on the occurrence time of the model update trigger event, the validity period of the results output by the nodes in the task graph, and the context length required by the target perception operator. Based on the sample extraction time interval, the input pattern of the target perception operator, and the target object and spatial range associated with the model update trigger event, target production data is extracted from the production data associated with the running evidence. Based on the coordinate transformation relationship recorded in the task map, coordinate mapping is performed on the target production data and the reference evidence with spatial coordinates; Organize the target production data after coordinate mapping according to the input pattern of the target perception operator, and generate corresponding labels according to the label pattern of the target perception operator based on the reference evidence after coordinate mapping, to form training samples.

6. The training data construction and model update method according to claim 5, characterized in that, The step of generating corresponding labels according to the labeling pattern of the target perception operator based on the reference evidence after coordinate mapping includes: Check whether the reference evidence can directly express the label required by the target perception operator. If it can be directly expressed, generate candidate labels based on the reference evidence, and determine whether to use the candidate labels as hard labels based on the reliability of the reference evidence. When conflicting labels are generated from different reference sources, they are aligned according to the time, target object, and spatial location of the corresponding reference evidence. If the conflict still exists after alignment, a training disabling flag is set for the conflicting labels, and hard labels with the training disabling flag are excluded during incremental training. When the reference evidence cannot directly express the label required by the target perception operator, multiple teacher models are used to generate candidate labels. When the consistency of the candidate labels reaches a consistency threshold, consensus pseudo-labels are generated, and training weights are determined based on the consistency level.

7. The training data construction and model update method according to claim 1, characterized in that, In the task graph, replacing the model used by the target perception operator node with the candidate model, performing business replay on historical production data, and deploying the candidate model as the running model of the target perception operator when the business indicators and historical performance maintenance indicators meet the corresponding verification thresholds, includes: While keeping the configuration of other nodes in the task graph, the model versions of other perception operators, and their dependencies unchanged, the same historical production data is used to perform business replays using the model before incremental training and the candidate model, respectively, to obtain baseline business results and candidate business results; for task graphs that use external production signals, the same external production signal is used for both business replays. The business metrics corresponding to the baseline business results and the candidate business results are compared, and the historical performance retention metrics are determined based on the performance changes of the candidate model relative to the model before incremental training in historical categories or historical operating conditions. When the business metrics and the historical performance retention metrics respectively meet the corresponding verification thresholds, the candidate model is deployed as the running model of the target perception operator; otherwise, the model before incremental training is retained.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.