Trusted display method based on real-time video analysis and production data linkage

Abstract

The application discloses a trusted display method based on real-time video analysis and production data linkage and belongs to the technical field of industrial intelligent monitoring. The method comprises the following steps: analyzing a factory monitoring video stream, outputting video semantic events; synchronously acquiring corresponding production data and performing space-time alignment; performing dynamic cross verification on the aligned events and data based on a rule base, calculating a comprehensive credibility C; when the credibility is insufficient, constructing a multi-source evidence reasoning graph to perform root cause tracing, generating an evidence chain abstract; and finally generating a fusion visualization view for display. Through multidimensional cross verification and deep reasoning on video information and production data, the application realizes quantitative evaluation of the credibility of monitoring information and auxiliary diagnosis of abnormal roots.

Description

Technical Field

[0001] This invention relates to the field of industrial intelligent monitoring technology, and more specifically, to a reliable display method based on real-time video analysis and production data linkage. Background Technology

[0002] In the construction of smart factories, real-time monitoring of the production site through video surveillance and its integration with the production management system is a key step in achieving visualized and intelligent management of the production process. Existing solutions often operate the video surveillance system and the production management system as two independent subsystems, resulting in low levels of information integration between them. Common practices include simply displaying the production data panel alongside the video feed, or analyzing the video stream and displaying the results separately. This approach suffers from information fragmentation; operators must manually compare events in the video feed with the status in the production data system to determine consistency. This not only increases cognitive load but also makes it difficult to quickly pinpoint the root cause of problems when discrepancies arise.

[0003] In existing technologies, a Chinese patent (publication number: CN120050397A) proposes a factory video optimization acquisition method, which achieves compressed transmission by sparse processing of video frames. Its focus is on alleviating network bandwidth pressure and balancing video quality, but it does not address the correlation analysis and credibility judgment between video content and production process data. Another Chinese patent (publication number: CN120602556A) proposes a real-time video analysis system based on edge computing, which optimizes video stream processing latency by dynamically scheduling computing resources. Its core is to improve the computing efficiency and task allocation rationality at the edge, but it also fails to solve the problem of how to effectively integrate video analysis results with structured production data and verify credibility. In summary, existing technologies struggle to automatically and intelligently verify the consistency between video analysis results and production data, and to provide explainable traceability information when inconsistencies arise. Therefore, a reliable display method based on real-time video analysis and production data linkage is proposed to address the above problems. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a reliable display method based on real-time video analysis and production data linkage, aiming to solve the problems of fragmented display of video information and production data, and the lack of automatic fusion and reliability verification mechanisms in existing factory monitoring systems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a reliable display method based on real-time video analysis and production data linkage, comprising the following steps: S1. Analyze the factory monitoring video stream and output video semantic events consisting of event type, timestamp, and confidence level, which are used to transform unstructured video information into a computable structured event stream; S2. Synchronously acquire production data corresponding to the monitoring video stream, consisting of device identifier, parameter values, and data timestamps, to provide structured status and process data from the control and information system; S3. Align the video semantic events with the production data in time and space to establish a unified association between the two types of heterogeneous data in time and physical space; S4. Based on the association mapping rule base, perform dynamic cross-validation on the aligned video semantic events and the production data. The dynamic cross-validation includes factual consistency verification to check the direct consistency between visual perception state and sensor readings, process logic rationality verification to review whether the occurrence of the event conforms to the production plan and process flow, and environmental reliability assessment to evaluate the reliability of the video acquisition source itself. S5. Based on the results of the dynamic cross-validation, calculate a comprehensive confidence level C to quantify the overall reliability of the fused information after multi-dimensional verification; S6. In response to the inconsistency found by the dynamic cross-validation or the overall credibility C being lower than the first preset threshold, a multi-source evidence reasoning graph is constructed to perform root cause reasoning, and an evidence chain summary is generated to automatically diagnose alarms with insufficient credibility and provide interpretable tracing evidence. S7. Generate a fused visualization view; the fused visualization view overlays graphic labels on the screen of the monitoring video stream at the positions corresponding to the video semantic events, the graphic labels containing the event type, the associated parameter values, and a visual representation of the overall credibility C; when the overall credibility C is lower than a second preset threshold, the display style of the graphic labels is changed for visual highlighting, and the evidence chain summary can be displayed in response to user interaction, thereby integrating the verification, evaluation, and diagnostic results into a unified monitoring interface for display.

[0006] Furthermore, the association mapping rule base includes deterministic rules and statistical association rules; the statistical association rules are automatically generated by the system during operation by analyzing the co-occurrence relationship between the video semantic events and the production data items in historical data, and are used to adaptively mine and utilize the implicit associations between data to improve the verification logic; each rule is associated with a confidence score, which is used to measure the reliability of different rules when there is a rule conflict.

[0007] Furthermore, the verification of the rationality of the process logic includes: S41. Compare the event type, occurrence time and location information in the video semantic event with the currently effective production plan schedule to verify the plan compliance of the event occurrence, and to determine whether the event is authorized to be executed at a specific time and place; S42. Compare the video semantic events with the process flow in the production process specification knowledge base to verify the temporal compliance between multiple events, which is used to check whether the event sequence conforms to the predetermined process sequence constraints. S43. When the video semantic event involves materials, tools or personnel, the entity identifier identified therein is compared with the identifier specified in the production plan or bill of materials to verify the consistency of materials, so as to ensure that the resources used in the production process meet the requirements.

[0008] Furthermore, the environmental reliability assessment includes: S44. Perform quality analysis on the monitoring video stream to obtain quantitative indicators of sharpness, illumination uniformity, and screen occlusion rate, which are used to objectively evaluate the signal-to-noise ratio and availability of the video source; S45. The quantitative indicators are compared with preset quality standard thresholds to determine whether the video acquisition conditions meet the basic requirements of the analysis. S46. Generate a correction factor based on the comparison results. The correction factor is used to adjust the initial confidence of subsequent related video semantic events, thereby reducing the risk of misjudgment due to poor video quality.

[0009] Furthermore, the formula for calculating the overall credibility C is as follows: ,in, The data consistency score is calculated based on the results of factual consistency verification and is used to characterize the degree of consistency at the direct observation level. The logical rationality score is calculated based on the results of the process logic rationality verification and is used to characterize the degree of compliance of the event at the business logic level. The environmental reliability score is calculated based on the environmental reliability assessment results and is used to characterize the reliability of the data source itself. , , These are preset weighting coefficients used to balance the contribution of verification results from different dimensions to the overall credibility.

[0010] Furthermore, constructing the multi-source evidence reasoning graph includes: S61. Mark inconsistencies or low-confidence points as conflict points to locate issues requiring in-depth diagnostics; S62. Extract video keyframes, video semantic event recognition confidence, related sensor data sequences, production database status change logs and triggered verification rules within a set time window before and after the conflict time point, as evidence nodes, to collect multi-source heterogeneous original information related to the conflict. S63. Based on chronological, logical, or causal relationships, directed edges are established between the evidence nodes to form a reasoning graph structure, which is used to formally represent the mutual support or contradiction relationships between evidence. S64. Perform probability propagation analysis on the inference graph structure to calculate the probability of different failure hypotheses, in order to quantitatively assess the possibility of various potential root causes; S65. Select fault hypotheses with probabilities exceeding a set value and trace back their key evidence paths to generate the evidence chain summary, which is used to provide a diagnostic conclusion supported by probability ranking and key evidence.

[0011] Furthermore, the time alignment specifically involves using a dynamic time warping algorithm to perform non-linear matching between the timestamp sequence of the video semantic event stream and the data timestamp sequence of the production data stream, in order to compensate for and correct non-fixed time delays caused by different transmission and processing links.

[0012] Furthermore, the spatial alignment specifically involves: converting the pixel coordinates of the video semantic event in the video frame into coordinates in the factory world coordinate system based on the camera calibration parameters and the factory digital model, thereby determining the corresponding logical workstation identifier, and associating the logical workstation identifier with the equipment identifier in the production data through a predefined mapping table, so as to achieve accurate mapping between the position in the video frame and the equipment or area in the physical world.

[0013] Furthermore, it also includes rule evolution steps: when a verification conflict occurs, the confidence score of the relevant rules is updated according to the conflict resolution result, so that the rule base can self-correct based on the new resolution evidence; when the confidence score of a statistical association rule is consistently higher than the upgrade threshold, it is marked as a rule that can be upgraded to a deterministic rule, so as to solidify the fully verified data-driven pattern into domain knowledge.

[0014] Furthermore, it also includes feedback on the execution steps: When the main fault hypothesis output by the multi-source evidence reasoning graph is a video analysis error, the relevant video segments are automatically saved and added to the model retraining dataset to drive the iterative optimization of the video analysis model to reduce similar errors in the future. When the primary fault is assumed to be abnormal sensor data, a mark indicating that manual verification is required is added to the corresponding data monitoring interface. This guides maintenance personnel to conduct targeted checks on the suspected faulty data sources, thereby forming a closed loop from problem diagnosis to system maintenance.

[0015] The technical effects and advantages of this invention are as follows: This invention designs and executes a dynamic cross-validation process that aligns video semantic events with multi-source production data spatiotemporally, performing parallel verification at three levels: factual, logical, and data source quality. First, factual consistency verification is performed based on a predefined rule base, directly comparing visual events with sensor readings to determine if they contradict each other. Second, logical rationality verification is conducted by introducing production plans and process knowledge bases, examining whether the timing, location, and materials of the events conform to established procedures. Finally, the reliability of the perception source is assessed through real-time analysis of the video stream quality. The verification results at these three levels are quantified into independent scores, which are then weighted and fused into a comprehensive credibility score. This process enables the system to automatically identify problems of different natures, such as simple sensor malfunctions, violations of process procedures, or poor video quality, providing a classification basis for subsequent processing.

[0016] When the system detects factual conflicts, logical inconsistencies, or a comprehensive credibility level below a set threshold, it automatically triggers the construction and diagnosis process of a multi-source evidence reasoning graph. This method centers on the conflict point, extracting relevant video keyframes, raw data sequences, rule trigger records, and knowledge base state snapshots within a specific time window to construct a reasoning network containing multiple evidence nodes. By performing probability propagation calculations on this network, the likelihood of different fault hypotheses (such as video analysis errors or sensor drift) is assessed, and the hypothesis with the highest probability and its key evidence path are output, forming a structured evidence chain summary. This mechanism mimics the reasoning process of experts in fault diagnosis, enabling the correlation analysis of scattered clues and providing verifiable auxiliary information for operators to locate the root cause of problems.

[0017] This invention presents the aforementioned verification and diagnostic results intuitively by generating a fused visual view. Graphical labels are overlaid on the original video footage at the location of the event, dynamically displaying the event type, related data, and a color-coded overall credibility. .when When the value is too low, the visual style of the label will change to attract attention, and users can view a detailed summary of the evidence chain after interaction. This display method organically integrates verified information, its credibility metrics, and optional diagnostic analysis results, helping operators quickly focus on events with questionable credibility and make decisions based on the evidence provided by the system, thereby improving the efficiency of identifying true anomalies from massive amounts of monitoring information.

[0018] The method described in this invention continuously optimizes the system through a closed-loop feedback mechanism. Based on the diagnostic conclusions derived from the multi-source evidence reasoning graph, if the problem points to an error in the video analysis model, relevant video clips are automatically included for model retraining; if it points to an anomaly in specific sensor data, a check mark is added to that data point to prompt maintenance. This design, which feeds online diagnostic results back to the offline optimization process, allows the system to learn from actual misjudgments or conflicts, helping to gradually reduce the recurrence of similar problems, thereby improving the long-term stability and accuracy of the entire monitoring and data analysis system. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall workflow of the method of the present invention; Figure 2 This is a schematic diagram of the core logic of dynamic cross-validation in this invention; Figure 3 This is a flowchart of the low-reliability intelligent diagnosis and traceability process of the present invention; Figure 4 This is a flowchart illustrating the process of generating and interactively displaying the integrated visual view according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 As attached Figures 1 to 4 The trusted display method based on real-time video analysis and production data linkage, as shown, has the following specific implementation details: I. Specific Implementation Process Step S1. Perform real-time analysis on the factory monitoring video stream and output video semantic events consisting of event type, timestamp, and confidence level.

[0022] The video analytics module, deployed on edge computing nodes, receives real-time video streams from high-definition network cameras (e.g., 1920×1080 resolution, 25 frames per second). This module incorporates a deep learning model trained on extensive factory scene data. For example, it uses the YOLOv5 model for object detection, the DeepSORT algorithm for object tracking, and a 3DCNN model to identify specific action types. This 3DCNN model takes consecutive multi-frame video segments (e.g., 16 frames) as input and outputs the main action categories occurring within those segments.

[0023] In implementation, the module processes the video stream using either a fixed time window (e.g., 1 second) or dynamic keyframe extraction. For each processing unit, the model outputs a series of detection results. Each result is structured into a triple: .

[0024] : Represents the event type, which is a predefined enumeration value, for example Represents "conveyor belt operation", Represents "robotic arm grasping". This means "personnel have entered a danger zone". Examples include "flames and smoke." The event type definitions cover common safety, operational, and material handling aspects in production monitoring.

[0025] : Represents a timestamp, accurate to milliseconds, recording the absolute time when the event was identified, providing a benchmark for subsequent time alignment with production data.

[0026] : Represents confidence level, which is the probability value of the model's output that the recognition result is correct, and is a value between 0 and 10. arrive A floating-point number between 0 and 0.95, for example, will be used as one of the initial inputs for subsequent credibility calculations.

[0027] all Triples constitute the video semantic event stream, which serves as input for subsequent processing.

[0028] Step S2. Synchronously acquire production data corresponding to the monitoring video stream, consisting of device identifier, parameter values, and data timestamps.

[0029] Production data is concurrently acquired from the Manufacturing Execution System (MES) database and the real-time data bus of the Monitoring and Data Acquisition System via an Industrial IoT gateway or direct data interface. Data acquisition follows a predetermined data point table, ensuring coverage of all critical equipment and process parameters relevant to the video surveillance area. Each piece of production data is also structured as a triple: .

[0030] : Represents the equipment identifier, which is a unique string, such as "WS_001" (welding station No. 1) or "CV_LineA" (A-line conveyor belt). This identifier strictly corresponds to the physical equipment or logical workstation in the factory.

[0031] : Represents a parameter value, which can be numerical (e.g., speed 1200) or Boolean (e.g., equipment status). The ID can be either a running symbol (e.g., a string) or a current workpiece ID "PART-2024-XYZ", reflecting the operating status or production information of the equipment at a specific moment.

[0032] T1: Represents the data timestamp, accurate to milliseconds, identifying the moment the data was acquired by the sensor or generated by the system. Its time base must be synchronized with or be convertible to the video stream time base.

[0033] Production data is continuously pushed in the form of message queues (such as Apache Kafka) to ensure the real-time and sequential transmission of data.

[0034] Step S3. Align the video semantic events with the production data in time and space based on the timestamp and device identifier.

[0035] This step aims to address the heterogeneity between video data and production data in the temporal and spatial domains, establishing a unified foundation for subsequent fusion analysis.

[0036] Time Alignment: For events originating from the same logical source, the system maintains a video event cache queue and a production data cache queue. A dynamic time warping algorithm is used to align the two timestamp sequences. and And their respective associated numerical sequences (such as the confidence sequence of video events and the parameter value sequence of production data).

[0037] This algorithm finds the path with the minimum cumulative distance, performs a non-linear mapping between two sequences of potentially different lengths, and outputs an optimal path mapping table, thus effectively compensating for non-fixed delays caused by asynchrony in network transmission and system processing. In implementation, the maximum normalization window is set to 1000 milliseconds to accommodate common latency fluctuations. After alignment, each video event... It will be linked to a set of production data that best matches in time. .

[0038] Spatial alignment: This operation maps image pixel coordinates to the factory physical coordinate system, enabling the binding of visual events to physical devices.

[0039] 1. Offline calibration stage: Calibrate each camera and obtain its intrinsic parameter matrix. (Including parameters such as focal length and principal point) and the extrinsic parameter matrix [R|t] relative to the world coordinate system (including rotation and translation). Simultaneously, based on the factory digital twin model or CAD drawings, define the boundary 3D coordinate range of each key piece of equipment (such as a robotic arm or workbench), and assign a unique logical workstation identifier to each range, denoted as... (e.g., L1, L2), and establish To equipment identification mapping table .

[0040] 2. Online mapping phase: For a video event, first determine its pixel coordinates in the image. (Typically the center point of the detection frame). Assume the event occurs on the ground plane ( Using perspective projection formulas Solve for the corresponding world coordinates .in, It is a non-zero scale factor.

[0041] Finally, by judgment The logical workstation identifier is obtained by determining which device's preset boundary polygon the device falls into. By querying the mapping table (like This allows visual events to be associated with specific device identifiers. Complete spatial alignment.

[0042] Step S4. Based on a predefined association mapping rule base, perform dynamic cross-validation on the aligned video semantic events and production data.

[0043] Association mapping rule base It is the core knowledge base for storing verification logic, containing several rules. Each rule It can be formally represented as: {IF condition THEN expectation}, with an accompanying strength coefficient characterizing its current reliability. (For rule r1, its strength is denoted as w1; for rule r2, its strength is denoted as w2, and so on.) The verification process performs checks at the following three levels in parallel: 1. Fact Consistency Verification: Directly compare whether the physical state perceived by the video matches the sensor readings.

[0044] Implementation example: Define rule r1: IF T==1(conveyor belt running visual event) AND D==“CV_LineA”THEN S(motor current)>A0, where A0=10 amperes is the motor no-load current threshold. .

[0045] Calculation process: When a visual event triggers r1, the matching is checked. Value. If Ampere, then it is judged as consistent; if If the value is less than ampere, it is considered inconsistent. Record the degree of inconsistency. Then, the factual consistency score is calculated. .

[0046] 2. Verification of the rationality of process logic: Based on the production plan and process knowledge, review the logical compliance of the occurrence of events.

[0047] Knowledge Dependency: This validation relies on three external knowledge sources: production planning schedule P0, process specification library K0, and bill of materials M0.

[0048] Sub-step S41 (Plan Compliance): Query P0 to confirm the time... workstation Is the planned task to be For example, P0 specifies that L1 is... "Welding" should be performed at all times, if detected If it is "spraying", then record it as "spraying". This indicates that it does not conform to the plan.

[0049] Sub-step S42 (Sequence Compliance): Query K0 to check if the event sequence conforms to the process order. For example, K0 specifies that "material loading" must precede "inspection". If the identified event sequence is reversed, record it. .

[0050] Sub-step S43 (Material Consistency): When Material handling involves video or related data. Parse the material code Compare this with the material m0 specified for the current task in M0 or P0. If they do not match, record the discrepancy. .

[0051] Result calculation: If B1, B2, and B3 are all... Then the logical rationality score ;otherwise, .

[0052] 3. Environmental reliability assessment: Assess the quality of the video analytics input source (camera footage) itself and quantify its reliability.

[0053] Sub-step S44 (Quality Index Calculation): Before the video stream is input into the analysis model, three objective quality indices are calculated in parallel: I1 (Clarity): Calculates the variance of the Laplacian operator of the grayscale image. The higher the value, the clearer the image.

[0054] I2 (Illumination Uniformity): Divides the image into... For each region, the standard deviation of the average brightness is calculated. The lower the value, the more uniform the illumination.

[0055] I3 (Occlusion Rate): Calculated using background subtraction or frame difference method, it represents the proportion of the moving foreground pixel area to the total image area. The lower the value, the less occlusion there is.

[0056] Sub-step S45 (threshold comparison): Preset quality standard threshold , , These thresholds are derived from statistical analysis of historical normal images.

[0057] Sub-step S46 (correction factor generation): Calculate the correction factor F2 by considering the exceedance of the three indicators.

[0058] ; Limit the value of F2 to the range [0.3, 1]. This indicates the highest video quality. Environmental reliability score. .

[0059] Step S5. Calculate the overall confidence level C based on the results of dynamic cross-validation.

[0060] This step aggregates the results of the triple verification into a unified scalar metric to quantify the overall credibility of the information fused from the video event and production data.

[0061] Aggregate calculation formula: ; Variable description: : Overall credibility, with a value range of [0, 1]. The higher the value, the higher the credibility.

[0062] U1: Factual consistency score from step S4.

[0063] U2: Logical rationality score from step S4.

[0064] U3: Environmental reliability score from step S4.

[0065] W1, W2, W3: Preset weighting coefficients used to balance the contribution of different dimensions to the overall credibility, satisfying... This embodiment uses a typical configuration: , , This reflects the highest weight given to consistency of direct facts.

[0066] Step S6. In response to the discovery of factual inconsistencies, logical inconsistencies, or overall credibility C being lower than the first preset threshold Z1 by dynamic cross-validation, trigger the construction of a multi-source evidence reasoning graph.

[0067] When the system determines that the credibility of the information is questionable (a first preset threshold is set). It automatically enters deep diagnostic mode and constructs a reasoning graph to analyze the root cause.

[0068] S61. Mark conflict points: This will trigger any of the above conditions. The combined markers are the points of conflict in this diagnosis. .

[0069] S62. Extracting Evidence Nodes: Based on Conflict Time Points Expand a time window forward and backward from the center. Seconds. Within this time window, the system collects all relevant raw and intermediate data, forming a set of evidence nodes: E1: Video keyframe image.

[0070] E2: Video event list and its original model confidence .

[0071] E3: Related equipment The sequence of raw sensor readings.

[0072] E4: Snapshot of associated records from P0, K0, and M0.

[0073] E5: One or more rules triggered during the verification process. and its current strength (e.g., w1).

[0074] S63. Establish the reasoning graph structure: Organize the evidence nodes and points of conflict As vertices of the graph, directed edges are established between vertices based on data generation relationships, logical dependencies, temporal sequences, or physical causal relationships, forming a network structure that represents the correlation between evidence and conflict.

[0075] S64. Probability Propagation Calculation: Define a set of mutually exclusive failure hypotheses For example, J1: video analysis error, J2: sensor data anomaly, J3: network communication delay, etc. Define the state and conditional probability table for each node in the graph (especially evidence nodes). This table defines the probability of evidence appearing given a given hypothesis. Probability propagation calculations are performed on the constructed graph using Bayesian network inference algorithms (such as the connection tree algorithm) to ultimately obtain each fault hypothesis. After observing all current evidence Posterior probability under given conditions .

[0076] S65. Generate a summary of the chain of evidence: Calculate all failure hypotheses according to their posterior probabilities. Sort in descending order. Select the first [number] with the highest probability. One assumption (in this embodiment) For each adopted hypothesis, the key evidence path that contributes the most to its probability is traced back from the inference graph.

[0077] The system formats this analysis result into a text summary output, for example: "Main hypothesis: Video analysis error (probability 70%). Key evidence: Original confidence level of the event P=0.55 (low); concurrent video illumination uniformity index." (Reflection exceeds the threshold.) Step S7. Generate a merged visualization view.

[0078] This step integrates all the aforementioned processing results into the monitoring screen, forming an augmented reality display.

[0079] View generation: The system generates the view at the pixel position corresponding to the video event on the original video stream. Draw a semi-transparent (e.g., 70% transparency) rectangular graphic label at that location.

[0080] Tag content: The tag dynamically displays three parts of information: (1) Event Type The text description; (2) Core associated production data parameter values (e.g., "speed: 1200"); (3) Overall credibility The visualization uses intuitive color coding: Displayed in green, Displayed in yellow, It is displayed in red.

[0081] Enhanced alert for anomalies: Set a second preset threshold. (generally ).when When the system automatically triggers visual enhancement: the border of the graphic label turns red and flashes at a frequency of about 2Hz, while a conspicuous warning icon (such as an exclamation mark) is added next to the label to attract the operator's attention.

[0082] Interactive Diagnosis: When the operator clicks on the visually enhanced label mentioned above, the monitoring interface sidebar or pop-up window will display a summary of the evidence chain generated in step S65, providing the user with a basis for further diagnosis and decision-making.

[0083] II. Implementation and Evolution of the Association Mapping Rule Base rule base The design supports incremental knowledge accumulation.

[0084] Rule structure: Deterministic rules: These are predefined by domain experts based on process principles and safety specifications, and their strength coefficients are... The initial value is set to 1.0, representing the highest authority, as mentioned above with r1.

[0085] Statistical Association Rules: During long-term operation, the system automatically analyzes historical data using association rule mining algorithms (such as the Apriori algorithm) to discover stable co-occurrence relationships between video events and production data patterns. For example, the system statistically discovers events... After it occurs, the data pattern exist If this occurs, a statistical rule r2 can be automatically generated, with its initial strength w2 set to 0.95.

[0086] Rule evolution mechanism: When new and old rules conflict with the expectations of the same scenario and cause verification failure, the system will record the conflict and wait for the "final decision" (which may come from manual review or instructions from a higher-level system).

[0087] After the ruling, the system will update the strength of the relevant rules based on the ruling result: the strength of the rules expected to be correct will be... The value is increased by a fixed increment (e.g., 0.05, with a maximum of 1.0). The rule of expected error, its The value is reduced by a fixed penalty (e.g., 0.1, with a lower limit of 0.1). When a statistical rule... When a value consistently and stably exceeds a high upgrade threshold (such as 0.98) during long-term operation, the system will mark it as "upgradeable to a deterministic rule," prompt experts to review it, and include it in the deterministic knowledge base.

[0088] III. Implementation of Closed-Loop Feedback on Model and Data Quality The system uses diagnostic results to form an optimization closed loop in two directions to improve the accuracy of long-term operation.

[0089] S8. Model Retraining Feedback: When the evidence chain summary indicates that the main fault hypothesis is J1 (video analysis error), and this is confirmed by manual verification, the system automatically re-selects the video keyframe E1 that triggered the conflict and a period of time before and after it (e.g., ...). (seconds) of video clips, with manually corrected event labels. Together, they are packaged and stored in a dedicated incremental learning sample pool. This sample pool is used periodically to fine-tune the training of the video analysis model, thereby specifically improving the model's recognition performance in specific, easily confused scenarios.

[0090] S9. Data Quality Tagging Feedback: When the evidence chain summary indicates that the primary fault hypothesis is J2 (sensor data anomaly), the system will monitor the corresponding sensors in the data acquisition system. On the historical trend curve, at a certain point in time Add a visual anchor point marker (such as a highlighted spot) and attach the annotation "Suspected drift, pending verification". This provides equipment maintenance personnel with clear clues for data quality verification, guiding them to prioritize checking relevant sensors and facilitating the discovery and resolution of data source issues.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A reliable display method based on real-time video analysis and production data linkage, characterized in that, Includes the following steps: S1. Analyze the factory monitoring video stream and output video semantic events consisting of event type, timestamp, and confidence level; S2. Synchronously acquire production data corresponding to the monitoring video stream, consisting of device identifier, parameter values, and data timestamp; S3. Align the video semantic events with the production data in time and space; S4. Based on the association mapping rule base, perform dynamic cross-validation on the aligned video semantic events and the production data; the dynamic cross-validation includes fact consistency verification, process logic rationality verification, and environmental reliability assessment; S5. Calculate a comprehensive confidence level C based on the results of the dynamic cross-validation; S6. In response to the inconsistency found by the dynamic cross-validation or the overall credibility C being lower than the first preset threshold, construct a multi-source evidence reasoning graph to perform root cause reasoning and generate an evidence chain summary. S7. Generate a merged visualization view; The fusion visualization view overlays graphic labels on the screen of the monitoring video stream at the positions corresponding to the video semantic events. The graphic labels include the event type, the associated parameter values, and a visual representation of the overall credibility C. When the overall credibility C is lower than the second preset threshold, the display style of the graphic label is changed, and the evidence chain summary can be displayed in response to user interaction.

2. The reliable display method based on real-time video analysis and production data linkage according to claim 1, characterized in that, The association mapping rule base includes deterministic rules and statistical association rules; the statistical association rules are automatically generated by the system during operation by analyzing the co-occurrence relationship between the video semantic events and the production data items in historical data; each rule is associated with a confidence score.

3. The reliable display method based on real-time video analysis and production data linkage according to claim 1, characterized in that, The verification of the rationality of the process logic includes: S41. Compare the event type, occurrence time, and location information in the video semantic event with the currently effective production plan schedule to verify the plan compliance of the event occurrence; S42. Compare the video semantic events with the process flow in the production process specification knowledge base to verify the temporal compliance between multiple events; S43. When the video semantic event involves materials, tools or personnel, the identified entity identifier is compared with the identifier specified in the production plan or bill of materials to verify the consistency of the materials.

4. The reliable display method based on real-time video analysis and production data linkage according to claim 1, characterized in that, The environmental reliability assessment includes: S44. Perform quality analysis on the monitoring video stream to obtain quantitative indicators of sharpness, illumination uniformity, and screen occlusion rate; S45. Compare the quantitative indicators with preset quality standard thresholds; S46. Generate a correction factor based on the comparison results, the correction factor being used to adjust the initial confidence of subsequent related video semantic events.

5. The reliable display method based on real-time video analysis and production data linkage according to claim 1 or 6, characterized in that, The formula for calculating the overall credibility C is as follows: ,in, The data consistency score is calculated based on the results of the factual consistency verification. The logical rationality score is calculated based on the results of the process logic rationality verification. The environmental reliability score is calculated based on the environmental reliability assessment results. , , These are the preset weighting coefficients.

6. The reliable display method based on real-time video analysis and production data linkage according to claim 1, characterized in that, Constructing the multi-source evidence reasoning graph includes: S61. Mark inconsistencies or points of low confidence as conflict points; S62. Extract video keyframes, video semantic event recognition confidence, relevant sensor data sequences, production database status change logs, and triggered verification rules within a set time window before and after the conflict time point as evidence nodes; S63. Based on temporal, logical, or causal relationships, directed edges are established between the evidence nodes to form a reasoning graph structure; S64. Perform probability propagation analysis on the inference graph structure to calculate the probability of different fault assumptions; S65. Select a fault hypothesis with a probability exceeding a set value and trace back its key evidence path to generate the evidence chain summary.

7. The reliable display method based on real-time video analysis and production data linkage according to claim 1, characterized in that, The time alignment specifically involves using a dynamic time warping algorithm to perform non-linear matching between the timestamp sequence of the video semantic event stream and the data timestamp sequence of the production data stream.

8. The reliable display method based on real-time video analysis and production data linkage according to claim 1 or 7, characterized in that, The spatial alignment specifically involves: converting the pixel coordinates of the video semantic events in the video frame into coordinates in the factory world coordinate system based on the camera calibration parameters and the factory digital model, thereby determining the corresponding logical workstation identifier, and associating the logical workstation identifier with the equipment identifier in the production data through a predefined mapping table.

9. The reliable display method based on real-time video analysis and production data linkage according to claim 2, characterized in that, It also includes rule evolution steps: when a verification conflict occurs, the confidence score of the relevant rule is updated based on the conflict resolution result; when the confidence score of a statistical association rule is consistently higher than the upgrade threshold, it is marked as a rule that can be upgraded to a deterministic rule.

10. The reliable display method based on real-time video analysis and production data linkage according to claim 6, characterized in that, It also includes feedback execution steps: When the main fault hypothesis output by the multi-source evidence reasoning graph is a video analysis error, the relevant video segments are automatically saved and added to the model retraining dataset. When the primary fault is assumed to be abnormal sensor data, a mark indicating that manual verification is required is added to the corresponding data monitoring interface.

Images