External operation inspection method and system

Abstract

The application relates to the technical field of image processing, and discloses an external operation inspection method and system. The external operation inspection method comprises the following steps: collecting a field image of a target inspection point; extracting a quality characteristic factor from the field image to obtain an image quality evaluation result; extracting a state characteristic of a target object from the field image, and converting the state characteristic and the image quality evaluation result into a field visual reading of the target object; acquiring online data of a control system corresponding to the target inspection point; comparing the field visual reading with the online data of the control system to obtain a deviation characteristic, and fusing the deviation characteristic and the image quality evaluation result to generate an inspection result of the target inspection point. The application makes the reliability of the inspection result adapt to the image acquisition quality, reduces the misjudgment risk caused by insufficient image quality, and improves the reliability of the inspection result.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an external operation inspection method and system. Background Technology

[0002] Field inspections are a crucial aspect of safety management in industrial sites. They typically involve collecting status information from field instruments and comparing it with online data from the control system to determine if any abnormalities exist. With the development of video recognition technology, existing solutions use cameras to capture field images, employing image recognition algorithms to extract the status features of target objects as visual readings, and then comparing these readings with online data from the control system to generate inspection results. However, the quality of field images often fluctuates significantly due to factors such as changing lighting conditions, target obstruction, reflections, and fog. Current technologies struggle to guarantee the reliability of inspection results when image quality varies.

[0003] Therefore, there is an urgent need for an external inspection method to solve the problem of insufficient reliability of inspection results in related technologies. Summary of the Invention

[0004] This application provides an external operation inspection method and system to solve the problem of insufficient reliability of inspection results in related technologies.

[0005] Firstly, this application provides an external operator inspection method, which includes: Acquire on-site images of the target inspection points; Extract quality feature factors from the on-site images to obtain the image quality evaluation results of the on-site images; Extract the state features of the target object from the on-site images, and combine them with the image quality evaluation results to convert the state features into on-site visual readings of the target object. Acquire online control system data corresponding to the target inspection points; By comparing on-site visual readings with online data from the control system, deviation characteristics are obtained. These deviation characteristics are then fused with image quality evaluation results to generate inspection results for the target inspection points.

[0006] In one optional implementation, the deviation features are fused with the image quality assessment results to generate inspection results for the target inspection points, including: The normalized deviation is obtained by taking the absolute value of the difference between the on-site visual readings and the online data of the control system as the numerator and the allowable error band including the image quality penalty term as the denominator; the image quality penalty term increases as the image quality evaluation result decreases. The inspection results for the target inspection points are generated based on the normalized deviation.

[0007] In one alternative implementation, the method further includes: Obtain the historical stability factor of the target inspection point; Based on the normalized deviation calculation, a deviation decay term that monotonically decreases as the normalized deviation increases is calculated. The deviation decay term is then multiplied by the image quality evaluation result and the historical stability factor to obtain the internal and external operation consistency score. When the consistency score between internal and external operations is consistently lower than the preset consistency threshold, it is determined that there is a continuous inconsistency between internal and external operation data at the target inspection point, and the determination result is included in the inspection results.

[0008] In one alternative implementation, the method further includes: When inconsistencies are detected between internal and external control data, the category with the highest conditional probability is determined from multiple preset candidate anomaly categories based on normalization deviation, internal and external control consistency score, image quality evaluation results, historical stability factor, and data change trend characteristics of the control system. This category is then used as the current source of anomalies at the target inspection point and incorporated into the inspection results.

[0009] In an alternative implementation, prior to the step of extracting state features of the target object from the scene image, the method further includes: Compare the image quality evaluation results with the preset image quality threshold; When the image quality evaluation result is lower than the preset image quality threshold, the on-site image of the target inspection point is re-acquired until the image quality evaluation result is not lower than the preset image quality threshold. Re-capturing includes at least one of the following operations: gimbal adjustment, zoom, or fill light.

[0010] In one optional implementation, acquiring online control system data corresponding to the target inspection point includes: Based on the acquisition time of the on-site images, the historical sampling value sequence of the control system corresponding to the target inspection point is obtained within a preset time window; According to the preset weighting rules, the historical sampled value sequence is weighted and fused, and unit conversion and range mapping are performed to obtain the online control system data after time window alignment; among them, the preset weighting rule is that the closer each sampling time is to the acquisition time, the greater the weight.

[0011] In one alternative implementation, the method further includes: For multiple consecutive frames of on-site images of the target inspection point, extract multiple image features. These multiple image features include at least the regional image change features, liquid accumulation features, splash and gas disturbance features, abnormal area expansion features, and abnormal duration features. Multiple image features are weighted and accumulated using the frame weights corresponding to each historical frame to obtain a spatiotemporal evidence score; the frame weights are negatively correlated with the time interval between the historical frame and the current time. Spatiotemporal evidence scoring will be incorporated into the generation of inspection results.

[0012] In one alternative implementation, the method further includes: Evidence of visual anomalies is accumulated from multiple consecutive frames of on-site images of the target inspection point to obtain a spatiotemporal evidence score; The inconsistency risk, spatiotemporal evidence score, and sensor anomaly intensity corresponding to the internal and external operation consistency score are weighted and summed according to their respective preset weights to obtain the comprehensive anomaly index. When the comprehensive anomaly index exceeds the preset trigger threshold, and the duration of exceeding the preset trigger threshold is not less than the preset minimum duration, an alarm message is generated or a review command is initiated.

[0013] In one optional implementation, quality feature factors of the scene image are extracted to obtain the image quality evaluation result of the scene image, including: Extract the sharpness, target stability, and shooting angle effectiveness of the scene image, and sum the sharpness, target stability, and shooting angle effectiveness according to the preset positive weighting coefficient to obtain the positive weighted result; Extract the degree of occlusion and the degree of light interference from the scene image, and sum the degree of occlusion and the degree of light interference according to the preset negative weight coefficient to obtain the negative weighted result; The image quality evaluation result is obtained by subtracting the positive weighted result from the negative weighted result and then normalizing it.

[0014] Secondly, this application provides an external operation inspection system, which includes: The front-end perception layer is used to collect on-site images of the target inspection points; The image quality control layer is used to extract quality feature factors of the on-site image and obtain the image quality evaluation result of the on-site image. The video intelligent recognition layer is used to extract the state features of the target object from the on-site image, and combine the image quality evaluation results to convert the state features into on-site visual readings of the target object. The control system time window alignment layer is used to acquire online control system data corresponding to the target inspection point. The multi-source mutual verification layer is used to compare on-site visual readings with online data from the control system to obtain deviation characteristics. The deviation characteristics are then fused with image quality evaluation results to generate inspection results for the target inspection points.

[0015] The external inspection method of this application extracts quality feature factors from the on-site image to obtain an image quality evaluation result, which is then incorporated into two stages. In the visual reading generation stage, the state features of the target object are combined with the image quality evaluation result. This allows the conversion process from state features to on-site visual readings to perceive the current image quality status, thereby reducing the impact of state feature extraction deviations on subsequent comparisons when image quality is poor. In the inspection result generation stage, the deviation features obtained from comparing on-site visual readings with online data from the control system are fused with the image quality evaluation result. This allows the final inspection conclusion to comprehensively reflect both deviation information and image quality status. This makes the inspection results adaptable to changes in image quality, reducing the risk of misjudgment due to unstable image quality. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of the first type of external operation inspection method according to an embodiment of this application; Figure 2 This is a second flowchart illustrating the external operation inspection method according to an embodiment of this application; Figure 3 This is a structural block diagram of an external operation and inspection system according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0020] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained.

[0021] External operator inspection: In industrial production, operators are generally divided into two categories: internal operators and external operators. Internal operators are those who monitor data and operate equipment through a monitoring system in the central control room; external operators are those who need to enter the production site regularly to conduct on-site inspections and operations of equipment, instruments, etc. External operator inspection is the work of external operators to check and record the equipment and instruments on site according to a plan.

[0022] Major hazard sources refer to locations or facility units that produce, store, use, or operate hazardous chemicals on a long-term or temporary basis, and whose quantities are equal to or exceed the critical quantity. Safety risks are concentrated in areas with major hazard sources, requiring significantly higher frequency and accuracy of inspections compared to ordinary areas.

[0023] DCS (Distributed Control System): An automated control system widely used in industrial production. It collects and monitors process parameters in real time through sensors and transmitters distributed on the production site, and then aggregates the data to the central control room.

[0024] SIS (Safety Instrumented System): A dedicated instrument and control system for safety protection. It can automatically trigger protective actions (such as emergency shutdown) when process parameters exceed safety limits. It is independent of DCS and together they form a safety protection system for the industrial site.

[0025] Multi-source verification: This method involves simultaneously collecting information from multiple independent data sources, such as video recognition, DCS / SIS online data, and sensor sampling, and improving the reliability of the final judgment through mutual verification and cross-validation among these sources.

[0026] PTZ Camera: PTZ is an abbreviation for Pan (horizontal rotation), Tilt (vertical tilt), and Zoom (zoom). A PTZ camera is a camera device that can respond to remote control commands, rotate freely in both horizontal and vertical directions, and adjust its focus. It is suitable for scanning and acquiring data from multiple targets in turn.

[0027] Explosion-proof camera: A camera device that meets explosion-proof certification standards and can be used safely in environments containing flammable gases or explosive dust. Its housing and structural design can prevent internal electrical faults from igniting external explosive mixtures.

[0028] Image quality gating: Before an image enters the subsequent recognition or comparison process, its quality is quantitatively evaluated and thresholded. Images that do not meet the quality standards are paused and active improvement operations are triggered (such as adjusting the camera angle, zooming, and adding light) to ensure that the images entering the recognition process have a sufficient quality foundation.

[0029] The Sigmoid function is a commonly used non-linear mathematical function with an S-shaped curve. It can map and compress any real number input value to between 0 and 1, and is often used for normalization.

[0030] Leaks and spills: A common term used in industrial settings to describe material leakage phenomena, which include four types of abnormal appearances: gas escaping from equipment or pipelines, large amounts of liquid overflowing, continuous small drips of liquid, and slight seepage of liquid from connections.

[0031] Mechanical seal: A sealing device used in rotating machinery (such as centrifugal pumps) to prevent liquid or gas leakage at the shaft seal. It consists of a stationary sealing ring and a rotating sealing ring. When a mechanical seal leaks, visible changes such as water stains or liquid accumulation can usually be observed in the shaft seal area.

[0032] Magnetic float level gauge: This instrument uses changes in liquid level to move a magnetic float up and down, and magnetic coupling drives a magnetic flip plate on an external display bar to flip, thus displaying the liquid level height. It is commonly used for measuring liquid levels in storage tanks.

[0033] U-tube manometer (U-tube differential pressure gauge): A simple differential pressure measuring instrument consisting of a U-shaped transparent tube filled with indicating fluid. Two pressure measuring points are connected to the two ends of the tube, and the differential pressure value is calculated by reading the difference in height between the liquid columns on both sides.

[0034] Electrolytic cell: An electrochemical device that drives a chemical reaction between two electrodes by passing direct current through them. It is widely used in production processes such as chlor-alkali chemical industry.

[0035] Temperature and vibration sensors: These are composite sensors that can simultaneously collect temperature and vibration signals. They are typically installed on the bearing housing or body of rotating machinery to monitor the health status of the equipment in real time. Abnormal temperature or vibration is an early warning signal of equipment failure.

[0036] Normalized deviation: The original deviation values ​​with different dimensions and ranges are converted into dimensionless relative deviation values ​​after standardization, so that the deviations of instruments of different types and precisions can be compared on a unified scale.

[0037] Spatiotemporal evidence chain: An evidence sequence formed by accumulating image features from multiple consecutive time points in chronological order. It can reflect the development and change of anomalies over time and is used to reliably identify gradual anomalies.

[0038] Dual prevention mechanism: a safety management system that organically combines safety risk classification and control with hidden danger investigation and management. The former focuses on the early identification and classification and control of various risks, while the latter focuses on the investigation and closed-loop rectification of discovered hidden dangers.

[0039] Internal and external operator consistency score: A quantitative indicator that measures the degree of consistency between on-site visual recognition data (external operator data) and DCS / SIS online data (internal operator data). The higher the score, the better the consistency between the two.

[0040] Anomaly Classification: After an anomaly is detected, the source of the anomaly is further classified and judged to distinguish whether it is a field instrument failure, DCS sensor drift, algorithm recognition deviation or real process anomaly, so as to provide a basis for subsequent targeted handling.

[0041] Time window alignment: Based on a certain reference time, multiple sampling data are collected within a certain time range before and after that time and then weighted and fused to eliminate time deviations caused by asynchronous sampling times from different data sources.

[0042] As an optional application scenario of this application embodiment, the external operation inspection method can be applied to an inspection system including at least one image acquisition device and at least one processing server. For example, the system may include an image acquisition device, a processing server and a control system. The image acquisition device is connected to the processing server via a network, and the processing server is also connected to the control system to obtain online data corresponding to the inspection points.

[0043] Specifically, the image acquisition equipment can be industrial cameras, explosion-proof cameras, PTZ cameras, or other devices with image acquisition capabilities, or it can be an edge computing terminal that integrates image acquisition and preliminary processing functions. The processing server can be a standalone physical server, a server cluster or distributed system, or an edge computing node deployed in an industrial site. The control system can be an automated system with online data acquisition and storage capabilities, such as a distributed control system (DCS) or a safety instrumented system (SIS) deployed in an industrial site. The network can be a wired network or a wireless network, examples of which include, but are not limited to, industrial Ethernet, local area networks, and combinations thereof.

[0044] It should be noted that the above optional application scenarios are merely examples of one application scenario and do not limit the scope of protection of this application.

[0045] In related technologies, field operator inspections involve capturing on-site images using cameras and extracting the state features of target objects using image recognition algorithms. These images are then compared with online data from the control system to generate inspection results. However, the quality of on-site images fluctuates due to factors such as changes in lighting, occlusion, and reflection. Existing technologies do not consider image quality factors in the generation of visual readings and the comparison of deviations. To address the technical problem of the reliability of inspection results being affected by fluctuations in image quality, this application provides a field operator inspection method to improve the adaptability of inspection results to changes in image quality and reduce the risk of misjudgment caused by unstable image quality.

[0046] According to an embodiment of this application, an embodiment of an external operation inspection method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0047] This embodiment provides an external inspection method, which can be used for the aforementioned processing server or edge computing node. Figure 1 This is a flowchart of an external operation inspection method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain on-site images of the target inspection points.

[0048] The target inspection point refers to the specific on-site location where status monitoring needs to be carried out during the inspection task, such as the installation location of a pressure gauge, a pipeline interface, or the location of a tank level gauge.

[0049] On-site images refer to images or video frames that reflect the current on-site status of a target inspection point, collected by image acquisition equipment deployed near the target inspection point.

[0050] Acquisition refers to the processing server receiving image data transmitted by image acquisition devices over the network. Alternatively, it can occur after an inspection task is triggered, by sending an acquisition command to the image acquisition device and receiving the returned image data. During this step, the processing server, based on the inspection task configuration, sends an acquisition request to the image acquisition device corresponding to the target inspection point, or directly receives images actively uploaded by the device at a preset period, thereby obtaining images reflecting the current site status as input for subsequent processing steps.

[0051] Step S102: Extract the quality feature factors of the on-site image to obtain the image quality evaluation result of the on-site image.

[0052] Quality feature factors refer to a set of features that can be quantified and extracted from an image to reflect its quality status. These may include image sharpness (reflecting the sharpness of the image focus), image stability (reflecting the degree of jitter between consecutive frames), shooting angle effectiveness (reflecting the degree of orientation matching of the target object relative to the camera), occlusion degree (reflecting the proportion of the target object being occluded by other objects), and lighting interference degree (reflecting the intensity of lighting interference such as reflections, overexposure, or fog in the image).

[0053] The image quality assessment result is the overall quality assessment value obtained by comprehensively weighting the above-mentioned multiple quality feature factors, which can be expressed as a normalized quality score.

[0054] Extraction refers to the process by which the server analyzes the on-site image and quantifies each quality characteristic factor. For example, it measures sharpness by calculating the image frequency domain energy, measures the degree of occlusion by statistically analyzing the pixel occlusion ratio of the target area, and measures illumination interference by analyzing the image brightness distribution.

[0055] The image quality assessment results will play a constraining role in subsequent steps, enabling the image quality status to influence the generation of visual readings and the final inspection conclusions.

[0056] Step S103: Extract the state features of the target object from the scene image, and combine the image quality evaluation results to convert the state features into the scene visual readings of the target object.

[0057] The target object refers to the specific object at the target inspection point that needs to be identified in terms of status. It can be instruments and equipment such as pressure gauges, level gauges, and flow meters, or structural components such as valves and pipe interfaces.

[0058] State features refer to image features extracted from the target object area of ​​a field image that reflect its current state, such as the position features of an instrument pointer on the dial or the character area features of a digital display instrument.

[0059] On-site visual readings refer to converting state characteristics into quantifiable values ​​with actual physical meaning, such as the current reading of an instrument.

[0060] Extraction refers to the processing server using image recognition algorithms to locate and analyze the area where the target object is located, obtaining a quantitative expression of its state features. Conversion refers to mapping the state features to corresponding physical readings by combining preset parameters of the target object (such as measurement range and unit information), and incorporating the image quality evaluation results obtained in step S102 during the conversion process. The image quality evaluation results participate in the state feature conversion process, ensuring that the conversion from state features to on-site visual readings has high confidence when the image quality is high; and that the conversion results reflect the uncertainty caused by poor image quality when the image quality is low, thus providing quality-aware reading input for subsequent deviation fusion.

[0061] Step S104: Obtain online control system data corresponding to the target inspection point.

[0062] Online control system data refers to process parameter data that the control system collects and stores in real time through sensors, corresponding to the physical indicators monitored at the target inspection points. For example, the pressure sensor value corresponding to the pressure gauge on site, or the liquid level sensor value corresponding to the liquid level gauge on site.

[0063] Acquisition refers to the process server actively reading online data corresponding to the process parameters configured at the target inspection point through the data interface established between it and the control system.

[0064] Correspondence means that the physical quantity reflected by the online data of the control system and the target object at the target inspection point belongs to the same process measurement point, and the two reflect the actual state of the same location from different paths. The online data of the control system will be compared with the field visual readings in the next step, and the two together serve as the data basis for generating the inspection results.

[0065] Step S105: Compare the on-site visual readings with the online data of the control system to obtain the deviation characteristics, and fuse the deviation characteristics with the image quality evaluation results to generate the inspection results of the target inspection points.

[0066] Deviation characteristics refer to the quantitative expression of the degree of difference between field visual readings and online data from the control system, reflecting the consistency between the two data sources.

[0067] Inspection results refer to the judgment conclusions made on the current status of the target inspection point after processing the server's comprehensive deviation characteristics and image quality evaluation results.

[0068] Comparison refers to numerically comparing on-site visual readings with online data from the control system, calculating the differences between the two, and obtaining deviation characteristics. Fusion refers to comprehensively calculating the deviation characteristics with the image quality evaluation results obtained in step S102, so that the image quality status constrains the interpretation of the deviation characteristics. When the image quality evaluation result is high, the deviation characteristics have a strong indicative role in the inspection conclusion; when the image quality evaluation result is low, the deviation characteristics are corrected by quality factors during the fusion process to avoid the recognition deviation introduced by poor image quality being directly amplified into an abnormal judgment. For example, when there is a deviation between the on-site visual readings and the online data from the control system, and the deviation still exceeds the preset range after correction by the image quality evaluation result, the inspection result is marked as abnormal; when the deviation is within the acceptable range after quality correction, the inspection result is marked as normal.

[0069] In summary, the external inspection method provided in this application extracts quality feature factors from the field image to obtain the image quality evaluation result, introduces the result into the conversion process of state features to field visual readings, and further fuses it with deviation features to generate inspection results. This allows the image quality status to play a constraining role in both the visual reading generation and inspection conclusion judgment stages, thereby reducing the impact of image quality fluctuations on the accuracy of inspection results and improving the reliability of inspection results.

[0070] This embodiment provides an external inspection method, which can be used for the aforementioned processing server or edge computing node. Figure 2This is a flowchart of an external operation inspection method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain on-site images of the target inspection points. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0071] Step S202: Extract the quality feature factors of the on-site image to obtain the image quality evaluation result of the on-site image.

[0072] To comprehensively measure image quality from multiple dimensions, in some embodiments, the steps of extracting quality feature factors of the scene image and obtaining image quality evaluation results can be performed as follows: The image quality is evaluated by extracting the image's sharpness, target stability, and shooting angle effectiveness. These factors are then weighted and summed according to preset positive weighting coefficients to obtain a positive weighted result. The image quality is also evaluated by extracting the image's occlusion and illumination interference levels. These factors are then weighted and summed according to preset negative weighting coefficients to obtain a negative weighted result. The difference between the positive and negative weighted results is calculated and normalized to obtain the final image quality evaluation result.

[0073] Specifically, sharpness reflects the sharpness of the image focus and can be quantified by calculating the high-frequency energy of the target area (e.g., the mean of the Laplacian operator response); a higher value indicates a sharper image. Target stability reflects the degree of jitter in the target area between consecutive acquisition frames and can be measured by comparing the displacement of feature points in the target area between adjacent frames; higher stability indicates that the image is more suitable for target recognition. Shooting angle effectiveness reflects the degree of orientation matching between the instrument or equipment target and the camera; for example, the angle effectiveness is higher when the pressure gauge dial faces the camera directly and lower when it faces to the side. All three factors contribute positively to image quality; therefore, they are weighted and summed using positive weighting coefficients to form a positively weighted result.

[0074] The degree of occlusion reflects the proportion of the target object obscured by pipes, equipment components, or other objects; the more severe the occlusion, the less usable information about the target. The degree of lighting interference reflects the intensity of interference such as strong reflections, overexposure, uneven lighting, or steam / mist in the image; the stronger the interference, the greater the difficulty of recognition. Both of these factors have a negative impact on image quality; therefore, they are weighted and summed using negative weighting coefficients to form a negatively weighted result.

[0075] The difference between the positive weighted result and the negative weighted result is used to obtain the original quality score that combines the positive and negative factors. After normalization (for example, by using a Sigmoid function to compress the score to the range of 0 to 1), the image quality evaluation result can be obtained.

[0076] By comprehensively calculating multiple quality factors such as sharpness, target stability, shooting angle effectiveness, degree of occlusion, and degree of lighting interference, and distinguishing between positive and negative factors and assigning them weights, the image quality evaluation results can comprehensively reflect the quality status of various factors affecting the recognition effect, providing a more accurate quality reference for subsequent processing steps.

[0077] After obtaining the image quality evaluation results, in order to avoid errors caused by low-quality images entering the recognition process, in some embodiments, the following steps may be performed before extracting the state features of the target object from the scene image: The image quality evaluation result is compared with the preset image quality threshold. When the image quality evaluation result is lower than the preset image quality threshold, the on-site image of the target inspection point is re-acquired until the image quality evaluation result is not lower than the preset image quality threshold. The re-acquisition includes at least one operation among gimbal adjustment, zoom or supplemental lighting.

[0078] The preset image quality threshold refers to the lower limit of the pre-configured image quality evaluation result, used to determine whether the current image meets the minimum quality requirements for entering the recognition process. It can be configured according to the detection accuracy requirements of the target inspection point and the instrument characteristics (such as the fineness of the scale and the size of the target). Different thresholds can be set for different inspection points. When the image quality evaluation result is lower than the preset image quality threshold, the target recognition process is not directly entered, but a re-acquisition operation is triggered.

[0079] Re-acquisition can include at least one of the following methods: pan-tilt adjustment, which involves driving the pan-tilt camera to rotate and adjust the shooting angle or orientation to reduce the impact of obstruction or improve the effectiveness of the shooting angle; zoom, which involves adjusting the camera's focal length to bring the target closer and improve the clarity and resolution of the target in the image; and supplementary lighting, which involves turning on or adjusting the on-site supplementary lighting device to improve lighting interference caused by insufficient light or strong reflections.

[0080] After re-acquisition, the newly acquired images undergo quality evaluation and comparison again until the image quality evaluation result is not lower than the preset image quality threshold before proceeding to the target recognition process. By introducing image quality gating and active re-acquisition mechanisms before entering the recognition process, the acquisition conditions are proactively improved when the image quality does not meet the requirements. This reduces the adverse impact of low-quality images on subsequent recognition and comparison results from the source, and lowers the risk of misjudgment due to poor image quality.

[0081] Step S203: Extract the state features of the target object from the scene image, and combine them with the image quality assessment results to convert the state features into a scene visual reading of the target object. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0082] Step S204: Obtain online control system data corresponding to the target inspection point.

[0083] To eliminate the time difference between video acquisition and control system sampling, and to address unit range differences, in some embodiments, the step of acquiring online control system data corresponding to the target inspection point can be performed as follows: Based on the acquisition time of the on-site images, the historical sampling value sequence of the control system corresponding to the target inspection point is obtained within a preset time window; according to the preset weighting rules, the historical sampling value sequence is weighted and fused, and unit conversion and range mapping are performed to obtain the online data of the control system after time window alignment; among them, the preset weighting rule is that the closer each sampling time is to the acquisition time, the greater the weight.

[0084] A preset time window refers to a time interval that extends forward or backward from the moment the on-site image was acquired, for example, 30 to 60 seconds before the acquisition moment. Within the preset time window, the multiple discrete sampled values ​​stored by the control system constitute the historical sampled value sequence of the control system.

[0085] Weighted fusion refers to assigning weights to each sample value in the historical sample value sequence and then performing a weighted average. The preset weighting rules stipulate that the closer each sampling time is to the acquisition time, the greater the weight. The closer the sample value is to the acquisition time, the more it can reflect the real working conditions that are consistent with the acquisition time of the video image. Assigning a larger weight can make the fusion result closer to the real value at the acquisition time of the image, thereby reducing the time misalignment deviation caused by the sampling time difference of the control system or the lag of sensor filtering.

[0086] Unit conversion refers to converting historical sampled values ​​of the control system from units stored internally by the control system (e.g., kilopascals) to units consistent with field visual readings (e.g., megapascals). Range mapping refers to mapping the numerical range of the control system's sampled values ​​to an interval consistent with the range of field instruments, based on unit unification. For example, if the field image acquisition time for a pressure measurement point is 15:00:00, the system takes the sequence of control system sampled values ​​from 14:59:30 to 15:00:00, performs a weighted average of the sampled points according to the rule that the closer the points are, the greater the weight, and then performs unit conversion (kilopascal to megapascal) and range mapping to finally obtain the online control system data after time window alignment.

[0087] By weighting and fusing historical sampled value sequences with a preset weighting rule that gives higher weight to closer samples, and combining unit conversion and range mapping, the online data of the control system is made consistent with the on-site visual readings in three dimensions: time, unit, and range. This reduces misjudgments caused by sampling time differences, filtering lag, and dimensional differences.

[0088] Step S205: Compare the on-site visual readings with the online data of the control system to obtain the deviation characteristics, and fuse the deviation characteristics with the image quality evaluation results to generate the inspection results of the target inspection points.

[0089] Regarding the fusion method of deviation characteristics and image quality evaluation results, in some embodiments, a more accurate fusion calculation can be achieved by introducing a normalized deviation. Specifically, the normalized deviation is obtained by using the absolute value of the difference between the on-site visual reading and the online data of the control system as the numerator, and the allowable error band including the image quality penalty term as the denominator; the image quality penalty term increases as the image quality evaluation result decreases; and the inspection results of the target inspection points are generated based on the normalized deviation.

[0090] Normalized deviation refers to converting the deviations of instruments with different ranges and accuracies into dimensionless deviation values ​​that can be compared horizontally between different instruments. The numerator is the absolute value of the difference between the field visual reading and the online data of the control system, directly reflecting the original deviation magnitude between the two data sources. The denominator is the allowable error band, which is the upper limit of the deviation judged to be within the normal tolerance range under the current conditions. It may include three items: fixed allowable error (reflecting the instrument's own accuracy, a fixed value), proportional tolerance (adjusted proportionally with the reading or range), and image quality penalty (increased as the image quality evaluation result decreases).

[0091] The image quality penalty increases as the image quality evaluation result decreases, meaning that the allowable error band expands accordingly when the image quality is poor. Poor image quality leads to significant recognition uncertainty in visual readings. If a smaller allowable error band is used, this uncertainty may be misjudged as a process anomaly. Expanding the allowable error band makes the judgment of deviations when the quality is poor more conservative, thus reducing false alarms. When the normalized deviation is no greater than 1, it indicates that the deviation is within the allowable error band; when the normalized deviation is greater than 1, it indicates that the deviation exceeds the allowable error band, further triggering the subsequent anomaly judgment process.

[0092] By introducing an allowable error band that includes an image quality penalty term to calculate the normalized deviation, the deviations of instruments with different ranges and accuracies are uniformly converted into comparable normalized values, and the allowable error band can be dynamically adjusted with changes in image quality, reducing false alarms when there are fluctuations in image quality.

[0093] Based on the normalized bias, in order to further enhance the inspection conclusions by integrating historical operational information, the following steps may also be performed in some embodiments: Obtain the historical stability factor of the target inspection point; calculate the deviation decay term based on the normalized deviation, which monotonically decreases as the normalized deviation increases; multiply the deviation decay term with the image quality evaluation result and the historical stability factor to obtain the internal and external operation consistency score; when the internal and external operation consistency score is continuously lower than the preset consistency threshold, it is determined that there is a continuous inconsistency between the internal and external operation data of the target inspection point, and the determination result is included in the inspection result.

[0094] The historical stability factor is a quantitative value that reflects the stability of the target inspection point in history. It can be calculated by comprehensively considering information such as the fluctuation of historical readings, the historical manual review pass rate, the recent false alarm rate, or the long-term stable operation performance of the point. The higher the historical stability factor, the more stable the historical performance of the point and the higher the reliability of the current data.

[0095] The deviation decay term is a function value that decreases monotonically as the normalized deviation increases. It reflects the relationship that the greater the deviation, the worse the consistency between internal and external operational data. It can be achieved by an exponential decay function (e.g., a function with the natural constant e as the base and the negative of the square of the normalized deviation as the exponent). The deviation decay term takes its maximum value when the normalized deviation is zero, and it decreases monotonically towards zero as the normalized deviation increases.

[0096] Multiplying the deviation attenuation term with the image quality evaluation result and the historical stability factor means performing a multiplication operation on the three values ​​to obtain the internal and external operation consistency score. When the normalization deviation is large (the deviation attenuation term is small), the image quality evaluation result is low, or the historical stability factor is low, the internal and external operation consistency score will decrease accordingly, indicating that the consistency of the internal and external operation data at the current location is poor.

[0097] A consistent score below the preset consistency threshold means that the consistency score for both internal and external operations has not exceeded the threshold for multiple consecutive inspection cycles, rather than a single low value. This consistent condition filters out occasional low scores caused by temporary fluctuations in the process, making the judgment more robust. For example, if the consistency score for both internal and external operations at a certain level measuring point is below the preset consistency threshold for five consecutive inspection cycles, it is determined that there is a persistent inconsistency between the internal and external operation data at that point. This judgment is then incorporated into the inspection results and triggers the subsequent review process.

[0098] By incorporating the historical stability factor into the calculation of the internal and external operation consistency score, and using a sustained value below the threshold as the judgment condition, the score can integrate information from three dimensions: current deviation, image quality, and historical stability of the location, thereby improving the robustness of the judgment conclusion.

[0099] After determining that there is inconsistency between internal and external control data, simply outputting "inconsistency" is insufficient to support subsequent handling decisions. Therefore, in some embodiments, the source of the anomaly can be further classified. Specifically, when inconsistency between internal and external control data is detected, based on normalization deviation, internal and external control consistency score, image quality evaluation results, historical stability factors, and the trend characteristics of control system data changes, the category with the highest conditional probability is determined from multiple preset candidate anomaly categories and used as the current source of the anomaly at the target inspection point, and included in the inspection results.

[0100] The data change trend characteristics of a control system refer to the regularity of changes in the online data of the control system over a period of time. This can include the slope of the data trend (whether there is a slow upward or downward trend), the fluctuation range, whether there are sudden changes, and whether the sampled values ​​drift or become unstable.

[0101] Multiple preset candidate anomaly categories refer to several predefined reasons that may lead to inconsistencies between internal and external operating data, such as actual process anomalies (process parameters have indeed changed), field instrument jamming (field meters have fixed readings due to mechanical reasons), control system acquisition drift (control system sensor drift leads to inaccurate online data), visual misidentification (image recognition algorithm misjudges the current image), and field leakage anomalies (leaks occur on-site, causing changes in operating conditions), etc.

[0102] Conditional probability refers to the probability of each candidate anomaly category occurring given multiple characteristic observations such as normalization deviation, internal and external operator consistency score, image quality evaluation results, historical stability factor, and control system data change trend characteristics. Determining the category with the highest conditional probability involves calculating the conditional probability of each candidate anomaly category using a classification model and selecting the category with the highest probability as the most likely source of the anomaly. For example, if the normalization deviation is large, but the historical stability factor is high (indicating that the location has historically performed stably), the control system data change trend is stable (excluding actual process changes), and the image quality evaluation result is high (excluding image quality factors), then the conditional probability of visual misidentification or on-site instrument malfunction may be high. The system will include the category with the highest conditional probability as the current source of the anomaly in the inspection results to support subsequent targeted handling.

[0103] By comprehensively classifying and judging the sources of anomalies based on multidimensional features, the inspection results can provide specific categories of anomaly sources, rather than just outputting general inconsistency alarms, thus supporting more targeted subsequent handling decisions.

[0104] Step S206: Determine reliability by observing continuous image changes.

[0105] In addition to the inspection logic based on single-data comparison mentioned above, for appearance anomalies such as leaks and spills that require continuous image changes for reliable judgment, some embodiments may introduce a spatiotemporal evidence accumulation mechanism to improve detection reliability. Specifically, for multiple consecutive frames of on-site images of the target inspection point, multiple image features are extracted. These features include at least regional image change features, liquid accumulation features, splash and gas disturbance features, abnormal area expansion features, and abnormal duration features. The multiple image features are weighted and accumulated using the frame weights corresponding to each historical frame to obtain a spatiotemporal evidence score. The frame weights are negatively correlated with the time interval between the historical frame and the current moment. The spatiotemporal evidence score is incorporated into the generation of the inspection results.

[0106] Regional image change characteristics refer to the degree of image change of a target area relative to historical frames or background frames. These can be calculated using frame differencing or background subtraction algorithms to reflect whether new appearance changes have occurred in the target area. Liquid accumulation characteristics refer to water stains, liquid accumulation areas, or liquid reflection features in an image, which can be detected through color analysis or changes in brightness distribution.

[0107] Splash and gas disturbance features refer to liquid splash trajectories, atomization diffusion areas, or gas disturbance areas appearing in images, which can be extracted through motion detection and optical flow analysis.

[0108] Anomaly region expansion features refer to the trend of the area or outline of detected anomaly regions (such as liquid traces) continuously expanding over time, which can be quantified by comparing the area of ​​regions in consecutive frames.

[0109] The duration of anomalies refers to the number of frames or duration during which a certain type of anomaly occurs continuously in a series of frames, reflecting the stability and persistence of the anomaly.

[0110] Multiple image features are weighted and accumulated using the frame weights corresponding to each historical frame. This means that various features extracted from multiple consecutive frames are multiplied by the weights of their corresponding historical frames and then summed to obtain a feature accumulation value that integrates temporal information. The frame weights are negatively correlated with the time interval between the historical frames and the current time; that is, the closer the historical frame is to the current time, the greater the weight, and the older the historical frame, the smaller the weight. This design makes recent frames contribute more to the spatiotemporal evidence scoring while still retaining the reference value of historical frames.

[0111] After normalization, the weighted cumulative results are used to obtain a spatiotemporal evidence score, which comprehensively reflects the strength of evidence of abnormal appearance in the target area over a period of time. For example, in the past few frames, the area of ​​a pump seal gradually changes in size, and the surface liquid feature appears and expands continuously. The corresponding spatiotemporal evidence score increases accordingly. When the spatiotemporal evidence score exceeds a preset threshold, it is included in the inspection results as evidence of abnormal appearance. By extracting multiple types of image features from consecutive frames of on-site images and weighting them with a higher weight for more recent frames, gradual appearance anomalies that are difficult to judge in a single frame can form a higher spatiotemporal evidence score through multi-frame accumulation, thereby improving the reliability of detecting such appearance anomalies.

[0112] To further improve the accuracy of alarm triggering and avoid false alarms caused by occasional exceedances of a single indicator, some embodiments can also generate a comprehensive anomaly index by integrating multi-dimensional information and trigger alarms under composite conditions. Specifically, multiple consecutive frames of on-site images of the target inspection point are used to accumulate evidence of appearance anomalies to obtain a spatiotemporal evidence score; the inconsistency risk corresponding to the internal and external operation consistency score, the spatiotemporal evidence score, and the sensor anomaly intensity are weighted and summed according to their respective preset weights to obtain a comprehensive anomaly index; when the comprehensive anomaly index exceeds a preset trigger threshold, and the duration of exceeding the preset trigger threshold is not less than a preset minimum duration, an alarm message is generated or a review command is initiated.

[0113] The inconsistency risk corresponding to the internal and external operator consistency score refers to the quantitative value of the degree of inconsistency, represented by the inverse value of the internal and external operator consistency score (e.g., 1 minus the internal and external operator consistency score). The larger the value, the lower the degree of consistency between the internal and external operator data. Sensor anomaly intensity refers to the quantitative value of the degree of anomaly detected by other types of sensors (e.g., temperature sensors, vibration sensors, gas concentration sensors, etc.) other than the vision channel and the control system acquisition channel. It can reflect a wider range of anomaly information near the target inspection point.

[0114] The weighted summation, calculated by multiplying the inconsistency risk, spatiotemporal evidence score, and sensor anomaly intensity by their respective preset weights, yields the comprehensive anomaly index. Each weight can be set based on location characteristics and sensor configuration. The preset trigger threshold is the lower limit for triggering an alarm based on the comprehensive anomaly index; the preset minimum duration is the shortest time the comprehensive anomaly index must exceed the preset trigger threshold, used to filter out occasional, brief exceedances.

[0115] Generating alarm information means issuing an abnormal prompt to the operator, which may include information such as the location number, the time of the abnormality, the visual reading, the online data of the control system, the deviation value, and the classification of the abnormality source; initiating a review instruction means issuing an instruction to business systems such as the dual prevention system, intelligent inspection system, or central control interface to trigger the manual review process so that subsequent hazard investigation records or inspection records can be generated.

[0116] By weighting and integrating three types of information—internal and external operational inconsistency risk, spatiotemporal evidence scoring, and sensor anomaly intensity—into a comprehensive anomaly index, and using exceeding a preset trigger threshold and a duration not less than a preset minimum duration as dual triggering conditions, alarm triggering can integrate multi-dimensional anomaly signals and filter occasional short-term fluctuations, thereby improving alarm accuracy and reducing false alarms.

[0117] In summary, the external inspection method provided in this application simultaneously incorporates image quality evaluation results into the conversion process from state features to on-site visual readings and the fusion process of deviation features with inspection results. This ensures that image quality constrains the inspection conclusions in both key stages. By extracting quality feature factors from multiple dimensions such as sharpness, target stability, shooting angle effectiveness, occlusion degree, and illumination interference degree, and weighting positive and negative factors separately, the image quality evaluation results more comprehensively reflect various quality conditions affecting recognition performance, providing a more accurate quality reference for the constraint mechanism. Furthermore, by introducing image quality gating and active resampling mechanisms before the recognition process, active intervention methods such as gimbal adjustment, zoom, or supplemental lighting are used to proactively improve acquisition conditions when image quality is insufficient, further reducing the possibility of low-quality images entering subsequent processes. These three elements together constitute a quality control system centered on image quality evaluation results, ensuring the reliability of visual reading generation and deviation comparison. By weighting and fusing the historical sampled value sequence of the control system within a time window with a rule that the closer the sample, the greater the weight, and combining unit conversion and range mapping, the online data of the control system is made consistent with the field visual readings in three dimensions: time, unit, and range. This eliminates the impact of sampling time difference and dimensional differences on deviation comparison. On this basis, a normalized deviation is calculated by introducing an allowable error band that includes an image quality penalty term. This unifies the deviations of instruments with different ranges and accuracies into comparable normalized values, and the allowable error band is dynamically expanded as the image quality decreases, further reducing false alarms when there are fluctuations in image quality. The historical stability factor is introduced into the calculation of the internal and external operation consistency score, so that the score comprehensively reflects the information of the current deviation, image quality, and historical stability of the location. Occasional fluctuations are filtered out by using a continuous value below a threshold as a judgment condition, which improves the robustness of the judgment conclusion. Based on the judgment inconsistency, the sources of anomalies are classified so that the inspection results can provide specific judgments on the sources of anomalies, supporting subsequent targeted handling. By extracting multiple types of appearance anomaly features from consecutive multi-frame images and weighting them with frames that are closer in weight, gradual appearance anomalies can be accumulated over multiple frames to form a reliable spatiotemporal evidence score, compensating for the shortcomings of single-frame detection for such anomalies. Based on the consistency score and spatiotemporal evidence score, three types of information—inconsistency risk between internal and external operations, spatiotemporal evidence score, and sensor anomaly intensity—are weighted and fused into a comprehensive anomaly index. An alarm is triggered under the dual conditions of exceeding a threshold and meeting the required duration, further integrating multi-dimensional information and filtering out occasional fluctuations, making alarm triggering more accurate. This application's embodiment can achieve reliable external operator inspection judgment in complex industrial scenarios where image quality fluctuates, control system sampling has time differences, and anomaly types are diverse.

[0118] This embodiment provides an external operation inspection system. Figure 3 This is a schematic diagram of the structure of the external operation and inspection system according to an embodiment of this application, as shown below. Figure 3 As shown, the system includes a front-end perception layer 301, an image quality control layer 302, a video intelligent recognition layer 303, a control system time window alignment layer 304, and a multi-source mutual verification layer 305.

[0119] The front-end perception layer 301 is used to collect on-site images of the target inspection points.

[0120] The front-end perception layer 301 is the data source layer of the external operation inspection system, responsible for acquiring images of target inspection points in the industrial field and providing raw image data for subsequent layers. The front-end perception layer 301 may include explosion-proof cameras, PTZ cameras, or other types of industrial image acquisition equipment deployed near the target inspection points, as well as supporting field communication devices. The PTZ camera has horizontal rotation and pitch adjustment capabilities and can rotate to a preset position in response to control commands to align with the specific object to be inspected at the target inspection point.

[0121] Target inspection points refer to specific locations in an industrial site that are pre-configured and require periodic status monitoring, such as pressure gauge installation locations, pipe interface areas, or level gauge locations. On-site images refer to image data or video frames captured by image acquisition equipment that reflect the current real-time status of the point. Acquisition refers to the front-end sensing layer 301 responding to acquisition commands from the upper-layer system (or automatically triggered according to a preset cycle), controlling the image acquisition equipment to capture images of the target inspection points, and transmitting the acquired on-site images to the subsequent processing layer. Furthermore, the front-end sensing layer 301 may also include supplementary lighting devices to provide auxiliary illumination when lighting conditions are insufficient, thereby improving image acquisition quality; it may also include auxiliary sensing devices such as temperature and vibration sensors to collect supplementary on-site status information for use by the multi-source mutual verification layer 305.

[0122] Image quality control layer 302 is used to extract quality feature factors of the scene image and obtain the image quality evaluation result of the scene image.

[0123] The image quality control layer 302 is responsible for evaluating the quality of the on-site images acquired by the front-end perception layer 301, providing image quality evaluation results for subsequent layers to support each processing stage in perceiving and constraining the image quality status. The image quality control layer 302 can be an image quality evaluation function module in a processing server or edge computing node, analyzing the on-site images through image processing algorithms.

[0124] Quality feature factors refer to a set of features that can be quantified and extracted from an image to reflect its quality. These features can be quantified from multiple dimensions, such as sharpness, target stability, shooting angle effectiveness, occlusion level, and illumination interference. The image quality evaluation result is an overall image quality evaluation value calculated based on the quality feature factors, for example, a value normalized to the range of 0 to 1, with values ​​closer to 1 indicating better image quality. Extraction refers to the image quality control layer 302 using image processing algorithms to quantify each quality dimension. For example, it measures sharpness by calculating the high-frequency energy of the target area and measures the degree of occlusion by analyzing the occlusion ratio.

[0125] The image quality control layer 302 can further send a re-sampling command to the front-end perception layer 301 when the image quality does not meet the requirements, based on the comparison between the image quality evaluation result and the preset image quality threshold. This triggers re-sampling operations such as gimbal adjustment, zooming, or supplemental lighting until the quality of the newly acquired image meets the requirements. Then, the qualified image and the corresponding image quality evaluation result are transmitted to the video intelligent recognition layer 303 and the multi-source mutual verification layer 305 for use.

[0126] The video intelligent recognition layer 303 is used to extract the state features of the target object from the on-site image, and combine the image quality evaluation results to convert the state features into on-site visual readings of the target object.

[0127] The video intelligent recognition layer 303 is responsible for identifying target objects in qualified on-site images, generating on-site visual readings with image quality constraints, and transmitting the readings and their associated image quality information to the multi-source mutual verification layer 305. The video intelligent recognition layer 303 can be an image recognition function module in a processing server or edge computing node, integrating recognition algorithms for different instrument types (such as pointer instruments, digital display instruments, magnetic float level gauges, etc.) and different anomaly types (such as liquid accumulation, splashing, crystallization, etc.).

[0128] The target object refers to the specific object at the target inspection point that needs to be identified in terms of its status. This can be various industrial instruments (such as pressure gauges, level gauges, flow meters, etc.), equipment components (such as pump seals, valves, etc.), or specific areas (such as ground areas, pipe joints, etc.). Status features refer to image features extracted from the target object area of ​​the on-site image that reflect the current status of the target, such as pointer angle, character shape, or liquid column height. Extraction refers to analyzing the target object area using methods such as target detection and feature encoding to quantify the status features. On-site visual readings refer to quantified values ​​with actual physical meaning obtained after converting the status features, such as pressure readings or liquid level heights. Combining image quality evaluation results means that during the conversion process from status features to on-site visual readings, the image quality evaluation results output by the image quality control layer 302 are used as constraints in the conversion calculation. This allows the image quality status to influence the confidence level of the final visual reading, thus providing quality-aware reading input for the subsequent fusion calculation of the multi-source mutual verification layer 305.

[0129] The control system time window alignment layer 304 is used to acquire online control system data corresponding to the target inspection point.

[0130] The control system time window alignment layer 304 is responsible for acquiring online process parameter data corresponding to the target inspection point from the control system, and performing time alignment, unit conversion, and range mapping on it to eliminate time differences and dimensional differences between the data and the on-site visual readings. This provides the multi-source mutual verification layer 305 with online control system data that can be directly compared. The control system time window alignment layer 304 can read historical sampled values ​​of specified process variables in the target inspection point configuration file through a pre-established data interface with the distributed control system (DCS) or safety instrumented system (SIS).

[0131] The acquisition process involves the control system's time window alignment layer 304 using the acquisition time of the field image as a benchmark. Within a preset time window, it reads the historical sampling value sequence of the control system and performs weighted fusion according to a preset weighting rule that the closer the data is, the greater its weight. Then, it performs unit conversion and range mapping to finally obtain the time window aligned online data of the control system. By using the rule of giving greater weight to closer data, the fusion result is closer to the true value at the image acquisition time, thereby reducing time misalignment caused by sampling time differences or sensor filtering lag. Through unit conversion and range mapping, the online data of the control system is kept consistent with the field visual readings in terms of dimensionality, providing a reliable reference benchmark for comparison by the multi-source mutual verification layer 305.

[0132] The multi-source mutual verification layer 305 is used to compare the on-site visual readings with the online data of the control system to obtain the deviation characteristics, and then fuse the deviation characteristics with the image quality evaluation results to generate the inspection results of the target inspection points.

[0133] The multi-source mutual verification layer 305 is the core judgment layer of the external operation inspection system. It integrates the on-site visual readings output by the video intelligent recognition layer 303, the online data of the control system output by the control system time window alignment layer 304, and the image quality evaluation results output by the image quality control layer 302. Through deviation comparison and multi-source information fusion, it generates the inspection results of the target inspection point.

[0134] The comparison process involves the multi-source verification layer 305 calculating the numerical differences between on-site visual readings and online data from the control system to obtain a deviation feature reflecting the degree of agreement between the two data sources. The deviation feature is a quantitative expression of the difference between the on-site visual readings and the online data from the control system; it can be the absolute value of the difference or its normalized form. Fusion involves comprehensively calculating the deviation feature with the image quality evaluation results, incorporating image quality into the interpretation of the deviation feature. For example, the multi-source verification layer 305 can introduce an allowable error that includes an image quality penalty term to calculate the normalized deviation, allowing the allowable error band to dynamically expand as image quality decreases, reducing false alarms when image quality fluctuates. Generation involves the multi-source verification layer 305 outputting an inspection judgment of the current status of the target inspection point based on the fusion calculation results. This judgment includes categories such as normal, abnormal, or pending review, and includes supporting evidence information such as on-site visual readings, online data from the control system, deviation values, and image quality evaluation results.

[0135] The multi-source mutual verification layer 305 can further combine the historical stability factors of the target inspection points and the trend characteristics of the control system data changes to classify and judge the sources of inconsistencies between internal and external operation data, so that the inspection results can provide specific abnormality source categories; it can also combine the spatiotemporal evidence scoring of continuous multi-frame field images to perform multi-frame cumulative detection of gradual appearance anomalies such as running, leaking, and dripping; and it can generate alarm information or initiate review instructions when the comprehensive anomaly index continuously exceeds the preset trigger threshold, so as to drive the subsequent closed-loop business process such as dual prevention recording, intelligent inspection recording and central control interface alarm.

[0136] By fusing on-site visual readings, online data from the control system, and image quality evaluation results at the multi-source mutual verification layer 305, rather than relying solely on a single data source, the inspection results can comprehensively reflect information from multiple data channels, improving the reliability of inspection conclusions and enabling credible judgment of the status of target inspection points in industrial sites.

[0137] To better illustrate the external operation inspection method and system of this application, a preferred embodiment will be provided below. This embodiment is intended to describe the implementation process of this application in detail, but is not intended to limit the scope of protection of this application.

[0138] This embodiment uses an unmanned field inspection scenario in a major hazard source area of ​​a hazardous chemical enterprise as a specific example for illustration. The reasons for choosing this scenario are: firstly, the types of inspection objects in this scenario are diverse, covering various instruments such as pressure gauges, level gauges, and flow meters, as well as various structural components such as pump seals, electrolytic cells, and pipelines; secondly, the on-site environment is complex, with factors such as steam mist, strong reflections, and high-temperature and high-pressure equipment significantly affecting image quality; and thirdly, safety management has strict requirements for verifying the consistency between on-site and central control room data. These characteristics enable this scenario to fully demonstrate the various technical features of the technical solution in this embodiment in complex industrial sites. It should be understood that the technical solution provided in this embodiment is not limited to major hazard source areas of hazardous chemical enterprises, but is also applicable to the unmanned field inspection needs of other industrial sites. Furthermore, this embodiment covers both the field inspection method and the system, which will be comprehensively explained below from four aspects: system architecture, workflow, key technology implementation, and specific scenario examples.

[0139] In hazardous chemical enterprises, areas such as major hazard source areas, reaction vessel areas, pipeline areas, storage tank areas, loading and unloading areas, electrolytic cell areas, and liquid chlorine tank areas typically contain flammable, explosive, toxic, harmful, or high-temperature and high-pressure media, making them the areas with the highest concentration of safety risks. External operator inspections of these areas not only require high frequency but also demand consistency between field instrument readings and online data from the central control room's DCS / SIS. Significant discrepancies between the two may indicate instrument malfunctions, sensor signal drift, abnormal process fluctuations, or equipment leaks, necessitating timely investigation and handling.

[0140] Traditional field inspection methods rely on personnel entering the site periodically to visually check the readings of instruments such as pressure gauges, flow meters, and level gauges, manually compare them with DCS data in the central control room, and record the equipment status. This method exposes inspection personnel to high-risk environments frequently, making it difficult to guarantee the quality of duty at night or in inclement weather conditions. It also suffers from inaccurate reading records and delayed anomaly detection. Existing unmanned video recognition solutions use cameras and image recognition algorithms to collect and identify on-site instrument readings and compare them with DCS data. While this reduces the frequency of personnel entering high-risk areas to some extent, image quality fluctuations under complex conditions such as light interference, steam obstruction, and strong reflections lead to unstable reliability of the recognition results. The sampling time difference and dimensional differences between DCS data and video recognition make simple difference comparisons prone to false alarms. Furthermore, single-frame image recognition is insufficient for reliable judgments regarding gradual appearance anomalies such as leaks and spills.

[0141] To address the above situation, this embodiment provides an unmanned inspection system based on multi-source mutual verification. This system introduces a multi-source mutual verification trusted verification mechanism, which integrates visual recognition readings, image quality evaluation results, DCS / SIS online data, sensor trends, historical stability factors, spatiotemporal leakage evidence, business task status, and manual review feedback into the judgment system, forming a trusted judgment chain for unmanned external operation in high-risk sites.

[0142] This system includes a front-end perception layer, an image quality control layer, a video intelligent recognition layer, a DCS time window alignment layer, a multi-source mutual verification layer, an anomaly classification layer, a business closed-loop layer, and a feedback learning layer. The functions of each layer are as follows.

[0143] The front-end sensing layer includes explosion-proof cameras, PTZ cameras (i.e., cameras with horizontal rotation, vertical tilt, and zoom functions), temperature and vibration sensors, supplementary lighting devices, and field communication equipment. The explosion-proof cameras and PTZ cameras acquire images of the field instruments and target areas at preset positions; the temperature and vibration sensors are installed on rotating equipment to collect temperature and vibration data during equipment operation; the supplementary lighting devices provide auxiliary illumination when lighting conditions are insufficient; and the field communication equipment is responsible for transmitting the acquired data to the back-end processing system.

[0144] The image quality control layer evaluates the quality of the images acquired by the front end, checking the image clarity, target occlusion, shooting angle effectiveness, and lighting interference, and outputs the image quality evaluation results. When the image quality does not meet the preset requirements, it sends a control command to the front-end perception layer, triggering active improvement operations such as gimbal fine-tuning, zooming, or supplemental lighting before re-acquiring the image.

[0145] The video intelligent recognition layer performs target recognition on images that have passed quality inspection, including: recognizing readings from various instruments such as pressure gauges, flow meters, level gauges, pH meters (hydrogen ion concentration measuring instruments), densitometers, U-tube manometers, and magnetic float level gauges; recognizing leaks, drips, flow interruptions, crystallization, and abnormal image changes in areas such as the mechanical seal area of ​​circulating pumps, the ground below electrolytic cells, densely piped areas, transparent sections of overflow pipes, and feed pipe areas; and outputting information such as reading values, units, recognition area label boxes, screenshots, timestamps, recognition credibility, and abnormal evidence.

[0146] The DCS time window alignment layer reads historical sampling values ​​of online process parameters corresponding to the target inspection point through the data interface with DCS (Distributed Control System) / SIS (Safety Instrumented System), performs time window alignment, unit conversion, range mapping, and process fluctuation tolerance correction, and generates aligned reference values ​​that can be directly compared with visual readings.

[0147] The multi-source mutual verification layer integrates visual readings, image quality evaluation results, DCS alignment reference values, sensor status, historical stability factors, and spatiotemporal leakage evidence to calculate the internal and external operator consistency score and output the reliable verification conclusion of internal and external operator data.

[0148] When the anomaly classification layer detects inconsistencies between internal and external operating data or on-site anomalies, it comprehensively considers multiple features to classify and judge the source of the anomaly, and outputs the most likely anomaly source category (such as actual process anomaly, on-site instrument malfunction, DCS acquisition drift, visual misidentification, on-site leakage anomaly, etc.).

[0149] The business closed-loop layer synchronizes the verification conclusions and anomaly classification results to the dual prevention system, intelligent inspection system and DCS alarm interface, automatically generating hidden danger investigation records, inspection records or review alarms, forming a complete business closed loop for anomaly discovery and handling.

[0150] The feedback learning layer collects the results of manual review and writes them back to the corresponding point's recognition threshold, image quality requirements, and inspection strategy configuration, so that the system can continuously optimize the point parameter settings as it runs.

[0151] The execution flow of the unmanned external inspection system based on multi-source mutual verification in this embodiment is executed in the following order: First, before the system is put into operation, the platform completes the establishment of the inspection point database, and enters the camera number, preset position number, detection object type, algorithm type, DCS indicator code, allowable error range, inspection cycle and business system task number for each target inspection point, and establishes a multi-dimensional mapping relationship between on-site objects, video points, DCS indicators and business tasks.

[0152] Once the task is triggered, the system sends a data acquisition command to the front-end perception layer, where explosion-proof cameras, PTZ cameras, and temperature and vibration sensors collect on-site images and sensor data.

[0153] Subsequently, the image quality control layer calculates the quality evaluation result of the acquired on-site images. When the quality evaluation result is lower than a preset threshold, the system triggers operations such as supplemental lighting, zooming, or gimbal fine-tuning until the image quality meets the requirements.

[0154] Once the image quality meets the requirements, the video intelligent recognition layer identifies instrument readings and on-site appearance anomalies, outputting visual recognition results. Simultaneously, the DCS time window alignment layer reads and processes the corresponding DCS / SIS online data.

[0155] Subsequently, the multi-source mutual verification layer comprehensively compares the two sets of data, taking into account image credibility, DCS deviation, sensor trends, historical stability, and spatiotemporal leakage evidence, to generate a credible verification conclusion.

[0156] When an anomaly is detected, the anomaly classification layer further determines the category of the anomaly source; the system saves the video before and after the anomaly, keyframes, recognition boxes, reading values, DCS values ​​and deviation records, forming a video review record; after review and confirmation, the business closed-loop layer synchronizes the conclusion to each business system; finally, the feedback learning layer writes the manual review results back to the model.

[0157] The following is a detailed explanation of the execution process: This system establishes a multi-element binding file for each target inspection point, containing multiple related elements, to achieve unified management of on-site objects, video channels, recognition algorithms, DCS data, and business tasks. For the i-th inspection point, its multi-element binding relationship can be represented as: ; in, Indicates the first A collection of multiple bound objects for each inspection point; Indicates the inspection point number; This indicates the objects to be inspected on-site, such as pressure gauges, flow meters, level gauges, valves, pipelines, pump seals, local areas of electrolytic cells, or level gauges in liquid chlorine tanks. Indicates the camera, video channel, or pan / tilt preset position associated with this location; This indicates the image recognition region, algorithm interest region, or target detection box corresponding to this location; This indicates the DCS / SIS indicators, central control variables, or online process data corresponding to that point. This indicates the dual prevention task, intelligent inspection task, or other business system task associated with this location; This indicates the verification rules, alarm thresholds, tolerance ranges, manual verification strategies, or closed-loop handling rules for this location; parentheses indicate that multiple related objects are combined into an ordered tuple.

[0158] By establishing the aforementioned multi-dimensional binding files for each inspection point, the system can uniformly manage the mapping relationships between on-site entities, video channels, recognition algorithms, DCS variables, and business tasks on a point-by-point basis.

[0159] Before entering the recognition and comparison process, the system calculates the image quality evaluation results of the acquired images. : ; in, Indicates the first Each inspection point is at a constant time Image quality gate value; Indicates the inspection point number; Indicates the sampling or identification time; This represents a Sigmoid-type nonlinear normalization function used to compress quality assessment results to the range of 0 to 1. Indicates the sharpness or focus of an image; A reverse evaluation indicating the stability of the target or the degree of jitter in consecutive frames; Indicates the validity of the shooting angle, the readable angle of the instrument, or the degree of target orientation matching; This indicates the degree of occlusion; a larger value means the target is more severely occluded. Indicates the degree of glare, reflection, overexposure due to fill light, or light interference; to These are the weighting coefficients for each quality factor; This indicates the minimum image quality threshold allowed for formal identification and data comparison. If The system triggers gimbal fine-tuning, zooming, supplemental lighting, or secondary acquisition, instead of directly outputting the recognition conclusion.

[0160] Once the image quality meets the requirements, the video intelligent recognition layer performs reading recognition on the target object in the on-site image, and combines the image algorithm output, instrument type parameters, scale template, and image quality evaluation results to generate the final visual reading: ; in, Indicates the first Each inspection point is at a constant time On-site visual readings obtained from video recognition, where superscript Represents visual data; This indicates that the point is at time [time]. The captured images or video frames from the scene; This indicates the visual recognition algorithm operator applicable to this instrument or target type; This indicates the scale template, range parameters, unit parameters, or calibration parameters of the instrument. This represents the image quality gate value; Indicates the first This function converts image recognition output, calibration templates, and quality-gated results into final field readings. The formula generates reliable visual readings with image quality constraints.

[0161] Considering the sampling delay and filtering lag between DCS / SIS data and video recognition, the system does not directly compare single-point values, but rather compares them within a time window. Internally calculated aligned DCS reference value: ; in, Indicates the first Each inspection point is at a constant time DCS / SIS reference values ​​after time window alignment and range correction, where the superscript... This represents data from the DCS / SIS side; Indicated by time Centered on or by time DCS time alignment window with endpoint; Indicates the sampling time within the time window; Indicates the first Each point at the sampling time The original online values ​​of DCS / SIS; Indicates the sampling time Relative to the current time The time weighting is generally such that the closer to the current moment, the greater the weight. Indicates the first Unit conversion, range mapping, filter correction, or signal hysteresis compensation function for each point; summation symbol for time window. Multiple DCS sample values ​​are weighted and fused. This formula is used to avoid misjudgments caused by sampling delay, filtering lag, or inconsistent unit range.

[0162] After obtaining the visual readings and the DCS alignment reference value, the system calculates the normalized deviation between the two. : ; in, Indicates the first Each inspection point is at a constant time The normalization deviation between internal and external operations; Indicates on-site visual readings; Indicates the DCS / SIS alignment reference value; the absolute value sign indicates the magnitude of the deviation between the calculated visual reading and the DCS reference value; Indicates the first The fixed permissible error, meter accuracy error, or minimum tolerance of an instrument or indicator; This represents the proportional tolerance factor related to the range or reading size. This represents the image quality penalty coefficient; This represents the image quality gate value; The first term represents the penalty for insufficient image quality; the denominator represents the allowable error band for that point under the current quality conditions. This formula is used to convert instrument deviations of different ranges and accuracies into comparable normalized deviations.

[0163] Based on the normalized bias, the system further calculates the internal and external operator consistency score: ; in, Indicates the first Each inspection point is at a constant time The internal and external operation consistency score is such that the higher the score, the more consistent the visual reading is with the DCS / SIS reference value. Represents the natural exponential function; Indicates normalization bias; This represents the image quality gate value; This indicates that the point is at time [time]. The historical stability factor can be calculated from historical reading fluctuations, historical review pass rates, recent false alarm rates, or point stability. This indicates that the greater the deviation, the faster the consistency score decays. When If the data remains below the preset threshold, the system determines that there is a risk of continuous inconsistency between internal and external operation data at that location.

[0164] For anomalies such as leaks, mechanical seal leaks, pipeline leaks, electrolytic cell leaks, flow interruptions, and crystallization, the system does not rely on single-frame judgments, but instead introduces regional spatiotemporal evidence. : ; in, Indicates the first Each inspection point is at a constant time Scoring of spatiotemporal evidence of leaks or abnormal appearance; This represents a Sigmoid-type nonlinear normalization function; Indicates the historical frame index; Indicates the number of historical frames or the length of the time window involved in the calculation; Indicates the first The decay weight of each historical frame is usually greater the closer it is to the current time. to Indicates the weight of various types of evidence of visual abnormalities; Indicates the first Each point at time The characteristics of regional image changes relative to the background or preceding frames; Indicates water stains, liquid accumulation, or reflective features; This describes the characteristics of atomized, splashed, gas turbulent, or leaking plumes. This indicates an abnormal region expansion trend, liquid stain area growth, or outline expansion characteristics. This indicates the duration of the anomaly, the number of consecutive occurrences, or the degree of sustained stability. This formula is used to upgrade a single-frame visual anomaly to a multi-frame spatiotemporal evidence assessment.

[0165] When the system detects inconsistencies between internal and external data or on-site anomalies, it does not directly output a single alarm, but instead classifies the source of the anomaly: ; in, Indicates the first Each inspection point is at a constant time The most likely source of the anomaly was identified. This represents the set of candidate anomaly categories, which may include real process anomalies, field instrument jamming, DCS data acquisition drift, visual misidentification, field leakage anomalies, sensor anomalies, etc. Represents a set Any candidate anomaly category in; Represents the category given multiple features. Conditional probability of occurrence or classification confidence level; This indicates selecting the anomaly category that maximizes the conditional probability. Indicates normalization bias; Indicates the consistency score between internal and external operations; Indicates the spatiotemporal evidence score for leakage and spillage; This represents the image quality gate value; Indicates the historical stability factor; This indicates DCS / SIS trend characteristics, such as trend slope, fluctuation amplitude, drift trend, or sampling stability. This formula is used to further determine the source of anomalies after inconsistencies or field anomalies are detected.

[0166] For high-risk locations, the system dynamically calculates the priority of unmanned inspections based on the risk level, the time of the last inspection, the intensity of recent anomalies, and false alarm feedback. : ; in, Indicates the first Each inspection point is at a constant time Dynamic unmanned inspection priority; to This represents the weighting coefficient of each priority-affected item; This indicates the inherent risk level, major hazard source level, or high-risk process weight of the location. Indicates the current time; Indicates the first The time when the last inspection was completed at each location; This indicates the planned inspection cycle for that location; This indicates the degree of delay or time urgency since the last inspection; Indicates the consistency score between internal and external operations; Indicates the contribution of inconsistent risk; The score represents the spatiotemporal evidence of leaks, spills, or abnormal appearance. This indicates recently manually verified false alarm corrections, model instability penalties, or confidence reductions. This formula is used to dynamically adjust inspection priorities based on risk and recent anomalies.

[0167] Based on dynamic priority, the system calculates the next unattended inspection interval: ; in, Indicates the first Each inspection point is at a constant time The calculated interval for the next unmanned inspection; This represents a cutoff or limiting function, used to restrict the calculation results to an allowable range; This indicates the baseline or planned inspection cycle for that location. Indicates the priority of dynamic unmanned inspection; Indicates the minimum permissible unattended inspection interval; Indicates the maximum permissible unattended inspection interval; This indicates that the inspection interval will be automatically reduced when the inspection priority is increased; this formula is used to dynamically adjust the frequency of unmanned inspections based on the risk and abnormal status of the location.

[0168] When the unmanned system operates continuously and stably and the pass rate of manual review is high, the proportion of unmanned inspections at that location can be gradually increased; when the false alarm rate increases or the image quality at the location decreases, the proportion of unmanned inspections should be reduced and manual review should be increased.

[0169] The system ultimately calculates the comprehensive anomaly index. It is used to determine whether to trigger alarms, video review, and business system linkage: ; in, Indicates the first Each inspection point is at a constant time The comprehensive abnormality index; to This represents the weighting coefficient of each factor in the comprehensive anomaly index; Indicates the consistency score between internal and external operations; This indicates a risk of inconsistency between internal and external operations; The score represents the spatiotemporal evidence of leaks, spills, or abnormal appearance. Indicates the intensity of abnormalities in temperature, vibration, gas concentration, liquid level, or other sensors; This indicates the importance of business tasks, the task weight of locations in dual prevention or intelligent inspection, or the business priority of major hazard sources; Indicates the priority of dynamic unmanned inspection; This indicates the threshold for triggering the comprehensive anomaly index; This indicates the minimum time required for the anomaly index to be met after exceeding the threshold. This formula is used to determine whether to trigger alarms, video verification, dual prevention recording, intelligent inspection anomaly recording, or DCS verification alarms.

[0170] The following will provide a detailed explanation using specific application examples: Taking the circulating pump area as an example, the system pre-enters configuration information such as camera numbers, pan-tilt preset position numbers, detection object types (pressure gauge readings and mechanical seal water stains), DCS pressure variable codes, allowable errors, and inspection cycles in the point database for the circulating pump pressure gauge and mechanical seal area, and simultaneously configures the data reading path for the temperature and vibration sensors. After the task is triggered, the system drives the explosion-proof zoom PTZ camera to adjust to the pressure gauge preset position to acquire image frames, while simultaneously reading data from the temperature and vibration sensors. The system first calculates the image quality threshold value. If the image quality is insufficient due to steam obstruction, strong reflection, or angular offset, a command is sent to trigger pan-tilt fine-tuning and zoom compensation before re-acquiring the image. After the image quality passes, the system identifies the pressure gauge pointer position, combines the scale template and quality threshold value to generate a field visual pressure reading, and reads the pressure variable corresponding to the same circulating pump from the DCS. After time window alignment and range correction, the DCS reference pressure value is obtained, and the normalized deviation and consistency score are calculated. At the same time, the system extracts water stain features, liquid accumulation expansion trends, and abnormal duration from multiple consecutive frames of images in the area near the mechanical seal, and calculates the spatiotemporal evidence score. If the consistency score remains consistently low, the system triggers anomaly classification: if the DCS pressure trend is stable but the visual readings continue to deviate, it may indicate a faulty pressure gauge in the field; if the visual readings are stable but the DCS data shows a drifting trend, it may indicate DCS sensor drift; if both change and there is evidence of liquid accumulation and expansion in the mechanical seal area, it may indicate a real process anomaly accompanied by leakage. The classification conclusion, along with video clips, screenshots, and DCS data, is pushed to the video review entry point. After confirmation, it is synchronized to the dual prevention system to form a hazard investigation record.

[0171] Taking the electrolytic cell area as an example, the system configures three types of detection objects in the point database: U-shaped pressure gauge differential pressure, liquid flow status in the transparent section of the overflow pipe, and the appearance of the ground at the bottom of the electrolytic cell. For the U-shaped pressure gauge, the system identifies the height of the water column on both sides and calculates the differential pressure through range conversion. After aligning the visual differential pressure value with the corresponding differential pressure variable on the DCS side through a time window, it performs normalized deviation calculation and consistency scoring. For the transparent section of the overflow pipe, the system identifies whether there is liquid flow inside the pipe. When there is an abnormal flow interruption, it records the abnormal evidence and includes it in the spatiotemporal evidence chain accumulation. For the ground at the bottom of the electrolytic cell, the system performs time-series weighted accumulation of the regional image change characteristics, water stain accumulation characteristics, liquid stain area growth trend, and abnormal duration over multiple consecutive acquisition cycles to determine whether there is dripping, water stain accumulation, or crystallization. Multi-frame spatiotemporal evidence chain replaces single-frame judgment, effectively avoiding false alarms caused by steam mist or occasional noise in a single frame. When the comprehensive abnormal index meets the triggering conditions, it links the business system to form a closed-loop processing.

[0172] Taking a liquid chlorine tank area as an example, the system configures two sets of detection points for magnetic float level gauges (an instrument that uses a magnetic float to move up and down with the liquid surface, driving an external indicator bar to flip and display the liquid level) and tank area pressure gauges. For magnetic float level gauges, the system identifies the upper edge height of the yellow indicator area and converts it into a liquid level scale value through range conversion as the visual liquid level reading; for pressure gauges, the system identifies the pointer position to obtain the visual pressure reading. The two types of visual readings are aligned with the corresponding liquid level and pressure variables on the DCS side through a time window, and then normalized deviation and consistency scores are calculated respectively. If the visual liquid level reading and the DCS liquid level data consistently deviate beyond the allowable error range, the system triggers anomaly classification, combining DCS liquid level trend characteristics and historical stability factors for classification judgment: if the DCS liquid level trend is stable but the visual reading shows a step deviation, it may be due to magnetic float jamming; if the visual reading continuously tracks but the DCS data shows a drifting trend, it may be due to DCS liquid level sensor drift; if the visual and DCS data change synchronously, it may be due to actual process liquid level changes. The system generates an alarm for inconsistency between internal and external operating data, saves screenshots, identification boxes, DCS data, and review conclusions. After confirmation, it is synchronized to the dual prevention system to form a hidden danger investigation record, and the review conclusions are written back to the learning layer for continuous optimization.

[0173] In summary, this embodiment integrates visual recognition readings, image quality evaluation results, DCS / SIS online data, sensor anomaly intensity, historical stability factors, and spatiotemporal leakage evidence into a multi-source mutual verification judgment system. It dynamically adjusts the allowable error band through image quality penalty terms to reduce false alarms when image quality fluctuates. A continuous multi-frame spatiotemporal evidence accumulation mechanism enables reliable detection of gradual leakage anomalies that are difficult to judge from a single frame image. An anomaly source classification mechanism provides specific anomaly source judgments when inconsistencies are found between internal and external operational data, offering more valuable diagnostic information for on-site handling. A comprehensive anomaly index continuously exceeding threshold trigger conditions filters out occasional short-term fluctuations, improving alarm robustness. A manual review and feedback mechanism continuously optimizes system parameters as operation accumulates, gradually increasing the coverage and reliability of unmanned inspections.

[0174] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An external inspection method, characterized in that, The method includes: Acquire on-site images of the target inspection points; Extract the quality feature factors of the on-site image to obtain the image quality evaluation result of the on-site image; Extract the state features of the target object from the scene image, and combine them with the image quality evaluation results to convert the state features into a scene visual reading of the target object; Acquire online control system data corresponding to the target inspection point; wherein, the online control system data is process parameter data corresponding to the physical indicators monitored at the target inspection point, which is collected and stored in real time by the control system through sensors. The on-site visual readings are compared with the online data of the control system to obtain deviation characteristics. The deviation characteristics are then fused with the image quality evaluation results to generate the inspection results for the target inspection point. The step of fusing the deviation features with the image quality evaluation results to generate the inspection results for the target inspection point includes: The normalized deviation is obtained by taking the absolute value of the difference between the on-site visual reading and the online data of the control system as the numerator and the allowable error band including the image quality penalty term as the denominator; the image quality penalty term increases as the image quality evaluation result decreases; wherein, the allowable error band is the upper limit of the deviation that is judged to be within the normal tolerance range under the current conditions. Obtain the historical stability factor of the target inspection point; wherein, the historical stability factor is a quantitative value reflecting the degree of stability of the target inspection point in history; Based on the normalized deviation, a deviation decay term that monotonically decreases as the normalized deviation increases is calculated. The deviation decay term is then multiplied by the image quality evaluation result and the historical stability factor to obtain the internal and external operation consistency score. When the internal and external operation consistency score is continuously lower than the preset consistency threshold, it is determined that there is a continuous inconsistency between the internal and external operation data at the target inspection point. Based on the normalization deviation, the internal and external operation consistency score, the image quality evaluation result, the historical stability factor, and the trend characteristics of the control system data change, the category with the highest conditional probability is determined from multiple preset candidate anomaly categories as the current anomaly source of the target inspection point, and the determination result and the anomaly source are included in the inspection result.

2. The external inspection method according to claim 1, characterized in that, Prior to the step of extracting the state features of the target object from the scene image, the method further includes: The image quality evaluation result is compared with a preset image quality threshold. When the image quality evaluation result is lower than the preset image quality threshold, the on-site image of the target inspection point is re-acquired until the image quality evaluation result is not lower than the preset image quality threshold. The re-acquisition includes at least one of the following operations: gimbal adjustment, zoom, or supplemental lighting.

3. The external inspection method according to claim 1, characterized in that, The acquisition of online control system data corresponding to the target inspection point includes: Based on the acquisition time of the on-site images, the historical sampling value sequence of the control system corresponding to the target inspection point is obtained within a preset time window; According to a preset weighting rule, the historical sampled value sequence is weighted and fused, and unit conversion and range mapping are performed to obtain the online control system data after time window alignment; wherein, the preset weighting rule is that the closer each sampling time is to the acquisition time, the greater the weight.

4. The external inspection method according to claim 1, characterized in that, The method further includes: For multiple consecutive frames of on-site images of the target inspection point, multiple image features are extracted. The multiple image features include at least the regional image change features, liquid accumulation features, splash and gas disturbance features, abnormal area expansion features, and abnormal duration features. The multiple image features are weighted and accumulated using the frame weights corresponding to each historical frame to obtain a spatiotemporal evidence score; the frame weights are negatively correlated with the time interval between the historical frame and the current time. The spatiotemporal evidence score is incorporated into the generation of the inspection results.

5. The external inspection method according to claim 1, characterized in that, The method further includes: The spatiotemporal evidence score is obtained by accumulating visual anomaly evidence from multiple consecutive frames of on-site images of the target inspection point. The inconsistency risk corresponding to the internal and external operation consistency score, the spatiotemporal evidence score, and the sensor anomaly intensity are weighted and summed according to their respective preset weights to obtain the comprehensive anomaly index. When the comprehensive anomaly index exceeds a preset trigger threshold, and the duration of exceeding the preset trigger threshold is not less than a preset minimum duration, an alarm message is generated or a review instruction is initiated.

6. The method according to claim 1, characterized in that, The step of extracting quality feature factors from the scene image to obtain the image quality evaluation result of the scene image includes: The sharpness, target stability, and shooting angle effectiveness of the scene image are extracted, and the sharpness, target stability, and shooting angle effectiveness are weighted and summed according to a preset positive weighting coefficient to obtain a positive weighting result; The degree of occlusion and the degree of light interference in the scene image are extracted, and the degree of occlusion and the degree of light interference are weighted and summed according to a preset negative weighting coefficient to obtain a negative weighted result; The image quality evaluation result is obtained by subtracting the positive weighting result from the negative weighting result and then normalizing the result.

7. An external operation inspection system, characterized in that, The system includes: The front-end perception layer is used to collect on-site images of the target inspection points; An image quality control layer is used to extract quality feature factors of the on-site image and obtain the image quality evaluation result of the on-site image. The video intelligent recognition layer is used to extract the state features of the target object from the scene image, and combine the image quality evaluation results to convert the state features into the scene visual readings of the target object; The control system time window alignment layer is used to acquire online control system data corresponding to the target inspection point; wherein, the online control system data is process parameter data corresponding to the physical indicators monitored by the target inspection point, which is collected and stored in real time by the control system through sensors. A multi-source mutual verification layer is used to compare the on-site visual readings with the online data of the control system to obtain deviation features, and to fuse the deviation features with the image quality evaluation results to generate the inspection results of the target inspection point. The multi-source mutual verification layer is used for: The normalized deviation is obtained by taking the absolute value of the difference between the on-site visual reading and the online data of the control system as the numerator and the allowable error band including the image quality penalty term as the denominator; the image quality penalty term increases as the image quality evaluation result decreases; wherein, the allowable error band is the upper limit of the deviation that is judged to be within the normal tolerance range under the current conditions. Obtain the historical stability factor of the target inspection point; wherein, the historical stability factor is a quantitative value reflecting the degree of stability of the target inspection point in history; Based on the normalized deviation, a deviation decay term that monotonically decreases as the normalized deviation increases is calculated. The deviation decay term is then multiplied by the image quality evaluation result and the historical stability factor to obtain the internal and external operation consistency score. When the internal and external operation consistency score is continuously lower than the preset consistency threshold, it is determined that there is a continuous inconsistency between the internal and external operation data at the target inspection point. Based on the normalization deviation, the internal and external operation consistency score, the image quality evaluation result, the historical stability factor, and the trend characteristics of the control system data change, the category with the highest conditional probability is determined from multiple preset candidate anomaly categories as the current anomaly source of the target inspection point, and the determination result and the anomaly source are included in the inspection result.

Images