Data processing method and system for patrolling device

Abstract

The application discloses a data processing method and system of an inspection device. The method comprises the following steps: obtaining a plurality of sensing data obtained by a plurality of sensors sensing an inspection scene; cross- verifying the plurality of sensing data to obtain a verification result; in response to the verification result indicating that the plurality of sensing data are abnormally matched, determining at least one target sensor from the plurality of sensors; updating confidence information of the inspection device based on an abnormal type to which an abnormal state corresponding to the target sensor belongs; and in response to the updated confidence information being lower than a confidence information threshold, performing a repair operation on the target sensor. The application solves the technical problem of low inspection efficiency of the inspection device.

Description

Technical Field

[0001] This application relates to the field of inspection equipment technology, and more specifically, to a data processing method and system for inspection equipment. Background Technology

[0002] Currently, anomalies in different inspection equipment at the inspection site can only be detected when the aforementioned inspection equipment is completely destroyed.

[0003] However, before the aforementioned inspection equipment was completely destroyed, it had already exhibited varying degrees of abnormalities, leading to a degradation in its inspection capabilities. Furthermore, the different types of sensor failures prevent the use of a uniform detection method, meaning that sensors can only passively accept problems after they occur, without proactive repair. Therefore, the technical problem of low inspection efficiency persists.

[0004] There is currently no effective solution to the aforementioned technical problems. Summary of the Invention

[0005] This application provides a data processing method and system for inspection equipment, which at least solves the technical problem of low inspection efficiency of inspection equipment.

[0006] According to one aspect of the embodiments of this application, a data processing method for an inspection device is provided. The inspection device is equipped with multiple sensors and operates in an inspection scenario. The method may include: acquiring multiple sensing data obtained by the multiple sensors sensing the inspection scenario, wherein different sensors correspond to different sensing data; performing cross-validation on the multiple sensing data to obtain a validation result; in response to the validation result indicating an anomaly in the matching between the multiple sensing data, determining at least one target sensor from the multiple sensors, wherein the sensing performance of the target sensor is in an abnormal state; updating the confidence information of the inspection device based on the anomaly type to which the abnormal state corresponding to the target sensor belongs, wherein the confidence information is used to represent the reliability of the inspection device in performing inspections in the inspection scenario; and in response to the updated confidence information being lower than a confidence information threshold, performing a repair operation on the target sensor, wherein the sensing performance of the repaired target sensor is in a normal state.

[0007] Optionally, the confidence information includes the confidence level of the 3D map of the inspection scene. The confidence level is used to represent the reliability of the inspection equipment in different spatial areas of the inspection scene. The confidence information threshold includes the confidence threshold corresponding to the confidence level. In response to the updated confidence information being lower than the confidence information threshold, a repair operation is performed on the target sensor, including: in response to the updated confidence level being lower than the confidence threshold, determining the degradation level of the target sensor, wherein the degradation level is used to represent the degree of degradation of the target sensor; determining a repair strategy matching the degradation level, wherein the repair strategy is used to represent the rules for performing repair operations on the target sensor, different degradation levels correspond to different repair strategies, and the repair strategy corresponding to the degradation level is cross-modal information reconstruction based on a physical model, wherein the physical model is used to characterize the real sensing data that the target sensor needs to sense; and performing a repair operation on the target sensor according to the repair strategy, wherein the modified target sensor is used to sense the real sensing data.

[0008] Optionally, cross-validation is performed on multiple sensing data to obtain validation results, including: performing consistency detection on multiple sensing data in a target map to obtain validation results, wherein the target map consists of multiple edges, which are used to represent the matching relationship between two sensors among multiple sensors. Each edge consists of two nodes in the target map, which correspond to two sensors. The matching relationship is used to represent the matching relationship based on physical knowledge and / or domain knowledge of the field to which the inspection equipment belongs.

[0009] Optionally, the target map can be updated. Consistency checks are performed on the target map for multiple sensing data to obtain verification results. This includes: initializing the weights of edges in the target map, where the weights represent the correlation between the two sensors corresponding to the edges; evaluating the edges in the initialized target map using multiple sensing data to obtain evaluation results, where the evaluation results represent the matching degree between the sensing data of the two sensors corresponding to the edges; and determining the verification results using the evaluation results.

[0010] Optionally, the method further includes: compensating for the differences in sampling information and response delay information among multiple sensors; evaluating the edges in the initialized target graph using multiple sensing data to obtain evaluation results, including: evaluating the edges in the initialized target graph using multiple sensing data, the compensated differences in sampling information, and the compensated response delay information to obtain evaluation results.

[0011] Optionally, the various sensors include: a carbon monoxide sensor, a lidar sensor, and a carbon dioxide sensor. The matching relationships corresponding to the edges are used to represent the matching relationship between thermal imaging and visible light, the matching relationship between thermal imaging and the spatiotemporal information of the carbon monoxide sensor, the matching relationship between visible light and the depth parallax of the lidar sensor, and the matching relationship between the carbon dioxide sensor and the combustion information of thermal imaging in the inspection scenario.

[0012] Optionally, in response to the verification result indicating an anomaly in the matching among multiple sensing data, determining at least one target sensor from multiple sensors includes: in response to the verification result indicating an anomaly in the matching among multiple sensing data, performing a voting tracing process on multiple sensors to obtain the target sensor.

[0013] Optionally, the method further includes: storing confidence information according to the target resolution, wherein the target resolution is inversely proportional to the integrity of the spatial region in the inspection scenario.

[0014] Optionally, the method further includes: determining the repaired target sensor's sensing data as sensing data among multiple sensing data, and returning to the step of performing cross-validation on the multiple sensing data to obtain the validation result.

[0015] According to another aspect of the embodiments of this application, a data processing apparatus for inspection equipment is also provided. The apparatus may include: an acquisition unit for acquiring multiple sensing data obtained by multiple sensors sensing an inspection scene, wherein different sensors correspond to different sensing data; a verification unit for cross-verifying the multiple sensing data to obtain a verification result; a determination unit for determining at least one target sensor from the multiple sensors in response to the verification result indicating an anomaly in the matching between the multiple sensing data, wherein the sensing performance of the target sensor is in an abnormal state; an update unit for updating the confidence information of the inspection equipment based on the anomaly type to which the abnormal state corresponding to the target sensor belongs, wherein the confidence information is used to represent the reliability of the inspection equipment performing inspections in the inspection scene; and a repair unit for performing a repair operation on the target sensor in response to the updated confidence information being lower than a confidence information threshold, wherein the sensing performance of the repaired target sensor is in a normal state.

[0016] According to another aspect of the embodiments of this application, a data processing system for inspection equipment is also provided. The system may include: a verification engine for acquiring multiple sensing data obtained by multiple sensors sensing an inspection scene, wherein different sensors correspond to different sensing data; an information generator for cross-validating the multiple sensing data to obtain a verification result; in response to the verification result indicating an anomaly in the matching between the multiple sensing data, determining at least one target sensor from the multiple sensors, wherein the sensing performance of the target sensor is in an abnormal state; updating the confidence information of the inspection equipment based on the anomaly type to which the abnormal state corresponding to the target sensor belongs, wherein the confidence information is used to represent the degree of credibility of the inspection equipment performing inspections in the inspection scene; and a reconstructor for performing a repair operation on the target sensor in response to the updated confidence information being lower than a confidence information threshold, wherein the sensing performance of the repaired target sensor is in a normal state.

[0017] According to another aspect of the embodiments of this application, a processor is also provided. The processor is used to run a program, wherein the program executes the methods of the embodiments of this application during runtime.

[0018] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the method of the embodiments of this application when it runs.

[0019] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method of the embodiments of this application.

[0020] In this embodiment, by acquiring multiple sensing data from multiple sensors and cross-validating these data, an anomaly can be identified when matching anomalies occurs among the multiple sensing data. The target sensor exhibiting an abnormal sensing performance can then be determined. Confidence information is updated based on the anomaly type of the abnormal state. When the confidence information falls below a threshold, the target sensor is repaired. This achieves the detection and performance recovery of the sensor before it is completely destroyed. Furthermore, by utilizing the cross-validation mechanism between multiple sensors during collaborative sensing, the target sensor's anomaly is identified during the performance degradation stage rather than the complete destruction stage. Based on the anomaly type of the target sensor's abnormal state, the confidence information is dynamically updated to trigger repair operations, thereby restoring the target sensor's normal sensing performance. Because the above method utilizes cross-validation of multiple sensing data to replace direct reliance on the hardware state of a single sensor, the target sensor can be identified when the sensor only shows performance degradation and there are mismatches between multiple sensing data but it has not yet been completely destroyed. Therefore, it overcomes the obstacles in related technologies that can only detect anomalies when the inspection equipment is completely destroyed, and cannot detect that the inspection equipment has actually experienced different degrees of anomalies before it is completely destroyed, which leads to the degradation of the inspection equipment's ability to perform inspections. Thus, it solves the technical problem of low inspection efficiency of inspection equipment and achieves the technical effect of improving the inspection efficiency of inspection equipment. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a flowchart of a data processing method for an inspection device according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of another data processing method for inspection equipment according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of a four-legged inspection robot dog's self-healing system for perceiving integrity, according to an embodiment of this application.

[0025] Figure 4 This is a schematic diagram of a data processing system for an inspection device according to an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of a data processing device for an inspection equipment according to an embodiment of this application;

[0027] Figure 6This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] According to the embodiments of this application, an embodiment is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 1 This is a flowchart of a data processing method for an inspection device according to an embodiment of this application, such as... Figure 1 As shown, the method may include the following steps.

[0032] Step S102: Obtain various sensing data obtained by sensing the inspection scene using multiple sensors.

[0033] In the technical solution provided by step S102 of this application, different sensors correspond to different sensing data.

[0034] In this embodiment, the inspection scenario can be an industrial setting such as a chemical plant, oil and gas field, or substation. The sensing data can be data representing the attributes or state of objects obtained after inspecting them in the inspection scenario. The sensors mentioned include, but are not limited to, visible light cameras, thermal imaging pan-tilt units, lidar, inertial measurement units, and electrochemical gas sensors.

[0035] Optionally, the inspection scene can be sensed using the aforementioned multiple sensors, and the data collected by these sensors can be aligned and denoised to obtain various sensing data. These multiple sensing data can be mapped to a unified spatiotemporal coordinate system to achieve consistency. For example, the sensing data can be image frames, temperature field data, point cloud information, and gas concentration readings. In the inspection scene, image frames can be acquired using a visible light camera, temperature field data can be acquired using a thermal imaging pan-tilt unit, point cloud information can be acquired using a lidar, and gas concentration readings can be acquired using a gas sensor.

[0036] In this embodiment of the application, the above step S102 can obtain sensing data for multiple dimensions of the inspection scenario and ensure that the data between different sensors are consistent in time and space.

[0037] Step S104: Cross-validate multiple sensing data to obtain the validation results.

[0038] In the technical solution provided in step S104 of this application, after obtaining various sensing data obtained by sensing the inspection scene by multiple sensors, the various sensing data can be cross-validated to obtain the verification result.

[0039] In this embodiment, the verification result can be used to represent the matching degree between multiple sensing data, which can be used to represent the consistency of multiple sensing data, or whether there is a perception contradiction caused by sensor abnormality.

[0040] Optionally, this embodiment performs cross-validation on multiple sensing data, which can be achieved by constructing a cross-modal sensing consistency map. It can utilize the physical association between different sensors and domain knowledge to define consistency relationships, calculate the consistency score of each relation edge in the cross-modal sensing consistency map in real time, and detect the matching status of sensor data based on the consistency score.

[0041] The aforementioned relational edges can be used to represent the constraints and correlations established between different sensors based on physical laws or domain knowledge, such as the correspondence between thermal radiation and visible light characteristics, or the correlation between gas diffusion and temperature fields. The aforementioned consistency score can be used to represent the degree of matching or confidence level of different sensor data under a specific correlation at the current moment. The lower the consistency score, the greater the contradiction between the data, and the more likely there is sensor degradation or anomaly.

[0042] For example, a cross-modal sensing consistency map can first be established, including relational edges such as edge consistency between thermal imaging and visible light, spatiotemporal correlation between thermal imaging and gas sensors, and depth parallax consistency between visible light and lidar. This cross-modal sensing consistency map can be a directed weighted graph. The directed weighted graph can be used to represent a topology with different sensors as nodes and physical or domain consistency relationships between sensors as edges, for online maintenance and updating of mutual verification relationships between sensors.

[0043] Optionally, this embodiment employs a dynamic time warping algorithm to compensate for sampling rate differences and response delays between different sensors, ensuring alignment of sampling rate differences and response delays among different sensors. Subsequently, a sliding window is used to perform statistical analysis on the consistency score to detect abnormal fluctuations, and the reliability of each aligned sensor in a specific spatial region and time series is comprehensively evaluated to obtain verification results reflecting the degree of matching of multiple sensing data.

[0044] In this embodiment of the application, the above step S104 can be used to cross-verify the data contradictions caused by sensor degradation or environmental interference by utilizing the physical correlation between different sensors and the consistency of domain knowledge definitions.

[0045] Step S106: In response to the verification result indicating an anomaly in the matching between multiple sensing data, at least one target sensor is identified from the multiple sensors.

[0046] In the technical solution of step S106 of this application, after cross-validating multiple sensing data and obtaining the verification result, in response to the verification result indicating an anomaly in the matching between the multiple sensing data, at least one target sensor can be determined from the multiple sensors. The sensing performance of the target sensor is in an abnormal state.

[0047] In this embodiment, the aforementioned at least one target sensor can be used to represent at least one sensor identified during cross-validation as causing inconsistencies or exhibiting signs of degradation.

[0048] Optionally, in response to verification results indicating anomalies in the matching among various sensing data, a source tracing analysis is performed based on the topological structure of the anomaly's relational edges to identify the set of sensors causing the anomalies. By statistically analyzing the weights of different sensors in the anomaly's relational edges, the persistence and severity of sensor performance degradation are determined, thereby obtaining an evaluation result. The above evaluation result can be compared with a preset degradation detection threshold to filter out target sensors whose evaluation results are below the degradation detection threshold or whose degradation trend is significant.

[0049] For example, when the consistency score between thermal imaging and visible light drops sharply, the thermal imaging gimbal and visible light camera can be traced back to the thermal imaging gimbal and visible light camera. It was found that the thermal imaging continuously produced artifacts that did not match the edge of the visible light in multiple time windows, while the gas sensor readings were normal. Thus, it was determined that the thermal imaging was the target sensor that was interfered with by sunlight reflection.

[0050] In this embodiment of the application, the above step S106 can achieve early positioning before the sensor completely fails, and transform the vague anomaly into a specific sensor-level fault diagnosis.

[0051] Step S108: Update the confidence information of the inspection equipment based on the anomaly type of the abnormal state corresponding to the target sensor.

[0052] In the technical solution of step S108 of this application, after determining at least one target sensor from multiple sensors in response to the verification result indicating an anomaly in the matching among multiple sensing data, the confidence information of the inspection equipment can be updated based on the anomaly type to which the anomaly state corresponding to the target sensor belongs. The confidence information is used to represent the degree of reliability of the inspection equipment in the inspection scenario.

[0053] In this embodiment, the aforementioned anomaly types can be used to represent the degradation type or failure mode of the target sensor's sensing performance. For example, optical anomalies could include lens contamination, overexposure, or sunlight reflection interference; thermal anomalies could include temperature drift or temperature measurement distortion; gas-related anomalies could include zero-point drift or sensor poisoning; and lidar anomalies could include point cloud holes or signal attenuation. The aforementioned confidence information can be used to represent the reliability of the inspection equipment's sensing results at the current moment or within a specific spatial area.

[0054] In this embodiment, when updating the confidence information of the inspection equipment based on the anomaly type corresponding to the abnormal state of the target sensor, a mapping relationship between the anomaly type and the confidence information attenuation strategy can be established first. The confidence information attenuation strategy can represent a preset confidence information reduction rule for different sensor anomalies. This confidence information reduction rule defines the correspondence between the anomaly type and the numerical change of the confidence information. Based on the influence range of the anomaly type, the confidence information of the inspection equipment in the corresponding area is dynamically corrected.

[0055] For example, when the target sensor is determined to have slight zero-point drift, the weight of the target sensor can be gradually reduced using a linear function to retain some of the effective information of the target sensor; when the target sensor is determined to have severe optical obstruction or hardware failure, the confidence information of the target sensor can be marked as invalid.

[0056] In this embodiment of the application, the above step S108 can transform the local anomaly of the sensor into an overall reliability assessment of the detection device, thereby avoiding false alarms or missed detections caused by blindly trusting erroneous data.

[0057] Step S110: In response to the updated confidence information being lower than the confidence information threshold, a repair operation is performed on the target sensor.

[0058] In the technical solution of step S110 of this application, after updating the confidence information of the inspection equipment based on the anomaly type of the abnormal state corresponding to the target sensor, a repair operation can be performed on the target sensor in response to the updated confidence information being lower than the confidence information threshold. The sensing performance of the repaired target sensor is in a normal state.

[0059] In this embodiment, the confidence information threshold can be used to indicate whether the sensor has the minimum reliability standard required to support the inspection. The repair operation includes, but is not limited to, adjusting, enhancing, replacing, and reconstructing the sensor's sensing information.

[0060] Optionally, in response to the updated confidence information falling below the confidence information threshold, a corresponding reconstruction strategy is matched based on the anomaly type of the target sensor. If the target sensor is severely degraded, data from other normal sensors in the cross-modal sensing consistency map can be used in conjunction with a physical model to reconstruct cross-modal information and generate virtual sensing data to replace the abnormal data. If the target sensor is only slightly degraded, it can be calibrated based on historical benchmarks. Finally, the repaired data is input into the inspection equipment to restore the sensing performance of the target sensor to normal. For example, when the confidence information of a thermal imaging sensor falls below the confidence information threshold due to sunlight reflection, the true temperature of the pipe is calculated using the pipe geometry surface information extracted by the lidar and the emissivity of the stainless steel material, combined with ambient temperature data, through a thermal radiation reconstruction model. This reconstructed data is then used to replace the false high-temperature reading, thereby eliminating the risk of false alarms.

[0061] Steps S102 to S110 of this application involve acquiring multiple sensing data from multiple sensors and cross-validating these multiple sensing data. When an anomaly is found among the multiple sensing data, the target sensor whose sensing performance is in an abnormal state can be identified. The confidence information is updated according to the anomaly type to which the abnormal state belongs. When the confidence information is lower than the confidence information threshold, the target sensor is repaired. This achieves the detection and performance recovery of the abnormal state of the sensor before it is completely destroyed. In the process of collaborative sensing by multiple sensors, the cross-validation mechanism between multiple sensors is used to identify the abnormality of the target sensor in the performance degradation stage rather than the complete destruction stage. Based on the anomaly type to which the abnormal state of the target sensor belongs, the confidence information is dynamically updated to trigger the repair operation on the target sensor, thereby restoring the normal sensing performance of the target sensor. Because the above method utilizes cross-validation of multiple sensing data to replace direct reliance on the hardware state of a single sensor, the target sensor can be identified when the sensor only shows performance degradation and there are mismatches between multiple sensing data but it has not yet been completely destroyed. Therefore, it overcomes the obstacles in related technologies that can only detect anomalies when the inspection equipment is completely destroyed, and cannot detect that the inspection equipment has actually experienced different degrees of anomalies before it is completely destroyed, which leads to the degradation of the inspection equipment's ability to perform inspections. Thus, it solves the technical problem of low inspection efficiency of inspection equipment and achieves the technical effect of improving the inspection efficiency of inspection equipment.

[0062] The method described in this embodiment will be further described below.

[0063] As an optional embodiment, the confidence information includes the confidence level of the 3D map of the inspection scene. The confidence level is used to represent the reliability of the inspection equipment in different spatial areas of the inspection scene. The confidence information threshold includes the confidence threshold corresponding to the confidence level. In step S106, in response to the updated confidence information being lower than the confidence information threshold, a repair operation is performed on the target sensor, including: in response to the updated confidence level being lower than the confidence threshold, determining the degradation level of the target sensor, wherein the degradation level is used to represent the degree of degradation of the target sensor; determining a repair strategy matching the degradation level, wherein the repair strategy is used to represent the rules for performing repair operations on the target sensor, different degradation levels correspond to different repair strategies, and the repair strategy corresponding to the degradation level is cross-modal information reconstruction based on a physical model, wherein the physical model is used to characterize the real sensing data that the target sensor needs to sense; and performing a repair operation on the target sensor according to the repair strategy, wherein the modified target sensor is used to sense the real sensing data.

[0064] In this embodiment, the aforementioned 3D map can be a perception integrity map constructed within the inspection environment. The aforementioned degradation level can be used to represent the degree of degradation of the target sensor. The aforementioned repair strategy can be used to represent the rules for performing repair operations on the target sensor. The aforementioned physical model can be used to characterize the actual sensing data that the target sensor needs to sense; the aforementioned physical model may include, but is not limited to, a material emissivity mapping model, a steady-state thermal conduction model, or a diffusion model.

[0065] Optionally, in response to an updated confidence level falling below a confidence threshold, sensor performance degradation can be quantified and graded based on the degree of deviation from the threshold, the duration of the abnormal state, and the physical severity of the abnormality. For example, a grading assessment model can be established to determine multiple degradation levels. Level 1 degradation represents slight deviation, manifested as short-term fluctuations in the sensing data acquired by the target sensor, which can be recovered through simple calibration. Level 2 degradation represents moderate degradation, manifested as a continuous performance decline in the target sensor, requiring adjustment of the fusion weights. Level 3 degradation represents severe degradation, manifested as partial loss or distortion of information from the target sensor, requiring partial information reconstruction. Level 4 degradation represents complete failure or strong interference, manifested as the sensing data (i.e., perceived data) of the target sensor severely violating physical laws, requiring full physical model reconstruction. When extremely low confidence is detected accompanied by significant violations of physical laws, it can be directly identified as severe degradation.

[0066] Optionally, determining the remediation strategy matching the degradation level can be achieved by pre-setting a mapping library between degradation levels and remediation strategies. When the target sensor is determined to be at a level requiring cross-modal information reconstruction based on a physical model, a cross-modal sensing consistency map is retrieved, and normal sensor data with strong physical correlation or domain consistency with the target sensor is identified and extracted as a reference source. Simultaneously, the physical model corresponding to the target sensor's function is invoked, and the data sequence that the target sensor should output under ideal conditions is generated using normal sensor data and environmental parameters as constraints, thereby constructing a targeted remediation strategy.

[0067] Optionally, according to the repair strategy, the repair operation on the target sensor can be carried out by using a selected physical model combined with real-time environmental parameters, including but not limited to ambient temperature, material emissivity, light intensity and readings of other normal sensors, to calculate and generate virtual sensing data of the target sensor; and replace the original abnormal data and the output of the original sensor with the virtual sensing data.

[0068] In the embodiments of this application, the above method can reconstruct highly reliable sensing information by utilizing the physical correlation and prior knowledge between multiple sensors even when the sensor is severely degraded or subjected to strong interference, thus significantly reducing the missed detection and false alarm rates of the sensor.

[0069] As an optional embodiment, step S104 involves cross-validating multiple sensing data to obtain a validation result, including: performing consistency detection on multiple sensing data in a target map to obtain a validation result. The target map consists of multiple edges, which represent the matching relationship between two sensors among multiple sensors. Each edge consists of two nodes in the target map, which correspond to two sensors. The matching relationship represents the matching relationship based on physical knowledge and / or domain knowledge of the field to which the inspection equipment belongs.

[0070] In this embodiment, the target graph described above can be used to represent a graph structure model of physical associations or domain knowledge consistency between different sensors, including but not limited to cross-modal sensing consistency graphs and directed weighted graphs. The edges described above are used to represent the matching relationship between two sensors among multiple sensors.

[0071] Optionally, consistency detection is performed on multiple sensing data in the target map to obtain verification results. Multiple sensing data from various sensors are acquired in real time and mapped to corresponding nodes in the target map. Multiple edges in the target map are traversed, and for each edge connecting two nodes, the corresponding sensing data is extracted. A consistency score is calculated between these two sets of sensing data based on predefined physical rules or domain knowledge. Subsequently, the calculated consistency score is compared with a preset consistency threshold. If the consistency score is lower than the threshold, the matching relationship represented by that edge is considered abnormal. For example, if the target map is a cross-modal sensing consistency map, for the edges between thermal imaging and visible light nodes, temperature distribution features in the thermal imaging image and object edge features in the visible light image can be extracted. An edge consistency score is calculated based on the physical correlation between thermal radiation and object surface reflection. If the consistency score is lower than the threshold, it indicates that the high-temperature area in the thermal imaging does not match the object outline in the visible light, potentially indicating sunlight reflection artifacts or lens contamination.

[0072] In the embodiments of this application, the above method can be used to convert various sensing data into target maps and construct references using the physical constraint relationships between different sensors, thereby identifying sensing data that violate physical laws in the absence of external standard answers.

[0073] As an optional implementation, the target map can be updated. Consistency checks are performed on the target map for various sensing data to obtain verification results. This includes: initializing the weights of edges in the target map, where the weights represent the correlation between the two sensors corresponding to the edges; evaluating the edges in the initialized target map using various sensing data to obtain evaluation results, where the evaluation results represent the matching degree between the sensing data of the two sensors corresponding to the edges; and determining the verification results using the evaluation results.

[0074] In this embodiment, the aforementioned weights can be used to represent the correlation between the two types of sensors corresponding to an edge. The aforementioned evaluation results can be used to represent the degree of matching between the sensing data of the two types of sensors corresponding to two nodes on an edge.

[0075] Optionally, the weights of edges in the target graph can be initialized with the expected relevance weights of each consistent edge. For example, weights can be differentiated based on the physical coupling strength and environmental sensitivity of the sensors: sensors with strong physical correlation and high stability, such as the geometric consistency between lidar and visible light, are given higher initial weights to reflect their high confidence contribution in the fusion process; sensor pairs that are greatly affected by the environment or have relatively indirect correlation, such as gas sensors and temperature sensors, are given lower initial weights to reduce interference with overall consistency.

[0076] Optionally, for each edge in the target graph, the sensor data corresponding to the two nodes connected by each edge are extracted; based on predefined physical rules or domain knowledge, a consistency score is calculated between the two sets of sensor data. This consistency score reflects the degree of physical and logical agreement between the two sets of sensor data at the current moment. The calculated consistency score can be compared or fused with the initial weight of the edge to generate an evaluation result reflecting the current matching degree of the edge.

[0077] Optionally, after determining the evaluation results, the consistency score calculated in real time can be weighted and fused with the current weight of the edge, or a deviation calculation can be performed to obtain the confidence level of the edge. If the confidence level is lower than the preset anomaly judgment threshold, the edge is marked as an anomalous edge, and its anomaly type is recorded in detail. At the same time, graph-based source tracing analysis is performed to count the number of anomalous participations and weight contributions of nodes connected to the anomalous edge in the edge. If the number of anomalous edges connected to a node or the cumulative weight exceeds the preset judgment ratio, the node is judged as a degradation source, thereby generating a comprehensive verification result that includes the location of the anomalous edge, the distribution of the degree of anomalousness, and suspected degraded sensors.

[0078] As an optional embodiment, the method further includes: compensating for the differences in sampling information and response delay information among multiple sensors; evaluating the edges in the initialized target graph using multiple sensing data to obtain evaluation results, including: evaluating the edges in the initialized target graph using multiple sensing data, the compensated differences in sampling information, and the compensated response delay information to obtain evaluation results.

[0079] In this embodiment, the aforementioned sampling information differences can be used to represent inconsistencies between different sensors in data acquisition frequency, timestamp accuracy, or data refresh cycle. The aforementioned response delay information can be used to represent the inherent physical delay between a sensor's change in a physical quantity and its output electrical signal or digital data.

[0080] Optionally, a dynamic time warping algorithm or a timing alignment mechanism can be used to timestamp and resample the sensing data from different sensors, ensuring that the different sensing data are on a unified time reference. Simultaneously, a delay model is established based on the physical characteristic parameters of each sensor, and inverse time compensation or phase correction is performed on the acquired data. This eliminates data mismatch caused by sampling asynchrony and response lag, ensuring that the sensing data participating in consistency detection are temporally corresponding.

[0081] This embodiment utilizes various sensing data, compensated sampling information differences, and compensated response delay information to evaluate edges in an initialized target graph, obtaining evaluation results. This can be achieved by re-extracting the sensing data corresponding to the two nodes connected by each edge in the target graph, based on time alignment and delay compensation. A consistency score between these two calibrated data sets is calculated based on predefined physical rules or domain knowledge. Because temporal interference is eliminated, the resulting evaluation results more accurately reflect the sensor's matching degree under real physical conditions, thus generating high-precision evaluation results.

[0082] In the embodiments of this application, the above method can solve the problem of time asynchrony and response lag caused by differences in hardware characteristics of heterogeneous sensors, and avoid misjudging normal physical delays as sensing anomalies.

[0083] As an optional embodiment, multiple sensors include: a carbon monoxide sensor, a lidar sensor, and a carbon dioxide sensor. The matching relationship corresponding to the edge is used to represent the matching relationship between thermal imaging and visible light, the matching relationship between thermal imaging and the spatiotemporal information of the carbon monoxide sensor, the matching relationship between visible light and the depth parallax of the lidar sensor, and the matching relationship between the combustion information of the carbon dioxide sensor and the thermal imaging in the inspection scenario.

[0084] Optionally, edge consistency edges can be defined between thermal imaging nodes and visible light nodes to verify the spatial overlap between temperature anomaly regions and the outlines of visible light objects, thus eliminating sunlight reflection artifacts. Spatiotemporal correlation edges can be defined between thermal imaging nodes and carbon monoxide sensor nodes to correlate the spatiotemporal distribution of fire hotspots and toxic gas leaks, thus verifying combustion events. Depth parallax consistency edges can be defined between visible light nodes and lidar nodes to verify the correspondence between pixels in the visible light image and lidar point clouds in three-dimensional space through geometric projection relationships, thus detecting image distortion or occlusion. Combustion correlation edges can be defined between carbon dioxide nodes and thermal imaging nodes to verify whether high-temperature regions are accompanied by an increase in the concentration of carbon dioxide, a combustion product, thus distinguishing between normal thermal radiation and actual combustion.

[0085] As an optional embodiment, step S106, in response to the verification result indicating an anomaly in the matching among multiple sensing data, determines at least one target sensor from multiple sensors, including: in response to the verification result indicating an anomaly in the matching among multiple sensing data, performing a voting tracing process on multiple sensors to obtain the target sensor.

[0086] Optionally, in response to the verification results indicating anomalies in the matching among multiple sensing data, a voting tracing process is performed on multiple sensors to obtain the target sensor. This can be based on all the anomalous consistency edges identified in the verification results, constructing an anomalous contribution mapping relationship for sensor nodes. Then, the voting tracing process is performed on multiple sensors, assigning different voting weights to different anomalous edges based on their confidence weights or historical stability to reflect the credibility of the anomalous evidence. Subsequently, the total number of anomalous votes or weighted total score received by each sensor node is accumulated. Finally, the accumulated score is compared with a preset downgrade judgment threshold. If the accumulated score of a node exceeds the threshold, the sensor corresponding to that node is marked as the target sensor.

[0087] In the embodiments of this application, the above method can solve the problem of difficulty in locating anomalies of a single sensor. By pooling and weighting evidence from multiple sources, the interference caused by instantaneous environmental factors can be reduced. At the same time, by using a voting tracing mechanism, the systematic deviation of the sensor in multiple dimensions can be determined, thereby identifying the degradation state of the sensor before it completely fails.

[0088] As an optional embodiment, the method further includes: determining the repaired target sensor's sensing data as sensing data among multiple sensing data, and returning to the step of performing cross-validation on the multiple sensing data to obtain a validation result.

[0089] Optionally, after identifying the repaired target sensor's sensing data as the sensing data among multiple sensing data sets, the repaired sensing data replaces the corresponding target sensor's sensing data in the original data stream, constructing a new sensing dataset containing the repaired data. This sensing dataset can be re-input into the consistency graph to trigger a new round of cross-validation, which could be a new round of consistency detection loop. During this new round of consistency detection on the sensing data, the focus is on all consistency edges involving the target sensor, recalculating the consistency score using the repaired data. Finally, an updated verification result is generated based on the new score. If the score of the abnormal edge recovers to within a preset normal threshold range, and the voting traceability score of the relevant sensor node decreases to a safe level, the reconstruction is considered successful.

[0090] In this embodiment, by acquiring multiple sensing data from multiple sensors and cross-validating these data, an anomaly can be identified when matching anomalies occurs among the multiple sensing data. The target sensor exhibiting an abnormal sensing performance can then be determined. Confidence information is updated based on the anomaly type of the abnormal state. When the confidence information falls below a threshold, the target sensor is repaired. This achieves the detection and performance recovery of the sensor before it is completely destroyed. Furthermore, by utilizing the cross-validation mechanism among multiple sensors during collaborative sensing, the target sensor's anomaly can be identified during the performance degradation stage rather than the complete destruction stage. Based on the anomaly type of the target sensor's abnormal state, the confidence information is dynamically updated to trigger repair operations, thereby restoring the target sensor's normal sensing performance. Because the above method utilizes cross-validation of multiple sensing data to replace direct reliance on the hardware state of a single sensor, the target sensor can be identified when the sensor only shows performance degradation and there are mismatches between multiple sensing data but it has not yet been completely destroyed. Therefore, it overcomes the obstacles in related technologies that can only detect anomalies when the inspection equipment is completely destroyed, and cannot detect that the inspection equipment has actually experienced different degrees of anomalies before it is completely destroyed, which leads to the degradation of the inspection equipment's ability to perform inspections. Thus, it solves the technical problem of low inspection efficiency of inspection equipment and achieves the technical effect of improving the inspection efficiency of inspection equipment.

[0091] The technical solutions of the embodiments of this application will be illustrated below with reference to preferred embodiments.

[0092] Currently, when quadrupedal inspection robot dogs perform inspection tasks in industrial settings such as chemical plants, oil and gas fields, and substations, they are typically equipped with a variety of different types of sensors, including dual-light gimbals (thermal imaging plus visible light), hydrocarbon gas sensors (used to detect carbon monoxide, hydrogen sulfide, oxygen, combustible gases, carbon dioxide, nitrogen oxides, etc.), lidar, positioning modules, and inertial measurement units.

[0093] The various sensor data processing technologies in related fields mainly fall into several categories: multimodal fusion navigation technology, which fuses data from different sensors using deep neural networks for terrain perception and path planning; multi-sensor filtering fusion technology, which processes perception state information through distributed filtering models; dynamic confidence weighting technology, which dynamically adjusts the weight of each sensor in the fusion result by analyzing the quality of each sensor's data; and perception self-healing technology, which tracks sensor confidence and ignores data when a sensor fails. In these technologies, sensors deteriorate gradually, but current methods fail to detect this. Sensor degradation in industrial environments is a gradual process. Visible light lenses gradually become covered in dust, thermal imaging suffers from decreased temperature measurement accuracy due to temperature drift, electrochemical gas sensors gradually become poisoned or dry out, and lidar accumulates errors due to interference from reflective surfaces. Existing confidence weighting methods only passively reduce the weight of deteriorating sensors during the data fusion stage, lacking a mechanism to actively detect degradation, often only discovering it when the sensor is completely broken.

[0094] Alternatively, because there are no standard answers at the inspection site, no one knows when a perception error occurs. In a real petrochemical plant or substation inspection environment, there is no standard answer to refer to. When thermal imaging shows an abnormal temperature at a certain point, the operator cannot determine whether it is a genuine equipment overheating, a sensor malfunction, or environmental interference (such as sunlight reflection). The relevant technology cannot provide an independent judgment on whether the perception results are reliable or not.

[0095] Optionally, each sensor exhibits different failure patterns, making a uniform detection method impossible. The failure modes of different sensors vary significantly. Visible light cameras may experience lens contamination, low-light failure, or overexposure; thermal imaging may suffer from solar reflection interference and temperature drift; lidar may display point cloud holes in front of transparent or mirrored objects and experience signal attenuation in rain and fog; electrochemical gases may be poisoned or experience zero-point drift. No single self-testing method can cover all these failure modes. Existing sensor self-testing modules can only detect hardware-level faults (e.g., open circuits or short circuits) and cannot detect degradation at the perceived content level.

[0096] Alternatively, when a sensor malfunctions, the system can only passively accept the degradation and will not actively repair it. When a sensor degrades, the system in related technologies either ignores that sensor or reduces its fusion weight. Both approaches are essentially passively accepting degradation, causing irreversible loss at the information level. They lack the ability to reconstruct degraded information using the physical relationships between different sensors.

[0097] This application's embodiments achieve early detection of progressive sensor degradation. Through continuous cross-validation among multiple sensors, early warnings are issued in the early stages of sensor degradation, before complete failure, accurately pinpointing which sensor is malfunctioning and what the problem is. Perception reliability is assessed without external standard answers. Utilizing the physical correlation between different sensors (e.g., the physical correlation between thermal imaging and gas sensors in detecting leaks), independent perception reliability indicators are constructed through mutual verification between sensors, requiring no external standard answers. Reliability is scaled down from the sensor level to the spatial region level, allowing the inspection robot to know which areas show reliable data and which areas may have problematic data. Proactive self-healing of perception degradation is achieved. When sensor degradation is detected, other sensors and physical models are automatically used to reconstruct the perception information that the degraded sensor should have provided, achieving dynamic reconstruction of the perception pipeline instead of passively accepting information loss. Comprehensive solutions are provided for inspection reliability issues in complex industrial environments, including perception uncertainties caused by multiple factors such as dust, rain, fog, drastic changes in lighting, and electromagnetic interference in scenarios such as chemical plants and oil and gas fields.

[0098] Optionally, regarding inspection reliability, it is expected to reduce the rate of missed detections and false alarms caused by sensor degradation by more than 70%. In terms of unattended operation capability, the perception system possesses self-healing capabilities, which can extend the cycle of single manual intervention and reduce operation and maintenance costs. Regarding environmental adaptability, it can maintain stable perception quality under conditions such as severe weather, dust, and drastic changes in lighting. In terms of interpretability, the perception integrity map provides operators with intuitive visualization of reliable areas, assisting in decision-making. Regarding generalization, it does not rely on pre-training for specific sensor models or specific scenarios, possessing plug-and-play capability across different scenarios. Regarding safety hazard detection, early warning of sensor degradation can prevent safety accidents such as collisions and missed detections caused by sudden perception failures.

[0099] Optionally, if the scenario is an afternoon inspection, the sunlight shining obliquely on the surface of the stainless steel pipe will produce strong reflections, causing the thermal imaging gimbal mounted on the robot dog to show a large area of ​​false high temperature readings in that area, while the visible light image will be overexposed. According to traditional solutions, the direct output of false high temperatures by thermal imaging will trigger numerous false alarms of overheating in the equipment, and the overexposure of visible light will cause detection failure, requiring operators to rush to the site to determine the truth. According to the solution of this invention, the system's processing procedure is as follows: The cross-modal perception consistency map first detects an anomaly: the edge of the high-temperature area displayed by the thermal imaging does not match the edge of the object in the visible light image severely, while the lidar echo shows that the pipe surface is smooth and has high reflectivity, which highly matches the pattern of reflection artifacts. The system determines that the thermal imaging is interfered with by sunlight reflection, and the visible light is also overexposed and degraded. The perception integrity map is then updated. The integrity score of the spatial area where the stainless steel pipe is located drops sharply from 0.9 to 0.31, and the map marks this area as unreliable in both thermal imaging and visible light. The perception pipeline dynamic reconstructor triggers the highest level of physical reconstruction. The system extracts the pipe's geometric surface from the lidar point cloud, queries a historical material database to determine the pipe is stainless steel with an emissivity of approximately 0.15, and combines this with an ambient temperature of 25 degrees Celsius and normal gas sensor readings (no abnormalities in carbon monoxide and carbon dioxide, indicating no signs of combustion). The system reconstructs the pipe's actual temperature to be approximately 35-42 degrees Celsius, which falls under normal operating conditions and poses no risk of overheating. During the verification phase, the reconstructed data is re-injected into the consistency map, and all consistency edges return to normal levels. The inspection report uses the reconstructed data for this area, with a confidence level of 0.85. Ultimately, the sensor's false alarm rate is reduced to 5% of that of traditional methods.

[0100] The embodiments of this application will be further described below.

[0101] Figure 2 This is a flowchart of another data processing method for inspection equipment according to an embodiment of this application, such as... Figure 2 As shown, the method may include the following steps.

[0102] Step S202: Sensor data acquisition.

[0103] Optionally, the time synchronization module can be used to synchronously acquire visible light camera images, thermal imaging gimbal temperature data, lidar point cloud data, positioning information, inertial measurement unit attitude data, and gas concentration readings from electrochemical gas sensors; the raw data can be denoised, distorted, and converted to a unified coordinate system to transform all sensor data into a unified robot body coordinate system or global coordinate system, providing a unified data foundation for subsequent consistency calculations.

[0104] Step S204: Continuously check the consistency between sensors.

[0105] Optionally, a consistent edge set is constructed based on physical laws and domain knowledge, with each sensor as a node. The consistent edge includes calculating the spatial overlap between the thermal imaging edge and the visible light object edge; using the Dynamic Time Warping (DTW) algorithm to compensate for the thermal imaging response delay and gas diffusion delay, and calculating the spatiotemporal correlation between the high-temperature region and the peak concentration of combustible / toxic gases; comparing the depth information of the visible light image with the depth information of the lidar point cloud to detect parallax anomalies; and when high temperature is detected, simultaneously monitoring whether the CO2 concentration rises abnormally to distinguish between real combustion and false heat sources.

[0106] Step S206: Is there a contradiction?

[0107] Optionally, if there is a conflict among multiple sensors, step S208 is executed; if there is no conflict among multiple sensors, step S204 is executed.

[0108] Step S208: Locate the problematic sensor and generate a spatial reliability map.

[0109] Optionally, the number of negative votes for all sensor nodes involved in the anomalous consistency edges is counted. If a sensor node is marked as an anomalous source in multiple consistency edges and has the highest voting weight, then the sensor is determined to be a degradation source. The specific degradation pattern is inferred by combining the degradation type knowledge base.

[0110] Step S210: Is the credibility of the key area insufficient?

[0111] Optionally, if yes, then step S212 is executed; otherwise, step S204 is executed.

[0112] Step S212: Automatically repair the degraded sensor.

[0113] Optionally, depending on the number and severity of the degraded sensors, if the degradation is minor, only the weight of that sensor in the fusion algorithm is reduced, while the data from the remaining normal sensors is retained. If the degraded sensor is a non-critical sensor, interpolation or time-series prediction reconstruction is performed using data from its neighboring healthy sensors at the same time or the sensor's historical normal data. If the degraded sensor is a critical sensor and there is supporting data from other modalities, then physical model reconstruction is performed.

[0114] Step S214: Re-verify consistency.

[0115] Optionally, the reconstructed data can be used as virtual sensor data and re-input into the cross-modal sensing consistency map. A new score for all consistency edges, including the reconstructed data, is then calculated using the consistency verification engine.

[0116] Step S216: Restore or not?

[0117] Optionally, if yes, then step S208 is executed; otherwise, step S204 is executed.

[0118] According to an embodiment of this application, a self-healing system for perceiving integrity in a quadrupedal inspection robot dog is also provided. It should be noted that this system can be used to execute the data processing method of the inspection equipment in the embodiments.

[0119] Figure 3 This is a schematic diagram of a four-legged inspection robot dog's self-healing system for perceiving integrity, according to an embodiment of this application. Figure 3 As shown, the system may include: a cross-modal perception consistency map module 302, an online consistency verification engine 304, a perception integrity map generator 306, and a perception pipeline dynamic reconstructor 308.

[0120] The cross-modal sensing consistency graph module 302 is used to construct and maintain a directed weighted graph with heterogeneous sensors as nodes and physical or domain consistency relationships between sensors as edges.

[0121] The online consistency verification engine 304 is used to calculate the score of each consistent edge in real time, detect consistency anomalies, and locate degradation sensors.

[0122] The perceived integrity map generator 306 is used to map degradation detection results to spatial voxels to generate a spatiotemporal integrity map containing integrity confidence.

[0123] The sensing pipeline dynamic reconstructor 308 is used to recover key information of degraded sensing channels through hierarchical reconstruction strategies and cross-modal physical reconstruction algorithms based on reconstruction decisions triggered by the integrity map.

[0124] According to an embodiment of this application, a data processing system for inspection equipment is also provided. It should be noted that this system can be used to execute the data processing method for the inspection equipment described in the embodiments.

[0125] Figure 4 This is a schematic diagram of a data processing system for an inspection device according to an embodiment of this application, as shown below. Figure 4 As shown, the system includes a verification engine 402, an information generator 404, and a refactoring unit 406.

[0126] Verification engine 402 is used to acquire various sensing data obtained by multiple sensors sensing the inspection scene, where different sensors correspond to different sensing data.

[0127] Information generator 404 is used to cross-validate multiple sensing data to obtain a validation result; in response to the validation result indicating an anomaly in the matching between multiple sensing data, at least one target sensor is identified from multiple sensors, wherein the sensing performance of the target sensor is in an abnormal state; based on the anomaly type to which the abnormal state corresponding to the target sensor belongs, the confidence information of the inspection equipment is updated, wherein the confidence information is used to represent the degree of credibility of the inspection equipment in the inspection scenario.

[0128] The reconstructor 406 is used to perform a repair operation on the target sensor in response to the updated confidence information being lower than the confidence information threshold, wherein the sensing performance of the target sensor is in a normal state after the repair.

[0129] Figure 5 This is a schematic diagram of a data processing device for an inspection equipment according to an embodiment of this application, as shown below. Figure 5 As shown, the device includes an acquisition unit 502, a determination unit 504, an update unit 506, and a repair unit 508.

[0130] The acquisition unit 502 is used to acquire various sensing data obtained by multiple sensors sensing the inspection scene, wherein different sensors correspond to different sensing data; the verification unit is used to cross-verify the various sensing data to obtain the verification result.

[0131] The determining unit 504 is configured to determine at least one target sensor from multiple sensors, wherein the sensing performance of the target sensor is in an abnormal state, in response to a verification result indicating an anomaly in the matching between multiple sensing data.

[0132] The update unit 506 is used to update the confidence information of the inspection equipment based on the anomaly type to which the abnormal state corresponding to the target sensor belongs. The confidence information is used to represent the degree of credibility of the inspection equipment in the inspection scenario.

[0133] The repair unit 508 is used to perform a repair operation on the target sensor in response to the updated confidence information being lower than the confidence information threshold, wherein the sensing performance of the target sensor is in a normal state after the repair.

[0134] In this embodiment, the acquisition unit 502 acquires various sensing data obtained by sensing the inspection scene using multiple sensors, where different sensors correspond to different sensing data; the verification unit performs cross-validation on the various sensing data to obtain a verification result; the determination unit 504, in response to the verification result indicating an anomaly in the matching between the various sensing data, determines at least one target sensor from the multiple sensors, where the sensing performance of the target sensor is in an abnormal state; the update unit 506 updates the confidence information of the inspection equipment based on the anomaly type to which the abnormal state of the target sensor belongs, where the confidence information is used to represent the reliability of the inspection equipment in the inspection scene; the repair unit 508, in response to the updated confidence information being lower than the confidence information threshold, performs a repair operation on the target sensor, where the sensing performance of the repaired target sensor is in a normal state, thereby solving the technical problem of low inspection efficiency of the inspection equipment and achieving the technical effect of improving the inspection efficiency of the inspection equipment.

[0135] This application also provides an electronic device 60, please refer to... Figure 6 , Figure 6 This is a structural diagram of an electronic device provided in one embodiment of the present application, including a processor 610 and a memory 620, wherein the memory 620 is used to store computer programs; the processor 610 is used to execute the programs stored in the memory 620 to implement the methods described in any embodiment of the present application.

[0136] According to another aspect of the embodiments of this application, a processor is also provided. The processor is used to run a program, wherein the program executes the methods of the embodiments of this application during runtime.

[0137] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method of the embodiments of this application.

[0138] According to another aspect of the embodiments of this application, a vehicle is also provided. The vehicle includes a memory and a processor. The memory stores an executable program; the processor is used to run the program, which, when running, implements the methods described in the embodiments of this application.

[0139] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0144] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method for inspection equipment, characterized in that, The inspection equipment is equipped with multiple sensors, and the inspection equipment operates in an inspection scenario. The method includes: The inspection scene is sensed by a variety of sensors, and the resulting sensing data is obtained, wherein different sensors correspond to different sensing data. Cross-validation is performed on the various types of sensing data to obtain the validation results; In response to the verification result indicating an anomaly in the matching among the various sensing data, at least one target sensor is identified from the various sensors, wherein the sensing performance of the target sensor is in an abnormal state; Based on the anomaly type of the abnormal state corresponding to the target sensor, the confidence information of the inspection equipment is updated, wherein the confidence information is used to represent the degree of credibility of the inspection equipment in the inspection scenario. In response to the updated confidence information being lower than the confidence information threshold, a repair operation is performed on the target sensor, wherein the sensing performance of the repaired target sensor is in a normal state.

2. The method according to claim 1, characterized in that, The confidence information includes the confidence level of the 3D map of the inspection scene. The confidence level represents the reliability of the inspection equipment's inspection in different spatial regions within the inspection scene. The confidence information threshold includes a confidence threshold corresponding to the confidence level. The step of performing a repair operation on the target sensor in response to the updated confidence information being lower than the confidence information threshold includes: In response to the updated confidence level being lower than the confidence threshold, a degradation level of the target sensor is determined, wherein the degradation level is used to represent the degree of degradation of the target sensor; Determine a repair strategy matching the degradation level, wherein the repair strategy is used to represent the rules for repairing the target sensor, different degradation levels correspond to different repair strategies, and the repair strategy corresponding to the degradation level is cross-modal information reconstruction based on a physical model, wherein the physical model is used to characterize the real sensing data that the target sensor needs to sense; According to the repair strategy, the target sensor is repaired, wherein the modified target sensor is used to sense the real sensing data.

3. The method according to claim 1, characterized in that, The cross-validation of multiple types of sensing data to obtain validation results includes: For the various types of sensing data, a consistency detection is performed in the target map to obtain the verification result. The target map consists of multiple edges, which represent the matching relationship between two of the various sensors. Each edge consists of two nodes in the target map, which correspond to the two types of sensors. The matching relationship represents the matching relationship based on physical knowledge and / or domain knowledge of the field to which the inspection equipment belongs.

4. The method according to claim 3, characterized in that, The target map is updatable, and the verification result is obtained by performing consistency detection on multiple types of sensed data in the target map, including: The weights of the edges in the target graph are initialized, wherein the weights are used to represent the degree of correlation between the two sensors corresponding to the edges; Using various types of sensing data, the edges in the initialized target map are evaluated to obtain evaluation results, wherein the evaluation results are used to represent the degree of matching between the sensing data of the two types of sensors corresponding to the edges; The verification result is determined using the evaluation results.

5. The method according to claim 4, characterized in that, The method further includes: Compensation is provided for differences in sampling information and response delays among the various sensors described; The method of evaluating the edges in the initialized target map using various sensing data to obtain evaluation results includes: The edges in the initialized target graph are evaluated using various types of sensing data, the differences in the compensated sampling information, and the compensated response delay information to obtain the evaluation results.

6. The method according to claim 3, characterized in that, The various sensors include a carbon monoxide sensor, a lidar sensor, and a carbon dioxide sensor. The matching relationship corresponding to the edge is used to represent the matching relationship between thermal imaging and visible light in the inspection scenario, the matching relationship between thermal imaging and the spatiotemporal information of the carbon monoxide sensor, the matching relationship between visible light and the depth parallax of the lidar sensor, and the matching relationship between the carbon dioxide sensor and the combustion information of the thermal imaging.

7. The method according to claim 1, characterized in that, The response to the verification result indicating an anomaly in the matching among the various sensing data, identifies at least one target sensor from the various sensors, including: In response to the verification result indicating an anomaly in the matching among multiple sensing data, a voting tracing process is performed on multiple sensors to obtain the target sensor.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The confidence information is stored according to the target resolution, wherein the target resolution is inversely proportional to the integrity of the spatial area in the inspection scenario.

9. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The repaired sensing data of the target sensor is identified as sensing data among various types of sensing data, and the process returns to start from the following steps: Cross-validation is performed on the various types of sensing data to obtain the validation results.

10. A data processing system for inspection equipment, characterized in that, The inspection equipment is equipped with multiple sensors and operates in an inspection scenario. The system includes: A verification engine is used to acquire various sensing data obtained by sensing the inspection scene by various sensors, wherein different sensors correspond to different sensing data; An information generator is used to cross-validate multiple types of sensing data to obtain a validation result; in response to the validation result indicating an anomaly in the matching among the multiple types of sensing data, at least one target sensor is identified from the multiple sensors, wherein the sensing performance of the target sensor is in an abnormal state; based on the anomaly type to which the abnormal state corresponding to the target sensor belongs, the confidence information of the inspection device is updated, wherein the confidence information is used to represent the reliability of the inspection device in performing inspections in the inspection scenario; A reconstructor is configured to perform a repair operation on the target sensor in response to the updated confidence information being lower than a confidence information threshold, wherein the sensing performance of the target sensor is normal after the repair.

Images