Atlas construction method and fault detection method

Abstract

The invention provides a graph construction method and a fault detection method, and relates to the technical field of equipment operation and maintenance. The graph construction method comprises the following steps: preprocessing historical information of tested equipment to obtain storage data; performing entity extraction and association processing based on the stored data to obtain a knowledge graph; performing graph feature extraction on the knowledge graph to obtain first-class feature data and second-class feature data; and performing fusion processing on the first type of feature data and the second type of feature data to obtain a target map. The fault detection method comprises the following steps: performing time sequence and / or visual feature extraction according to to-be-detected data of to-be-detected equipment to obtain to-be-detected feature data; based on the to-be-detected feature data, screening is carried out in the second type of feature data in the target map, and a first target feature is determined; and determining a second target feature in the third type of feature data of the target map according to the target node corresponding to the first target feature, so as to determine a fault detection result according to the second target feature.

Description

Technical Field

[0001] The present application relates to the technical field of equipment operation and maintenance, and specifically, to a graph construction method and a fault detection method. Background Art

[0002] As the complexity and quantity of industrial equipment continue to increase, the difficulty of equipment maintenance and troubleshooting is also gradually increasing. Traditional manual maintenance methods have problems such as low efficiency and reliance on manual experience, and cannot meet the needs of fast, accurate and sustainable production.

[0003] In order to improve the efficiency of fault detection, intelligent operation and maintenance technology solutions are used in existing technologies. Intelligent operation and maintenance technology solutions mainly model the corresponding sensor data on industrial equipment, and on this basis, manually design indicators to achieve alarm processing, and then push the alarm information to engineers for manual diagnosis. Finally, the engineers give the conclusion of the fault diagnosis and the corresponding diagnostic opinions.

[0004] However, this processing method requires manual processing of the alarms given by intelligent operation and maintenance, and provides diagnostic conclusions and corresponding inspection and maintenance opinions. The labor cost is high. In addition, the alarm processing mainly relies on structured data (such as sensor data, operating parameters, etc.) for analysis, and cannot be processed based on unstructured data that characterizes the operating status and fault information of the equipment. The fault detection cost is high, the efficiency is low, and the accuracy is low. The fault detection effect is poor and cannot meet the current detection needs of various types of equipment. Summary of the invention

[0005] In view of this, the purpose of the embodiments of the present application is to provide a graph construction method and a fault detection method to improve the problem of poor fault detection effect of equipment existing in the prior art.

[0006] In order to solve the above problems, in a first aspect, an embodiment of the present application provides a method for constructing a graph, the method comprising: Preprocess the historical information of the device under test to obtain storage data; Perform entity extraction and association processing based on the stored data to obtain a knowledge graph; Extracting graph features from the knowledge graph to obtain first-category feature data and second-category feature data; wherein the first-category feature data includes graph node features, and the second-category feature data includes temporal features and / or visual features; The first type of feature data and the second type of feature data are fused to obtain a target map.

[0007] In the above implementation process, various types of historical information of the device can be processed accordingly to improve the validity of the stored data, and entity extraction and association processing are performed based on the stored data to construct a corresponding knowledge graph. Two different types of feature extraction processing are performed on the basis of the knowledge graph to obtain a first type of feature data including graph node features corresponding to the situation of the graph itself, and a second type of feature data including time series features and / or visual features determined according to the actual situation of the historical information. Considering the differences between the two types of feature data, the two types of features can be fused to obtain a target graph that combines the graph feature situation and the actual situation of the data. Feature extraction can be performed according to the actual situation of the historical information, and fusion processing can be performed in combination with the feature situation of the knowledge graph constructed by entity extraction to obtain a multimodal target graph including multiple different types of data, which effectively improves the validity of the target graph and the accuracy of fault analysis.

[0008] Optionally, the extracting graph features from the knowledge graph to obtain the first type of feature data and the second type of feature data includes: Performing semantic analysis on multiple entity nodes in the knowledge graph based on word embedding technology to obtain the first type of feature data; Extracting time series and / or visual features from a plurality of entity nodes in the knowledge graph to obtain the second type of feature data; The first type of feature data and the second type of feature data are associated based on each of the entity nodes.

[0009] In the above implementation process, when extracting features from the constructed knowledge graph, semantic analysis methods such as word embedding technology can be used to perform semantic analysis on each entity node in the knowledge graph, and graph node features representing the relationship between each node in the knowledge graph are obtained as the corresponding first-category feature data. In addition, considering that each entity node in the knowledge graph is obtained by entity extraction based on historical storage data, in order to determine the actual situation corresponding to each entity node, time series and / or visual feature extraction can also be performed on multiple entity nodes to obtain time series features and / or visual features representing the time series or visual situation of the data as the corresponding second-category feature data. Feature extraction can be performed separately for the node structure and actual content of the entity nodes in the graph, and two different types of related feature data can be obtained through natural language processing, computer vision, and time series analysis technologies, which effectively improves the correlation between feature data and entity nodes and the effectiveness of feature data.

[0010] Optionally, extracting time series and / or visual features from a plurality of entity nodes in the knowledge graph to obtain the second type of feature data includes: Classifying the plurality of entity nodes in the knowledge graph to obtain image entity nodes and time series entity nodes; Performing image recognition on the image entity nodes in the knowledge graph through an image recognition model to obtain the visual features; And / or, the time series entity nodes in the knowledge graph are encoded through a deep learning model to obtain the time series features.

[0011] In the above implementation process, due to the diversity of historical information, the multiple entity nodes obtained by preprocessing and entity extraction based on historical information are also diverse. When extracting the features of the second type of feature data, considering that unstructured data such as maintenance data, image data, video data, and text data contain rich equipment operating status and fault information, the multiple entity nodes in the knowledge graph can be classified first to obtain corresponding image entity nodes and time series entity nodes, and the image entity nodes are processed by pre-trained image recognition models to obtain corresponding visual features, and the time series entity nodes are encoded by pre-trained deep learning models to obtain time series features. Different types of models can be used to classify different types of entity nodes to extract different feature data that characterize time series conditions and visual conditions, so that the second type of feature data can contain the corresponding features of time series conditions and visual conditions, effectively improving the comprehensiveness and effectiveness of the second type of feature data.

[0012] Optionally, if the entity node has the visual feature and the temporal feature, the method further includes: performing weighted processing on the visual feature and the temporal feature.

[0013] In the above implementation process, considering that data such as video data has both temporal characteristics and visual characteristics, the same entity node may be used as both an image entity node and a temporal entity node. Accordingly, the second type of feature data obtained after feature extraction includes visual features and temporal features. Considering the actual situation of entity nodes represented by multiple types of feature data, a weighted algorithm can be used to perform weighted calculation processing on visual features and temporal features to obtain the corresponding second type of feature data. The ability to perform comprehensive processing when there are multiple different types of features effectively improves the effectiveness and comprehensiveness of the second type of feature data.

[0014] Optionally, the fusing the first type of feature data and the second type of feature data to obtain a target map includes: Determining a matching degree between the first type of feature data and the second type of feature data of each of the entity nodes; The first type of feature data is adjusted based on a fusion algorithm and the matching degree to obtain third type of feature data corresponding to the second type of feature data, and the target map including the second type of feature data and the third type of feature data.

[0015] In the above implementation process, the first type of feature data and the second type of feature data corresponding to each entity node are associated. When performing fusion processing, the matching degree between the two types of feature data corresponding to each entity node can be determined first, and the first type of feature data can be adjusted based on the corresponding fusion algorithm and the actual matching degree to make it close to the second type of feature data, and obtain the corresponding third type of feature data, so that each entity node has the corresponding second type of feature data and the associated third type of feature data, and the multi-modal data is fused into a unified feature to obtain the corresponding target map. The first type of feature data can be adjusted accordingly according to the multi-modal second type of feature data so that the two different types of feature data can be fused, which effectively improves the effectiveness of the obtained target map and the accuracy of fault analysis.

[0016] Optionally, preprocessing the historical information of the device under test to obtain storage data includes: Acquire the historical information of multiple abnormal records corresponding to the multiple devices under test; wherein the historical information includes structured data and unstructured data, the structured data includes at least one of sensor data and operating parameters, and the unstructured data includes at least one of maintenance data, image data, video data, and text data; Extracting keywords from the historical information according to a preset extraction template to obtain key data; Screening the key data to obtain valid data; wherein the screening process includes: at least one of redundant deletion, error correction, and filtering; The multiple valid data are classified and converted according to the multiple abnormal records to obtain the multiple stored data.

[0017] In the above implementation process, the historical information may be related data of multiple historical records of one or more devices under test, and the historical information may include structured data of the type of sensor data, operating parameters, and unstructured data of the type of maintenance data, image data, video data, text data, and other types of multimodal data. During preprocessing, keywords can be extracted from the acquired historical information based on a preset extraction template to obtain corresponding key data, and then the key data can be screened and classified and converted to obtain corresponding storage data. By preprocessing the historical information, data of different modes can be uniformly stored and managed, the storage path of the data can be determined, and the stored data can be stored in the corresponding database for subsequent calls.

[0018] Optionally, the performing entity extraction and association processing based on the stored data to obtain a knowledge graph includes: Performing entity extraction on each of the stored data based on semantic analysis to obtain a plurality of entity nodes in each of the stored data and an entity type of each of the entity nodes; Determine an association relationship between a plurality of the entity nodes based on a plurality of the entity types; The knowledge graph is constructed based on the multiple entity nodes and the association relationships.

[0019] In the above implementation process, due to the diversity of stored data, in order to determine the correlation between multiple data, entity extraction can be performed on each stored data based on semantic analysis technology to extract corresponding entity nodes from multiple data sources, determine the entity type of the entity node, and the association relationship between multiple entity nodes, and construct a corresponding knowledge graph based on multiple entity nodes and association relationships. Entity extraction can be performed based on the actual situation of the stored data to construct a corresponding knowledge graph based on the actual situation of the entity node, thereby placing scattered data in a unified knowledge graph, effectively improving the effectiveness and accuracy of equipment fault analysis based on the knowledge graph.

[0020] In a second aspect, an embodiment of the present application further provides a fault detection method, the method comprising: Extracting time series and / or visual features according to the test data of the device under test to obtain the test feature data; Based on the feature data to be tested, the second type of feature data in the target map is screened to determine the first target feature; wherein the target map is a map constructed by the method described in any one of the first aspects above; A second target feature is determined in the third type of feature data of the target map according to the target node corresponding to the first target feature, so as to determine a fault detection result according to the second target feature.

[0021] In the above implementation process, when the device under test is fault-detected, the data under test when the device under test is detected can be obtained, and the time series and / or visual feature extraction of the data under test can be performed to obtain the corresponding feature data under test. By comparing the feature data under test with the second type of feature data in the target map, the first target feature similar to the feature data under test is screened in the second type of feature data, and the second target feature corresponding to the target node is determined according to the target node corresponding to the first target feature and the correlation between the second type of feature data and the third type of feature data, so as to determine the fault detection result of the device under test on the basis of the second target feature. It is possible to perform feature matching in the target map according to the actual situation of the device under test, and determine the corresponding fault detection result by comparing and screening the data features, which effectively improves the accuracy and effectiveness of the fault detection result, so as to improve the accuracy and robustness of fault prediction. It is possible to automatically complete the fault diagnosis reasoning of the device under test without manual judgment, effectively reduce the labor cost required for fault detection, optimize the effect of fault detection, and simultaneously perform fault detection on multiple types of devices under test to meet the detection needs of multiple different types of devices.

[0022] Optionally, the determining the first target feature by screening the second type of feature data in the target map based on the feature data to be measured includes: Performing a similarity comparison between the feature data to be tested and a plurality of feature data in the second category of feature data to obtain a plurality of similarity data; The plurality of similarity data are screened based on a preset similarity threshold to obtain the first target feature corresponding to the similarity data having a similarity greater than or equal to the similarity threshold.

[0023] In the above implementation process, the feature data to be measured can be compared with multiple feature data in the second category of feature data for similarity to obtain similarity data between the feature data to be measured and each feature data. When processing, a corresponding similarity threshold can be set in advance to screen multiple similarity data based on the similarity threshold. When the similarity data is greater than or equal to the similarity threshold, it means that the similarity between the feature data to be measured and the corresponding feature data is high, and the feature data corresponding to the similarity data can be selected as the required first target feature. The first target feature can be determined by feature comparison and threshold screening to ensure the similarity between the first target feature and the feature data to be measured, thereby effectively improving the effectiveness and accuracy of the first target feature.

[0024] Optionally, if there are multiple first target features, the method further includes: Determine a sub-detection result corresponding to each of the first target features; The fault detection result is determined based on the overlap degree and / or distribution ratio of the plurality of sub-detection results.

[0025] In the above implementation process, when performing similarity comparison and screening, there may be multiple similarity data that are greater than or equal to the similarity threshold. Accordingly, multiple first target features will be obtained. In order to improve the accuracy of the final fault detection result, the corresponding second target feature can be determined according to each first target feature, and the corresponding sub-detection result can be determined according to each second target feature. The final fault detection result is obtained by performing coincidence analysis and / or distribution ratio analysis on the multiple sub-detection results. When there are multiple first target features, the sub-detection result corresponding to each first target feature can be determined, and the fault detection result with the highest probability can be determined according to the actual distribution of the sub-detection results, further improving the effectiveness and accuracy of the fault detection result.

[0026] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any implementation of the above-mentioned graph construction method and / or fault detection method.

[0027] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions. When the computer program instructions are read and executed by a processor, the steps in any implementation of the above-mentioned graph construction method and / or fault detection method are executed.

[0028] In summary, the embodiments of the present application provide a graph construction method and a fault detection method, which can perform feature extraction according to the actual situation of historical information, and perform fusion processing based on the feature situation of the knowledge graph constructed by entity extraction, so as to obtain a multimodal target graph including multiple different types of data, and can perform feature matching in the target graph according to the actual situation of the equipment to be tested, and determine the corresponding fault detection results by comparing and screening the data features, thereby effectively improving the accuracy and effectiveness of the fault detection results, so as to improve the accuracy and robustness of fault prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1A block diagram of an electronic device provided in an embodiment of the present application; Figure 2 A schematic diagram of a flow chart of a method for constructing a graph provided in an embodiment of the present application; Figure 3 A detailed flowchart of step S230 provided in an embodiment of the present application; Figure 4 A detailed flow chart of step S232 provided in an embodiment of the present application; Figure 5 A detailed flow chart of step S240 provided in an embodiment of the present application; Figure 6 A detailed flowchart of step S210 provided in an embodiment of the present application; Figure 7 A detailed flow chart of step S220 provided in an embodiment of the present application; Figure 8 A flowchart of a fault detection method provided in an embodiment of the present application; Fig. 9 A detailed flow chart of step S320 provided in an embodiment of the present application; Fig.10 A schematic diagram of the structure of a graph construction device provided in an embodiment of the present application; Fig.11 A schematic diagram of the structure of a fault detection device provided in an embodiment of the present application.

[0031] Icons: 100-electronic device; 111-memory; 112-storage controller; 113-processor; 114-peripheral interface; 115-input and output unit; 116-display unit; 400-map construction device; 410-preprocessing module; 420-entity module; 430-feature extraction module; 440-fusion module; 500-fault detection device; 510-data module; 520-screening module; 530-determination module. DETAILED DESCRIPTION

[0032] The technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.

[0033] The existing intelligent operation and maintenance technical solutions are mainly: using the data from equipment sensors to perform data modeling, and then manually designing indicators to achieve intelligent alarms. The alarm information is then pushed to engineers for diagnosis, and finally the engineers give the conclusion of the fault diagnosis and corresponding diagnostic opinions. At the same time, there are other technical solutions such as integrating historical inspection and maintenance suggestion diagnostic data, and comparing it with historical data when a new fault occurs in the equipment to give the final operation and maintenance recommendation.

[0034] However, the applicant has found that there are many problems with the current technical solutions for industrial equipment operation and maintenance. First, the labor cost of fault detection is high, and it is still necessary to manually process a variety of alarm information, and manually give diagnostic conclusions and corresponding inspection and maintenance opinions; second, the existing solutions mainly rely on structured data (such as sensor data, operating parameters, etc.) for analysis, and cannot be processed based on unstructured data that characterizes the operating status and fault information of the equipment. In addition, the prediction accuracy of the existing technical solutions is not high, and most of them are based on simple statistical analysis or traditional machine learning algorithms. The complexity and prediction accuracy of the model are limited, resulting in the current fault detection scheme. The fault detection cost is high, the efficiency is low, and the accuracy is low, and the fault detection effect is poor, which cannot meet the current detection needs of various types of equipment.

[0035] In order to solve the above problems, the embodiments of the present application provide a graph construction method and a fault detection method, which are applied to electronic devices. The electronic devices can be servers, personal computers (PCs), tablet computers, smart phones, personal digital assistants (PDAs), and other electronic devices with logical computing functions. The corresponding target graphs can be constructed according to various types of structured data and unstructured data of the devices, and the faults of the devices can be automatically detected based on the target graphs to improve the accuracy and robustness of fault prediction.

[0036] Optionally, see Figure 1 , Figure 1 The block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 may include a memory 111, a storage controller 112, a processor 113, a peripheral interface 114, an input and output unit 115, and a display unit 116. A person skilled in the art may understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the electronic device 100. For example, the electronic device 100 may further include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0037] The memory 111, storage controller 112, processor 113, peripheral interface 114, input / output unit 115 and display unit 116 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The processor 113 is used to execute the executable module stored in the memory.

[0038] The memory 111 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 111 is used to store programs, and the processor 113 executes the program after receiving the execution instruction. The method executed by the electronic device 100 defined by the process disclosed in any embodiment of the present application can be applied to the processor 113, or implemented by the processor 113.

[0039] The processor 113 may be an integrated circuit chip with signal processing capability. The processor 113 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor.

[0040] The peripheral interface 114 couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113 and the memory controller 112 can be implemented in a single chip. In other embodiments, they can be implemented by separate chips.

[0041] The input / output unit 115 is used to provide input data to the user. The input / output unit 115 may be, but is not limited to, a mouse and a keyboard.

[0042] The above-mentioned display unit 116 provides an interactive interface (such as a user operation interface) between the electronic device 100 and the user or is used to display image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display or a touch display. If it is a touch display, it can be a capacitive touch screen or a resistive touch screen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more positions on the touch display, and the sensed touch operations are handed over to the processor for calculation and processing. In an embodiment of the present application, the display unit 116 can display various types of data such as the constructed knowledge graph, the target graph, the extracted multiple feature data, and the obtained fault detection results.

[0043] The electronic device in this embodiment can be used to execute each step in each graph construction method and / or fault detection method provided in the embodiment of the present application. The implementation process of the graph construction method and / or fault detection method is described in detail below through several embodiments.

[0044] See also Figure 2 , Figure 2 A flowchart of a graph construction method provided in an embodiment of the present application, the method may include steps S210-S240.

[0045] Step S210, pre-processing the historical information of the device under test to obtain storage data.

[0046] The device under test may be various types of industrial equipment or equipment in other fields, and the historical information may be various types of data collected when the device under test fails or is abnormal. The historical information may include sensor data, operating parameters, maintenance data, image data, video data, text data, and other types of data representing different conditions of the device. The various types of historical information of the device can be processed accordingly to improve the validity of the stored data.

[0047] Optionally, multi-source data integration technology can be used to pre-process the data. For example, based on the ETL (Extract, Transform, Load) architecture, various types of historical information (such as sensor data, historical maintenance records, text records, images, videos, etc.) scattered in different systems can be extracted, transformed, and loaded, and then uniformly stored in a data lake or data warehouse as storage data.

[0048] Optionally, the stored data may be stored in various types of databases. In addition to some common data warehouses, databases such as graph vector databases may also be used to store the corresponding stored data.

[0049] Step S220: perform entity extraction and association processing based on the stored data to obtain a knowledge graph.

[0050] Among them, entity extraction and association processing can be performed based on the stored data to construct the corresponding knowledge graph.

[0051] Optionally, for multimodal storage data stored in the database in the intelligent operation and maintenance scenario, the RDF (Resource Description Framework) semantic web technology can be used to extract entities and relationships in the data. The basic idea of ​​RDF is that any network resource can be uniquely represented by a URI (Uniform Resource Identifier). Based on this identifier, other characteristics of network resources are represented in the form of "Property" = "Property value". In the specific use process, for a piece of storage data describing the status of a device, according to the definition of RDF, the device is the subject, the property name status is the predicate, and the specific actual status is the object. Then, the storage data can be defined as a subject-predicate-object form through RDF semantic technology. In this way, the entity information and entity relationships in the data can be extracted to construct the corresponding knowledge graph.

[0052] It should be noted that the knowledge graph is a technical method that uses a graph model to describe knowledge and model the relationship between all things in the world. The core of the knowledge graph lies in entities and relationships, which together constitute the abstraction and representation of the real world. It represents entities (Entity) or concepts (Concept) through nodes (Point), and represents various semantic relationships (Relation) between entities / concepts through edges (Edge), thus forming a mesh knowledge structure composed of nodes and edges. This structure can efficiently express, store, organize and query various types of information, so that machines can understand and interpret complex data, and thus support the application of equipment fault detection in this application. The knowledge graph may include multiple entity nodes and the relationship between entity nodes. Each entity node may also include corresponding descriptive content, and the descriptive content may include various fault-related content such as fault type.

[0053] Step S230, extract graph features from the knowledge graph to obtain first-category feature data and second-category feature data.

[0054] Among them, the first type of feature data includes graph node features, and the second type of feature data includes temporal features and / or visual features. Two different types of feature extraction processing can be performed on the basis of the knowledge graph to obtain the first type of feature data including graph node features corresponding to the situation of the graph itself, and the second type of feature data including temporal features and / or visual features determined according to the actual situation of historical information.

[0055] Optionally, both the first category feature data and the second category feature data can be corresponding feature vector data. The graph node features in the first category feature data can characterize the position, relationship, etc. of the entity node in the knowledge graph. The timing features in the second category feature data can characterize the time conditions such as the data acquisition time. The visual features can characterize the actual situation contained in the image of the image or video data.

[0056] Step S240, fusing the first type of feature data and the second type of feature data to obtain a target map.

[0057] Among them, considering the difference between the two types of feature data, a fusion process can be performed based on the obtained two types of features to obtain a target atlas that combines the atlas feature situation and the actual data situation.

[0058] Optionally, a fusion algorithm such as cosine similarity matching may be used to perform fusion calculation on the two types of feature data, and the feature data obtained by the fusion calculation may be used to construct a corresponding target map.

[0059] exist Figure 2 In the illustrated embodiment, feature extraction can be performed based on the actual situation of historical information, and fusion processing can be performed in combination with the feature situation of the knowledge graph constructed by entity extraction to obtain a multimodal target graph including a variety of different types of data, thereby effectively improving the effectiveness of the target graph and the accuracy of fault analysis.

[0060] Optionally, see Figure 3 , Figure 3 A detailed flow chart of step S230 is provided for an embodiment of the present application. Step S230 may include steps S231-S232.

[0061] Step S231, perform semantic analysis on multiple entity nodes in the knowledge graph based on word embedding technology to obtain the first type of feature data.

[0062] Among them, when extracting features for the constructed knowledge graph, semantic analysis methods such as word embedding technology can be used to perform semantic analysis on each entity node in the knowledge graph, and graph node features that represent the relationships between each node in the knowledge graph can be obtained as the corresponding first-category feature data.

[0063] Optionally, for the descriptive content corresponding to each entity node in the knowledge graph, key entity nodes and relationships can be extracted using a variety of methods such as word segmentation, part-of-speech tagging, named entity recognition, and syntactic analysis technology. Subsequently, the words in the text are converted into vector representations using Word2Ve word embedding technology to capture the semantic relationship between words and obtain the corresponding graph node features.

[0064] Optionally, the first type of characteristic data may include specific pointing relationships between entity nodes, such as parent-child relationships, tree relationships, etc. of entities.

[0065] Step S232, extracting time series and / or visual features from multiple entity nodes in the knowledge graph to obtain second-category feature data.

[0066] Among them, considering that each entity node in the knowledge graph is obtained by entity extraction based on historical storage data, in order to determine the actual situation corresponding to each entity node, time series and / or visual features of multiple entity nodes can also be extracted to obtain time series features and / or visual features that characterize the time series situation or visual situation of the data as the corresponding second-category feature data.

[0067] It should be noted that the first type of feature data and the second type of feature data are associated based on each entity node, that is, each entity node has the first type of feature data and the corresponding second type of feature data.

[0068] Optionally, for the storage data in image mode or time mode, considering that in the intelligent operation and maintenance scenario, it is not only necessary to classify and extract features based on the device morphology, but also to consider the acceleration spectrum diagram generated by the device operation. Therefore, the corresponding image recognition model and deep learning model can be used to extract time series and / or visual features of multiple entity nodes to obtain multiple types of second-category feature data.

[0069] exist Figure 3 In the illustrated embodiment, feature extraction can be performed for the node structure and actual content of the entity nodes in the graph respectively, and two different types of related feature data are obtained through natural language processing, computer vision, time series analysis and other technologies, thereby effectively improving the correlation between the feature data and the entity nodes and the validity of the feature data.

[0070] Optionally, see Figure 4 , Figure 4 A detailed flow chart of step S232 is provided for an embodiment of the present application. Step S232 may include steps S2321-S2323.

[0071] Step S2321, classify multiple entity nodes in the knowledge graph to obtain image entity nodes and time series entity nodes.

[0072] Among them, due to the diversity of historical information, the multiple entity nodes obtained by preprocessing and entity extraction based on historical information are also diverse. When extracting features of the second type of feature data, considering that unstructured data such as maintenance data, image data, video data, and text data contain rich equipment operating status and fault information, the multiple entity nodes in the knowledge graph can be classified first to obtain corresponding image entity nodes and time series entity nodes.

[0073] It should be noted that, considering that data such as video data has both temporal characteristics and visual characteristics, the same entity node may serve as an image entity node and a temporal entity node at the same time.

[0074] Step S2322, image recognition is performed on the image entity nodes in the knowledge graph through the image recognition model to obtain visual features. And / or, step S2323, encoding is performed on the time series entity nodes in the knowledge graph through the deep learning model to obtain time series features.

[0075] Among them, the image entity nodes can be processed by image recognition through a pre-trained image recognition model to obtain corresponding visual features, and the time series entity nodes can be encoded through a pre-trained deep learning model to obtain time series features.

[0076] For example, when performing visual feature extraction, various types of image recognition models can be used, for example, using segmented operation and maintenance data to pre-train a CNN (Convolutional Neural Networks) model with a ResNet (Residual Network) 18 structure, or an ImageNet, Vit, and other models, to extract visual features of image entity nodes, and the visual features can include high-dimensional feature vectors of the image obtained by the convolution layer and pooling layer of the ResNet18 model. Pre-training an image recognition model with segmented operation and maintenance data can improve the recognition accuracy of the image recognition model, thereby improving the effectiveness of the extracted visual features.

[0077] For example, when extracting time series features, various types of deep learning models can be used. For example, a pre-trained transformer architecture is used to encode data such as acceleration and spectrum in a time series entity node to extract the time series features of the time series entity node. Pre-training a deep learning model can improve the encoding accuracy of the deep learning model, thereby improving the effectiveness of the extracted time series features.

[0078] It should be noted that if the entity node has visual features and temporal features, the method also includes: weighted processing of the visual features and the temporal features. The same entity node may serve as both an image entity node and a temporal entity node. Accordingly, the second type of feature data obtained after feature extraction includes visual features and temporal features. Taking into account the actual situation of entity nodes represented by various types of feature data, a weighted algorithm can be used to perform weighted calculation processing on the visual features and the temporal features to obtain the corresponding second type of feature data. The ability to perform comprehensive processing when there are multiple different types of features effectively improves the effectiveness and comprehensiveness of the second type of feature data.

[0079] Optionally, for a single image entity node or a time series entity node, the weighting coefficient of the corresponding visual feature or time series feature is 1.

[0080] Optionally, the corresponding weighting coefficients can be configured according to the actual conditions of the visual features and the temporal features. For example, when the time is short, the weighting coefficient of the temporal features is small, and the weighting coefficient of the visual features is large. When the image changes slightly, the weighting coefficient of the temporal features is large, and the weighting coefficient of the visual features is small. The image recognition capability of the CNN model and the ability of the new attention-based model to recognize temporal data can be combined to obtain fused features as the second type of feature data.

[0081] Optionally, for the image entity nodes and time series entity nodes of the video data, they can be decomposed into frame sequences and then converted into images, and then image features can be extracted. For the time series entity nodes of the sensor data, the Fourier transform technology can be used to perform frequency domain analysis on the sensor data, extract the time series features, and then apply algorithms such as the random forest machine learning algorithm to perform feature selection and importance analysis.

[0082] exist Figure 4 In the illustrated embodiment, different types of models can be used to classify different types of entity nodes to extract different feature data characterizing temporal conditions and visual conditions, so that the second type of feature data can contain corresponding features of the temporal conditions and visual conditions, thereby effectively improving the comprehensiveness and effectiveness of the second type of feature data.

[0083] Optionally, see Figure 5 , Figure 5 A detailed flowchart of step S240 is provided for an embodiment of the present application. Step S240 may include steps S241-S242.

[0084] Step S241 : determining the matching degree between the first type of feature data and the second type of feature data of each entity node.

[0085] There is a correlation between the first type of feature data and the second type of feature data corresponding to each entity node. When performing fusion processing, the matching degree between the two types of feature data corresponding to each entity node can be determined first.

[0086] Optionally, the matching degree may represent the similarity between the first type of feature data and the second type of feature data.

[0087] Step S242, adjusting the first type of feature data based on the fusion algorithm and the matching degree to obtain the third type of feature data corresponding to the second type of feature data, and a target map including the second type of feature data and the third type of feature data.

[0088] Among them, based on the corresponding fusion algorithm and the actual matching degree, the first type of feature data can be adjusted to be close to the second type of feature data to obtain the corresponding third type of feature data, so that each entity node has the corresponding second type of feature data and the associated third type of feature data, and the multimodal data is fused into a unified feature to obtain the corresponding target map.

[0089] Optionally, when performing fusion processing, Node2Vec and DeepWalk graph embedding technology can be used to embed and vectorize entity nodes and edges, and then the first type of feature data and the second type of feature data are fused by vector space mapping matching, so that the first type of feature data is close to the corresponding second type of special data, and the corresponding third type of feature data is obtained.

[0090] Optionally, feature data of different modalities can be fused to construct a unified feature vector. The attention mechanism can be used to weight the fusion of graph node features, as well as the visual features and temporal features of various entity nodes such as text, images, videos, and sensor data, to obtain the second and third feature data as the final multimodal feature vector, which is then embedded into the knowledge graph to obtain the corresponding target graph.

[0091] exist Figure 5 In the illustrated embodiment, the first type of feature data can be adjusted accordingly based on the multimodal second type of feature data so that two different types of feature data can be fused, thereby effectively improving the validity of the obtained target map and the accuracy of fault analysis.

[0092] Optionally, see Figure 6 , Figure 6 A detailed flowchart of step S210 is provided for an embodiment of the present application. Step S210 may include steps S211-S214.

[0093] Step S211, obtaining historical information of multiple abnormal records corresponding to multiple tested devices.

[0094] The historical information may include structured data and unstructured data. The structured data includes sensor data, operating parameters and other data. The unstructured data includes maintenance data, image data, video data, text data and other data. The historical information of the device under test may be obtained by extracting from a database.

[0095] Optionally, in order to improve the validity of historical information for the current time node, recent data, such as data within one month, may be selected as historical information.

[0096] Step S212, extracting keywords from the historical information according to a preset extraction template to obtain key data.

[0097] During the preprocessing, keywords may be extracted from the acquired historical information based on a preset extraction template to obtain corresponding key data.

[0098] For example, for data extraction, structured data can be directly stored in a database, and unstructured data can be first stored in a specified database. Then, for different data modalities, keywords can be extracted according to pre-set regulations, such as data templates that are pre-set according to the data modality type, to obtain the corresponding key data.

[0099] For example, taking video data as an example, key data may include the identity information of the device to which the video data belongs (such as number, device QR code, barcode, etc.), the address where the video data is stored, the time when the video data was collected, the abnormal record number and fault type corresponding to the video data, and other types of data.

[0100] Step S213, screening the key data to obtain valid data.

[0101] Among them, the screening process can include: redundant deletion, error correction, filtering processing and other operations, which can clean the key data, filter incomplete data, correct erroneous data and delete redundant data to improve the effectiveness of the final valid data.

[0102] Optionally, incomplete data may include data lacking identity information or actual content, erroneous data may be data whose fault type does not match the actual content, and redundant data may be repeated data with exactly the same content.

[0103] Step S214, performing classified data conversion on the multiple valid data according to the multiple abnormal records to obtain multiple storage data.

[0104] The effective data obtained by screening can be classified and converted to obtain corresponding storage data, and the storage data can be stored in one or more corresponding databases.

[0105] Optionally, the data granularity of valid data can be aligned and converted. For example, when classifying, all valid data with the same exception record number can be converted to the granularity of the same exception record. Data conversion can also be performed for inconsistent data. For example, the same device may be named differently in different systems, so different aliases and designations of the same device need to be converted into unified identity information. After all data has been classified and converted, it can be loaded and stored in the target database as storage data using a unified data standard format.

[0106] exist Figure 6 In the illustrated embodiment, by preprocessing historical information, data of different modalities can be uniformly stored and managed, the storage path of the data can be determined, and the stored data can be stored in a corresponding database for subsequent calls.

[0107] Optionally, see Figure 7 , Figure 7 A detailed flow chart of step S220 is provided for an embodiment of the present application. Step S220 may include steps S221-S223.

[0108] Step S221 , extracting entities from each stored data based on semantic analysis to obtain multiple entity nodes in each stored data and the entity type of each entity node.

[0109] Among them, due to the diversity of stored data, in order to determine the correlation between multiple data, entity extraction can be performed on each stored data based on semantic analysis technology to extract corresponding entity nodes from multiple data sources and determine the entity type of the entity node.

[0110] Optionally, the entity node may include multiple types of nodes such as the device's identity information, fault type, type of stored data, and actual content contained. The entity type is the content of the standard entity node type, such as device node, fault node, data node, etc. The type of the entity node can be labeled according to the specific description content of the entity node.

[0111] Optionally, the stored data of the image modality and the video modality may be taken as separate entity nodes, and the text description may be taken as an attribute of the specific fault.

[0112] Step S222: determining association relationships between multiple entity nodes based on multiple entity types.

[0113] Step S223, constructing a knowledge graph based on multiple entity nodes and association relationships.

[0114] Among them, the association relationship between multiple entity nodes can be determined according to the labeled entity type. The association relationship can be a triple relationship between two entity nodes, etc., and the corresponding knowledge graph can be constructed based on multiple entity nodes and the association relationship.

[0115] exist Figure 7 In the illustrated embodiment, entity extraction can be performed based on the actual situation of the stored data, so as to construct a corresponding knowledge graph based on the actual situation of the entity nodes, thereby placing scattered data in a unified knowledge graph, effectively improving the effectiveness and accuracy of fault analysis of the equipment based on the knowledge graph.

[0116] See also Figure 8 , Figure 8 A flowchart of a fault detection method provided in an embodiment of the present application, the method may include steps S310-S330.

[0117] Step S310, extracting timing and / or visual features according to the test data of the device under test to obtain the test feature data.

[0118] When fault detection is performed on the device under test, the test data of the device under test can be obtained, and time series and / or visual features can be extracted from the test data to obtain corresponding test feature data.

[0119] Optionally, a feature extraction method for extracting time series and / or visual features from the data to be measured is similar to the feature extraction method described above and will not be described in detail.

[0120] Step S320, screening the second type of feature data in the target map based on the feature data to be measured to determine the first target feature.

[0121] The target map is a map constructed based on the above-mentioned map construction method. By comparing the feature data to be tested with the second type of feature data in the target map, the first target feature similar to the feature data to be tested is screened out in the second type of feature data.

[0122] Step S330, determining a second target feature in the third type of feature data of the target map according to the target node corresponding to the first target feature, so as to determine a fault detection result according to the second target feature.

[0123] Among them, since there is a correlation between the first type of feature data and the third type of feature data of the entity node, the second target feature corresponding to the target node can be determined according to the target node corresponding to the first target feature and the correlation between the second type of feature data and the third type of feature data, thereby determining the fault detection result of the equipment under test based on the second target feature.

[0124] Optionally, the fault detection result may include the fault type and solution corresponding to the result node pointed to by the second target feature.

[0125] exist Figure 8 In the embodiment shown, feature matching can be performed in the target map according to the actual situation of the device under test, and the corresponding fault detection result can be determined by comparing and screening the data features, which effectively improves the accuracy and effectiveness of the fault detection result, so as to improve the accuracy and robustness of fault prediction. The fault diagnosis reasoning of the device under test can be completed automatically without manual judgment, which effectively reduces the labor cost required for fault detection, optimizes the effect of fault detection, and can simultaneously perform fault detection on multiple types of devices under test to meet the detection needs of multiple different types of devices.

[0126] Optionally, see Fig. 9 , Fig. 9 A detailed flow chart of step S320 is provided for an embodiment of the present application. Step S320 may include steps S321-S322.

[0127] Step S321 : performing similarity comparison between the feature data to be tested and a plurality of feature data in the second type of feature data to obtain a plurality of similarity data.

[0128] The feature data to be measured may be compared with a plurality of feature data in the second type of feature data in terms of similarity, so as to obtain similarity data between the feature data to be measured and each feature data.

[0129] Optionally, the distances between the feature vectors of the feature data to be measured and a plurality of feature data in the second type of feature data may be compared to obtain similarity data.

[0130] Step S322: screening multiple similarity data based on a preset similarity threshold to obtain a first target feature corresponding to similarity data having a similarity greater than or equal to the similarity threshold.

[0131] Among them, when processing, the corresponding similarity threshold can be set in advance. In order to improve the accuracy of fault detection, a higher similarity threshold can be set, such as 0.95, etc., to screen multiple similarity data based on the similarity threshold. When the similarity data is greater than or equal to the similarity threshold, it means that the similarity between the feature data to be tested and the corresponding feature data is high, and the feature data corresponding to the similarity data can be selected as the required first target feature.

[0132] It should be noted that when performing similarity comparison and screening, there may be a situation where multiple similarity data are greater than or equal to the similarity threshold. Accordingly, multiple first target features will be obtained. If there are multiple first target features, the method may also include: determining the sub-detection result corresponding to each first target feature; determining the fault detection result based on the overlap and / or distribution ratio of the multiple sub-detection results. In order to improve the accuracy of the final fault detection result, the corresponding second target feature can be determined according to each first target feature, and the corresponding sub-detection result can be determined according to each second target feature. The final fault detection result is obtained by performing overlap analysis and / or distribution ratio analysis on the multiple sub-detection results. When there are multiple first target features, the sub-detection result corresponding to each first target feature can be determined, and the fault detection result with the highest probability can be determined based on the actual distribution of the sub-detection results, which further improves the effectiveness and accuracy of the fault detection result.

[0133] For example, when there are three sub-detection results A, B, and C, if A and B are the same and the overlap between A and C is high, the sub-detection result corresponding to A can be used as the fault detection result; if there are multiple sub-detection results of different types, the sub-detection result with the highest distribution ratio can be used as the fault detection result.

[0134] exist Fig. 9 In the illustrated embodiment, the first target feature can be determined by feature comparison and threshold screening to ensure the similarity between the first target feature and the feature data to be measured, thereby effectively improving the validity and accuracy of the first target feature.

[0135] See also Fig.10 , Fig.10 A structural schematic diagram of a graph construction device provided in an embodiment of the present application, wherein the graph construction device 400 may include: a preprocessing module 410, an entity module 420, a feature extraction module 430 and a fusion module 440.

[0136] The preprocessing module 410 is used to preprocess the historical information of the device under test to obtain storage data; The entity module 420 is used to perform entity extraction and association processing based on the stored data to obtain a knowledge graph; The feature extraction module 430 is used to extract graph features from the knowledge graph to obtain first-category feature data and second-category feature data; wherein the first-category feature data includes graph node features, and the second-category feature data includes temporal features and / or visual features; The fusion module 440 is used to fuse the first type of feature data and the second type of feature data to obtain a target map.

[0137] In an optional embodiment, the feature extraction module 430 is specifically used to: perform semantic analysis on multiple entity nodes in the knowledge graph based on word embedding technology to obtain a first type of feature data; perform temporal and / or visual feature extraction on multiple entity nodes in the knowledge graph to obtain a second type of feature data; wherein the first type of feature data and the second type of feature data are correlated based on each entity node.

[0138] In an optional embodiment, the feature extraction module 430 is specifically used to: classify multiple entity nodes in the knowledge graph to obtain image entity nodes and time series entity nodes; perform image recognition on the image entity nodes in the knowledge graph through an image recognition model to obtain visual features; and / or, encode the time series entity nodes in the knowledge graph through a deep learning model to obtain time series features.

[0139] In an optional implementation, if the entity node has visual features and temporal features, the feature extraction module 430 is further used to: perform weighted processing on the visual features and the temporal features.

[0140] In an optional embodiment, the fusion module 440 is specifically used to: determine the matching degree between the first type of feature data and the second type of feature data of each entity node; adjust the first type of feature data based on the fusion algorithm and the matching degree to obtain the third type of feature data corresponding to the second type of feature data, and a target map containing the second type of feature data and the third type of feature data.

[0141] In an optional embodiment, the preprocessing module 410 is specifically used to: obtain historical information of multiple abnormal records corresponding to multiple tested devices; wherein the historical information includes structured data and unstructured data, the structured data includes at least one of sensor data and operating parameters, and the unstructured data includes at least one of maintenance data, image data, video data, and text data; extract keywords from the historical information according to a preset extraction template to obtain key data; filter the key data to obtain valid data; wherein the filtering process includes: at least one of redundant deletion, error correction, and filtering processing; classify and convert multiple valid data according to multiple abnormal records to obtain multiple stored data.

[0142] In an optional embodiment, the entity module 420 is specifically used to: extract entities from each stored data based on semantic analysis to obtain multiple entity nodes in each stored data and the entity type of each entity node; determine the association relationship between multiple entity nodes based on multiple entity types; and construct a knowledge graph based on multiple entity nodes and association relationships.

[0143] Since the principle of solving the problem by the map construction device 400 in the embodiment of the present application is similar to that of the embodiment of the aforementioned map construction method, the implementation of the map construction device 400 in this embodiment can refer to the description in the embodiment of the aforementioned map construction method, and the repeated parts will not be repeated.

[0144] See also Fig.11 , Fig.11 A structural diagram of a fault detection device provided in an embodiment of the present application, wherein the fault detection device 500 may include: a data module 510, a screening module 520 and a determination module 530; The data module 510 is used to extract the timing and / or visual features according to the test data of the device under test to obtain the feature data under test; The screening module 520 is used to screen the second type of feature data in the target map based on the feature data to be tested to determine the first target feature; wherein the target map is a map constructed based on any of the above-mentioned map construction methods; The determination module 530 is used to determine the second target feature in the third type of feature data of the target map according to the target node corresponding to the first target feature, so as to determine the fault detection result according to the second target feature.

[0145] In an optional embodiment, the screening module 520 is specifically used to: perform similarity comparison between the feature data to be tested and multiple feature data in the second category of feature data to obtain multiple similarity data; screen the multiple similarity data based on a preset similarity threshold to obtain the first target feature corresponding to the similarity data having a similarity greater than or equal to the similarity threshold.

[0146] In an optional embodiment, if there are multiple first target features, the determination module 530 is further used to: determine the sub-detection result corresponding to each first target feature; and determine the fault detection result based on the overlap and / or distribution ratio of the multiple sub-detection results.

[0147] Since the principle of solving the problem by the fault detection device 500 in the embodiment of the present application is similar to that of the embodiment of the aforementioned fault detection method, the implementation of the fault detection device 500 in this embodiment can refer to the description in the embodiment of the aforementioned fault detection method, and the repeated parts will not be repeated.

[0148] An embodiment of the present application also provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps of any one of the graph construction method and / or fault detection method provided in the present embodiment are executed.

[0149] In several embodiments provided in the present application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely schematic, for example, the block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices according to the multiple embodiments of the present application. In this regard, each box in the block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram, and the combination of the block diagrams, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0150] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0151] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0152] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0153] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

[0154] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for constructing a graph, characterized in that: The method comprises: Preprocess the historical information of the device under test to obtain storage data; Perform entity extraction and association processing based on the stored data to obtain a knowledge graph; Extracting graph features from the knowledge graph to obtain first-category feature data and second-category feature data; wherein the first-category feature data includes graph node features, and the second-category feature data includes temporal features and / or visual features; The first type of feature data and the second type of feature data are fused to obtain a target map.

2. The method according to claim 1, characterized in that The extracting graph features from the knowledge graph to obtain the first type of feature data and the second type of feature data includes: Performing semantic analysis on multiple entity nodes in the knowledge graph based on word embedding technology to obtain the first type of feature data; Extracting time series and / or visual features from a plurality of entity nodes in the knowledge graph to obtain the second type of feature data; The first type of feature data and the second type of feature data are associated based on each of the entity nodes.

3. The method according to claim 2, characterized in that The step of extracting time series and / or visual features from a plurality of entity nodes in the knowledge graph to obtain the second type of feature data includes: Classifying the plurality of entity nodes in the knowledge graph to obtain image entity nodes and time series entity nodes; Performing image recognition on the image entity nodes in the knowledge graph through an image recognition model to obtain the visual features; And / or, the time series entity nodes in the knowledge graph are encoded through a deep learning model to obtain the time series features.

4. The method according to claim 3, characterized in that in, If the entity node has the visual feature and the temporal feature, the method further includes: performing weighted processing on the visual feature and the temporal feature.

5. The method according to claim 2, characterized in that: The fusing the first type of feature data and the second type of feature data to obtain a target map includes: Determining a matching degree between the first type of feature data and the second type of feature data of each of the entity nodes; The first type of feature data is adjusted based on a fusion algorithm and the matching degree to obtain third type of feature data corresponding to the second type of feature data, and the target map including the second type of feature data and the third type of feature data.

6. The method according to any one of claims 1 to 5, characterized in that The preprocessing of the historical information of the device under test to obtain the stored data includes: Acquire the historical information of multiple abnormal records corresponding to the multiple devices under test; wherein the historical information includes structured data and unstructured data, the structured data includes at least one of sensor data and operating parameters, and the unstructured data includes at least one of maintenance data, image data, video data, and text data; Extracting keywords from the historical information according to a preset extraction template to obtain key data; Screening the key data to obtain valid data; wherein the screening process includes: at least one of redundant deletion, error correction, and filtering; The multiple valid data are classified and converted according to the multiple abnormal records to obtain the multiple stored data.

7. The method according to claim 1, characterized in that The entity extraction and association processing based on the stored data to obtain a knowledge graph includes: Performing entity extraction on each of the stored data based on semantic analysis to obtain a plurality of entity nodes in each of the stored data and an entity type of each of the entity nodes; Determine an association relationship between a plurality of the entity nodes based on a plurality of the entity types; The knowledge graph is constructed based on the multiple entity nodes and the association relationships.

8. A fault detection method, characterized in that: The method comprises: Extracting time series and / or visual features according to the test data of the device under test to obtain the test feature data; Based on the feature data to be tested, the second type of feature data in the target map is screened to determine the first target feature; wherein the target map is a map constructed based on the method according to any one of claims 1 to 7; A second target feature is determined in the third type of feature data of the target map according to the target node corresponding to the first target feature, so as to determine a fault detection result according to the second target feature.

9. The method according to claim 8, characterized in that The step of screening the second type of feature data in the target map based on the feature data to be measured to determine the first target feature includes: Performing a similarity comparison between the feature data to be tested and a plurality of feature data in the second category of feature data to obtain a plurality of similarity data; The plurality of similarity data are screened based on a preset similarity threshold to obtain the first target feature corresponding to the similarity data having a similarity greater than or equal to the similarity threshold.

10. The method according to claim 8, characterized in that in, If there are a plurality of the first target features, the method further comprises: Determine a sub-detection result corresponding to each of the first target features; The fault detection result is determined based on the overlap degree and / or distribution ratio of the plurality of sub-detection results.

Images