Ship fault classification method and system based on word segmentation matching of fuzzy search
Through the word segmentation matching method based on fuzzy search and combined with the Jieba word segmentation tool, the problems of insufficient data and difficulty in vocabulary recognition in ship failure classification are solved, efficient ship failure classification is achieved, and classification accuracy and work efficiency are improved.
Patent Information
- Application Number
- CN202410888982.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-04
- Publication Date
- 2024-11-22
AI Technical Summary
The lack of massive training data in the classification of ship failures in the existing technology has led to high cost of artificial intelligence algorithms based on natural language processing, and the general fuzzy search algorithm cannot effectively distinguish important words from identification abbreviations, alias, models, code names or informal spoken names. Influence classification accuracy.
Using a word segmentation matching method based on fuzzy search, by obtaining maintenance guarantee history and ship classification structure tree, the similarity score of the fault description string and the leaf node path is calculated using the fuzzy search algorithm and Jieba word segmentation tool, and combined with the word segmentation matching algorithm to weight average. Output leaf node paths with a score higher than the threshold to improve classification accuracy.
The accuracy rate of the first-class classification structure tree nodes of about 75% and the success rate of the top 10 including correct nodes was achieved, which significantly improved the efficiency of ship system classification and reduced the error results of manual confirmation.
Smart Images

Figure CN119004276A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of ship fault classification, and more specifically, relates to a ship fault classification method and system based on fuzzy search and word segmentation matching. Background Art
[0002] When analyzing and processing historical information of ship maintenance and support, it is necessary to identify the system, subsystem, equipment and accessories to which the faulty part belongs for classification statistics and reliability assessment. However, there are often only 1-2 sentences to describe the fault phenomenon, and different people may call the same object differently, with abbreviations, aliases, models, codes or informal spoken names, etc. In addition, the complex ship system classification structure tree has more than a thousand leaf nodes. Therefore, it is necessary to develop a ship system classification software and algorithm that can find the appropriate node on the classification structure tree according to a spoken expression describing the fault phenomenon.
[0003] Currently, the best algorithms for classification problems should be artificial intelligence algorithms based on natural language processing, such as BERT and GPT. These algorithms require massive training data. Each training data contains a fault description and a corresponding node in the classification structure tree. However, we lack massive training data, and this solution is too costly.
[0004] A simple solution is to use a general fuzzy search algorithm, which generally uses the shortest edit distance method to compare the similarity of two strings. This algorithm can basically solve the problem, but the accuracy is not high. There are two main problems. One is that it cannot distinguish the importance of matching words. For example, finding some common words and characters, such as "system", "device", "control valve", "valve", etc., will seriously affect the accuracy of classification. The other is that it cannot identify various abbreviations, aliases, models, codes or informal spoken names well. Summary of the invention
[0005] In order to solve the above technical problems, the present invention proposes a ship fault classification method based on fuzzy search and word matching, comprising:
[0006] Obtaining maintenance and support history records and a classification structure tree of the ship, and calculating the fuzzy search similarity score of the fault description string in the maintenance and support history records and each leaf node path in the classification structure tree by using a fuzzy search algorithm;
[0007] Performing word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculating the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm;
[0008] The fuzzy search similarity score and the word segmentation matching similarity score are weighted averaged, and are sorted from high to low according to the scores, and the leaf node paths of the classification structure tree with scores higher than a threshold are output.
[0009] Furthermore, the fault description character string is segmented by using Jieba word segmentation tool.
[0010] Further, outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the leaf node paths with top 10 scores corresponding to the fault description character strings.
[0011] Furthermore, calculating the fuzzy search similarity score between the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score between the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
[0012] Further, calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by a segmentation matching algorithm includes: segmenting the fault description string by a Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list;
[0013] The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list.
[0014] The present invention also proposes a ship fault classification system based on fuzzy search and word segmentation matching, comprising:
[0015] A fuzzy search module is used to obtain the maintenance and support history records and the classification structure tree of the ship, and calculate the fuzzy search similarity score of the fault description string in the maintenance and support history records and each leaf node path in the classification structure tree through a fuzzy search algorithm;
[0016] A word segmentation matching module is used to perform word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculate the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm;
[0017] The output module is used to perform a weighted average of the fuzzy search similarity score and the word segmentation matching similarity score, sort them from high to low according to the score, and output the leaf node path of the classification structure tree with a score higher than a threshold.
[0018] Furthermore, the fault description character string is segmented by using Jieba word segmentation tool.
[0019] Further, outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the leaf node paths with top 10 scores corresponding to the fault description character strings.
[0020] Furthermore, calculating the fuzzy search similarity score between the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score between the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
[0021] Further, calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by a segmentation matching algorithm includes: segmenting the fault description string by a Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list;
[0022] The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list.
[0023] Compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
[0024] 1. In general, the correctness rate of the nodes in the first classification structure tree recommended by the system is about 75%, and the success rate of the first 10 containing correct nodes is about 90%. The system has been used to complete the system classification of more than 30,000 maintenance and support history records, which can greatly improve the efficiency of system classification. Users only need to manually confirm the 10 system classification results recommended by the system for each maintenance and support history record, delete the wrong results, and keep the correct results.
[0025] 2. The factors that affect the success rate of the top 10 correct nodes mainly include two aspects: about half are newly appeared names, codes or equipment, accessories; the other half are incorrect system names or professional names in the maintenance and support history records, which may be due to inconsistencies between the crew system classification and the designer classification.
[0026] 3. In addition to the factors that affect the success rate of the top 10 correct nodes, the factors that affect the accuracy of the first place also include the following factors: multiple maintenance and support history records are combined together to describe each other, resulting in mutual interference; algorithm defects can only perform string matching, and cannot correctly recognize and understand syntax and semantics like the human brain.
[0027] 4. It takes about 25 seconds for the system to process one maintenance record. The operating environment is a microcomputer, Intel Core i7-8700 CPU@3.20GHz, 16GB memory. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 is a flow chart of the method of embodiment 5 of the present invention;
[0029] Figure 2 is a flow chart of a module for calculating a similarity score by using a fuzzy search algorithm according to Embodiment 5 of the present invention;
[0030] Figure 3 is a schematic diagram of a process of calculating a similarity score by using word segmentation matching according to Embodiment 5 of the present invention;
[0031] Figure 4 is a system structure diagram of Embodiment 2 of the present invention;
[0032] Figure 5 Is a flow chart of the method of Example 1 of the present invention. DETAILED DESCRIPTION
[0033] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0034] The method provided by the present invention can be implemented in the following terminal environment, and the terminal may include one or more of the following components: a processor, a storage medium, and a display screen. The storage medium stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method described in the following embodiment.
[0035] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the storage medium, and calling data stored in the storage medium.
[0036] The storage medium may include a random access memory (RAM) or a read-only memory (ROM). The storage medium may be used to store instructions, programs, codes, code sets or instructions.
[0037] The display is used to show the interactive cross-section of each application.
[0038] All subscripts in the formulas of the present invention are only used to distinguish parameters and have no actual meaning.
[0039] In addition, those skilled in the art can understand that the structure of the above terminal does not constitute a limitation on the terminal, and the terminal may include more or fewer components, or combine certain components, or arrange the components differently. For example, the terminal also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, and a power supply, which will not be described in detail here.
[0040] Example 1
[0041] like Figure 5 As shown, an embodiment of the present invention provides a ship fault classification method based on fuzzy search and word segmentation matching, comprising:
[0042] Obtaining maintenance and support history records and a classification structure tree of the ship, and calculating the fuzzy search similarity score of the fault description string in the maintenance and support history records and each leaf node path in the classification structure tree by using a fuzzy search algorithm;
[0043] Specifically, calculating the fuzzy search similarity score of the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score of the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
[0044] Performing word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculating the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm;
[0045] Specifically, the fault description character string is segmented using the Jieba word segmentation tool.
[0046] Specifically, calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by the segmentation matching algorithm includes: segmenting the fault description string by the Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list;
[0047] The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list, and the weighting coefficients of the similarity score are set basically the same.
[0048] The fuzzy search similarity score and the word segmentation matching similarity score are weighted averaged, and are sorted from high to low according to the scores, and the leaf node paths of the classification structure tree with scores higher than a threshold are output.
[0049] Specifically, outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the top 10 leaf node paths with scores corresponding to the fault description character strings.
[0050] Example 2
[0051] like Figure 4 As shown, the embodiment of the present invention also proposes a ship fault classification system based on fuzzy search and word segmentation matching, including:
[0052] A fuzzy search module is used to obtain the maintenance and support history records and the classification structure tree of the ship, and calculate the fuzzy search similarity score of the fault description string in the maintenance and support history records and each leaf node path in the classification structure tree through a fuzzy search algorithm;
[0053] Specifically, calculating the fuzzy search similarity score of the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score of the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
[0054] A word segmentation matching module is used to perform word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculate the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm;
[0055] Specifically, the fault description character string is segmented using the Jieba word segmentation tool.
[0056] Specifically, calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by the segmentation matching algorithm includes: segmenting the fault description string by the Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list;
[0057] The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list.
[0058] The output module is used to perform a weighted average of the fuzzy search similarity score and the word segmentation matching similarity score, sort them from high to low according to the score, and output the leaf node path of the classification structure tree with a score higher than a threshold.
[0059] Specifically, outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the top 10 leaf node paths with scores corresponding to the fault description character strings.
[0060] Example 3
[0061] The embodiment of the present invention further provides a storage medium storing a plurality of instructions, wherein the instructions are used to implement the ship fault classification method based on fuzzy search and word segmentation matching.
[0062] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.
[0063] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: obtaining a maintenance and support history record and a classification structure tree of the ship, and calculating a fuzzy search similarity score between a fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by a fuzzy search algorithm;
[0064] Specifically, calculating the fuzzy search similarity score of the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score of the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
[0065] Performing word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculating the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm;
[0066] Specifically, the fault description character string is segmented using the Jieba word segmentation tool.
[0067] Specifically, calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by the segmentation matching algorithm includes: segmenting the fault description string by the Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list;
[0068] The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list.
[0069] The fuzzy search similarity score and the word segmentation matching similarity score are weighted averaged, and are sorted from high to low according to the scores, and the leaf node paths of the classification structure tree with scores higher than a threshold are output.
[0070] Specifically, outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the top 10 leaf node paths with scores corresponding to the fault description character strings.
[0071] Example 4
[0072] An embodiment of the present invention also proposes an electronic device, including a processor and a storage medium connected to the processor, wherein the storage medium stores multiple instructions, and the instructions can be loaded and executed by the processor to enable the processor to execute a ship fault classification method based on fuzzy search and word segmentation matching.
[0073] Specifically, the electronic device of this embodiment may be a computer terminal, and the computer terminal may include: one or more processors, and a storage medium.
[0074] Among them, the storage medium can be used to store software programs and modules, such as a ship fault classification method based on fuzzy search and word segmentation matching in an embodiment of the present invention, and the corresponding program instructions / modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the storage medium, that is, realizing the above-mentioned ship fault classification method based on fuzzy search and word segmentation matching. The storage medium may include a high-speed random storage medium, and may also include a non-volatile storage medium, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state storage media. In some instances, the storage medium may further include a storage medium remotely arranged relative to the processor, and these remote storage media may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0075] The processor can call the information and application program stored in the storage medium through the transmission system to execute the steps of: obtaining the maintenance and support history record and the classification structure tree of the ship, and calculating the fuzzy search similarity score of the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree through a fuzzy search algorithm;
[0076] Specifically, calculating the fuzzy search similarity score of the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score of the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
[0077] Performing word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculating the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm;
[0078] Specifically, the fault description character string is segmented using the Jieba word segmentation tool.
[0079] Specifically, calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by the segmentation matching algorithm includes: segmenting the fault description string by the Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list;
[0080] The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list.
[0081] The fuzzy search similarity score and the word segmentation matching similarity score are weighted averaged, and are sorted from high to low according to the scores, and the leaf node paths of the classification structure tree with scores higher than a threshold are output.
[0082] Specifically, outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the top 10 leaf node paths with scores corresponding to the fault description character strings.
[0083] Example 5
[0084] like Figure 1 As shown, the method of the present invention specifically comprises:
[0085] S101, read in the maintenance support history record. Generally, the maintenance support history record Excel file is read in, and the file contains multiple maintenance support history records, each record includes fields such as fault description, specialty, engineering order number, maintenance unit, and reporting date.
[0086] S102, read in the classification structure tree. Generally, the classification structure tree Excel file is read in, which is used to describe the standard names and hierarchical structures of various systems and equipment of the ship, and mainly includes 5 columns, namely "primary system", "secondary system", "third-level system", "equipment name" and "accessory name". Due to the inconsistency of the levels of each system, in order to ensure the continuity of the structure tree, the consistency of the names of each column is not required here. The lowest level node is called a leaf node, and the leaf node path includes the nodes from the first level to the leaf node.
[0087] S103, read in the synonym table. Generally, the synonym table Excel file is read in, which mainly includes 8 columns such as keywords, processing mode, primary system, secondary system, tertiary system, equipment name, accessory name, and conditions. The processing mode can be 1 and 0, 1 indicates an exact match of the string, and 0 indicates a fuzzy match of the string. The match is successful when the similarity ratio is greater than 0.75, which is equivalent to 3 out of 4 characters must be the same; the condition is used to specify the ship type.
[0088] The synonym table file was initially created to solve the problem of inconsistent names for systems and equipment, by replacing informal names and codes with formal names. For example, the formal name for "air cooler" is "air cooler". At this time, the keyword is "air cooler", and the first-level system is filled with "air cooler", and all subsequent columns are empty; later it was expanded to specify the node path of the classification structure tree based on keywords, for example, sewage pump refers to the support system-drainage system-submersible sewage pump. This file is the accumulation of expert experience in classification issues. The number of records has now increased to more than a thousand, which can solve the problem of inconsistent names for most systems, equipment, and accessories.
[0089] S104. Read in the professional comparison table. Generally, the professional comparison table Excel file is read in to describe the correspondence between the crew team profession and the first-level system in the classification structure tree. It can be a one-to-many relationship. It contains 2 columns, one column header is "Professional Name" and the other column is "System Name". It can contain multiple systems separated by ",". For example, the "Navigation" profession contains the first-level systems such as "Navigation, Communication, Steering".
[0090] The maintenance history may contain a professional field to guide the system classification work. For example, when the recommended system belongs to a specified profession, the leaf node of the system will be scored *1.4.
[0091] S105, read in the secondary vocabulary table. Generally, the secondary vocabulary table Excel file is read in. The table contains 2 columns, one column header is "vocabulary", which is used to fill in the secondary vocabulary; the other column is "flag", 1 represents an exact match, indicating that the vocabulary must be the same as the name of the classification structure tree node; 0 represents a fuzzy match, requiring a similarity ratio of 100%, indicating that the vocabulary is included in the name of the classification structure tree node.
[0092] The secondary vocabulary file is used to list the equipment and accessory name terms that appear multiple times in the classification structure tree, and it is difficult to determine the location of the tree node through this term. For example, there are multiple safety valves in the classification structure tree, and it is difficult to determine the system and equipment names through this term. When this file is used for fuzzy search algorithm to calculate the score, when the matching string is not included in the secondary vocabulary, the score is *1.4; when this file is used for word segmentation matching algorithm to calculate the score, when the matching word is included in the secondary vocabulary, the score is *0.8.
[0093] S106, read in the Jieba word segmentation user-defined dictionary, which is used to describe the user-defined vocabulary part of speech and frequency of occurrence. Generally, it is read in a txt file, and the encoding format is UTF-8. The file has no header, and each line contains 3 columns, namely, phrase string, word frequency and part of speech. The word frequency is generally defined as more than 70,000; the part of speech is "n", which means a noun.
[0094] Here, all the main words appearing in the classification structure tree are listed, and the part of speech is defined as noun, the frequency is greater than or equal to 70,000, and it appears in the shortest phrase. This is mainly because the Jieba word segmentation package gives priority to finding the longest word, but long words will lead to larger string matching granularity and lower calculation accuracy. In order to avoid this, the word frequency of short phrases is set to more than 70,000, which is much larger than the default word frequency of long phrases.
[0095] S107, using a fuzzy search algorithm to calculate a similarity score. For each fault description string of the maintenance and assurance history record, a general fuzzy search algorithm is used to calculate the similarity score of each leaf node path (each row) of the classification structure tree and the node name of each layer compared with the fault description string, and then the sum is added. The score is adjusted according to factors such as the specified profession, the system, equipment, and attachment strings, the strings containing synonyms, and the strings containing secondary words, and output to the fuzzy search worksheet.
[0096] Specifically, the algorithm formula of the improved fuzzy search algorithm is:
[0097]
[0098] Among them, ScoreFuzzy(s, t) is the fuzzy search similarity score between the fault description string S and the classification structure leaf node path t, weight(n) is the weight of the classification structure leaf node n, SimilarityRatio(n, S) is the similarity between the fault description string S and the string in the classification structure leaf node n, N is the total number of classification structure leaf nodes, freq(n) is the frequency of the classification structure leaf node n in the classification structure tree, len(t) is the string length of the classification structure leaf node path t, and α is the penalty factor used to penalize path strings that exceed the threshold length.
[0099] The fuzzy search worksheet is a worksheet in the system memory. Each maintenance and assurance history record is assigned a fuzzy search worksheet, which records the paths and scores of the top 50 classification structure tree nodes recommended by each maintenance and assurance history record, and sorts them by score.
[0100] S108, using the word segmentation matching algorithm to calculate the similarity score. In order to avoid the impact of the same but unrelated text on the score, the Chinese word segmentation toolkit Jieba is used here to perform word segmentation on the fault description string, and then the similarity score is calculated based on the word segmentation list and the ratio of the string matching of each leaf node path in the classification structure tree, and the score is adjusted according to factors such as the specified profession, the string containing the system, equipment, and attachment, the string containing synonyms, and the string containing secondary vocabulary, and the word segmentation matching worksheet is output.
[0101] Specifically, the algorithm formula of the improved word segmentation matching algorithm is:
[0102]
[0103]
[0104] Among them, ScoreToken(s, t) is the word segmentation matching similarity score between the fault description string s and the classification structure leaf node path t, MatchRatio(token, n) is the matching ratio between the word token in the phrase list and the name string of the classification structure leaf node n, freq(token) is the frequency of the word token in the phrase list in the classification structure tree, ScaleFactor is the adjustment factor, len(tokens(s)) is the character length of the phrase list after the fault description string s is segmented, β is the penalty factor used to penalize the word segmentation list that exceeds the threshold length, PositionDecay(n) is the position decay factor of the classification structure leaf node n in the classification structure leaf node path t, pos(n) is the position sequence number of the classification structure leaf node n in the classification structure leaf node path t, and λ is the position decay factor.
[0105] The word segmentation matching worksheet is a worksheet in the system memory. It is copied to the fuzzy search worksheet, but the score is reset to 0. Each maintenance and guarantee history record is assigned a word segmentation matching worksheet, which records the path and score of the top 50 classification structure tree nodes recommended by each maintenance and guarantee history record, and the sorting is the same as the fuzzy search worksheet.
[0106] S109, weighted average and score sorting to output the top 10. The fuzzy search worksheet and the word segmentation matching worksheet are merged, the scores are calculated by summing and averaging, and then the scores are sorted from high to low, and the node paths of the top 10 classification structure trees with the highest scores for each fault description string are output to an Excel table file in a fixed format.
[0107] Figure 2 Figure 1 is a flow chart of the module for calculating similarity scores using the fuzzy search algorithm. Figure 2 As shown, the method includes:
[0108] S201, using a fuzzy search algorithm to calculate the similarity score of the node name and fault description string of each layer of each leaf node path of the classification structure tree. The general fuzzy search algorithm Fuzzywuzzymit is used here to search for the longest identical continuous string between two strings, and then calculate the length of the identical continuous string / the shortest length of the two source strings as the similarity score, for example, the similarity score between the strings "submersible sewage pump leaking" and "sewage pump" is 1.00, while the similarity score between "submersible sewage pump leaking" and "submersible pump" is 0.67.
[0109] Before performing a fuzzy search, the keywords in the synonym table used for replacement in each node name string need to be directly replaced with the corresponding formal names; words such as "system", "equipment", and "device" in each node name string need to be temporarily deleted to avoid the influence of these common words on the score.
[0110] S202, if a node name of a certain layer is completely included in the fault description string and is not included in the secondary vocabulary, the node score of this layer is * 1.4. The fuzzy search score is generally below 1.0, and a score of 1.0 indicates that there is a complete mutual inclusion relationship between the two comparison strings, and a weighted reward is required to distinguish the case of partially identical scores. However, if the node name belongs to the secondary vocabulary, no weighted reward will be given, such as "pipes and accessories", "valves", "control cabinets", etc. These objects are configured in all major systems and have little effect on system classification.
[0111] S203. If the node name of a certain layer and the fault description string have only one identical character, the score of the node of this layer is increased by *0.3 to avoid interference of identical and irrelevant characters in the score.
[0112] S204. After the scoring of the node names at each layer is completed, the consecutive characters that are identical to the node name at that layer and appear for the first time in the fault description string are deleted to avoid interference of the identical and consecutive characters in the node name strings at each layer on the scoring.
[0113] S205. The scores of the nodes at each level of the leaf node path of the classification structure are added together to serve as the score of the leaf node.
[0114] S206, delete the nodes of each layer on the classification structure tree that are irrelevant to the fault description string. For the leaf node path with a score greater than 0, first find the node with the most identical characters among the 5 level nodes, keep it and do not delete it, then start from the leaf node, when the similarity score is less than 0.5, and the number of identical continuous characters is less than 3, and the node name string is not included in the fault description string, and the ratio of the number of identical characters to the length of the node name string is less than 80%, delete the node until a node that does not meet the condition is encountered. When only the first layer node is left at last, and the length of the identical characters of the node is less than 2, delete the layer node, and the leaf node score is set to 0. In order to facilitate user selection, considering that deleting a node is much easier than adding a node, the condition setting for deleting irrelevant nodes is relatively loose, and some irrelevant nodes may be retained on the leaf node.
[0115] S207, search for keywords that meet the requirements in the synonym table, score the similarity of the structure tree node path corresponding to the keyword, write it into the fuzzy search worksheet and sort it by the score. If the keyword used for extended use in the synonym table is included in the fault description string, or the similarity score is greater than 0.75, fill the structure tree node path corresponding to the keyword into the fuzzy search worksheet, and score it as 3.0*keyword similarity score+the similarity score of the structure tree node path. This method is used to strengthen the role of the keyword used for extended use in the synonym table, because as long as the keyword appears in the fault description string, it means that the structure tree node path corresponding to the keyword may be one of the answers to the system classification.
[0116] The similarity scoring algorithm of the structure tree node path refers to S201, S202 and S203, except that the weighting system in S202 is 1.1.
[0117] S208, write all the node paths of the classification structure tree with a score greater than 0 into the fuzzy search work table according to the score. The table only accepts the top 50 nodes with the same leaf node path, and the score is scored according to the high score. The classification structure tree has thousands of leaf nodes, and this is to reduce the amount of calculation.
[0118] S209. In the fuzzy search worksheet, if a primary system exists in a primary system included in a specified major, all leaf nodes of the system will be scored *1.4. This method is used to strengthen the role of all leaf nodes under the primary system that meets the professional requirements.
[0119] S210. In the fuzzy search worksheet, merge the included leaf nodes, that is, a leaf node path includes another leaf node path, leaving the leaf node with the longer path, and score according to the higher score of the two.
[0120] Figure 3 The following is a flowchart of calculating similarity scores using word segmentation matching. Figure 3 As shown, the method includes:
[0121] S301, copy from the fuzzy search worksheet to the word segmentation matching worksheet, and reset the score to zero. Carrying out work based on the fuzzy search worksheet can reduce the amount of calculation.
[0122] S302. Use Jieba Chinese word segmentation algorithm to segment the fault description string to obtain a phrase list and a part-of-speech list. Here, the word segmentation list only considers nouns and quantifiers, because this system defines all the words in the node names of the classification structure tree as nouns and quantifiers in the Jieba word segmentation user-defined dictionary.
[0123] S303, summarize the phrases in the word segmentation list that are included in the path string of each structural leaf node of the word segmentation matching worksheet, obtain the word segmentation matching string of each leaf node, and calculate the matching string length * weighting coefficient * effect factor / total length of the word segmentation list string as the score of the leaf node. When a certain word segmentation is included in the name of the first-level system node, the weighting coefficient of the word segmentation length is 1.5; when a certain word segmentation is included in the name of the second-level system node, the weighting coefficient of the word segmentation length is 1.4; when a certain word segmentation is included in the name of the third-level system node, the weighting coefficient of the word segmentation length is 1.2; when a certain word segmentation is included in the name of the device node, the weighting coefficient of the word segmentation length is 1.0; when a certain word segmentation is included in the name of the attachment node, the weighting coefficient of the word segmentation length is 0.8. This is mainly due to the fact that the matching word segmentation of the first-level system, the second-level system, and the third-level system has a greater effect on the system classification, while the matching word segmentation of the equipment and attachment has a smaller effect on the classification.
[0124] The action factor is 3.0, which is the weight of the relative fuzzy search score. Without considering various weighting coefficients, the maximum value of the fuzzy search score is generally around 5.0, and the word segmentation match score is around 1.0, so it is necessary to multiply the action factor to balance the impact of both parties on the score.
[0125] After calculating the score of each word segment, the string where the word appears for the first time is deleted from the path string of the structure leaf node to avoid interference of the same matching word segment on the score.
[0126] S304: If a word in the secondary vocabulary is included in a word segmentation matching string of a leaf node, the score of the leaf node is subtracted by the score of the word*0.2 to reduce the influence of the common words.
[0127] S305. If a keyword in the synonym table is included in a leaf node segmentation matching string, a score of 1.25 is given to all leaf node paths in the segmentation matching worksheet that contain the structure tree node path corresponding to the keyword, so as to strengthen the role of the matching keyword in the synonym table.
[0128] S306. In the word segmentation matching worksheet, if the primary system is included in the primary system included in the specified major, all leaf nodes of the system will be scored *1.25 to strengthen the role of all leaf nodes under the primary system that meets the professional requirements.
[0129] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0130] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0131] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0132] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0133] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only storage medium (ROM, Read-Only Memory), random access storage medium (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.
[0135] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.
Claims
1. A ship fault classification method based on fuzzy search and word matching, characterized in that: include: Obtaining maintenance and support history records and a classification structure tree of the ship, and calculating the fuzzy search similarity score of the fault description string in the maintenance and support history records and each leaf node path in the classification structure tree by using a fuzzy search algorithm; Performing word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculating the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm; The fuzzy search similarity score and the word segmentation matching similarity score are weighted averaged, and are sorted from high to low according to the scores, and the leaf node paths of the classification structure tree with scores higher than a threshold are output.
2. A ship fault classification method based on fuzzy search and word matching as claimed in claim 1, characterized in that: The fault description character string is segmented using the Jieba word segmentation tool.
3. A ship fault classification method based on fuzzy search and word matching as claimed in claim 1, characterized in that: Outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the top 10 leaf node paths with scores corresponding to the fault description character string.
4. A ship fault classification method based on fuzzy search and word matching as claimed in claim 1, characterized in that: Calculating the fuzzy search similarity score of the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score of the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
5. A ship fault classification method based on fuzzy search and word matching as claimed in claim 2, characterized in that: Calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by a segmentation matching algorithm includes: segmenting the fault description string by a Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list; The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list.
6. A ship fault classification system based on fuzzy search and word matching, characterized in that: include: A fuzzy search module is used to obtain the maintenance and support history records and the classification structure tree of the ship, and calculate the fuzzy search similarity score of the fault description string in the maintenance and support history records and each leaf node path in the classification structure tree through a fuzzy search algorithm; A word segmentation matching module is used to perform word segmentation processing on the fault description string to obtain the fault description string after word segmentation, and calculate the word segmentation matching similarity score between the fault description string after word segmentation and each leaf node path in the classification structure tree through a word segmentation matching algorithm; The output module is used to perform a weighted average of the fuzzy search similarity score and the word segmentation matching similarity score, sort them from high to low according to the score, and output the leaf node path of the classification structure tree with a score higher than a threshold.
7. A ship fault classification system based on fuzzy search and word segmentation matching as claimed in claim 6, characterized in that: The fault description character string is segmented using the Jieba word segmentation tool.
8. A ship fault classification system based on fuzzy search and word segmentation matching as claimed in claim 6, characterized in that: Outputting the leaf node paths of the classification structure tree with scores higher than a threshold includes: outputting the top 10 leaf node paths with scores corresponding to the fault description character string.
9. A ship fault classification system based on fuzzy search and word segmentation matching as claimed in claim 6, characterized in that: Calculating the fuzzy search similarity score of the fault description string in the maintenance and support history record and each leaf node path in the classification structure tree by using a fuzzy search algorithm includes: calculating the similarity score of the node name of each layer of each leaf node path in the classification structure tree and the fault description string by using a fuzzy search algorithm.
10. A ship fault classification system based on fuzzy search and word segmentation matching as claimed in claim 7, characterized in that: Calculating the segmentation matching similarity score of the fault description string after segmentation and each leaf node path in the classification structure tree by a segmentation matching algorithm includes: segmenting the fault description string by a Jieba segmentation tool to obtain a segmentation list, wherein the segmentation list includes a phrase list and a part-of-speech list; The phrases in the word segmentation list that contain each leaf node path string are summarized to obtain the word segmentation matching string of each leaf node path, and the similarity score is calculated by the ratio of the word segmentation matching string length to the total length of the word segmentation list.
Citation Information
Patent Citations
Method for quickly searching tree structure data in database and storage medium
CN108681603A
Search result reordering method and device, equipment and storage medium
CN114880459A
Fuzzy matching character string searching method and system for nuclear fuel data of nuclear power
CN117453855A
