An automobile accessory name word segmentation method and system

Abstract

The application belongs to the technical field of computers, and particularly relates to a kind of automobile accessory name word segmentation method and system.The method comprises: creating a list of accessory names, creating a containment relationship graph, and using the containment relationship graph.The list of accessory names is created by manual annotation, the created list of accessory names is sorted by accessory name length, and a processed accessory name list with an initial state of empty is created;for each accessory name in the sorted list of accessory names, the following operations are performed: finding a containment relationship, generating a directed acyclic graph, setting a weight, finding the optimal path in all paths according to a dynamic programming algorithm, and updating the containment relationship graph.The application uses a containment relationship graph to save the word segmentation results in a graph structure, not only completes Chinese word segmentation, but also saves the knowledge of the automobile accessory field contained in the word segmentation results, provides an important source of knowledge for creating a knowledge graph in the field of automobile accessories, and can improve the accuracy and efficiency of Chinese word segmentation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer, and particularly relates to a method and system for segmenting names of automobile parts. BACKGROUND

[0002] In the field of automobile parts, the name of a part is usually used to determine which category the part belongs to. For example, a list of automobile parts produced by a manufacturer is provided, but different manufacturers have their own naming style for automobile parts. Therefore, there may be multiple names for the same automobile part. One of the goals of an automobile parts ERP system is to integrate data from different sources and convert them into a unified and standardized system. For part names from different sources, the automobile parts ERP system needs to create a standard parts category library and map each part name to a standard part category.

[0003] One characteristic of automobile part names is that they are mainly composed of short texts. In a sample of several million part names, 80% of the names are less than ten characters long, and 86% of the names are less than twelve characters long. The brevity of automobile part names determines that there is not much context to refer to in the names.

[0004] The main goal of natural language processing of automobile part names is to assign the names to a standard part category. To achieve this goal, an important step is to accurately segment the names. Common Chinese segmentation software such as Jieba cannot accurately segment automobile part names because these names contain professional terms specific to the field of automobile parts, such as "air cleaner", "shock absorber", "ignition coil", etc.

[0005] Therefore, a method and system for segmenting automobile part names in the field of automobile parts are needed. The method accurately segments automobile part names, providing a basis for accurate classification of automobile parts. SUMMARY

[0006] To solve the problem of accurate segmentation of automobile part names, the present application provides a method and system for segmenting names of automobile parts, which uses a recursive method to segment short texts and saves the segmentation results in a part name inclusion graph to accurately segment automobile part names.

[0007] The present application uses the following technical solutions:

[0008] A method for segmenting an automobile accessory name, comprising the following steps:

[0009] I. creating an accessory name list:

[0010] An accessory name list is created by manual annotation.

[0011] II. creating a containing relationship graph:

[0012] The created accessory name list is sorted according to the length of the accessory name, and an initially empty processed accessory name list is created. The following operations are performed on each accessory name in the sorted accessory name list:

[0013] Finding the containing relationship; generating a directed acyclic graph; setting the weight, finding the optimal path in all paths according to the dynamic programming algorithm, updating the containing relationship graph, and obtaining the segmentation result of the executed accessory name;

[0014] The currently executed accessory name is added to the processed accessory name list, and the above operations are performed on all accessory names in the sorted accessory name list to complete the creation of the containing relationship graph of the accessory name.

[0015] III. using the containing relationship graph:

[0016] The created containing relationship graph of the accessory name is used to segment the accessory product name.

[0017] As a further scheme of the present application, when the accessory name list is created by manual annotation, a domain expert manually annotates or extracts it, and one accessory name must correspond to one automobile accessory category, and all words in one accessory name are related to automobile accessories.

[0018] As a further scheme of the present application, the accessory name in the accessory name list is composed of two or more characters, and one accessory name corresponds to one automobile accessory category.

[0019] As a further scheme of the present application, before creating the containing relationship graph of the accessory name, the method for sorting the created accessory name list according to the length of the accessory name is to arrange the created accessory name list according to the length of the accessory name from short to long.

[0020] As a further scheme of the present application, when finding the containing relationship, for the accessory name currently cycled to, all accessory names containing the accessory name currently cycled to are found in the processed accessory name list and marked as a sub-list of the accessory name currently cycled to, including the following steps:

[0021] The current executed accessory name in the sorted accessory name list is obtained, and the length of the current executed accessory name is determined; when the length of the current executed accessory name is 2, the current executed accessory name is defined as a basic accessory name and added to the processed accessory name list; when the length of the current executed accessory name is greater than 2, all accessory names containing the current executed accessory name in the sorted accessory name list are queried, and a sub-list of the current executed accessory name is generated.

[0022] As a further scheme of the present application, in the sorted accessory name list arranged from short to long, the basic accessory name with a length of 2 is added to the processed accessory name list, and when the accessory name with a length greater than 2 is executed, all accessory names with a length of 2 are contained in the processed accessory name list, and all accessory names containing the accessory name with a length greater than 2 in the processed accessory name list are searched.

[0023] As a further scheme of the present application, when a directed acyclic graph is generated, a directed acyclic graph is generated according to the generated sub-list of the accessory name, including the following steps:

[0024] An ID is set for each character of the accessory name in the sub-list of the current executed accessory name, and an ID is added at the end of each accessory name as an end identifier; the ID in the accessory name is taken as a point of the directed acyclic graph, and each accessory name in the sub-list is taken as an edge of the directed acyclic graph of the current executed accessory name, so as to construct the directed acyclic graph of the current executed accessory name.

[0025] As a further scheme of the present application, a weight is set, including the following steps:

[0026] According to a monotonically decreasing function, a weight is set for each character in the current executed accessory name in order, and an average weight value containing all characters in the current executed accessory name is obtained;

[0027] The obtained average weight value and the calculation result of the length weight value set for the corresponding accessory name are taken as a total weight.

[0028] As a further scheme of the present application, the dynamic programming algorithm finds the optimal path in all paths according to the total weight, and the path with the highest total weight is the optimal path, wherein the total weight of a path is the sum of the weights of all edges contained in the path.

[0029] As a further scheme of the present application, when the containing relationship graph is updated, after the dynamic programming algorithm calculates the optimal path, the word segmentation result of the current executed accessory name is obtained, and the word segmentation result is saved in the containing relationship graph; wherein the weight of the accessory name is in a positive proportional relationship with the length of the accessory name.

[0030] As a further scheme of the present application, when a containing relationship graph is used, the step of segmenting the accessory commodity name is:

[0031] An accessory commodity name is given, all accessory names contained in the accessory commodity name are found by searching for a containing relationship, a directed acyclic graph of the given accessory commodity name is generated by generating a directed acyclic graph, the weights of the edges in the directed acyclic graph are set by setting weights, and the optimal path of the given accessory commodity name is found by a dynamic programming algorithm, and the optimal path obtained is the segmentation result of the given accessory commodity name.

[0032] The present application also includes an automobile accessory name segmentation system, which uses the aforementioned automobile accessory name segmentation method to segment the automobile accessory commodity name, and includes an accessory name list creation module, a containing relationship graph creation module, and a containing relationship graph use module.

[0033] The accessory name list creation module is used to create an accessory name list by manual annotation.

[0034] The containing relationship graph creation module is used to perform the operation of finding a containing relationship for each accessory name in the accessory name list sorted by accessory name length, generating a directed acyclic graph, setting weights, finding the optimal path in all paths according to a dynamic programming algorithm, updating the containing relationship graph, and obtaining the segmentation result of the executed accessory name, and is also used to add the currently executed accessory name to a created accessory name list with an initial state of empty, and perform the above operation on all accessory names in the sorted accessory name list to complete the creation of the containing relationship graph of the accessory name.

[0035] The containing relationship graph use module is used to segment the accessory commodity name using the created containing relationship graph of the accessory name.

[0036] The present application also includes a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the automobile accessory name segmentation method when executing the computer program.

[0037] The present application also includes a storage medium, which stores a computer program, and the computer program implements the steps of the automobile accessory name segmentation method when executed by a processor.

[0038] The technical scheme provided by the present application has the following beneficial effects:

[0039] The automobile accessory name segmentation method and system of the present application uses a containment relationship graph to save the segmentation results in a graph structure, not only completes Chinese segmentation, but also saves the automobile accessory field knowledge contained in the segmentation results, providing an important knowledge source for creating a knowledge graph in the automobile accessory field. When the containment relationship graph and other knowledge are combined, new knowledge and relationships can be automatically generated. Moreover, the segmentation results of the accessory name are saved in the containment relationship graph, so the containment relationship graph and the standard accessory category can be obtained simultaneously.

[0040] The automobile accessory name segmentation method and system of the present application also defines the hierarchical relationship between accessory names through the containment relationship graph, greatly reducing the possibility of segmentation errors caused by the diversity and uncertainty of the combination of accessory names, and improving the accuracy of Chinese segmentation.

[0041] The automobile accessory name segmentation method and system of the present application can also improve the efficiency of Chinese segmentation through the containment relationship graph. When the containment relationship graph accumulates more and more accessory names, the segmentation results of most common accessory names are already saved in the containment relationship graph, and when the same accessory name appears again, the segmentation result can be directly found through the directed edge of the containment relationship graph. The directed edge of the containment relationship graph can be saved in a dictionary data structure, so the segmentation result will be returned with constant time complexity, improving the efficiency of Chinese segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application together with the embodiments thereof, and explain the present application, but do not limit the present application. In the drawings:

[0043] Figure 1 A flowchart of an automobile accessory name segmentation method according to an embodiment of the present application.

[0044] Figure 2 An ID diagram of an accessory name in an automobile accessory name segmentation method according to an embodiment of the present application.

[0045] Figure 3 A diagram of five edges of a directed acyclic graph generated in an automobile accessory name segmentation method according to an embodiment of the present application.

[0046] Figure 4 A diagram of calculating an optimal path in an automobile accessory name segmentation method according to an embodiment of the present application.

[0047] Figure 5 A system block diagram of an automobile accessory name segmentation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not used to limit the present application.

[0049] The present application provides a kind of automobile accessory name word segmentation method and system, using recursive method to short text is segmented, by a accessory name inclusion relation diagram to save segmentation result.In the present application, segmentation algorithm can be divided into two steps, the first step is to create accessory name inclusion relation diagram, the second step is to use accessory name inclusion relation diagram.Accurate segmentation is realized to automobile accessory commodity name by these two steps.The following will be described in combination with specific examples.

[0050] As shown in Figure 1 An embodiment of the present application provides an automobile accessory name word segmentation method, which is used for segmenting automobile accessory commodity name, and includes the following steps:

[0051] S1, create accessory name list.

[0052] In the embodiment, the accessory name list is created by manual annotation. Generally, it is manually annotated or extracted by experts in the field.

[0053] To avoid ambiguity of single word accessory name, one accessory name must correspond to one automobile accessory category, and all words in one accessory name are related to automobile accessory.In the embodiment, all accessory names are composed of two or more words, and one accessory name corresponds to one automobile accessory category, thereby avoiding many ambiguities. Therefore, all single words are not used as accessory names.

[0054] In the embodiment, when defining accessory name, one accessory name must correspond to one automobile accessory category, and all words in one accessory name are related to automobile accessory. However, when the embodiment is applied to segment automobile accessory commodity name, the accessory commodity name can contain some information unrelated to automobile accessory. For example:

[0055] "engine produced by Shanghai Automotive Industry Group" can be an accessory commodity name, but when defining, "engine produced by Shanghai Automotive Industry Group" cannot be an accessory name, because it contains a lot of information unrelated to automobile accessory. In the above example, "Shanghai Automotive Industry Group", "produce" and "of" are not accessory names, and "engine" is an accessory name. Therefore, an accessory commodity name can contain accessory name and some non-accessory name words, and of course an accessory commodity name can only contain accessory name.

[0056] In addition, there are also inclusion relations in the accessory name, for example, "front wheel", "bearing" and "front wheel bearing" are accessory names, but "front wheel bearing" contains the first two accessory names. "Front wheel bearing" is a more specific classification of "bearing". It should be noted that non-accessory names are often included in accessory names. For example, "front brake pad" is an accessory name, which contains the accessory name "brake pad" because "front brake pad" is a kind of "brake pad", but "front brake pad" also contains the non-accessory name "front".

[0057] It should be particularly pointed out that the programming language used in the present application is Python. The program uses the parallel processing function concurrent.futures.ProcessPoolExecutor to perform parallel execution on a 4-core processor. Moreover, the obtained accessory name list is saved in a relational database, and the accessory name list does not contain duplicate accessory names.

[0058] S2, create an inclusion relation graph.

[0059] In the embodiment of the present application, before creating the inclusion relation graph of the accessory name, the method for sorting the created accessory name list by accessory name length is: arranging the created accessory name list by accessory name length from short to long.

[0060] The created accessory name list is sorted by accessory name length, and a processed accessory name list with an initial state of empty is created.

[0061] The reason is that after completing the accessory name list annotation, an inclusion relation graph of the accessory name needs to be created to save the word segmentation result. Before that, since the accessory name list may contain tens of thousands to hundreds of thousands of non-duplicate accessory names, the annotated accessory name list needs to be arranged by accessory name length from short to long. At the same time, a processed accessory name list is created, and the initial state of the processed accessory name list is empty.

[0062] In the loop of the sorted accessory name list, each accessory name in the sorted accessory name list is created and added to the created processed accessory name list after creating the inclusion relation graph.

[0063] In this embodiment, the following operations are performed on each accessory name in the sorted accessory name list:

[0064] Find the inclusion relation; generate a directed acyclic graph; set the weight, find the optimal path in all paths according to the dynamic programming algorithm, update the inclusion relation graph, and get the word segmentation result of the executed accessory name.

[0065] The current executing accessory name is added to the processed accessory name list, and the above operation is performed on all accessory names in the sorted accessory name list to complete the creation of the inclusion relationship graph of the accessory name.

[0066] Specifically, in creating the inclusion relationship graph of the accessory name, the first step is to find the inclusion relationship.

[0067] In finding the inclusion relationship, for the accessory name currently cycled to, all accessory names containing the accessory name currently cycled to are found in the processed accessory name list and marked as a child list of the accessory name currently cycled to, including the following steps:

[0068] The current executing accessory name in the sorted accessory name list is obtained, and the length of the current executing accessory name is determined; when the length of the current executing accessory name is 2, the current executing accessory name is defined as a basic accessory name and added to the processed accessory name list; when the length of the current executing accessory name is greater than 2, all accessory names containing the current executing accessory name are queried in the sorted accessory name list to generate a child list of the current executing accessory name.

[0069] Among the sorted accessory name list from short to long, the basic accessory name with a length of 2 is added to the processed accessory name list, and when the accessory name with a length greater than 2 is executed, all accessory names with a length of 2 are contained in the processed accessory name list, and all accessory names containing the accessory name with a length greater than 2 currently cycled to are found in the processed accessory name list containing all accessory names with a length of 2.

[0070] In this embodiment, for the accessory name currently cycled to in the sorted accessory name list (denoted as oe_name), all accessory names containing oe_name are found in the processed accessory name list, and these accessory names are denoted as a child list sub_names. When the length of oe_name is 2, no other accessory name is contained, so sub_names will be an empty list. When an accessory name does not contain any other accessory name, it is defined as a basic accessory name. Therefore, in this embodiment, all accessory names with a length of 2 are basic accessory names. When oe_name is a basic accessory name, it does not have any inclusion relationship and can be directly added to the processed accessory name list.

[0071] When the length of oe_name is greater than 2, it may contain other accessory names, and at this time the sub_names list is not empty. For example, when oe_name is “front axle”, the processed accessory name list contains all accessory names with a length of 2. In the processed accessory name list, “front wheel” and “axle” can be found to contain “front axle”. Therefore, the sub_names list is [“front wheel”, “axle”].

[0072] In the first step, a Trie data structure is used to improve the speed of string searching when finding all the accessory names contained in oe_name. The Trie is a prefix tree structure. All the accessory names in the list of processed accessory names are inserted into the Trie structure. When searching for the sub-strings of the oe_name string, the prefix tree structure of the Trie can improve the speed of the loop search by nearly 10 times.

[0073] The second step is to generate a directed acyclic graph.

[0074] When generating the directed acyclic graph of the current accessory name according to the inclusion relationship, an ID is set for each word of the accessory name in the sub-list of the current accessory name, and an ID is added at the end of each accessory name as an end identifier; the IDs in the accessory name are taken as the points of the directed acyclic graph, and each accessory name in the sub-list is taken as an edge in the directed acyclic graph of the current accessory name, to construct and generate the directed acyclic graph of the current accessory name.

[0075] In this embodiment, a directed acyclic graph (DAG) is generated according to the sub_names list in the first step. First, an ID is set for each word in the accessory name, and the index of each word in the string is taken as its ID, for example, the IDs of the words of “front axle” are 0, 1, and 2 respectively, and an ID representing the end is added at the end of each accessory name, so the ID list of the accessory name “front axle” is [0, 1, 2, 3], where ID=3 represents the end identifier of the accessory name. These IDs will be taken as the points of the directed acyclic graph, that is, the directed acyclic graph has four points: 0, 1, 2, and 3.

[0076] Referring to Figure 2 , in the figure, 0, 1, 2, and 3 are the four points of the directed acyclic graph. “Front” is a directed edge of the directed acyclic graph (the arrow in the figure represents the direction of the directed edge), which points from “front” to the next word “axle” of “wheel”.

[0077] According to the first step, the sub_names list of “front axle” is [“front wheel”, “wheel axle”], and each accessory name in the sub_names list will be an edge in the directed acyclic graph. For example, “front wheel” will be represented as (0, 2) because “front wheel” connects “front” (ID=0) to “axle” (ID=2), and “wheel axle” will be represented as (1, 3) because “wheel axle” connects “wheel” (ID=1) to the end of the oe_name string (ID=3). Referring to Figure 3 , the five edges of the directed acyclic graph are “front wheel”, “wheel axle”, “front”, “wheel”, and “axle”.

[0078] Note that the above definition is just one of the forms of converting text into a directed acyclic graph, and any other valid conversion form is acceptable.

[0079] The purpose of constructing a directed acyclic graph is to find a path from the beginning to the end of the oe_name string. However, in the above example, there is no way to find a path from the beginning (ID = 0) to the end (ID = 3) from the existing two edges (0, 2) and (1, 3). In order to ensure that the path from the beginning of the oe_name to the end always exists, the single-word sequence of the oe_name is added to the sub_names list: ["front", "wheel", "axle"]. At this time, the sub_names list becomes ["front wheel", "wheel axle", "front", "wheel", "axle"], and the edges of the directed acyclic graph represented by ID are: [(0, 2), (1, 3), (0, 1), (1, 2), (2, 3)]. At this time, the path through the entire oe_name string is easy to find, for example, [(0, 2), (2, 3)] represents ["front wheel", "axle"], and [(0, 1), (1, 3)] represents ["front", "wheel axle"]. However, the problem is that we need to find an optimal path among these paths. In order to achieve this goal, a weight needs to be added to each edge of the directed acyclic graph.

[0080] Step 3 is to set the weight, and set a weight for each edge of the directed acyclic graph.

[0081] When setting the weight for each edge in the generated directed acyclic graph, the weight of each word in the currently executed accessory name is set in order according to the monotonic decreasing function, and the average weight of all words in the currently executed accessory name is obtained. The average weight obtained and the calculation result of the length weight value set for the corresponding accessory name are used as the total weight.

[0082] After Step 2 is completed, the created inclusion relationship graph of the accessory name will be saved in a relational database. When the inclusion relationship graph needs to be used, the data will be loaded from the relational database, and the loaded inclusion relationship graph will be placed in the memory.

[0083] In this step, the inclusion relationship graph is used for word segmentation, and a recursive function is used to search the inclusion relationship graph to obtain a fine-grained word segmentation result.

[0084] According to the second step, the elements in the list of sub_names are divided into two categories. One category is the words with length greater than or equal to 2, which are defined as the basic accessory names. The other category is the single-character sequences with length 1. In this embodiment, the single-character sequences with length 1 are ignored. Therefore, only the words with length greater than or equal to 2 in the tokenization result of oe_name are considered to determine the category of oe_name. Since the single-character sequences only play an auxiliary role in the tokenization process, a weight that can be almost ignored is set for each single-character, for example, the weight is set to 1 x 10 -10 . In the final tokenization result, only the words with length greater than or equal to 2 are considered, and the single-character sequences with length 1 are ignored.

[0085] In this embodiment, for the words with length greater than or equal to 2, the frequency of the word in the corpus is not used as the weight, but two other indicators are used. As described above, one feature of the accessory product name is that it is short and directly describes the accessory product itself. According to this feature, it is assumed that the characters closer to the right in the accessory product name are more important. Because the left of the accessory product name may be some adjectives describing the product, and the right is more likely to be a noun representing the product itself.

[0086] Therefore, for the characters contained in oe_name, a monotonically decreasing function is used to set the weight of each character in the order from right to left. For example, when oe_name is "front axle", the weight of each character may be "front" = 0.2, "wheel" = 0.6, "axle" = 1.0. At this time, when a word contains multiple characters, the average weight of all characters contained in the word is taken, for example, the weight of "front wheel" is (0.2 + 0.6) / 2 = 0.4, and the weight of "wheel axle" is (0.6 + 1.0) / 2 = 0.8. Another indicator for setting the weight is the length of the word. The longer the word, the higher the weight. For example, the final weight of the word can be obtained by multiplying the weight of the word just calculated by the length of the word raised to the power of λ, where λ is a number greater than 1. When λ = 2, the weight of "front wheel" is 0.4 x 2 λ = 1.6, and the weight of "wheel axle" is 0.8 x 2 λ = 3.2. Again, it is emphasized that the above two indicators for calculating the weight are only effective for words with length greater than or equal to 2. Therefore, for the words ["front wheel", "wheel axle", "front", "wheel", "axle"] in the list of sub_names, the final weights are [1.6, 3.2, 1 x 10 -10 , 1 x 10 -10 , 1 x 10 -10 ] respectively.

[0087] The fourth step is to find the optimal path in the directed acyclic graph using the dynamic programming algorithm.

[0088] After the weights of the elements in the sub_names list are set, the dynamic programming algorithm can be used to find the optimal path in the directed acyclic graph. The dynamic programming algorithm is a known algorithm and will not be described here. It should be noted that the dynamic programming algorithm finds the optimal path from right to left, and the total weight of a path is the sum of the weights of all the edges contained in the path. The dynamic programming algorithm will find the path with the highest total weight among all the paths.

[0089] Referring to Figure 4 The dynamic programming algorithm is used to calculate the optimal path: "front", "axle".

[0090] Step 5 is to update the containment graph.

[0091] In this embodiment, after the optimal path is calculated using the dynamic programming algorithm, the word segmentation result of the oe_name is obtained. In the above example, the optimal path is [(0, 1), (1, 3)], which represents ["front", "axle"], and the total weight of the path is 3.2 + 1 x 10 -10 . Next, the word segmentation result is saved in the containment graph. As described above, single characters in the word segmentation result will not be considered. Therefore, for the word segmentation result ["front", "axle"], only ["axle"] is retained. For this oe_name, the conclusion is that the "front axle" accessory name contains the "axle" accessory name. Therefore, the containment relationship needs to be inserted into the containment graph.

[0092] In this embodiment, the initial state of the containment graph is empty. The accessory names represent the points of the containment graph. The information of the points does not need to be saved. The edges of the containment graph can be represented by a relationship list. The relationship list is initialized as an empty list. It is specified that an accessory name can contain at most two sub-accessory names, for example, the "air conditioner filter" accessory name contains two sub-accessory names "air conditioner" and "filter", and the two sub-accessory names are in order, with "air conditioner" on the left and "filter" on the right. The data structure of ("air conditioner filter", ("air conditioner", "filter")) is used to represent an element in the relationship list, which contains two directed edges, one edge is the containment relationship between "air conditioner filter" and "air conditioner", and the other is the relationship between "air conditioner filter" and "filter". Therefore, the containment relationship between "front axle" and "axle" in the above example will be represented as ("front axle", (None, "axle")), because "front axle" contains only one sub-accessory name, and the left sub-accessory name is set to empty (None).

[0093] Since the list of accessory names in Step 1 is sorted by the length of the accessory names from short to long, and the weight of an accessory name is proportional to the length of the accessory name according to Step 3, the inclusion relationship of the accessory names has a recursive effect. In general, the oe_name will only contain two sub-accessory names in the segmentation result. However, in some cases, the oe_name may contain three or more sub-accessory names, for example, the accessory name A contains three sub-accessory names [D, C, B]. In this case, the accessory names C and B are concatenated into a new accessory name CB, and the inclusion relationship of CB and (C, B) is represented as (CB, (C, B)), and the inclusion relationship of A and (D, CB) is represented as (A, (D, CB)). After obtaining the segmentation result of the currently executed accessory name, the currently executed accessory name is added to the list of processed accessory names, and the loop of each accessory name in the sorted list is executed to create the inclusion relationship graph.

[0094] In Step 5, the directed edges of the inclusion relationship graph can be saved using the dictionary structure of Python. Assuming that the accessory name A contains two sub-accessory names B and C, their relationship can be saved as {A: [B, C]}, where A is the key in the dictionary, and the array [B, C] is the value, and the first element of the array is the left sub-accessory name, and the second element of the array is the right sub-accessory name. In this way, when searching for the inclusion relationship graph, the sub-accessory names of the accessory name can be quickly found by the accessory name.

[0095] Therefore, after the above five steps are completed, the current oe_name needs to be added to the list of processed accessory names. According to the above five steps, after the list of accessory names in Step 1 is looped, the inclusion relationship graph of the accessory names will be constructed.

[0096] S3, using the inclusion relationship graph.

[0097] In this embodiment, the inclusion relationship graph of the created accessory names is used to segment the accessory product names.

[0098] In this embodiment, when the inclusion relationship graph of the accessory names is created, the inclusion relationship graph can be used to segment the accessory product names. The steps of segmentation are consistent with the above steps 1 to 5. For a given accessory product name, all accessory names contained in the accessory product name are found using Step 1, a directed acyclic graph is generated using Step 2, the weights of each edge in the directed acyclic graph are set using Step 3, and the optimal path is found using the dynamic programming algorithm of Step 4. The found optimal path is the segmentation result. The difference is that Step 5 is not needed to update the inclusion relationship graph.

[0099] It should be noted that the word segmentation completed by the first to fourth steps is a relatively coarse-grained word segmentation result, for example, the word segmentation result of "driver seat back heater" is ["driver", "seat back heater"]. At this time, a more fine-grained word segmentation result can be obtained by using the inclusion relationship graph of the accessory name.

[0100] Because the inclusion relationship graph saves the word segmentation result of each accessory name. It can be searched from the inclusion relationship graph that "seat back heater" contains "seat" and "back heater", and "back heater" contains "back" and "heater", so the final word segmentation result obtained is ["driver", "seat", "back", "heater"].

[0101] The automobile accessory name word segmentation method of the present application uses the inclusion relationship graph to save the word segmentation result in a graph structure, not only completes Chinese word segmentation, but also saves the automobile accessory field knowledge contained in the word segmentation result, providing an important knowledge source for creating a knowledge graph in the automobile accessory field. When the inclusion relationship graph and other knowledge are combined, new knowledge and relationships can be automatically generated. Moreover, the word segmentation result of the accessory name is saved in the inclusion relationship graph, so the inclusion relationship graph and the standard accessory category can be obtained simultaneously. Moreover, the word segmentation result of the accessory name is saved in the inclusion relationship graph, so the inclusion relationship graph and the standard accessory category can be obtained simultaneously, which can improve the accuracy and efficiency of Chinese word segmentation.

[0102] As shown in FIG. 1, Figure 5 As shown in FIG. 1,

[0103] The accessory name list creation module 100 is used to create an accessory name list by manual annotation.

[0104] In this embodiment, the accessory name list is created by manual annotation by experts in the field, that is, a manual annotation or extraction is performed, one accessory name must correspond to one automobile accessory category, and all the words in one accessory name are related to automobile accessories.

[0105] In this embodiment, the accessory name list is created by manual annotation by experts in the field, that is, a manual annotation or extraction is performed, one accessory name must correspond to one automobile accessory category, and all the words in one accessory name are related to automobile accessories.

[0106] The containing relationship graph creation module 200 is configured to perform the operation of finding a containing relationship for each accessory name in the accessory name list sorted by accessory name length, generating a directed acyclic graph, setting a weight, finding an optimal path in all paths according to a dynamic programming algorithm, updating a containing relationship graph, and obtaining a segmentation result of an executed accessory name, and is further configured to add a currently executed accessory name to a created accessory name list with an initial state of null, and perform the above operation on all accessory names in the sorted accessory name list, to complete creation of a containing relationship graph of the accessory name.

[0107] In this embodiment, before the containing relationship graph of the accessory name is created, the method for sorting the created accessory name list by accessory name length is as follows: the created accessory name list is arranged by accessory name length from short to long.

[0108] When the containing relationship graph creation module 200 finds a containing relationship, for a currently cycled accessory name, all accessory names containing the currently cycled accessory name are found in the processed accessory name list, and are marked as a child list of the currently cycled accessory name, including the following steps.

[0109] The currently executed accessory name in the sorted accessory name list is obtained, and the length of the currently executed accessory name is determined; when the length of the currently executed accessory name is 2, the currently executed accessory name is defined as a basic accessory name and added to the processed accessory name list; when the length of the currently executed accessory name is greater than 2, all accessory names containing the currently executed accessory name are queried in the sorted accessory name list, and a child list of the currently executed accessory name is generated.

[0110] In the accessory name list arranged by length from short to long, the basic accessory name with a length of 2 is added to the processed accessory name list, and when the accessory name with a length greater than 2 is executed, all accessory names with a length of 2 are contained in the processed accessory name list, and all accessory names containing the currently cycled accessory name with a length greater than 2 are found in the processed accessory name list containing all accessory names with a length of 2.

[0111] When the containing relationship graph creation module 200 generates a directed acyclic graph, a directed acyclic graph is generated according to the generated child list of the accessory name, including the following steps.

[0112] An ID is set for each character of the accessory name in the child list of the currently executed accessory name, and an ID is added at the end of each accessory name as an end identifier; the ID in the accessory name is used as a point of the directed acyclic graph, and each accessory name in the child list is used as an edge in the directed acyclic graph of the currently executed accessory name, to construct the directed acyclic graph of the currently executed accessory name.

[0113] The weight setting in the containing relationship graph creating module 200 includes the following steps:

[0114] The weight of each word in the currently executed accessory name is set in order according to a monotone decreasing function, and an average weight of all words in the currently executed accessory name is obtained;

[0115] The average weight obtained and the length weight value set for the corresponding accessory name are taken as a total weight.

[0116] In the embodiment, the dynamic programming algorithm used by the containing relationship graph creating module 200 finds an optimal path in all paths according to the total weight, and the path with the highest total weight is the optimal path, wherein the total weight of a path is the sum of the weights of all edges contained in the path.

[0117] When updating the containing relationship graph, the dynamic programming algorithm calculates the optimal path, and the word segmentation result of the currently executed accessory name is obtained, and the word segmentation result is saved in the containing relationship graph; wherein the weight of the accessory name is in a proportional relationship with the length of the accessory name.

[0118] The containing relationship graph using module 300 is used to perform word segmentation on the accessory product name by using the created containing relationship graph of the accessory name.

[0119] As a further scheme of the application, the step of performing word segmentation on the accessory product name by using the containing relationship graph includes the following steps:

[0120] An accessory product name is obtained, all accessory names contained in the accessory product name are found by searching for the containing relationship, a directed acyclic graph of the accessory product name is generated by generating a directed acyclic graph, the weight of each edge in the directed acyclic graph is set by setting the weight, the optimal path of the accessory product name is found by using the dynamic programming algorithm, and the optimal path is taken as the word segmentation result of the accessory product name.

[0121] The automobile accessory name word segmentation system adopts the steps of the automobile accessory name word segmentation method of the foregoing embodiment when executed, and therefore, the running process of the automobile accessory name word segmentation system in the embodiment will not be described in detail.

[0122] In an embodiment of the application, a computer device is provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps in the foregoing method embodiments when executing the computer program.

[0123] An artificially annotated accessory name list is obtained, and a containing relationship graph of the accessory name is created.

[0124] The accessory name is segmented using the inclusion relationship graph of the accessory name to obtain a segmentation result.

[0125] In another embodiment of the present application, a storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the above-mentioned method embodiments.

[0126] An artificially labeled accessory name list is obtained, and an inclusion relationship graph of the accessory name is created.

[0127] The accessory name is segmented using the inclusion relationship graph of the accessory name to obtain a segmentation result.

[0128] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments can be included. Any reference to a memory, storage, database or other medium in the embodiments provided by the present application can include at least one of a non-volatile and volatile memory.

[0129] In summary, the present application uses an inclusion relationship graph to save the segmentation result in a graph structure. Not only is Chinese segmentation completed, but also the domain knowledge contained in the segmentation result is saved, which provides an important knowledge source for creating a knowledge graph in the field of automobile accessories. Moreover, when the inclusion relationship graph and its knowledge are combined, new knowledge and relationships can be automatically generated. For example, when the segmented accessory name is classified into a standard automobile accessory category library, a contrast relationship between the accessory name and the standard accessory category is obtained. The segmentation result of the accessory name is saved in the inclusion relationship graph, so a contrast relationship between the inclusion relationship graph and the standard accessory category is also obtained. Now, assume that accessory name A contains sub-accessory names B and C, and accessory name X contains sub-accessory names B and Y. If it is known that accessory names C and Y are mapped to the same standard accessory category, it can be inferred that accessory names A and X also represent the same standard accessory category.

[0130] In the present application, the inclusion relationship graph defines the hierarchical relationship between the accessory names. Generally, a longer accessory name has a higher weight, for example, the weight of "air conditioner filter" is greater than the sum of the weights of "air conditioner" and "filter". Therefore, the dynamic programming algorithm will tend to select a longer accessory name rather than a combination of many short accessory names. This greatly reduces the possibility of segmentation errors caused by the diversity and uncertainty of the combination of accessory names. That is, the inclusion relationship graph can improve the accuracy of Chinese segmentation.

[0131] The inclusion relation graph can also improve the efficiency of Chinese word segmentation. When more and more accessory names are accumulated in the inclusion relation graph, the word segmentation results of most common accessory names are saved in the inclusion relation graph. When the same accessory name appears again, the word segmentation result can be directly found through the directed edge of the inclusion relation graph. The directed edge of the inclusion relation graph can be saved in the dictionary data structure, so the word segmentation result is returned with constant time complexity.

[0132] The above merely provides the preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall fall within the scope of protection of the present application.

Claims

1. A method for word segmentation of automotive parts names, characterized in that, Includes the following steps: Step 1: Create a list of parts names: A list of parts names is created through manual labeling. Each part name in the list consists of two or more characters, and each part name corresponds to a category of automotive parts. Step 2: Create an inclusion diagram: Sort the created list of part names by part name length, and create an initially empty list of processed part names; perform the following operations on each part name in the sorted list of part names: Find inclusion relationships; generate a directed acyclic graph (DAG); set weights, find the optimal path among all paths using a dynamic programming algorithm, update the inclusion relationship graph, and obtain the word segmentation result of the executed part name. Specifically, when finding inclusion relationships, for the currently looped part name, search the list of processed part names for all part names that contain the currently looped part name, and mark them as a sublist of the currently looped part name. This includes the following steps: obtain the currently executed part name from the sorted part name list, determine the length of the currently executed part name; if the length of the currently executed part name is 2, define it as a base part name and add it to the list of processed part names; if the length of the currently executed part name is greater than 2, query the list of sorted part names for all part names that contain the currently executed part name, and generate a sublist of the currently executed part name; wherein, generating... When generating a directed acyclic graph (DAG), a DAG is generated based on the sublist of generated part names, including the following steps: Assign an ID to each character of the part name in the sublist of the currently executed part name, and add an ID to the end of each part name as an end marker; use the IDs in the part names as nodes in the DAG, and each part name in the sublist as an edge in the DAG of the currently executed part name, thus constructing the DAG of the currently executed part name; wherein, setting weights involves: setting weights for each edge in the generated DAG, including the following steps: setting weights for each character in the currently executed part name sequentially according to a monotonically decreasing function, and obtaining the average weight of all characters in the currently executed part name; the calculated result of the average weight and the length weight value set for the corresponding part name is used as the total weight; Add the currently executed part name to the list of processed part names, and perform the above operation on all part names in the sorted part name list to complete the creation of the part name inclusion relationship diagram; Step 3: Use an inclusion diagram: Use the inclusion relationship graph of the created accessory names to segment the accessory product names into words.

2. The method for segmenting automotive parts names as described in claim 1, characterized in that: The accessory name list is arranged from shortest to longest. Basic accessory names with a length of 2 are added to the processed accessory name list. When an accessory name with a length greater than 2 is executed, the processed accessory name list already contains all accessory names with a length of 2. The processed accessory name list that already contains all accessory names with a length of 2 searches for all accessory names with a length greater than 2 that are currently being looped.

3. The method for segmenting automotive parts names as described in claim 2, characterized in that: The dynamic programming algorithm finds the optimal path among all paths based on the total weight. The path with the highest total weight is the optimal path. The total weight of a path is the sum of the weights of all the edges contained in that path.

4. The method for segmenting automotive parts names as described in claim 3, characterized in that: When updating the inclusion graph, the dynamic programming algorithm calculates the optimal path and obtains the word segmentation result of the currently executed part name, and saves the word segmentation result in the inclusion graph; among them, the weight of the part name is proportional to the length of the part name.

5. The method for segmenting automotive parts names as described in claim 4, characterized in that: When using an inclusion diagram, the steps for segmenting accessory product names are as follows: Given a given accessory product name, find all accessory names contained within that accessory product name by searching for inclusion relationships. Generate a directed acyclic graph (DAG) of the given accessory product name by generating a directed acyclic graph (DAG). Set the weights of each edge in the DAG by setting weights. Use a dynamic programming algorithm to find the optimal path for the given accessory product name. The obtained optimal path is the word segmentation result of the given accessory product name.

6. A word segmentation system for automotive parts names, characterized in that: The automotive parts name segmentation system uses the automotive parts name segmentation method described in any one of claims 1-5 to segment automotive parts product names; the automotive parts name segmentation system includes: The Parts Name List Creation Module is used to create a parts name list through manual annotation. The inclusion graph creation module is used to perform operations such as finding inclusion relationships for each part name in a list of part names sorted by part name length, generating a directed acyclic graph, setting weights, finding the optimal path among all paths using a dynamic programming algorithm, updating the inclusion graph, and obtaining the word segmentation results of the executed part names. It also adds the currently executed part name to a newly created list of processed part names (initially empty), and performs the above operations on all part names in the sorted list of part names, thus completing the creation of the inclusion graph for the part names; and The inclusion graph module is used to segment accessory product names using the inclusion graph of the created accessory names.

Images