Natural language recognition method and device, equipment and storage medium

Abstract

The invention relates to a natural language recognition method and device, computer equipment and a computer storage medium, and the method comprises the steps: for a to-be-processed natural language, extracting key information in the natural language, the key information comprising information of at least one language; and in a standardized lexicon, according to a term lexicon association line, determining a format of standard data corresponding to the key information, the term lexicon association line being used for representing a corresponding relationship between the format of the natural language and the format of the standard data. According to the method, efficient distribution can be realized, and bed type descriptions provided by suppliers are accurately analyzed and converted into unified standard bed type data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a natural language recognition method, apparatus, computer device, and computer-readable storage medium. Background Art

[0002] With the rapid development of the company's hotel distribution system, in order to unify the distribution standards of hotel accommodation resources and improve resource integration efficiency, it is necessary to standardize the data of major hotel suppliers.

[0003] In the standardization process, the standardization of bed type data is a crucial step. However, the description of bed types in hotel rooms is usually presented in natural language and lacks a unified standard, which poses a huge challenge to data integration. In order to achieve efficient distribution, the bed type descriptions provided by suppliers must be accurately parsed and converted into unified standard bed type data. Summary of the invention

[0004] The embodiments of the present application provide a natural language recognition method, apparatus, computer device and storage medium, aiming to solve the problems of low distribution efficiency and low natural language recognition accuracy.

[0005] In a first aspect, an embodiment of the present application provides a natural language recognition method, which includes: for a natural language to be processed, extracting key information from the natural language, wherein the key information includes information of at least one language; in a standardized vocabulary, determining the format of standard data corresponding to the key information based on a terminology vocabulary associated row, wherein the terminology vocabulary associated row is used to represent the correspondence between the formats of the natural language and the standard data.

[0006] In a second aspect, an embodiment of the present application also provides a natural language recognition device, comprising an extraction unit for extracting key information from the natural language to be processed, wherein the key information includes information of at least one language; and a determination unit for determining, in a standardized vocabulary, the format of standard data corresponding to the key information according to a terminology vocabulary associated row, wherein the terminology vocabulary associated row is used to represent the correspondence between the formats of the natural language and the standard data.

[0007] In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0008] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program can implement the above method when executed by a processor.

[0009] The embodiment of the present application provides a natural language recognition method, apparatus, computer equipment and storage medium. The method includes: extracting key information from the natural language to be processed, wherein the key information includes information of at least one language; in a standardized word library, determining the format of the standard data corresponding to the key information according to the term word library associated line, wherein the term word library associated line is used to represent the correspondence between the format of the natural language and the standard data. It can achieve efficient distribution, accurately parse the bed type description provided by the supplier, and convert it into unified standard bed type data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0013] Figure 1 A flow chart of a natural language recognition method provided in an embodiment of the present application;

[0014] Figure 2 A structural diagram of a natural language recognition device provided in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of the computer device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0017] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0018] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0019] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0020] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0021] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0022] In order to solve the technical problem of low accuracy of natural language parsing in the prior art, especially low distribution efficiency in the hotel distribution system, the present application provides a natural language recognition device that can achieve efficient distribution, accurately parse the bed type description provided by the supplier, and convert it into unified standard bed type data.

[0023] Figure 1 A natural language recognition method is provided in an embodiment of the present application, and the specific processing flow of the method is as follows:

[0024] S101, obtaining a standardized vocabulary.

[0025] In this step, the data set to be trained is determined in advance, and then the keywords in the key information in the data set to be trained are determined, and the format of the standard data, the corresponding relationship between the terminology database associated lines and the keywords in the data set to be trained are determined.

[0026] In the embodiments proposed in this application, a hotel distribution system will be used as an example for detailed description. The hotel distribution system helps hotels achieve more efficient sales and reservation management and improve the market competitiveness and revenue of hotels by integrating multiple online channels and reservation platforms.

[0027] The hotel distribution system proposed in this application can be a hotel distribution system used by a single hotel, or it can be a hotel distribution system used by a distribution platform.

[0028] In the first implementation method, if used in a single hotel, the format of the standard data in the standardized vocabulary can be stored in the form of keywords, and the term vocabulary associated row can contain at least one of the following: distribution platform identification, room type information, bed type information, number of bed types, and hotel-related remarks.

[0029] The second implementation method is that if the hotel distribution system proposed in this application is applied in a distribution platform scenario, the format of the standard data in the standardized vocabulary can be stored in the form of keywords, and the term vocabulary associated row can include at least one of the following: hotel merchant logo, room type information, bed type information, number of bed types, and notes on other information related to the hotel.

[0030] In this step, the training dataset can be a vocabulary integration of bed type terms in the hotel industry:

[0031] Collect and organize the commonly used bed type terms in the hotel industry, extract keywords from the bed type terms, and then perform cluster analysis on the keywords to determine the format of the standard data, the correspondence between the associated rows of the terminology library and the keywords in the training data set, store the correspondence, and obtain the table conversion vocabulary.

[0032] Specifically, the standardized vocabulary library may include the hotel logo / distribution platform logo, serial number, English name, Chinese name or French name of various bed types, and detailed description of various bed types, room types, hotel geographical location, environment and other related information.

[0033] A preferred implementation is shown in Table 1 below. The standardized vocabulary in the hotel distribution system may include bed types (in Chinese and English) and notes on bed type information.

[0034] It should be noted that the ID in the following Table 1 can be a hotel ID, such as merchant 1, merchant 2, etc.; it can also be an ID of a distribution system, such as group buying website 1, distribution website 2, WeChat applet, etc.

[0035] Table 1

[0036]

[0037]

[0038]

[0039]

[0040]

[0041] The standardized word library proposed in the embodiment of the present invention also provides a definition of the labeled window type data, which is used to describe the basic data structure of bed type information, including bed type ID, bed width (unit: meter) and quantity, etc. At the same time, the data structure of bed type groups and bed type group combination relationships is also defined to support a variety of bed type configurations.

[0042] A better implementation method is to first define a basic data structure for describing bed type information. Bed type information is the most basic metadata. The metadata structure needs to provide detailed information of each bed type, bed size, quantity, etc.

[0043] The data structure of the C#Class class is defined as follows:

[0044]

[0045] Bed Type Group

[0046] Define the category of bed type group, which is used to identify the combination and association of multiple different bed types. A bed type group can have multiple bed type information. The bed type information in a bed type group is an AND relationship diagram.

[0047] The data structure of the C#Class class is defined as follows:

[0048]

[0049] Bed type group combination relationship

[0050] Describes the relationship between different bed type combinations and supports a variety of bed type configurations. Set the bed type information within a bed type group to be an AND relationship, and the relationship between bed type groups is an OR relationship.

[0051] The data structure of the C#Class class is defined as follows:

[0052]

[0053] Bed group combination examples

[0054] For example, to express "2 single beds or 1 large bed", the corresponding C# data structure converted to JSON is as follows:

[0055] {

[0056] "BedGroups":[

[0057] {"BedInfos":[{"ID":1,"Count":2}]},

[0058] {"BedInfos":[{"ID":3,"Count":1}]} ]

[0060] }

[0061] S102, receiving natural language.

[0062] The natural language can be sent by the merchant or the hotel distribution platform.

[0063] The present application embodiment will be described in detail by taking the hotel merchant sending natural language as an example, that is, the ID column in the above Table 1 is filled with the hotel merchant identifier, for example, it can be hotel merchant A, hotel merchant B...

[0064] S103, extracting key information from the received natural language.

[0065] The key information includes information in at least one language.

[0066] The key information can be bed type keywords or hotel business logos. Natural language can be the languages ​​of various countries. When processing natural language, it is necessary to first determine the country described by the natural language, and then extract the key information in the natural language according to the pre-set word segmentation processing method.

[0067] Key information includes bed type information and business logo;

[0068] For the natural language to be processed, key information in the natural language is extracted, including:

[0069] The bed type information and the business logo in the natural language are extracted through regular expressions and / or grammar state machine word segmentation judgment.

[0070] By parsing the supplier's bed type description text and extracting key bed type information in the description text according to the relevance of the bed type terminology vocabulary, a set of bed type type information contained in the supplier's bed type description text is obtained.

[0071] For example, the description provided by the supplier:

[0072] “business plus room,1 double or 2 single beds,courtyard view-breakfast”

[0073] According to the bed type vocabulary, through regular expressions and some grammatical state machine word segmentation judgment, the bed type information can be extracted as "double bed" and "single bed"

[0074] After extracting the bed type keywords to obtain the relevant bed type categories, it is necessary to further analyze the bed type quantity based on the additional description of the bed type.

[0075] For example, the description provided by the supplier:

[0076] "bus i ness pl us room,1doub le or 2s i ng le beds,courtyard view-breakfast"

[0077] According to the bed type vocabulary, after extracting the bed type, we need to get the number of bed types. Then read the expression judgment about quantity before and after the vocabulary text, and use the regular or quantity grammar state machine for analysis. We can identify 1doub le and 2s i ng le, and then analyze that the number of beds is "1doub le bed" and "2s i ng le bed".

[0078] After obtaining the bed type information, the next step is to determine the data relationship between multiple bed types and to parse the grammatical relationship in the description.

[0079] For example, the description provided by the supplier:

[0080] "bus i ness pl us room,1doub le or 2s i ng le beds,courtyard view-breakfast"

[0081] The key word "or" in natural grammar is identified, and the relationship between "1doub le bed" and "2single bed" is an or relationship. Similarly, in natural language, some common grammatical vocabulary needs to be sorted out, such as 'or', 'and', and even punctuation marks such as '-' and '\'.

[0082] S104, in the standardized word library, according to the term word library associated row, determine the format of the standard data corresponding to the key information.

[0083] The terminology vocabulary associated row is used to represent the correspondence between the natural language and the format of standard data. The key information includes the room type information that the merchant can provide; using the room type information as the index value, according to the terminology vocabulary associated row, determine the room type expression corresponding to the room type information in the standardized vocabulary. The key information includes the number of bed types that the merchant can provide; using the merchant ID as the index value, according to the terminology vocabulary associated row, determine the number of bed types corresponding to the merchant ID in the standardized vocabulary, if the standardized terminology vocabulary associated row includes at least one of the following: merchant ID; room type information; number of bed types. If the number of bed types corresponding to the merchant ID in the vocabulary is consistent with the number of bed types that the merchant can provide in the key information, it will not be modified, otherwise the number of bed types corresponding to the merchant ID in the standardized vocabulary is modified to the number of bed types that the merchant can provide in the key information.

[0084] A better implementation method for bed type keyword extraction is: by parsing the supplier's bed type description text, using the association of the standardized vocabulary, the key bed type information in the description text can be extracted. After extracting the bed type keywords, further analyze the additional information about the number of bed types in the description text to determine the specific number of each bed type. Analysis of multi-bed type relationship description text: Determine the data relationship between multiple bed types, parse the grammatical relationship in the description text, such as "or", "and" and other words, to determine the logical relationship between the bed types. Convert the above analysis results into a standard bed type data format. Using the defined data structure, integrate the bed type information, bed type groups and bed type group combination relationships into a unified JSON structure to obtain the final standard bed type data.

[0085] S105: updating the standardized vocabulary according to the key information.

[0086] If the number of bed types corresponding to the merchant logo in the standardized vocabulary is consistent with the number of bed types that the merchant can provide in the key information, then no modification is made; otherwise, the number of bed types corresponding to the merchant logo in the standardized vocabulary is modified to be the number of bed types that the merchant can provide in the key information.

[0087] The analysis results are converted into a standard bed type data format. The final standard bed type data obtained by integration is "2 single beds or 1 large bed". The JSON structure is as follows

[0088]

[0089] See also Figure 2 , Figure 2: is a schematic block diagram of a natural language recognition device provided in an embodiment of the present application. Corresponding to the above natural language recognition method, the present application also provides a natural language recognition device. The natural language recognition device includes a unit for executing the above natural language recognition method, and the natural language recognition device can be configured in a desktop computer, a tablet computer, a laptop computer, and other terminals. Specifically, the natural language recognition device includes an extraction unit and a determination unit.

[0090] An extraction unit 201 is used to extract key information from a natural language to be processed, wherein the key information includes information of at least one language;

[0091] The determination unit 202 is configured to determine the format of the standard data corresponding to the key information in the standardized word library according to the term word library associated row, wherein the term word library associated row is used to represent the correspondence between the natural language and the format of the standard data.

[0092] like Figure 3 As shown, the embodiment of the present application provides a computer device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0093] Memory 113, used for storing computer programs;

[0094] In one embodiment of the present application, the processor 111 is used to implement the natural language recognition method provided by any of the above method embodiments when executing the program stored in the memory 113. The method includes: extracting key information from the natural language to be processed, wherein the key information includes information of at least one language; in the standardized word library, determining the format of the standard data corresponding to the key information according to the term word library associated row, wherein the term word library associated row is used to indicate the corresponding relationship between the format of the natural language and the standard data.

[0095] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0096] Therefore, an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the natural language recognition method provided in any of the aforementioned method embodiments are implemented.

[0097] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which can store program codes. The computer-readable storage medium can be non-volatile or volatile.

[0098] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0099] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0100] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0101] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0102] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0103] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0104] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A natural language recognition method, characterized in that: include: For a natural language to be processed, extract key information from the natural language, wherein the key information includes information of at least one language; In the standardized word library, the format of the standard data corresponding to the key information is determined according to the term word library associated row, wherein the term word library associated row is used to represent the corresponding relationship between the natural language and the format of the standard data.

2. The method according to claim 1, characterized in that The standardized word library contains multilingual texts, and the standardized word library is established in the following manner: Determine the keywords in the key information of the training data set; Determine the format of the standard data, the correspondence between the terminology vocabulary associated rows and the keywords in the training data set.

3. The method according to claim 1 or 2, characterized in that: The key information includes bed type information and merchant logo; For the natural language to be processed, key information in the natural language is extracted, including: The bed type information and the merchant logo in the natural language are extracted through regular expressions and / or grammar state machine word segmentation judgment.

4. The method according to claim 3, characterized in that The method according to claim 1 or 2, characterized in that the key information includes room type information that the merchant can provide; In the standardized word library, the format of the standard data corresponding to the key information is determined according to the term word library associated row, including: The room type information is used as an index value, and the room type expression corresponding to the room type information in the standardized word library is determined according to the associated row of the term word library.

5. The method according to claim 3, characterized in that: The key information includes the number of bed types that the merchant can provide; In the standardized word library, the format of the standard data corresponding to the key information is determined according to the term word library associated row, including: Using the merchant ID as the index value, determine the number of bed types corresponding to the merchant ID in the standardized vocabulary according to the associated rows of the terminology library. If the number of bed types corresponding to the merchant ID in the standardized vocabulary is consistent with the number of bed types that the merchant can provide in the key information, then do not modify it; otherwise, modify the number of bed types corresponding to the merchant ID in the standardized vocabulary to the number of bed types that the merchant can provide in the key information.

6. The method according to claim 1, characterized in that The terminology library associated row includes at least one of the following: Business logo; room information; number of bed types.

7. The method according to claim 1, characterized in that Before extracting key information from the natural language to be processed, the method further includes: Receive natural language sent by the merchant; and After determining the format of the standard data corresponding to the key information, the method further includes: The standardized word library is updated according to the key information.

8. A natural language recognition device, characterized in that: include: An extraction unit, configured to extract key information from a natural language to be processed, wherein the key information includes information of at least one language; The determination unit is used to determine the format of the standard data corresponding to the key information in the standardized word library according to the term word library associated row, wherein the term word library associated row is used to represent the corresponding relationship between the natural language and the format of the standard data.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 can be implemented.

Images