Natural language data conversion method and device and electronic equipment

Abstract

The invention provides a natural language data conversion method and device and electronic equipment, and the method comprises the steps: obtaining to-be-converted data, and converting the to-be-converted data into an execution step text, the to-be-converted data at least comprises text data, and the execution step text is a text representing the conversion of the to-be-converted data; a data integration component corresponding to the execution step text is generated, and the data integration component is at least used for representing a data structure of the execution step text; the data integration components are connected according to the execution sequence, target components are obtained, the target components are controlled to execute, target conversion data corresponding to the to-be-converted data are generated, and the target conversion data are data obtained after the to-be-converted data are converted. Through the data conversion method and device, the problem of low efficiency of a data conversion method in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of data conversion, and in particular to a method, device, computer-readable storage medium and electronic device for converting natural language data. Background Art

[0002] With the popularity of the Internet, the Internet of Things, social media, mobile devices, etc., the speed and total amount of data generation are growing rapidly. Many companies and organizations use a variety of different systems and platforms to store and manage data. The data formats and structures between these systems may be different and need to be converted for unified management and analysis. When an enterprise upgrades or replaces an information system, the data in the old system needs to be migrated to the new system. This usually involves complex data conversion to ensure the correctness and consistency of the data in the new system. When business managers and decision makers rely on data-driven decisions to maintain a competitive advantage, in order to conduct effective data analysis, the raw data needs to be converted into a format suitable for analysis, which includes steps such as data cleaning, normalization, and aggregation to extract valuable information from the raw data.

[0003] The prior art discloses an intelligent query and visualization method based on natural language processing, which relates to the field of big data query analysis technology, including obtaining natural language input by a user and generating user text query data; generating corresponding database query statements based on the text query data; executing database queries and extracting data; generating personalized charts and data visualization. This processing method converts natural language into database query statements, obtains data from a single database, and visualizes it, which has the disadvantages of low efficiency and complex operation. Summary of the invention

[0004] The main purpose of the present application is to provide a natural language data conversion method, device, computer-readable storage medium and electronic device to at least solve the problem of low efficiency of data conversion methods in the prior art.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for converting natural language data is provided, comprising: obtaining data to be converted, converting the data to be converted into an execution step text, wherein the data to be converted includes at least text data, and the execution step text is a text representing the conversion of the data to be converted; generating a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text; connecting the data integration components in an execution order to obtain a target component, controlling the execution of the target component, and generating target conversion data corresponding to the data to be converted, wherein the target conversion data is data obtained after converting the data to be converted.

[0006] Optionally, converting the data to be converted into an execution step text comprises: converting the data to be converted into an execution step text comprises: splitting the data to be converted to obtain a plurality of sub-execution steps, wherein the sub-execution steps represent the minimum action steps for data conversion; extracting the operation type, operation object and execution condition of each sub-execution step, wherein the operation type represents the type of operation, the operation object represents the object of the operation, and the execution condition represents the condition for executing the sub-execution step; and splitting the sub-execution steps again to obtain the execution step text.

[0007] Optionally, the data to be converted is split, including: performing data preprocessing on the data to be converted to obtain preprocessed data, wherein the preprocessed data includes multiple words; performing part-of-speech classification on the preprocessed data to obtain a part-of-speech tag corresponding to each of the words; performing type classification according to the part-of-speech tags to obtain multiple text types; performing grammatical structure recognition on the data to be converted to obtain the grammatical structure of the data to be converted.

[0008] Optionally, after obtaining the execution step text, the method further includes: determining source data of the operation object, wherein the source data is a data set storing information of the operation object; and if the source data of the operation object does not exist, acquiring and importing the source data.

[0009] Optionally, generating a data integration component corresponding to the execution step text includes: constructing a word vector space, wherein the word vector space maps words in the execution step text to points in a real number vector space; in the word vector space, matching the operation object with the source data, and matching the operation type with the data integration component to obtain the data integration component corresponding to each of the execution steps.

[0010] Optionally, after generating the data integration component corresponding to the execution step text, the method further comprises: writing component parameters into the data integration component, wherein the component parameters at least include parameters for executing the execution step text.

[0011] Optionally, controlling the execution of the target component includes: visualizing the target component and previewing the execution result of the target component; and controlling the execution of the target component when the execution result is correct.

[0012] According to another aspect of the present application, a natural language data conversion device is provided, comprising: a conversion unit, used to obtain data to be converted, and convert the data to be converted into an execution step text, wherein the data to be converted includes at least text data, and the execution step text is a text representing the conversion of the data to be converted; a first generation unit, used to generate a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text; a second generation unit, used to connect the data integration components in an execution order to obtain a target component, control the execution of the target component, and generate target conversion data corresponding to the data to be converted, wherein the target conversion data is data obtained after converting the data to be converted.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the natural language data conversion methods.

[0014] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the natural language data conversion methods.

[0015] According to another aspect of the present application, a natural language data conversion system is provided, comprising: a data integration component, a data structure at least for representing an execution step text, wherein the execution step text is a text representing the conversion of data to be converted, the data integration component is connected in an execution order to obtain a target component, and the target component converts the data to be converted into target conversion data when executed; a controller, comprising one or more processors, and one or more programs, wherein the programs implement any one of the natural language data conversion methods when executed by the processors.

[0016] Apply the technical solution of the present application, obtain the data to be converted, convert the data to be converted into an execution step text, wherein the data to be converted includes at least text data, and the execution step text is a text representing the conversion of the data to be converted; generate a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text; connect the data integration components according to the execution order to obtain the target component, control the execution of the target component, and generate the target conversion data corresponding to the data to be converted, wherein the target conversion data is the data obtained after the data to be converted is converted. Compared with the prior art, in which the conversion steps between data are complicated and the conversion efficiency is low, the present application converts data based on the data integration component through the above steps, can extract data from multiple data sources, and then perform more complex data conversion work, to assist users in processing a large amount of data resources in a simpler and more efficient way. Therefore, it can solve the problem of low data conversion efficiency in the prior art and improve the efficiency of user data conversion processing work. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram of a process for converting natural language data provided by an embodiment of the present application is shown;

[0019] Figure 2 A specific natural language data conversion method provided by an embodiment of the present application is shown;

[0020] Figure 3 A text processing flow chart provided by an embodiment of the present application is shown;

[0021] Figure 4 A component generation flow chart provided by an embodiment of the present application is shown;

[0022] Figure 5 A structural block diagram of a natural language data conversion device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0027] ETL: It is the abbreviation of Extract, Transform, and Load, and is a commonly used data integration method.

[0028] word2Dec: It is a model that learns semantic knowledge from a large amount of text corpus in an unsupervised way and is widely used in NLP.

[0029] NLP: (Natural Language Processing, NLP for short) is an important research direction in the field of artificial intelligence.

[0030] As introduced in the background technology, the conversion steps between data in the prior art are complicated and the conversion efficiency is low. In order to solve the problem of complicated conversion steps between data and low conversion efficiency, the embodiments of the present application provide a natural language data conversion method, device, computer-readable storage medium and electronic device.

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0032] In this embodiment, a method for converting natural language data running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 1 : is a flow chart of a method for converting natural language data according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0034] Step S201, obtaining data to be converted, and converting the data to be converted into execution step text, wherein the data to be converted at least includes text data, and the execution step text is text representing the conversion of the data to be converted;

[0035] Specifically, the data to be converted can be a natural language text input by the user. According to the natural language processing technology, it is analyzed and processed to split the complete sentence text into execution step text. In the actual application process, in addition to text data, the data to be converted can also include data in text format such as numbers and symbols. The execution step text is the step of converting the data to be converted, for example: Step 1: data association; Step 2: data screening.

[0036] Step S202, generating a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text;

[0037] Specifically, according to the execution step text, based on the Word2Dec technical model, each execution step text generates a corresponding data integration component, which can be an ETL component. Word2Dec is a model that learns semantic knowledge from a large amount of text corpus in an unsupervised manner, and is widely used in NLP (Natural Language Processing, NLP for short). ETL is the abbreviation of Extract, Transform, and Load, and is a commonly used data integration method. Therefore, the data integration component is also used to extract the required data from the imported database or data table, analyze the data structure for data conversion, and load the converted data into the target system.

[0038] Step S203, connecting the data integration components in an execution order to obtain a target component, controlling the execution of the target component, and generating target conversion data corresponding to the data to be converted, wherein the target conversion data is data obtained after converting the data to be converted.

[0039] Specifically, each ETL component is connected in the order of execution and then visualized. Users can view the parameters of each ETL component, preview the execution results of the component, click Execute after confirmation, and select the data export method (save to database or excel, csv file) to complete the data conversion work and obtain the final target conversion data.

[0040] Through this embodiment, the data to be converted is obtained, and the data to be converted is converted into an execution step text, wherein the data to be converted includes at least text data, and the execution step text is a text representing the conversion of the data to be converted; a data integration component corresponding to the execution step text is generated, wherein the data integration component is at least used to represent the data structure of the execution step text; the data integration components are connected in an execution order to obtain a target component, the target component is controlled to execute, and the target conversion data corresponding to the data to be converted is generated, wherein the target conversion data is the data obtained after the data to be converted is converted. Compared with the prior art, in which the conversion steps between data are complicated and the conversion efficiency is low, the present application converts data based on the data integration component through the above steps, can extract data from a variety of data sources, and then perform more complex data conversion work, to assist users in processing a large amount of data resources in a simpler and more efficient way. Therefore, it can solve the problem of low data conversion efficiency in the prior art and improve the efficiency of user data conversion processing work.

[0041] In the specific implementation process, the above step S201 converts the data to be converted into the execution step text, which can be implemented by the following steps: Step S2011: split the data to be converted to obtain multiple sub-execution steps, wherein the sub-execution steps represent the minimum action steps for data conversion; Step S2012: extract the operation type, operation object and execution condition of each sub-execution step, wherein the operation type represents the type of operation, the operation object represents the object of operation, and the execution condition represents the condition for executing the sub-execution step; Step S2013: split the sub-execution steps again to obtain the execution step text. The method converts the data to be converted through the above steps and converts it into the execution step text.

[0042] Specifically, first, split the execution step text into each independent sub - execution step, and extract the operation type, operation object, and execution conditions of each sub - execution step. Then, split the sub - execution steps according to the above steps to obtain fields such as target objects, operation types, and operation values. Finally, obtain the total execution step text.

[0043] In some alternative embodiments, step S2011 of splitting the data to be converted includes the following steps: perform data pre - processing on the data to be converted to obtain pre - processed data, where the pre - processed data includes multiple words; perform part - of - speech tagging on the pre - processed data to obtain the part - of - speech tags corresponding to each word; perform type classification according to the part - of - speech tags to obtain multiple text types; and perform syntactic structure recognition on the data to be converted to obtain the syntactic structure of the data to be converted. By splitting through the above steps, the data to be processed after splitting is obtained, so that components can be matched according to the split data, and individual analysis and data conversion can be performed.

[0044] In the specific implementation process, data pre - processing is mainly divided into stop - word removal and text tokenization. First, stop - word removal: In natural language, many words have no actual meaning, such as "le", "de", etc., so they need to be removed. Second, text tokenization: Use the jieba tool to recombine the continuous character sequence into a word sequence according to certain specifications. Use the jieba tool to perform part - of - speech tagging and assign a part - of - speech tag to each word in the text. These tags usually indicate the grammatical function of the word in the sentence, such as nouns, verbs, adjectives, and numerals. Use the SpaCy tool to identify text types with special meanings from the text, perform type classification, and classify them into predefined categories, that is, text types, such as person names, place names, organization names, time, currency, percentages, etc. Use the SpaCy tool to analyze the syntactic structure of natural language sentences, perform syntactic structure recognition, and identify the components of the sentence and their mutual relationships.

[0045] In some alternative embodiments, after obtaining the execution step text, the method further includes: step S204: Determine the source data of the operation object, where the source data is a data set storing information about the operation object; step S205: In the case where the source data of the operation object does not exist, obtain and import the source data. By the above steps, the method imports the source data to perform data search and processing based on the source data.

[0046] Specifically, determine the source data of the operation object. For example, there are two steps: table association step and screening step. The operation objects of the table association step are the student information table and the data competition table, which come from the student management database and the competition results file. Therefore, two data import execution steps need to be added. The operation object of the screening step is the student, which comes from the data set after the table association step is executed, so no additional execution steps need to be added.

[0047] In some other optional implementations, the above step S202 generates the data integration component corresponding to the execution step text, which can be implemented by the following steps: Step S2021: Construct a word vector space, wherein the word vector space maps the words in the execution step text to points in the real vector space; Step S2022: In the word vector space, match the operation object with the source data, match the operation type with the data integration component, and obtain the data integration component corresponding to each execution step. The method generates the data integration component through the above steps, so that data conversion can be quickly realized by connecting the data integration components and controlling the execution of the components.

[0048] In the specific implementation process, the word embedding space is constructed through the Word2Dec technology model based on the data set imported by the user into the system (including the database connection information created by the user and the imported excel and csv files) and ETL components. According to the execution step text, semantic capture is performed based on the Word2Vec technology model, and the operation object is matched with the source data, the operation type and the ETL component one by one, and the data set and ETL component corresponding to each step are obtained. Word embedding is a technology in natural language processing (NLP), which maps words to points in the real vector space.

[0049] In some optional implementations, after generating the data integration component corresponding to the execution step text, the method further includes: writing component parameters into the data integration component, wherein the component parameters at least include parameters for executing the execution step text. The method improves the parameters of the component through the above steps, thus ensuring that the component is executed smoothly.

[0050] Specifically, after the component is constructed, parameters such as data sets and execution conditions are written into the above ETL component.

[0051] In order to quickly perform data conversion, the above step S203 controls the execution of the target component, including the following steps: step S2031: visualize the target component and preview the execution result of the target component; step S2032: control the execution of the target component if the execution result is correct. The method executes the target component if the above preview passes, so that accurate target conversion data after conversion can be obtained.

[0052] During the specific implementation process, users can view the parameters of each ETL component, preview the execution results of the component, click Execute after confirmation, select the data export method (save to database or excel, csv file), and the data conversion work can be completed.

[0053] The following is a specific embodiment:

[0054] After preliminary processing of "Associate the student information table in the student management database and the mathematics competition table in the competition score file by student number, and filter out students with scores greater than 95 points", the following execution steps can be obtained:

[0055] 1. Table association. Operation objects: student information table, data competition table. Execution conditions: table association based on student ID.

[0056] 2. Filter. Operation object: students. Execution condition: Mathematics score greater than 95 points.

[0057] Then the execution condition text is processed again to obtain the following execution steps:

[0058] 1. Table association. Operation objects: student information table, data competition table. Execution condition: the target object is "student number".

[0059] 2. Filter. Operation object: students. Execution condition: the target object is "math score", the operator is "greater than", and the operation value is "90".

[0060] Finally, determine the source of the operation object. Since the operation objects of the table association step are the student information table and the data competition table, which come from the student management database and the competition results file, two data import execution steps need to be added. The operation object of the screening step is the student, which comes from the data set after the table association step is executed, so no additional execution steps need to be added. So the final execution steps are:

[0061] 1. Table input. Operation object: student management database. Execution condition: the target object is "student information table".

[0062] 2. File input. Operation object: competition results file. Execution condition: target object is "data competition table".

[0063] 3. Table association. Operation objects: student information table, data competition table. Execution condition: target object is "student number".

[0064] 4. Filter. Operation object: students. Execution condition: target object is "math score", operator is "greater than", and operation value is "90".

[0065] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the natural language data conversion method of the present application will be described in detail below in conjunction with specific embodiments.

[0066] This embodiment relates to a specific method for converting natural language data, such as Figure 2 As shown, the following steps are included:

[0067] Step S1: Obtain the natural language input by the user, analyze and process it, and generate execution step text;

[0068] Step S2: Generate corresponding ETL components according to the execution step text;

[0069] Step S3: Connect the ETL components in series in the order of execution and visualize them.

[0070] Figure 3 A text processing flow chart is shown, comprising the following steps:

[0071] Step S11: text preprocessing;

[0072] Step S12: part-of-speech tagging;

[0073] Step S13: named entity recognition;

[0074] Step S14: Syntactic analysis;

[0075] Step S15: Information extraction.

[0076] Figure 4 A component generation flow chart is shown, comprising the following steps:

[0077] Step S21: Based on the data set imported into the system by the user (including the database connection information created by the user and the imported excel and csv files) and ETL components and other information, the word vector space is constructed through the Word2Dec technical model.

[0078] Step S22: According to the execution step text, semantics is captured based on the Word2Vec technology model, and the operation object and the data set, the operation type and the ETL component are matched one by one, and the data set and ETL component corresponding to each step are obtained;

[0079] Step S23: Write the data set, execution conditions and other parameters into the ETL component.

[0080] The embodiment of the present application also provides a conversion device for natural language data. It should be noted that the conversion device for natural language data of the embodiment of the present application can be used to execute the conversion method for natural language data provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0081] The following is an introduction to the natural language data conversion device provided in an embodiment of the present application.

[0082] Figure 5 Schematic diagram of a natural language data conversion device according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0083] The conversion unit 10 is used to obtain data to be converted, and convert the data to be converted into an execution step text, wherein the data to be converted at least includes text data, and the execution step text is a text representing the conversion of the data to be converted;

[0084] Specifically, the data to be converted can be a natural language text input by the user. According to the natural language processing technology, it is analyzed and processed to split the complete sentence text into execution step text. In the actual application process, in addition to text data, the data to be converted can also include data in text format such as numbers and symbols. The execution step text is the step of converting the data to be converted, for example: Step 1: data association; Step 2: data screening.

[0085] A first generating unit 20, configured to generate a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent a data structure of the execution step text;

[0086] Specifically, according to the execution step text, based on the Word2Dec technical model, each execution step text generates a corresponding data integration component, which can be an ETL component. Word2Dec is a model that learns semantic knowledge in an unsupervised manner from a large amount of text corpus, and is widely used in NLP (Natural Language Processing, NLP for short). ETL is the abbreviation of Extract, Transform, and Load, and is a commonly used data integration device. Therefore, the data integration component is also used to extract the required data from the imported database or data table, analyze the data structure for data conversion, and load the converted data into the target system.

[0087] The second generating unit 30 is used to connect the data integration components according to the execution order to obtain the target component, control the execution of the target component, and generate the target conversion data corresponding to the data to be converted, wherein the target conversion data is the data obtained after converting the data to be converted.

[0088] Specifically, each ETL component is connected in the order of execution and then visualized. Users can view the parameters of each ETL component, preview the execution results of the component, click Execute after confirmation, and select the data export method (save to database or excel, csv file) to complete the data conversion work and obtain the final target conversion data.

[0089] Through this embodiment, the data to be converted is obtained, and the data to be converted is converted into an execution step text, wherein the data to be converted includes at least text data, and the execution step text is a text representing the conversion of the data to be converted; a data integration component corresponding to the execution step text is generated, wherein the data integration component is at least used to represent the data structure of the execution step text; the data integration components are connected in an execution order to obtain a target component, the target component is controlled to execute, and the target conversion data corresponding to the data to be converted is generated, wherein the target conversion data is the data obtained after the data to be converted is converted. Compared with the prior art, in which the conversion steps between data are complicated and the conversion efficiency is low, the present application converts data based on the data integration component through the above steps, can extract data from a variety of data sources, and then perform more complex data conversion work, to assist users in processing a large amount of data resources in a simpler and more efficient way. Therefore, it can solve the problem of low data conversion efficiency in the prior art and improve the efficiency of user data conversion processing work.

[0090] In the specific implementation process, the conversion unit includes a first splitting module, an extraction module, and a second splitting module. The first splitting module is used to split the data to be converted to obtain a plurality of sub-execution steps, where the sub-execution steps represent the smallest action steps for data conversion; the extraction module is used to extract the operation type, operation object, and execution condition of each sub-execution step, where the operation type represents the type of operation, the operation object represents the object of the operation, and the execution condition represents the condition for executing the sub-execution step; the second splitting module is used to split the sub-execution steps again to obtain the execution step text. The device performs the conversion of the data to be converted through the above steps and converts it into the execution step text.

[0091] Specifically, first, the execution step text is split into each independent sub-execution step, and the operation type, operation object, and execution condition of each sub-execution step are extracted. Then, the sub-execution steps are split according to the above steps to obtain fields such as target objects, operation types, and operation values. Finally, the total execution step text is obtained.

[0092] In some optional implementation manners, the first splitting module includes a preprocessing sub-module, a first partitioning sub-module, a second partitioning sub-module, and an identification sub-module. The preprocessing sub-module is used to perform data preprocessing on the data to be converted to obtain preprocessed data, where the preprocessed data includes a plurality of words; the first partitioning sub-module is used to perform part-of-speech partitioning on the preprocessed data to obtain the corresponding part-of-speech tags for each word; the second partitioning sub-module is used to perform type partitioning according to the part-of-speech tags to obtain a plurality of text types; the identification sub-module is used to perform syntactic structure identification on the data to be converted to obtain the syntactic structure of the data to be converted. The device performs splitting through the above steps to obtain the data to be processed after splitting, so that components can be matched according to the split data, and individual analysis and data conversion can be performed.

[0093] In the specific implementation process, data preprocessing is mainly divided into stop word removal and text tokenization. 1. Stop word removal: In natural language, many words have no actual meaning, such as "le" and "de", so they need to be removed. 2. Text tokenization: Use the jieba tool to recombine the continuous character sequence into a word sequence according to certain specifications. Use the jieba tool to perform part-of-speech partitioning and assign a part-of-speech tag to each word in the text. These tags usually indicate the grammatical functions of the words in the sentence, such as nouns, verbs, adjectives, and numerals. Use the SpaCy tool to identify text types with special meanings from the text, perform type partitioning, and classify them into predefined categories, namely text types, such as personal names, place names, organization names, time, currency, percentages, etc. Use the SpaCy tool to analyze the syntactic structure of natural language sentences, perform syntactic structure identification, and identify the components of the sentence and their mutual relationships.

[0094] In some optional implementations, after obtaining the execution step text, the device further includes a judgment unit and an acquisition unit, the judgment unit is used to judge the source data of the operation object, wherein the source data is a data set storing information of the operation object; the acquisition unit is used to acquire and import the source data when the source data of the operation object does not exist. The device imports the source data through the above steps to search and process data based on the source data.

[0095] Specifically, determine the source data of the operation object. For example, there are two steps: table association step and screening step. The operation objects of the table association step are the student information table and the data competition table, which come from the student management database and the competition results file. Therefore, two data import execution steps need to be added. The operation object of the screening step is the student, which comes from the data set after the table association step is executed, so no additional execution steps need to be added.

[0096] In some other optional implementations, the first generation unit includes a construction module and a matching module, the construction module is used to construct a word vector space, wherein the word vector space maps the words in the execution step text to points in the real vector space; the matching module is used to match the operation object with the source data, and the operation type with the data integration component in the word vector space, to obtain the data integration component corresponding to each execution step. The device generates a data integration component through the above steps, so that data conversion can be quickly realized by connecting the data integration component and controlling the execution of the component.

[0097] In the specific implementation process, the word embedding space is constructed through the Word2Dec technology model based on the data set imported by the user into the system (including the database connection information created by the user and the imported excel and csv files) and ETL components. According to the execution step text, semantic capture is performed based on the Word2Vec technology model, and the operation object is matched with the source data, the operation type and the ETL component one by one, and the data set and ETL component corresponding to each step are obtained. Word embedding is a technology in natural language processing (NLP), which maps words to points in the real vector space.

[0098] In some optional implementations, after generating the data integration component corresponding to the execution step text, the device further includes a writing unit for writing component parameters into the data integration component, wherein the component parameters at least include parameters for executing the execution step text. The device improves the parameters of the component through the above steps, thus ensuring that the component is executed smoothly.

[0099] Specifically, after the component is constructed, parameters such as data sets and execution conditions are written into the above ETL component.

[0100] In order to quickly perform data conversion, the second generation unit includes a preview module and a control module. The preview module is used to visualize the target component and preview the execution result of the target component; the control module is used to control the execution of the target component if the execution result is correct. The device executes the target component if the preview passes, so that accurate target conversion data after conversion can be obtained.

[0101] During the specific implementation process, users can view the parameters of each ETL component, preview the execution results of the component, click Execute after confirmation, select the data export method (save to database or excel, csv file), and the data conversion work can be completed.

[0102] The natural language data conversion device includes a processor and a memory, wherein the conversion unit, the first generation unit, the second generation unit, etc. are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0103] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and data conversion is performed by adjusting kernel parameters.

[0104] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0105] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the natural language data conversion method.

[0106] Specifically, the natural language data conversion method includes:

[0107] Step S201, obtaining data to be converted, and converting the data to be converted into execution step text, wherein the data to be converted at least includes text data, and the execution step text is text representing the conversion of the data to be converted;

[0108] Specifically, the data to be converted can be a natural language text input by the user. According to the natural language processing technology, it is analyzed and processed to split the complete sentence text into execution step text. In the actual application process, in addition to text data, the data to be converted can also include data in text format such as numbers and symbols. The execution step text is the step of converting the data to be converted, for example: Step 1: data association; Step 2: data screening.

[0109] Step S202, generating a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text;

[0110] Specifically, according to the execution step text, based on the Word2Dec technical model, each execution step text generates a corresponding data integration component, which can be an ETL component. Word2Dec is a model that learns semantic knowledge from a large amount of text corpus in an unsupervised manner, and is widely used in NLP (Natural Language Processing, NLP for short). ETL is the abbreviation of Extract, Transform, and Load, and is a commonly used data integration method. Therefore, the data integration component is also used to extract the required data from the imported database or data table, analyze the data structure for data conversion, and load the converted data into the target system.

[0111] Step S203, connecting the data integration components in an execution order to obtain a target component, controlling the execution of the target component, and generating target conversion data corresponding to the data to be converted, wherein the target conversion data is data obtained after converting the data to be converted.

[0112] Specifically, each ETL component is connected in the order of execution and then visualized. Users can view the parameters of each ETL component, preview the execution results of the component, click Execute after confirmation, and select the data export method (save to database or excel, csv file) to complete the data conversion work and obtain the final target conversion data.

[0113] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, at least the following steps are implemented:

[0114] Step S201, obtaining data to be converted, and converting the data to be converted into execution step text, wherein the data to be converted at least includes text data, and the execution step text is text representing the conversion of the data to be converted;

[0115] Specifically, the data to be converted can be a natural language text input by the user. According to the natural language processing technology, it is analyzed and processed to split the complete sentence text into execution step text. In the actual application process, in addition to text data, the data to be converted can also include data in text format such as numbers and symbols. The execution step text is the step of converting the data to be converted, for example: Step 1: data association; Step 2: data screening.

[0116] Step S202, generating a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text;

[0117] Specifically, according to the execution step text, based on the Word2Dec technical model, each execution step text generates a corresponding data integration component, which can be an ETL component. Word2Dec is a model that learns semantic knowledge from a large amount of text corpus in an unsupervised manner, and is widely used in NLP (Natural Language Processing, NLP for short). ETL is the abbreviation of Extract, Transform, and Load, and is a commonly used data integration method. Therefore, the data integration component is also used to extract the required data from the imported database or data table, analyze the data structure for data conversion, and load the converted data into the target system.

[0118] Step S203, connecting the data integration components in an execution order to obtain a target component, controlling the execution of the target component, and generating target conversion data corresponding to the data to be converted, wherein the target conversion data is data obtained after converting the data to be converted.

[0119] Specifically, each ETL component is connected in the order of execution and then visualized. Users can view the parameters of each ETL component, preview the execution results of the component, click Execute after confirmation, and select the data export method (save to database or excel, csv file) to complete the data conversion work and obtain the final target conversion data.

[0120] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0121] The present application also provides a natural language data conversion system, including: a data integration component, a data structure at least used to represent an execution step text, the execution step text is a text representing the conversion of the data to be converted, the data integration component is connected in an execution order to obtain a target component, and the target component converts the data to be converted into target conversion data when executed; a controller, including one or more processors, and one or more programs, and the programs implement any natural language data conversion method when executed by the processor.

[0122] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in each embodiment of the present application:

[0123] Step S201, obtaining data to be converted, and converting the data to be converted into execution step text, wherein the data to be converted at least includes text data, and the execution step text is text representing the conversion of the data to be converted;

[0124] Specifically, the data to be converted can be a natural language text input by the user. According to the natural language processing technology, it is analyzed and processed to split the complete sentence text into execution step text. In the actual application process, in addition to text data, the data to be converted can also include data in text format such as numbers and symbols. The execution step text is the step of converting the data to be converted, for example: Step 1: data association; Step 2: data screening.

[0125] Step S202, generating a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text;

[0126] Specifically, according to the execution step text, based on the Word2Dec technical model, each execution step text generates a corresponding data integration component, which can be an ETL component. Word2Dec is a model that learns semantic knowledge from a large amount of text corpus in an unsupervised manner, and is widely used in NLP (Natural Language Processing, NLP for short). ETL is the abbreviation of Extract, Transform, and Load, and is a commonly used data integration method. Therefore, the data integration component is also used to extract the required data from the imported database or data table, analyze the data structure for data conversion, and load the converted data into the target system.

[0127] Step S203, connecting the data integration components in an execution order to obtain a target component, controlling the execution of the target component, and generating target conversion data corresponding to the data to be converted, wherein the target conversion data is data obtained after converting the data to be converted.

[0128] Specifically, each ETL component is connected in the order of execution and then visualized. Users can view the parameters of each ETL component, preview the execution results of the component, click Execute after confirmation, and select the data export method (save to database or excel, csv file) to complete the data conversion work and obtain the final target conversion data.

[0129] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0136] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0138] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0139] 1) In the natural language data conversion method of the present application, the data to be converted is obtained, and the data to be converted is converted into an execution step text, wherein the data to be converted includes at least text data, and the execution step text is a text representing the conversion of the data to be converted; a data integration component corresponding to the execution step text is generated, wherein the data integration component is at least used to represent the data structure of the execution step text; the data integration components are connected in an execution order to obtain a target component, the target component is controlled to execute, and the target conversion data corresponding to the data to be converted is generated, wherein the target conversion data is the data obtained after the data to be converted is converted. Compared with the prior art, in which the conversion steps between data are complicated and the conversion efficiency is low, the present application converts data based on the data integration component through the above steps, can extract data from multiple data sources, and then perform more complex data conversion work, to assist users in processing a large amount of data resources in a simpler and more efficient way. Therefore, it can solve the problem of low data conversion efficiency in the prior art and improve the efficiency of user data conversion processing work.

[0140] 2) In the natural language data conversion device of the present application, the data to be converted is obtained, and the data to be converted is converted into an execution step text, wherein the data to be converted includes at least text data, and the execution step text is a text representing the conversion of the data to be converted; a data integration component corresponding to the execution step text is generated, wherein the data integration component is at least used to represent the data structure of the execution step text; the data integration components are connected in an execution order to obtain a target component, the execution of the target component is controlled, and the target conversion data corresponding to the data to be converted is generated, wherein the target conversion data is the data obtained after the data to be converted is converted. Compared with the prior art, in which the conversion steps between data are complicated and the conversion efficiency is low, the present application converts data based on the data integration component through the above steps, can extract data from multiple data sources, and then perform more complex data conversion work, to assist users in processing a large amount of data resources in a simpler and more efficient way. Therefore, it can solve the problem of low data conversion efficiency in the prior art and improve the efficiency of user data conversion processing work.

[0141] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for converting natural language data, characterized in that: include: Acquire data to be converted, and convert the data to be converted into execution step text, wherein the data to be converted at least includes text data, and the execution step text is text representing the conversion of the data to be converted; Generate a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent the data structure of the execution step text; The data integration components are connected in an execution order to obtain a target component, and the target component is controlled to execute to generate target conversion data corresponding to the data to be converted, wherein the target conversion data is data obtained after converting the data to be converted.

2. The method for converting natural language data according to claim 1, characterized in that: Convert the data to be converted into execution step text, including: Splitting the data to be converted to obtain a plurality of sub-execution steps, wherein the sub-execution steps represent minimum action steps for performing data conversion; Extracting the operation type, operation object and execution condition of each of the sub-execution steps, wherein the operation type indicates the type of operation, the operation object indicates the object of the operation, and the execution condition indicates the condition for executing the sub-execution step; The sub-execution steps are split again to obtain the execution step text.

3. The method for converting natural language data according to claim 2, characterized in that: Splitting the data to be converted includes: Performing data preprocessing on the data to be converted to obtain preprocessed data, wherein the preprocessed data includes a plurality of words; Performing part-of-speech classification on the preprocessed data to obtain a part-of-speech tag corresponding to each of the words; Performing type division according to the part-of-speech tags to obtain multiple text types; The grammatical structure of the data to be converted is identified to obtain the grammatical structure of the data to be converted.

4. The method for converting natural language data according to claim 2, characterized in that: After obtaining the execution step text, the method further includes: Determining source data of the operation object, wherein the source data is a data set storing information of the operation object; When the source data of the operation object does not exist, the source data is acquired and imported.

5. The method for converting natural language data according to claim 4, characterized in that: Generating a data integration component corresponding to the execution step text includes: Constructing a word vector space, wherein the word vector space maps words in the execution step text to points in a real number vector space; In the word vector space, the operation object is matched with the source data, and the operation type is matched with the data integration component to obtain the data integration component corresponding to each execution step.

6. The method for converting natural language data according to claim 1, characterized in that: After generating the data integration component corresponding to the execution step text, the method further includes: Writing component parameters into the data integration component, wherein the component parameters at least include parameters for executing the execution step text.

7. The method for converting natural language data according to claim 1, characterized in that: Controlling the execution of the target component includes: Visualizing the target component and previewing the execution result of the target component; When the execution result is correct, the target component is controlled to execute.

8. A natural language data conversion device, characterized in that: include: A conversion unit, used for acquiring data to be converted, and converting the data to be converted into an execution step text, wherein the data to be converted at least includes text data, and the execution step text is a text representing the conversion of the data to be converted; A first generating unit, configured to generate a data integration component corresponding to the execution step text, wherein the data integration component is at least used to represent a data structure of the execution step text; The second generating unit is used to connect the data integration components according to the execution order to obtain the target component, control the execution of the target component, and generate the target conversion data corresponding to the data to be converted, wherein the target conversion data is the data obtained after converting the data to be converted.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the natural language data conversion method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the natural language data conversion method described in any one of claims 1 to 7.

11. A natural language data conversion system, characterized in that: include: A data integration component, at least a data structure for representing an execution step text, wherein the execution step text is a text representing the conversion of the data to be converted, the data integration component is connected in an execution order to obtain a target component, and the target component converts the data to be converted into target converted data when executed; A controller comprises one or more processors and one or more programs, wherein when the programs are executed by the processors, the method for converting natural language data as claimed in any one of claims 1 to 7 is implemented.

Images