Unstructured data processing method, device, system and storage medium

Abstract

The present invention provides a method, device, system and storage medium for processing unstructured data, the method comprising: reading unstructured data to be processed and paragraph configuration information; segmenting the unstructured data to obtain paragraph data; calling corresponding verification rules to verify the paragraph data according to the paragraph configuration information to obtain parsed target data; storing the target data in a database. Thus, the unstructured data can be segmented, simplifying the parsing process of the unstructured data to form a simple and unified data structure, facilitating the editing of the unstructured data, thereby improving the reusability and extensibility of the code.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, system and storage medium for processing unstructured data. Background Art

[0002] With the development of Internet technology, more and more fields involve Web applications. In Web projects, it is often necessary to interact with the front-end page and then store the unstructured data of the HTML page in a relational database.

[0003] Currently, the common processing solution is to use an unstructured data editing plug-in to store the original HTML code in the database, or to store the unstructured data of the HTML page in a structured manner in a table containing many fields after complex parsing.

[0004] However, in the above processing scheme, the parsing process of unstructured data is complicated, and it is difficult to abstract the HTML code into overall components, which is not conducive to the editing of unstructured data. Summary of the invention

[0005] The present invention provides a method, device, system and storage medium for processing unstructured data, which can segment unstructured data and simplify the parsing process of unstructured data to form a simple and unified data structure, facilitate editing of unstructured data, thereby improving the reusability and extensibility of code.

[0006] In a first aspect, an embodiment of the present invention provides a method for processing unstructured data, comprising:

[0007] Read unstructured data and paragraph configuration information to be processed;

[0008] Segmenting the unstructured data to obtain segment data;

[0009] According to the paragraph configuration information, the corresponding verification rules are called to verify the paragraph data to obtain the parsed target data;

[0010] The target data is stored in a database.

[0011] In a possible design, the paragraph configuration information includes: paragraph type, field name, stored field name, verification rule, and the correspondence between paragraph type and component.

[0012] In a possible design, the unstructured data is segmented to obtain segment data, including:

[0013] Reading each component in the unstructured data in sequence;

[0014] If the current component is a single component, convert the component content into paragraph data;

[0015] If the current component is an integral component, the integral component is split according to the parent-child relationship of the integral component to obtain split components;

[0016] Each split component content is converted into paragraph data, wherein the single component and the component content of the split components only contain one paragraph type.

[0017] In a possible design, according to the paragraph configuration information, the corresponding verification rules are called to verify the paragraph data to obtain the parsed target data, including:

[0018] generating additional information corresponding to the paragraph type according to the verification rule;

[0019] The additional information is supplemented in the paragraph data to obtain the target data.

[0020] In one possible design, it also includes:

[0021] Perform any one or more editing operations of deletion, modification, and addition on the paragraph configuration information to obtain updated paragraph configuration information.

[0022] In one possible design, it also includes:

[0023] Retrieving the target data from the database;

[0024] Create a single component or a whole component according to the paragraph type of the target data;

[0025] Using the target data as component content of the single component or the entire component;

[0026] The single component or the entire component is displayed on the client.

[0027] In a second aspect, an embodiment of the present invention provides a device for processing unstructured data, including:

[0028] A reading module, used to read unstructured data and paragraph configuration information to be processed;

[0029] A segmentation module, used for segmenting the unstructured data to obtain segment data;

[0030] A processing module, used to call corresponding verification rules to verify the paragraph data according to the paragraph configuration information to obtain the parsed target data;

[0031] The storage module is used to store the target data in a database.

[0032] In a possible design, the paragraph configuration information includes: paragraph type, field name, stored field name, verification rule, and the correspondence between paragraph type and component.

[0033] In a possible design, the segmentation module is specifically used to:

[0034] Reading each component in the unstructured data in sequence;

[0035] If the current component is a single component, convert the component content into paragraph data;

[0036] If the current component is an integral component, the integral component is split according to the parent-child relationship of the integral component to obtain split components;

[0037] Each split component content is converted into paragraph data, wherein the single component and the component content of the split components only contain one paragraph type.

[0038] In a possible design, the processing module is specifically used to:

[0039] generating additional information corresponding to the paragraph type according to the verification rule;

[0040] The additional information is supplemented in the paragraph data to obtain the target data.

[0041] In one possible design, it also includes:

[0042] The editing module is used to perform any one or more editing operations of deletion, modification, and addition on the paragraph configuration information to obtain updated paragraph configuration information.

[0043] In a possible design, it also includes: a display module, specifically used for:

[0044] Retrieving the target data from the database;

[0045] Create a single component or a whole component according to the paragraph type of the target data;

[0046] Using the target data as component content of the single component or the entire component;

[0047] The single component or the entire component is displayed on the client.

[0048] In a third aspect, an embodiment of the present invention provides a system for processing unstructured data, comprising: a memory and a processor, wherein the memory stores executable instructions of the processor; wherein the processor is configured to execute the method for processing unstructured data described in any one of the first aspects by executing the executable instructions.

[0049] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for processing unstructured data described in any one of the first aspects.

[0050] In a fifth aspect, an embodiment of the present invention provides a program product, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of a server can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the server executes any method for processing unstructured data described in the first aspect.

[0051] The present invention provides a method, device, system and storage medium for processing unstructured data, which reads unstructured data to be processed and paragraph configuration information; processes the unstructured data in sections to obtain paragraph data; verifies the paragraph data according to the paragraph configuration information by calling corresponding verification rules to obtain parsed target data; and stores the target data in a database. Thus, the unstructured data can be processed in sections, and the parsing process of the unstructured data can be simplified to form a simple and unified data structure, which is convenient for editing the unstructured data, thereby improving the reusability and extensibility of the code. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 This is a schematic diagram of the principle of an application scenario of the present invention;

[0054] Figure 2 A flowchart of a method for processing unstructured data provided in Embodiment 1 of the present invention;

[0055] Figure 3 A schematic diagram of paragraph configuration information provided by an embodiment of the present invention;

[0056] Figure 4A schematic diagram of paragraph data provided by an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of target data provided by an embodiment of the present invention;

[0058] FIG6( a) is a schematic diagram of segment data corresponding to video types provided by an embodiment of the present invention;

[0059] FIG6( b ) is a schematic diagram of target data corresponding to video types provided by an embodiment of the present invention;

[0060] Figure 7 A flowchart of a method for processing unstructured data provided in Embodiment 2 of the present invention;

[0061] Figure 8 A schematic diagram of the structure of an unstructured data processing device provided in Embodiment 3 of the present invention;

[0062] Fig. 9 A schematic diagram of the structure of an unstructured data processing device provided in Embodiment 4 of the present invention;

[0063] Fig.10 A schematic diagram of the structure of a system for processing unstructured data provided in Embodiment 5 of the present invention.

[0064] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0068] With the development of Internet technology, more and more fields involve Web applications. In Web projects, it is often necessary to interact with the front-end page, and then store the unstructured data of the HTML page in a relational database. At present, the common processing solution is to use an editing plug-in for unstructured data to store the original HTML code in the database, or to store the unstructured data of the HTML page in a structured form in a table containing many fields after complex parsing. However, in the above processing solution, the parsing process of unstructured data is complicated, and it is difficult to abstract the HTML code into a whole component, which is not conducive to the editing of unstructured data. Taking rich text on HTML as an example, the disadvantages of directly storing rich text mainly include the following aspects: 1. Rich text parsing is relatively difficult. There are many types of HTML controls, and a set of overall parsing solutions is not easy. 2. The HTML code in the rich text is not easy to abstract into a whole component. Even if a piece of text has a certain association with a picture or a link, it may not be clearly identified from the code due to different user operation methods. 3. The main advantage of rich text editing is the mixed typesetting of pictures and texts, as well as the personalized processing of some fonts, blank lines and other formats. However, for some special data, such as JD.com's product cards and coupons, if they are handed over to users for personalized editing, the format will be inconsistent, resulting in data disorder. The disadvantages of pure structured solutions are quite obvious. First, like rich text, the parsing process is more complicated. Second, even if it is parsed, it is not easy to expand. Adding new data types requires adding new database fields. In the long run, the number of fields in the table is too large. If different data types share a table, some data types have many fields, while others have fewer fields, which will result in field waste. Pure structured solutions are suitable for scenarios where the probability of field changes is low and data types are unified.

[0069] In view of the above technical problems, the present invention provides a method for segmenting unstructured data and simplifying the parsing process of unstructured data to form a simple and unified data structure, which is convenient for editing unstructured data and thus improves the reusability and scalability of the code. Figure 1 It is a schematic diagram of the principle of an application scenario of the present invention, such as Figure 1 As shown, the server 10 reads the unstructured data to be processed and the paragraph configuration information, and then performs segment processing on the read unstructured data according to the paragraph configuration information to obtain paragraph data; then the check rule corresponding to the paragraph type is called by the checker 11 to check the paragraph data to obtain the target data; finally, the target data is stored in the database 12. Further, when it is necessary to display the unstructured data on the client 20, the target data can be read from the server 10, and the target data can be reassembled into unstructured data by the client 20, so as to be displayed on the display interface of the client 20. Thus, the unstructured data can be segmented, and the parsing process of the unstructured data can be simplified to form a simple and unified data structure, which is convenient for editing the unstructured data, thereby improving the reusability and extensibility of the code.

[0070] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0071] Figure 2 This is a flowchart of a method for processing unstructured data provided in Embodiment 1 of the present invention, such as Figure 2 As shown, the method in this embodiment may include:

[0072] S101 : Read unstructured data and paragraph configuration information to be processed.

[0073] In this embodiment, the unstructured data to be processed is the data that needs to be processed by the present invention. The data may include basic components such as plain text, pictures, product skus, links, videos, etc., or it may be a combination of these basic components, such as a combination of text and pictures, a combination of links and text, and so on.

[0074] Optionally, the paragraph configuration information includes: paragraph type, field name, stored field name, verification rule, and the correspondence between paragraph type and component.

[0075] Specifically, the unstructured data to be processed can be considered to be composed of paragraphs one by one, and the parsing can be performed in segments and concurrently according to the paragraph configuration information. The paragraph configuration information defines different paragraph types, corresponding inspection rules, components, database corresponding fields, etc. Figure 3 A schematic diagram of paragraph configuration information provided by an embodiment of the present invention, such as Figure 3 As shown in the figure, the first column is the paragraph type. Different components can be mapped to the paragraph type. The client can decide the code format for adding this paragraph. The second column is the field name, which indicates the text displayed on the page, that is, the data content. The third column is the field name for storage. The fourth column is the validation rule, which can be used to determine the legality of the input, validate the data content, and add additional information. Figure 3 The paragraph configuration information shown has different basic components and verification rules for text and image data.

[0076] Optionally, the method further includes: performing any one or more editing operations of deletion, modification, and addition on the paragraph configuration information to obtain updated paragraph configuration information.

[0077] Specifically, the paragraph configuration information is editable and configurable. According to the needs of the unstructured data to be processed, any or multiple editing operations such as deletion, modification, and addition can be flexibly performed. For example, if a new component of live broadcast plus text is added to the unstructured data to be processed, only one paragraph type needs to be added, and the corresponding basic components and verification rules of the live broadcast can be used. If you want to add a timed online and offline function to the sku, you can expand the fields startTime and endTime on the original product card type.

[0078] S102: Segment the unstructured data to obtain segment data.

[0079] In this embodiment, each component in the unstructured data is read in turn; if the current component is a single component, the component content is converted into paragraph data; if the current component is an overall component, the overall component is split according to the parent-child relationship of the overall component to obtain the split components; the content of each split component is converted into paragraph data, wherein the component content of a single component and the split components only contains one paragraph type.

[0080] Specifically, the unstructured data to be processed is the data that needs to be processed by the present invention. The data may include basic components such as plain text, pictures, product skus, links, videos, etc., or it may be a combination of these basic components, such as a combination of text and pictures, a combination of links and text, and so on. This overall component can be regarded as an arrangement and combination of basic components, and a parent-child relationship can be established between components. If a single basic component is encountered, the component content is directly converted into paragraph data; if an overall component is encountered, it is split into multiple basic components for processing according to the parent-child relationship. The content of each split component is converted into paragraph data. The paragraph data has a unified format, including three fields: type, content, and childs. Type represents the paragraph type, content represents the paragraph data, and childs represents the configuration of the child node. Figure 4 A schematic diagram of paragraph data provided by an embodiment of the present invention, such as Figure 4 As shown, the first component is an overall component, and the paragraph type type=20 indicates a collection type, corresponding to a collection class validation rule. This overall component includes three basic components: product title, atmosphere map, and product sku. The paragraph types are 1 (plain text), 2 (picture), and 5 (sku), respectively. The validation rules are TextHandler, ImageHandler, and SkuHandler, which process product title, atmosphere map, and sku respectively. The second component is a basic component, and the paragraph type type=1 is a plain text, corresponding to TextHandler. The responsibility of the Handler is to verify whether the content of each paragraph meets the requirements according to the limited requirements of the field configuration, and fill in some additional data.

[0081] S103: According to the paragraph configuration information, the corresponding verification rules are called to verify the paragraph data to obtain the parsed target data.

[0082] In this embodiment, additional information corresponding to the paragraph type is generated according to the verification rule; the additional information is supplemented in the paragraph data to obtain the target data.

[0083] Specifically, the paragraph configuration information defines different paragraph types, corresponding verification rules, components, database corresponding fields, etc. The verification rules corresponding to the paragraph type can be called to perform concurrent verification and assembly data on the paragraph data. Figure 5 A schematic diagram of target data provided by an embodiment of the present invention, Figure 4 The paragraph data shown is processed and finally obtained as follows Figure 5The target data shown. The difference between target data and paragraph data is that the target data will have more supplementary data, which is based on the metadata in the paragraph data. Taking video type data processing as an example, Figure 6(a) is a schematic diagram of the paragraph data corresponding to the video type provided in an embodiment of the present invention; Figure 6(b) is a schematic diagram of the target data corresponding to the video type provided in an embodiment of the present invention; the paragraph data shown in Figure 6(a) only records the video ID, that is, the videoId field; and the target data is shown in Figure 6(b), which is supplemented with information such as image and video size.

[0084] S104: Store the target data in a database.

[0085] In this embodiment, the processed target data is stored in a relational database according to the parent-child relationship and field correspondence.

[0086] In this embodiment, the unstructured data to be processed and the paragraph configuration information are read; the unstructured data is segmented to obtain the paragraph data; the corresponding verification rules are called according to the paragraph configuration information to verify the paragraph data to obtain the parsed target data; the target data is stored in the database. Thus, the unstructured data can be segmented, and the parsing process of the unstructured data can be simplified to form a simple and unified data structure, which is convenient for editing the unstructured data, thereby improving the reusability and extensibility of the code.

[0087] Figure 7 A flowchart of a method for processing unstructured data provided in Embodiment 2 of the present invention is shown in FIG. Figure 7 As shown, the method in this embodiment may include:

[0088] S201, reading unstructured data and paragraph configuration information to be processed.

[0089] S202: Segment the unstructured data to obtain segment data.

[0090] S203: According to the paragraph configuration information, the corresponding verification rules are called to verify the paragraph data to obtain the parsed target data.

[0091] S204: Store the target data in a database.

[0092] In this embodiment, the specific implementation process and technical principles of steps S201 to S204 can be found in Figure 2 The relevant descriptions in steps S101 to S104 in the method shown are not repeated here.

[0093] S205: Retrieve target data from the database.

[0094] S206. Create a single component or an entire component according to the paragraph type of the target data.

[0095] S207. Use the target data as component content of a single component or an entire component.

[0096] S208: Display a single component or the entire component on the client.

[0097] In this embodiment, steps S201 to S204 process the unstructured data to obtain structured data and store them in a database. Steps S205 to S206 convert the structured data in the database into unstructured data and display them on the client.

[0098] Specifically, first, read the target data from the database. Then, create a single component according to the paragraph type of the target data. If the target data contains a parent-child relationship, assemble the corresponding single components to obtain the overall component. Use the target data as the component content of the single component or the overall component and display it on the client.

[0099] In this embodiment, the unstructured data to be processed and the paragraph configuration information are read; the unstructured data is segmented to obtain the paragraph data; the corresponding verification rules are called according to the paragraph configuration information to verify the paragraph data to obtain the parsed target data; the target data is stored in the database. Thus, the unstructured data can be segmented, and the parsing process of the unstructured data can be simplified to form a simple and unified data structure, which is convenient for editing the unstructured data, thereby improving the reusability and extensibility of the code.

[0100] In addition, this embodiment can also read structured data in the database, convert it into unstructured data, and display it on the client.

[0101] Figure 8 A schematic diagram of the structure of an unstructured data processing device provided in Embodiment 3 of the present invention is shown in FIG. Figure 8 As shown, the unstructured data processing device of this embodiment may include:

[0102] A reading module 31, used for reading unstructured data and paragraph configuration information to be processed;

[0103] A segmentation module 32, used for segmenting the unstructured data to obtain segment data;

[0104] The processing module 33 is used to call the corresponding verification rules to verify the paragraph data according to the paragraph configuration information to obtain the parsed target data;

[0105] The storage module 34 is used to store the target data in a database.

[0106] In a possible design, the paragraph configuration information includes: paragraph type, field name, stored field name, validation rules, and the correspondence between paragraph type and components.

[0107] In a possible design, the segmentation module 32 is specifically configured to:

[0108] Read each component in the unstructured data in turn;

[0109] If the current component is a single component, convert the component content into paragraph data;

[0110] If the current component is a whole component, split the whole component according to the parent-child relationship of the whole component to obtain the split components;

[0111] Each split component content is converted into paragraph data, wherein a single component and the component content of the split component only contain one paragraph type.

[0112] In a possible design, the processing module 33 is specifically configured to:

[0113] Generate additional information corresponding to the paragraph type according to the verification rules;

[0114] Supplement additional information to the paragraph data to obtain the target data.

[0115] The unstructured data processing device of this embodiment can execute Figure 2 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 2 The relevant descriptions in the method shown will not be repeated here.

[0116] In this embodiment, the unstructured data to be processed and the paragraph configuration information are read; the unstructured data is segmented to obtain the paragraph data; the corresponding verification rules are called according to the paragraph configuration information to verify the paragraph data to obtain the parsed target data; the target data is stored in the database. Thus, the unstructured data can be segmented, and the parsing process of the unstructured data can be simplified to form a simple and unified data structure, which is convenient for editing the unstructured data, thereby improving the reusability and extensibility of the code.

[0117] Fig. 9 A schematic diagram of the structure of an unstructured data processing device provided in Embodiment 4 of the present invention, as shown in FIG. Fig. 9 As shown, the unstructured data processing device of this embodiment is Figure 8 Based on the device shown, it can also include:

[0118] The editing module 35 is used to perform any one or more editing operations of deletion, modification, and addition on the paragraph configuration information to obtain updated paragraph configuration information.

[0119] In a possible design, it further includes: a display module 36, specifically used for:

[0120] Retrieve target data from the database;

[0121] Create a single component or a whole component according to the paragraph type of the target data;

[0122] The target data is the component content of a single component or an overall component;

[0123] Display a single component or an entire component on the client.

[0124] The unstructured data processing device of this embodiment can execute Figure 2 , Figure 7 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 2 , Figure 7 The relevant descriptions in the method shown will not be repeated here.

[0125] In this embodiment, the unstructured data to be processed and the paragraph configuration information are read; the unstructured data is segmented to obtain the paragraph data; the corresponding verification rules are called according to the paragraph configuration information to verify the paragraph data to obtain the parsed target data; the target data is stored in the database. Thus, the unstructured data can be segmented, and the parsing process of the unstructured data can be simplified to form a simple and unified data structure, which is convenient for editing the unstructured data, thereby improving the reusability and extensibility of the code.

[0126] In addition, this embodiment can also read structured data in the database, convert it into unstructured data, and display it on the client.

[0127] Fig.10 A structural diagram of a system for processing unstructured data provided in Embodiment 5 of the present invention is shown in FIG. Fig.10 As shown, the unstructured data processing system 40 of this embodiment may include: a processor 41 and a memory 42 .

[0128] The memory 42 is used to store programs; the memory 42 may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory 42 is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories 42. And the above computer programs, computer instructions, data, etc. can be called by the processor 41.

[0129] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories 42 in partitions. And the above-mentioned computer programs, computer instructions, data, etc. may be called by the processor 41.

[0130] The processor 41 is used to execute the computer program stored in the memory 42 to implement the various steps in the method involved in the above embodiment.

[0131] For details, please refer to the relevant description in the previous method embodiment.

[0132] The processor 41 and the memory 42 may be independent structures or integrated structures. When the processor 41 and the memory 42 are independent structures, the memory 42 and the processor 41 may be coupled and connected via a bus 43 .

[0133] The unstructured data processing system of this embodiment can execute Figure 2 , Figure 7 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figure 2 The relevant descriptions of the method shown in the figure will not be repeated here.

[0134] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When at least one processor of a user device executes the computer-executable instructions, the user device executes the above-mentioned various possible methods.

[0135] Among them, computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.

[0136] The present application also provides a program product, which includes a computer program. The computer program is stored in a readable storage medium. At least one processor of the server can read the computer program from the readable storage medium. At least one processor executes the computer program so that the server implements any method of the above-mentioned embodiments of the present invention.

[0137] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing unstructured data, characterized in that: include: Read unstructured data and paragraph configuration information to be processed; Segmenting the unstructured data to obtain segment data; According to the paragraph configuration information, the corresponding verification rules are called to verify each of the paragraph data after segmentation processing in parallel to obtain the parsed target data, wherein the verification rules are used to determine the legality of the input and to check whether the content of the paragraph data meets the preset conditions; storing the target data in a database; The unstructured data is segmented to obtain segment data, including: Reading each component in the unstructured data in sequence; If the current component is a single component, convert the component content into paragraph data; If the current component is an integral component, the integral component is split according to the parent-child relationship of the integral component to obtain split components; Each split component content is converted into paragraph data, wherein the single component and the component content of the split components only contain one paragraph type.

2. The method according to claim 1, characterized in that The paragraph configuration information includes: paragraph type, field name, stored field name, verification rules, and the correspondence between paragraph type and components.

3. The method according to claim 2, characterized in that According to the paragraph configuration information, the corresponding verification rules are called to verify each of the paragraph data after segmentation processing in parallel to obtain the parsed target data, including: generating additional information corresponding to the paragraph type according to the verification rule; The additional information is supplemented in the paragraph data to obtain the target data.

4. The method according to claim 1, characterized in that: Also includes: Perform any one or more editing operations of deletion, modification, and addition on the paragraph configuration information to obtain updated paragraph configuration information.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: Retrieving the target data from the database; Create a single component or a whole component according to the paragraph type of the target data; Using the target data as component content of the single component or the entire component; The single component or the entire component is displayed on the client.

6. A device for processing unstructured data, characterized in that: include: A reading module, used to read unstructured data and paragraph configuration information to be processed; A segmentation module, used for segmenting the unstructured data to obtain segment data; A processing module, used to call corresponding verification rules according to the paragraph configuration information and verify each of the paragraph data after segmentation processing in parallel to obtain the parsed target data, wherein the verification rules are used to determine the legality of the input and to check whether the content of the paragraph data meets the preset conditions; A storage module, used for storing the target data in a database; The segmentation module is specifically used for: Reading each component in the unstructured data in sequence; If the current component is a single component, convert the component content into paragraph data; If the current component is an integral component, the integral component is split according to the parent-child relationship of the integral component to obtain split components; Each split component content is converted into paragraph data, wherein the single component and the component content of the split components only contain one paragraph type.

7. The device according to claim 6, characterized in that The paragraph configuration information includes: paragraph type, field name, stored field name, verification rules, and the correspondence between paragraph type and components.

8. A system for processing unstructured data, characterized in that: include: A memory and a processor, wherein the memory stores executable instructions of the processor; wherein the processor is configured to execute the method for processing unstructured data according to any one of claims 1 to 5 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for processing unstructured data described in any one of claims 1 to 5 is implemented.

Images