Data processing method and device, electronic equipment and storage medium
By acquiring the attributes of the raw and target data, determining the target rule template, and parsing the data transformation rules, the problem of differences in data attributes among multiple partners was solved, thereby improving the accuracy and efficiency of data transformation.
Patent Information
- Application Number
- CN202310531755.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-11-12
AI Technical Summary
Existing data transformation methods cannot effectively meet the different data attribute requirements of multiple partners in different application scenarios, resulting in data attributes that do not meet the target requirements.
By acquiring the attributes of the original data and the target data, the target rule template is determined, the data transformation rules are parsed, and the transformed data is compared with the attributes of the target data to determine whether they meet the requirements. It supports the conversion of JSON format and image structured data.
It achieves accurate data transformation and consistent attribute comparison, ensuring that the transformed data attributes meet the target data attributes, thus improving the accuracy and efficiency of data transformation.
Smart Images

Figure CN118940737A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, device, electronic equipment and storage medium. Background Art
[0002] With the rapid development and widespread application of information technology, the scale of project data in various application scenarios has also expanded rapidly. However, since the same project data may be shared by multiple partners, and different partners and different application scenarios have different requirements for data attributes, the existing data conversion method cannot well obtain data attributes that meet the needs of all parties. Summary of the invention
[0003] In view of this, an embodiment of the present invention provides a data processing method, device, electronic device and storage medium to perform corresponding data conversion on data, and compare the converted data attributes with the target data attributes to determine whether the converted data attributes meet the required target data attributes.
[0004] In a first aspect, a data processing method is provided, the method comprising:
[0005] Get the original data to be converted and the target data attributes;
[0006] Determining a target rule template based on the original data and / or target data attributes, the target rule template being generated according to the selected at least one predefined rule;
[0007] Parsing the target rule template to determine at least one data conversion rule;
[0008] Convert the original data according to the data conversion rules to obtain converted data;
[0009] The converted data is compared with the target data attribute to determine a comparison result, wherein the comparison result is used to indicate whether the data attribute of the converted data satisfies the target data attribute.
[0010] In some embodiments, the original data is obtained through a web page;
[0011] The method further comprises:
[0012] The comparison result and / or conversion data are sent to a web page.
[0013] In some embodiments, the original data is Json format data or picture structured data.
[0014] In some embodiments, obtaining the raw data to be converted includes:
[0015] Get the image;
[0016] The image is formatted to obtain image structured data.
[0017] In some embodiments, generating the rule template specifically includes:
[0018] Receive selected predefined rules and rule processing order through the configuration rule template interface;
[0019] A rule template is generated according to the selected predefined rules and the rule processing sequence.
[0020] In some embodiments, the determining of the target rule template based on the original data and / or target data attributes is specifically:
[0021] A target rule template is determined from a pre-generated rule template library based on the original data and / or target data attributes.
[0022] In some embodiments, the predefined rules include key-value pair rule processing, keyword rule processing, and key value rule processing.
[0023] In some embodiments, the method further comprises:
[0024] The predefined rules are extended through an application program interface.
[0025] In a second aspect, a data processing device is provided, the device comprising:
[0026] a configuration management module configured to obtain original data to be converted and target data attributes and determine a target rule template based on the original data and / or the target data attributes, wherein the target rule template is generated according to at least one selected predefined rule;
[0027] A rule template conversion engine is configured to parse the target rule template and determine at least one data conversion rule;
[0028] The comparison module is configured to convert the original data according to the data conversion rule, obtain the converted data, compare the converted data with the target data attribute, and determine a comparison result, wherein the comparison result is used to characterize whether the data attribute of the converted data meets the target data attribute.
[0029] In some embodiments, the apparatus further comprises:
[0030] Image recognition module, used to perform image recognition to obtain image structured data;
[0031] A data access module is used to obtain target data attributes and send the target data attributes to the configuration management module;
[0032] An application program interface is used to extend the predefined rules.
[0033] According to a third aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect.
[0034] According to a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to the first aspect is implemented.
[0035] According to a fifth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer is caused to execute the method according to the first aspect.
[0036] The embodiment of the present invention determines a target rule template based on the acquired original data and / or target data attributes, the target rule template is generated based on at least one selected predefined rule, at least one data conversion rule is determined by parsing the target rule template, the original data is converted according to each of the data conversion rules to obtain conversion data, the conversion data is compared with the target data attributes, and it is determined whether the data attributes of the conversion data meet the target data attributes. Thus, the corresponding conversion of the data is achieved, and the consistency of the converted data attributes and the target data attributes can be compared, so as to determine whether the converted data attributes meet the target data attributes. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0038] Figure 1 is a schematic diagram of a data processing device according to an embodiment of the present invention;
[0039] Figure 2 is a flow chart of rule template generation according to an embodiment of the present invention;
[0040] Figure 3 is a schematic diagram of a configuration rule template interface of an embodiment of the present invention;
[0041] Figure 4 is a schematic diagram of a data processing device according to another embodiment of the present invention;
[0042] Figure 5 is a flow chart of a data processing method according to an embodiment of the present invention;
[0043] Figure 6 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0044] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.
[0045] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.
[0046] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".
[0047] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.
[0048] The solutions described in this specification and in the examples, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.
[0049] Figure 1 Schematic diagram of a data processing device according to an embodiment of the present invention. Figure 1 In the illustrated embodiment, the data processing device includes a configuration management module 11 , a rule template conversion engine 12 and a comparison module 13 .
[0050] The configuration management module 11 is configured to obtain original data and target data attributes to be converted and determine a target rule template based on the original data and / or target data attributes, wherein the target rule template is generated according to at least one selected predefined rule.
[0051] In some embodiments, the original data to be converted and the target data attributes can be obtained through the web page. The target data attributes represent the conditions that the target data (i.e., the converted data) must meet, for example, meeting the specified characteristics, format, or threshold range. Thus, this embodiment can compare the original data with the target data attributes after conversion, and then determine whether the converted data attributes meet the target data attributes. That is, this embodiment can perform data verification on the converted data to further improve the accuracy of the converted data.
[0052] Optionally, the original data is Json format data or picture structured data. Further, the picture structured data is generated after the acquired picture format is converted. Optionally, the present embodiment can perform text recognition on the acquired picture to obtain the corresponding picture structured data. Therefore, the present embodiment provides a text recognition operation of the picture before data conversion, so as to provide a function of data conversion on the picture. For example, the acquired picture can be converted into picture structured data by using OCR (Optical Character Recognition) technology. Through OCR technology, the image file of the text data can be analyzed and identified, and the text and layout information can be obtained, so that the picture can be automatically converted into an editable text document, such as Word, TXT, etc. The converted document can be saved as data, which greatly reduces labor and improves work efficiency. In some implementations, other methods can also be used to perform text recognition on the acquired picture, for example, deep learning models such as CRNN (Convolutional Recurrent Neural Network), which are not limited in the present embodiment.
[0053] In some embodiments, the configuration management module 11 can determine the target rule template from a pre-generated rule template library based on the original data and / or target data attributes. The pre-generated rule template library includes at least one rule template. Since the data conversion performed on a specific type of original data generally remains basically unchanged, the rule template required for data conversion of the type of original data remains basically unchanged. Moreover, for a specific target data attribute, the rule template applied when obtaining data that meets the target data attribute remains basically unchanged. In other words, for a specific rule template, the type of original data and the target data attribute corresponding to it remain basically unchanged. Therefore, the pre-generated rule module can be annotated with the type of original data and / or target data attribute to which it is applicable, and then the target rule template can be determined from the pre-generated rule template library based on the acquired original data and / or target data attributes.
[0054] Furthermore, in actual application scenarios, within a certain period of time, for the original data, the required converted data usually does not change significantly, so the required rule templates usually do not change significantly either. Therefore, the present embodiment can pre-generate commonly used rule templates and store them in the template library, so that the corresponding target conversion template can be directly called when performing the corresponding data conversion, thereby improving the data conversion efficiency. Optionally, the present embodiment manages and stores the rule template library through the configuration management module 11. Further optionally, the configuration management module 11 manages the rule template library including adding, modifying, and / or deleting the rule templates therein to adapt to changes in application scenarios.
[0055] In some implementations, the flowchart for generating the rule template is as follows: Figure 2 As shown, the following steps are included:
[0056] Step S210: receiving the selected predefined rules and rule processing order through the configuration rule template interface.
[0057] Specifically, Figure 3 As shown, a configuration rule template interface A pops up on the web page, and the user selects predefined rules and rule processing order in the configuration rule template interface A according to the target data attributes to be obtained. In one implementation, the configuration rule template interface A is as follows Figure 3 As shown, it includes a component area B and a configuration area C. The component area B contains various predefined rules for users to select and combine. The configuration area C is used to configure rule templates.
[0058] In some implementations, the predefined rules include key-value pair rule processing, keyword rule processing, and key value rule processing.
[0059] The key-value pair rule processing mainly deals with the key and value of the data at the same time. The key is the index and the value is the data. The corresponding value can be determined according to the key. The key-value pair rule processing includes blacklist key-value pair rules (hereinafter referred to as blacklist) and whitelist key-value pair rules (hereinafter referred to as whitelist). The predefined components of the blacklist and whitelist 31 are mainly applied to the overall data format. The blacklist 311 represents the key-value pairs that need to be filtered out, and the whitelist 312 represents the key-value pairs that must be included.
[0060] Keyword rule processing includes field processing 32, which is mainly applied to the keyword part of the data format. The field processing specifically includes processing methods such as case processing 321, regular processing 322 and mapping processing 323. The case processing 321 represents the case change of the character string of some keyword fields, the regular processing 322 represents the segmentation or processing of the keyword field part through the regular expression input, and the mapping processing 323 represents mapping some keyword fields into other keyword fields.
[0061] The key value rule processing includes numerical processing 33, which is mainly applied to the numerical part corresponding to the keyword field of the data format. The numerical processing 33 specifically includes processing methods such as precision processing 331, type processing 332 and conversion processing 333. The precision processing 331 indicates that the numerical values corresponding to some key fields of the data are retained with decimal places, such as retaining two decimal places. The type processing 332 indicates that the numerical values corresponding to some key fields of the data are replaced with a specified format string. The conversion processing 333 indicates that the numerical values corresponding to some key fields of the data are numerically calculated. The user determines the selected predefined rule by dragging the selected predefined rule to the placeholder position 34, 35 or 36 of the configuration area C. The rule processing order can be set by the placeholder position, for example, setting the predefined rules in the placeholder position 34, 35 and 36 to be executed in sequence. It should be understood that the present embodiment does not limit the number of selected predefined rules, and the number of selected predefined rules can be increased or decreased according to the actual conversion requirements. In some implementations, the number of placeholders can be increased by clicking the control 37 to facilitate the user to drag and select predefined rules. As a result, flexible configuration of rule templates is achieved, and users can adjust rule templates according to their needs, thereby improving the accuracy of converted data.
[0062] It should be understood that Figure 3 The configuration rule template interface A shown is only exemplary, and the present embodiment is not limited thereto. In some implementations, the user can also determine the selected predefined rule by clicking on a predetermined rule. For example, after the user clicks to select the corresponding predefined rule, the color of the selected predefined rule control is changed to prompt the user that the predefined rule has been selected, or a corresponding selection box can be set near each predefined rule, and after the user clicks to select the corresponding predefined rule or selection box, the selection of the predefined rule is prompted to the user by changing the display state of the selection box.
[0063] In some implementations, the rule processing order can also be determined based on the order in which the user selects the predefined rules, or the user can also pre-specify the priority of each predefined rule. After the user selects the predefined rule, each predefined rule is executed according to the priority order of the selected predefined rule to perform data conversion.
[0064] It should be understood that the predefined rules in this embodiment are merely examples of commonly used rules, and this embodiment does not limit the types of predefined rules. In some embodiments, the predefined rules may also include various other existing rule processing methods, or may also customize relevant rules based on the data to be processed and the data conversion effect to be achieved, and this embodiment does not limit this.
[0065] Thus, the configuration management module 11 can receive the selected predefined rules and rule processing order through the configuration rule template interface to perform subsequent data processing.
[0066] Step S220: generating a rule template according to the selected predefined rules and rule processing sequence.
[0067] In some implementations, after the user selects a predefined rule, he can send a rule template generation instruction by clicking the configuration template 38. The configuration template 38 is a button control. After receiving the rule template generation instruction, the configuration management module 11 generates a rule template according to the selected predefined rule and the rule processing order and stores it in the rule template library.
[0068] In some implementations, the user can configure corresponding rule templates according to multiple target data attributes in advance and store them in a rule template library, so that after obtaining the original data and the target data attributes, the corresponding target rule template can be determined from the rule template library. The user can also configure and generate corresponding rule templates in real time to perform data conversion after obtaining the original data and the target data attributes.
[0069] In some implementations, the user can determine the target rule template from the rule template library according to the target data attribute, or can determine the target rule template from the rule template library according to the original data to convert the original data, or can determine the target rule template from the rule template library according to the original data and the target data attribute. Specifically, the data processing device of this embodiment performs deep learning on the rules for determining the target rule template from the rule template library according to the original data and / or the target data attributes, so that after the configuration management module 11 obtains the original data or the target data attributes, it can automatically determine the target rule template according to the original data and / or the target data attributes based on the learned rules, or it can mark each rule template in the rule template library with its corresponding original data type or target data attribute, so that the target rule template can be determined according to at least one of the currently obtained original data or target data attributes, so that the original data converted based on the target rule template meets the corresponding target data attribute.
[0070] The rule template conversion engine 12 is configured to parse the target rule template and determine at least one data conversion rule. The data conversion rule includes predefined rules in the target template and the processing order of each predefined rule.
[0071] The comparison module 13 is configured to convert the original data according to the data conversion rule, obtain the converted data, compare the converted data with the target data attribute, and determine a comparison result, wherein the comparison result is used to characterize whether the data attribute of the converted data meets the target data attribute.
[0072] In some implementations, the comparison result may be a consistency result of the data attribute of the converted data and the target data attribute. For example, if the comparison result is consistent, it indicates that the data attribute of the converted data meets the target data attribute, and if the comparison result is inconsistent, it indicates that the data attribute of the converted data does not meet the target data attribute. Optionally, when the comparison result is inconsistent, the target rule template is re-determined from the rule template library or a new rule template is generated in real time, and the corresponding data processing flow is re-performed.
[0073] In some implementations, the comparison module 13 may send the comparison result and / or conversion data to the configuration management module 11 for storage, and when the user needs, the comparison result may be obtained from the configuration management module 11 through the web page. In some implementations, the comparison result and / or conversion data may be directly sent to the web page after the comparison is completed by the comparison module 13. The user may intuitively obtain the information on the conversion data and whether the data attributes of the conversion data meet the target data attributes through the web page.
[0074] For example, a user needs to obtain the "amount" in a certain original bill, and the "amount" he wants to obtain is in uppercase format and accurate to cents, while the "amount" in the original bill is in lowercase format. The user can drag the precision processing 331 and uppercase and lowercase processing 321 in the predefined rules of component area B to configuration area C, and click the configuration template 38 to generate a rule template and store it in the rule template library. After obtaining the corresponding original bill, the corresponding original data and target data attributes are determined, wherein the bill is an image file. The original data at this time is the image structured data obtained after the bill is formatted, and the target data attribute is the "amount" in uppercase format and accurate to cents. The configuration management module 11 determines the target rule template according to the original data and / or target data attributes. In some implementations, the data processing device can automatically determine the target rule template of the corresponding bill according to the acquired target data attributes, or automatically determine the target rule template according to the historical correspondence between the bill of the corresponding type and the target rule template. The user can also directly specify the target rule template corresponding to the bill, and then perform corresponding data processing through the data processing device, so that the matching between the bill and the target rule template can be more flexible.
[0075] Furthermore, in this embodiment, after determining the target rule template, the rule template conversion engine 12 parses the target rule template to determine the data conversion rule, and the comparison module 13 converts the original data according to the data conversion rule, and compares the "amount" obtained after the conversion with the target data attribute to determine whether the two are consistent, so as to determine the comparison result. At the same time, the comparison result and the converted "amount" can also be sent to the web page, so that the user can intuitively understand whether the converted data meets the target data attribute and can directly obtain the converted data.
[0076] Further optionally, when the user needs to perform corresponding data conversion processing on multiple types of bills separately, and different types of data bills have different data conversion processing methods, the data processing device can perform data conversion processing on each bill in sequence in the order in which the processing requests are received, or it can perform data conversion processing on each bill in parallel after receiving the processing requests.
[0077] The embodiment of the present invention determines a target rule template based on the acquired original data and / or target data attributes, the target rule template is generated based on at least one selected predefined rule, at least one data conversion rule is determined by parsing the target rule template, the original data is converted according to each of the data conversion rules to obtain conversion data, the conversion data is compared with the target data attributes, and it is determined whether the data attributes of the conversion data meet the target data attributes. Thus, the corresponding conversion of the data is achieved, and the consistency of the converted data attributes and the target data attributes can be compared, so as to determine whether the converted data attributes meet the target data attributes.
[0078] Figure 4 is a schematic diagram of a data processing device according to another embodiment of the present invention. Figure 4 As shown, the data processing device includes a configuration management module 41 , a rule template conversion engine 42 , a comparison module 43 , an image recognition module 44 , a data access module 45 and an application program interface 46 .
[0079] Among them, the configuration management module 41 includes a Json data unit 411, a picture structured data unit 412, a rule template management unit 413 and a comparison task management unit 414. The rule template conversion engine 42 includes a key-value pair rule processing unit 421, a keyword rule processing unit 422 and a key value rule processing unit 433. Among them, the Json data unit 411 is used to receive and store raw data of a Json data type. The picture structured data unit 412 is used to receive and store raw data of a structured data type extracted from the picture. The rule template management unit 413 is used to generate, store and manage rule templates. The comparison task management unit 414 is used to generate a comparison task between the corresponding conversion data and the target data attribute. It should be understood that the rule template conversion engine 42 is used to store the parsing methods of various predefined rules to parse out the data conversion methods corresponding to various predefined rules. For example, the key-value pair rule processing unit 421 is used to implement the parsing of key-value pair rules, and the key value rule processing unit 433 is used to implement the parsing of key value rules. It should be understood that the rule template conversion engine 42 is not limited to the key-value pair rule processing unit 421, the keyword rule processing unit 422 and the key value rule processing unit 423, and it can be configured based on all predefined rules supported by the data processing device of this embodiment.
[0080] In some embodiments, if the data uploaded by the user is a picture, the picture recognition module 44 receives the picture uploaded by the web page and performs text recognition on the obtained picture to obtain the corresponding picture structured data. The picture recognition module 44 sends the obtained picture structured data to the picture structured data unit 412 for storage. In some implementations, the configuration management module 41 can feedback the information that the picture structured data is successfully stored to the web page. Optionally, the picture recognition module 44 can use various existing picture recognition methods, such as OCR (Optical Character Recognition, optical character recognition) technology and CRNN (Convolutional Recurrent Neural Network, convolutional recurrent neural network) deep learning model, etc., to convert the obtained picture into picture structured data. In some embodiments, the target data attribute can be obtained through the data access module 45 and the target data attribute can be sent to the configuration management module 41 for storage. Optionally, the user can enter or select the target data attribute on the web page, and the target data attribute entered or selected on the web page can be sent to the configuration management module 41 through the data access module 45, so that the data processing device can perform subsequent data processing according to the target data attribute.
[0081] In some embodiments, the configuration management module 41 can determine the target rule template from a pre-generated rule template library based on the original data and / or target data attributes, or can configure the corresponding target rule template in real time after receiving the original data and target data attributes. Further, the user can select the predefined rules through the configuration rule template interface on the web page, and click the configuration template control to send the instruction to generate the rule template to the configuration management module 41 after the selection is completed, so that the configuration management module 41 can generate the rule template through the rule template management unit 413 after receiving the instruction to generate the rule template. The specific rule template generation steps can be referred to steps S210 and S220, which will not be repeated here. The application interface 46 is used to extend the predefined rules. Optionally, the user can extend the predefined rules through the application interface so that the extended predefined rules can be used to generate rule templates, thereby making the generated rule templates more diversified, and thus better meeting the user's data conversion needs.
[0082] In some embodiments, the comparison task management unit 414 can generate a comparison task based on the image structured data and / or target data attributes and the corresponding target rule template. The rule template conversion engine 42 parses the corresponding target rule template according to the comparison task and obtains the corresponding data conversion rule. The comparison module 43 converts the image structured data according to the parsed data conversion rule, obtains the converted data, and compares the converted data with the target data attributes to determine whether the converted data attributes meet the target data attributes.
[0083] The embodiment of the present invention illustrates the data processing flow with the original data being picture structured data. It should be noted that the original data of the present invention can also be Json data. When the original data is Json data, the Json data can be directly uploaded to the configuration management module through the web page. It should be understood that when the original data is Json data, except for the step of uploading the original data, the other data processing flows are similar to the processing flows when the original data is picture structured data. Therefore, the data processing device of this embodiment can support both Json data that can directly perform data conversion and data conversion of pictures.
[0084] The data processing device of the embodiment of the present invention determines the target rule template according to the acquired original data and / or target data attributes, the target rule template is generated according to at least one selected predefined rule, at least one data conversion rule is determined by parsing the target rule template, the original data is converted according to each of the data conversion rules to obtain conversion data, the conversion data is compared with the target data attributes, and it is determined whether the data attributes of the conversion data meet the target data attributes. Thus, the corresponding conversion of the data is achieved, and the consistency of the converted data attributes and the target data attributes can be compared, so as to determine whether the converted data attributes meet the target data attributes. Moreover, the data processing device of this embodiment can not only perform data conversion of Json data, but also perform data conversion of pictures. At the same time, the user can also expand the relevant rules according to the needs to generate or adjust the rule template, so as to make the rule template more diversified, thereby improving the accuracy of the converted data.
[0085] Figure 5 is a flow chart of the data processing method of the embodiment of the present invention. Figure 4 The data processing device shown in FIG. Figure 5 As shown, the data processing method of this embodiment includes the following steps:
[0086] Step S510, obtaining the original data to be converted and the target data attributes.
[0087] Wherein, the original data is Json format data or picture structured data. In some implementations, the picture recognition module 44 obtains the picture uploaded by the web page, and performs picture conversion on the obtained picture to obtain picture structured data. The picture recognition module 44 feeds back the picture structured data to the configuration management module 41, and stores it, so that the original data to be converted can be obtained through the configuration management module 41. Optionally, the configuration management module 41 can feed back the information of the successful storage of the original data to the web page. In some embodiments, the configuration management module 41 can also directly obtain Json format data from the web page and store it. The target data attribute represents the conditions that the target data (i.e., the converted data) must meet, for example, meet the specified characteristics, format or threshold range and other conditions. In some implementations, the target data attribute can be obtained by the data access module 45 and sent to the configuration management module 41 for storage, so that the target data attribute can be obtained by the configuration management module 41.
[0088] Step S520: determining a target rule template based on the original data and / or target data attributes, wherein the target rule template is generated according to at least one selected predefined rule.
[0089] In some embodiments, the predefined rules include key-value pair rule processing, keyword rule processing and key value rule processing. The key-value pair rule processing mainly processes the key and value of the data at the same time, and the key-value pair rule processing includes blacklist key-value pair rules and whitelist key-value pair rules. When the data contains a blacklist key-value pair, it means that the data needs to be filtered out, and when the data contains a whitelist key-value pair, it means that the data needs to be retained. The keyword rule processing includes case processing, regular processing and mapping processing. The key value rule processing includes precision processing, type processing and conversion processing. It should be understood that the predefined rules in this embodiment are only examples of commonly used rules, and this embodiment does not limit the types of predefined rules. In some embodiments, the predefined rules can also be extended through the application program interface 46. For example, relevant rules can be customized based on the data to be processed and the data conversion effect to be achieved, so that the extended predefined rules can be used to generate rule templates, so that the generated rule templates are more diverse, and thus can better meet the data conversion needs of users.
[0090] In some embodiments, the configuration management module 41 determines the target rule template from the rule template library according to the acquired original data and / or target data attributes. The rule template library includes a plurality of pre-generated rule templates. The generation method of the rule template can be referred to step S210 and step S220, which will not be described in detail here.
[0091] In some implementations, the user can configure corresponding rule templates according to multiple target data attributes in advance and store them in a rule template library, so that after obtaining the original data and the target data attributes, the corresponding target rule template can be determined from the rule template library. The user can also configure and generate corresponding rule templates in real time to perform data conversion after obtaining the original data and the target data attributes.
[0092] In some implementations, the user can determine the target rule template from the rule template library according to the target data attribute, or determine the target rule template from the rule template library according to the original data to convert the original data, or determine the target rule template from the rule template library according to the original data and the target data attribute. Specifically, by deeply learning the rules for determining the target rule template from the rule template library according to the original data and / or the target data attribute, after the configuration management module obtains the original data or the target data attribute, it can automatically determine the target rule template according to the original data and / or the target data attribute based on the learned rules, or each rule template in the rule template library can be labeled with its corresponding original data type or target data attribute, so that the target rule template can be determined according to at least one of the currently obtained original data or target data attributes, so that the original data converted based on the target rule template meets the target data attribute.
[0093] Step S530: parse the target rule template to determine at least one data conversion rule.
[0094] In some implementations, the target rule template is parsed by the rule template conversion engine 42 to determine the data conversion rules corresponding to the target rule template. The data conversion rules include predefined rules in the target template and the processing order of each predefined rule. Optionally, the rule template conversion engine 42 can send the parsed data conversion rules to the comparison module 43 for subsequent data processing.
[0095] Step S540: convert the original data according to each of the data conversion rules to obtain converted data.
[0096] In some implementations, the configuration management module 41 sends the original data and the target data attributes to the comparison module 43, and the comparison module 43 converts the original data according to each of the data conversion rules to obtain conversion data. In some implementations, the obtained conversion data can be stored in the configuration management module 41. The user can query the conversion data from the configuration management module 41 through the web page.
[0097] Step S550: compare the converted data with the target data attributes to determine a comparison result, wherein the comparison result is used to indicate whether the data attributes of the converted data meet the target data attributes.
[0098] In some implementations, the comparison module 43 determines the comparison result by comparing the conversion data with the target data attribute. In some implementations, the comparison result can be a consistency result between the data attribute of the conversion data and the target data attribute. For example, if the comparison result is consistent, it indicates that the data attribute of the conversion data meets the target data attribute, and if the comparison result is inconsistent, it indicates that the data attribute of the conversion data does not meet the target data attribute. Optionally, when the comparison result is inconsistent, the target rule template is re-determined from the rule template library or a new rule template is generated in real time, and the corresponding data processing flow is re-performed.
[0099] In some implementations, the comparison module 43 may send the comparison result and / or conversion data to the configuration management module 41 for storage, and when the user needs, the comparison result may be obtained from the configuration management module through the web page. In some implementations, the comparison result and / or conversion data may be directly sent to the web page after the comparison is completed by the comparison module. The user may intuitively obtain the information on the conversion data and whether the data attributes of the conversion data meet the target data attributes through the web page.
[0100] The embodiment of the present invention determines a target rule template based on the acquired original data and / or target data attributes, the target rule template is generated based on at least one selected predefined rule, at least one data conversion rule is determined by parsing the target rule template, the original data is converted according to each of the data conversion rules to obtain conversion data, the conversion data is compared with the target data attributes, and it is determined whether the data attributes of the conversion data meet the target data attributes. Thus, the corresponding conversion of the data is achieved, and the consistency of the converted data attributes and the target data attributes can be compared, so as to determine whether the converted data attributes meet the target data attributes.
[0101] An embodiment of the present invention further provides an electronic device. Figure 6 Schematic diagram of an electronic device according to an embodiment of the present invention. Figure 6 As shown, Figure 6 The electronic device shown is a universal address query device, which includes a universal computer hardware structure, which at least includes a processor 61 and a memory 62. The processor 61 and the memory 62 are connected via a bus 63. The memory 62 is suitable for storing instructions or programs executable by the processor 61. The processor 61 can be an independent microprocessor or a collection of one or more microprocessors. Therefore, the processor 61 executes the instructions stored in the memory 62 to perform the following operations: Figure 5The method flow of the embodiment shown realizes the processing of data and the control of other devices. The bus 63 connects the above-mentioned multiple components together, and connects the above-mentioned components to the display controller 64 and the display device and the input / output (I / O) device 65. The input / output (I / O) device 65 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 65 is connected to the system through an input / output (I / O) controller 66.
[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0103] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.
[0104] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.
[0105] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.
[0106] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.
[0107] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A data processing method, characterized in that: The method comprises: Get the original data to be converted and the target data attributes; Determining a target rule template based on the original data and / or target data attributes, the target rule template being generated according to the selected at least one predefined rule; Parsing the target rule template to determine at least one data conversion rule; Convert the original data according to the data conversion rules to obtain converted data; The converted data is compared with the target data attribute to determine a comparison result, wherein the comparison result is used to indicate whether the data attribute of the converted data satisfies the target data attribute.
2. The method according to claim 1, characterized in that The original data is obtained through the web page; The method further comprises: The comparison result and / or conversion data are sent to a web page.
3. The method according to claim 1, characterized in that The original data is Json format data or picture structured data.
4. The method according to claim 1, characterized in that The obtaining of the original data to be converted comprises: Get the image; The image is formatted to obtain image structured data.
5. The method according to claim 1, characterized in that The rule template is generated specifically as follows: Receive selected predefined rules and rule processing order through the configuration rule template interface; A rule template is generated according to the selected predefined rules and the rule processing sequence.
6. The method according to claim 1, characterized in that The determining of the target rule template based on the original data and / or target data attributes is specifically: A target rule template is determined from a pre-generated rule template library based on the original data and / or target data attributes.
7. The method according to claim 1, characterized in that The predefined rules include key-value pair rule processing, keyword rule processing and key value rule processing.
8. The method according to claim 1, characterized in that The method further comprises: The predefined rules are extended through an application program interface.
9. A data processing device, characterized in that: The device comprises: a configuration management module configured to obtain original data to be converted and target data attributes and determine a target rule template based on the original data and / or the target data attributes, wherein the target rule template is generated according to at least one selected predefined rule; A rule template conversion engine is configured to parse the target rule template and determine at least one data conversion rule; The comparison module is configured to convert the original data according to the data conversion rule, obtain the converted data, compare the converted data with the target data attribute, and determine a comparison result, wherein the comparison result is used to characterize whether the data attribute of the converted data meets the target data attribute.
10. The device according to claim 9, characterized in that The device also includes: Image recognition module, used to perform image recognition to obtain image structured data; A data access module is used to obtain target data attributes and send the target data attributes to the configuration management module; An application program interface is used to extend the predefined rules.
11. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 8.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
13. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.
