Method for binding CSV data to data object and related equipment

Abstract

The invention discloses a method and related equipment for binding CSV data to a data object, and the method comprises the steps: obtaining a target data object, defining the target data object, and obtaining the metadata of the target data object; according to the target data object metadata, constructing a binding rule of the target data object metadata; and obtaining to-be-bound CSV data, and binding the to-be-bound CSV data to the target data object according to the binding rule. The problem that in the prior art, when the CSV data is processed, due to the fact that the data are not bound into the data object, the data correspond to the data object through manual intervention in the data processing process is solved, automatic collection of the CSV data is achieved, collection of the CSV data is accelerated, and the collection labor cost is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of big data processing, and in particular to a method, system, intelligent terminal and computer-readable storage medium for binding CSV data to a data object. Background Art

[0002] CSV (Comma-Separated Values) format is a common way to store spreadsheet data. Its format is simple and easy to understand, and most spreadsheet applications support importing and exporting CSV files. It has become one of the standard formats for data exchange in many applications.

[0003] Currently, when CSV data is processed and stored, since the data itself is not bound to a data object, manual intervention is required to achieve the correspondence between data and data objects when processing the data, which makes CSV data extremely inconvenient to collect and affects users' use of CSV data.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present application is to provide a method, system, intelligent terminal and computer-readable storage medium for binding CSV data to data objects, aiming to solve the problem in the prior art that when processing CSV data, the data itself is not bound to the data object, resulting in manual intervention to achieve the correspondence between the data and the data object when processing the data, which makes it extremely inconvenient to collect CSV data and affects the user's use of the CSV data.

[0006] The first aspect of the present application provides a method for binding CSV data to a data object, comprising the following steps: obtaining a target data object, defining the target data object, and obtaining metadata of the target data object; constructing binding rules for the target data object metadata based on the target data object metadata; obtaining CSV data to be bound, and binding the CSV data to be bound to the target data object according to the binding rules.

[0007] According to the above-mentioned technical means, the embodiment of the present application can process the CSV data to be bound by defining the metadata of the target data object, so that the CSV data to be bound can be accurately bound to the target data object to be bound through the formulated binding rules, thereby solving the problem that when processing CSV data, the data itself is not bound to the data object, resulting in manual intervention to achieve the correspondence between the data and the data object when processing the data, which makes it extremely inconvenient to collect CSV data and affects the user's use of CSV data.

[0008] Optionally, in one embodiment of the present application, the acquiring the target data object, defining the target data object, and obtaining the target data object metadata specifically include: acquiring attribute information of the target data object according to the target data object, and calculating an encryption value of the target data object according to the attribute information of the target data object; acquiring a metadata storage table of the target data object, and traversing the metadata storage table according to the encrypted value to determine whether metadata information exists for the target data object; if metadata information exists in the target data object, acquiring the metadata information and defining the metadata information as target data object metadata; if metadata information does not exist in the target data object, defining the attribute information as target data object metadata.

[0009] According to the above-mentioned technical means, the embodiment of the present application can determine whether the target data object has metadata information through the encrypted value during the process of binding CSV data to the data object, so that the target data object metadata can be obtained through the existing metadata information, saving the power consumption of defining the target data object metadata; at the same time, the target data object that does not have metadata information is also defined to obtain the target data object metadata, ensuring that the target data object can be bound to all CSV data to be bound.

[0010] Optionally, in one embodiment of the present application, the target data object is acquired, the target data object is defined, and the target data object metadata is obtained, and then the following steps are further included: traversing the target data object metadata, profiling the attributes in the target data object metadata, obtaining an attribute portrait of the target data object metadata, and storing the attribute portrait in the metadata storage table, wherein the attributes of the target data object metadata include an attribute name, an attribute description, an attribute type, an attribute constraint rule, and an enumeration type.

[0011] According to the above technical means, the embodiment of the present application can label the data object by profiling the attributes in the metadata of the target data object, so that the attributes in the metadata of each data object are clearer and more specific.

[0012] Optionally, in one embodiment of the present application, constructing binding rules for the target data object metadata based on the target data object metadata specifically includes: determining whether the target data object contains a data architecture; if the target data object contains a data architecture, defining a first identification rule for the attribute portrait based on the data architecture corresponding to the target data object; if the target data object does not contain a data architecture, establishing labeling information for the target data object without a data architecture, and selecting or constructing an algorithm model based on the labeling information to define a second identification rule for the attribute portrait; using the first identification rule and the second identification rule as the binding rules; and storing the binding rules in the attribute portrait in the metadata storage table.

[0013] According to the above-mentioned technical means, the embodiments of the present application can construct binding rules for the target data object metadata for the target data object, and in the binding process, for the data with data architecture, the corresponding identification rules are defined through the data architecture, and for the target data object without data architecture, the corresponding identification rules are defined through the algorithm model after annotation. Through the formulation of the above-mentioned identification rules, the target data object can define comprehensive and complete binding rules for the target data object metadata, making the binding process more complete and accurate.

[0014] Optionally, in one embodiment of the present application, the obtaining of the CSV data to be bound and binding of the CSV data to be bound to the target data object according to the binding rules specifically include: converting the CSV data to be bound into an array; randomly sampling the array to obtain sampled data, and performing row-column conversion on the sampled data, treating each column of data with the same index as a group of analysis objects; matching the analysis object with the attribute portrait according to the binding rules to obtain a matching result; and binding the CSV data in the column where the analysis object is located to the corresponding target data object according to the matching result.

[0015] According to the above technical means, the embodiment of the present application does not need to identify the entire CSV data, but only needs to process the sampled data in each column to bind the column to the corresponding target data object, thereby saving processing time and reducing data processing consumption.

[0016] Optionally, in one embodiment of the present application, matching the analysis object with the attribute portrait according to the binding rule specifically includes: obtaining an index, matching the analysis object with the corresponding attribute portrait according to the attribute portrait in the metadata storage table to obtain a matching result, and marking the attributes of the target data object metadata through the index according to the matching result; marking the index that does not mark the attributes of the target data object metadata accordingly; recording the marking relationship between the index and the attribute, and matching the CSV data in the column where the analysis object is located with the attributes of the target data object metadata.

[0017] According to the above-mentioned technical means, the embodiment of the present application can process the analysis object through the attribute portrait in the metadata storage table, match the analysis object with the corresponding attribute, and mark the attribute of the target data object metadata through the index, so as to make the CSV data correspond one-to-one with the attribute of the target data object metadata through the index, thereby achieving the effect of binding the CSV data to be bound to the target data object.

[0018] Optionally, in one embodiment of the present application, the analysis object is matched with the corresponding attribute portrait according to the attribute portrait in the metadata storage table to obtain a matching result, which specifically includes: when the analysis object is matched with the attribute portrait, determining whether the adopted matching rule is the first identification rule or the second identification rule; if the first identification rule is adopted for matching, the match is successful when the data in the analysis object is consistent with the data structure of the target data object; if the second identification rule is adopted for matching, the match is successful when the matching accuracy between the data in the analysis object and the algorithm model reaches a predetermined threshold.

[0019] According to the above-mentioned technical means, the embodiment of the present application can directly match the identification rules of the attribute portrait corresponding to the data architecture when matching the analysis object with the attribute portrait, and the data matched by the algorithm model is considered to be successfully matched after it reaches a certain accuracy value, so that data without data architecture in the CSV data can also be flexibly automatically matched through the algorithm model.

[0020] A second aspect of the present application provides a system for binding CSV data to a data object, the system for binding CSV data to a data object comprising: a metadata definition module, for acquiring a target data object, defining the target data object, and obtaining metadata of the target data object; a rule construction module, for constructing binding rules for the target data object metadata based on the target data object metadata; and a data binding module, for acquiring CSV data to be bound, and binding the CSV data to be bound to the target data object according to the binding rules.

[0021] Optionally, in one embodiment of the present application, the metadata definition module includes: an encryption value calculation unit, used to obtain attribute information of the target data object according to the target data object, and calculate the encryption value of the target data object according to the attribute information of the target data object; an information determination unit, used to obtain the metadata storage table of the target data object, and traverse the metadata storage table according to the encryption value to determine whether the target data object has metadata information; a first definition unit, used to obtain the metadata information if there is metadata information in the target data object, and define the metadata information as target data object metadata; a second definition unit, used to define the attribute information as target data object metadata if there is no metadata information in the target data object.

[0022] Optionally, in one embodiment of the present application, the system for binding CSV data to a data object of the embodiment of the present application also includes: an attribute portrait unit, which is used to traverse the target data object metadata, portrait the attributes in the target data object metadata, obtain the attribute portrait of the target data object metadata, and store the attribute portrait in the metadata storage table, wherein the attributes of the target data object metadata include attribute name, attribute description, attribute type, attribute constraint rules and enumeration type.

[0023] Optionally, in one embodiment of the present application, the rule construction module includes: a data architecture judgment unit, used to judge whether the target data object contains a data architecture; a first portrait definition unit, used to define a first identification rule of the attribute portrait according to the data architecture corresponding to the target data object if the target data object contains a data architecture; a second portrait definition unit, used to establish annotation information for the target data object without a data architecture if the target data object does not contain a data architecture, and select or construct an algorithm model according to the annotation information to define a second identification rule of the attribute portrait; a binding unit, used to use the first identification rule and the second identification rule as the binding rule; and a binding storage unit, used to store the binding rule in the attribute portrait in the metadata storage table.

[0024] Optionally, in one embodiment of the present application, the data binding module includes: a conversion unit, used to convert the CSV data to be bound into an array; an analysis object generation unit, used to randomly sample the array to obtain sampled data, and perform row-column conversion on the sampled data, taking each column of data with the same index as a group of analysis objects; an analysis object matching unit, used to match the analysis object with the attribute portrait according to the binding rule to obtain a matching result; and an analysis object binding unit, used to bind the CSV data in the column where the analysis object is located with the corresponding target data object according to the matching result.

[0025] Optionally, in one embodiment of the present application, the analysis object matching unit includes: a first marking unit, used to obtain an index, match the analysis object with the corresponding attribute portrait according to the attribute portrait in the metadata storage table to obtain a matching result, and mark the attributes of the target data object metadata through the index according to the matching result; a second marking unit, used to mark the index that does not mark the attributes of the target data object metadata; a marking matching unit, used to record the marking relationship between the index and the attribute, and match the CSV data in the column where the analysis object is located with the attributes of the target data object metadata.

[0026] Optionally, in one embodiment of the present application, the first marking unit includes: a rule judgment unit, used to judge whether the matching rule adopted is the first identification rule or the second identification rule when matching the analysis object with the attribute portrait; a data architecture matching unit, used to determine if the first identification rule is adopted for matching, and if the data in the analysis object is consistent with the data architecture of the target data object, the match is successful; an algorithm model matching unit, used to determine if the second identification rule is adopted for matching, and if the matching accuracy between the data in the analysis object and the algorithm model reaches a predetermined threshold, the match is successful.

[0027] A third aspect of the present application provides an intelligent terminal, which includes a memory, a processor, and a program for binding CSV data to a data object stored in the memory and executable on the processor. When the program for binding CSV data to a data object is executed by the processor, the steps of the method for binding CSV data to a data object as described in the above embodiment are implemented.

[0028] The fourth aspect of the present application provides a computer-readable storage medium, which stores a program for binding CSV data to a data object. When the program for binding CSV data to a data object is executed by a processor, the steps of the method for binding CSV data to a data object as described in the above embodiment are implemented.

[0029] Beneficial effects of this application: The embodiment of the present application can use metadata as an intermediate bridge to bind the CSV data to be bound to the target data object through the binding rules obtained from the metadata of the target data object, thereby solving the problem that when processing CSV data, the data itself is not bound to the data object, which leads to the need for manual intervention to achieve the correspondence between the data and the data object when processing the data, making the CSV data extremely inconvenient to collect and affecting the user's use of the CSV data. The user can directly bind a large amount of CSV data to the target data object when using it, so that the user can conveniently and quickly perform subsequent storage or other processing processes. Furthermore, in the embodiment of the present application, when binding CSV data to a data object, when specifying the binding rules, for the target data object with a data structure, the binding rules are defined through the data structure, and for the target data object without a data structure, the binding rules are defined through the algorithm model, so that the binding rules corresponding to the data object can cover all the CSV data to be bound.

[0030] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 is a flow chart of a preferred embodiment of the method of binding CSV data to a data object of the present application; Figure 2 It is a flowchart of specific implementation steps of the process in a preferred embodiment of the method of binding CSV data to a data object in the present application; Figure 3 It is a schematic diagram of the structure of a preferred embodiment of the system for binding CSV data to a data object of the present application; Figure 4 This is a schematic diagram of the structure of a preferred embodiment of the smart terminal of the present application.

[0033] Among them, 10-a system for binding CSV data to a data object; 100-a metadata definition module, 200-a rule building module and 300-a data binding module; 502-a processor, 501-a memory and 503-a communication interface. DETAILED DESCRIPTION

[0034] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0035] The following describes the method and related equipment for binding CSV data to a data object according to an embodiment of the present application with reference to the accompanying drawings. In view of the fact that when CSV data is processed and stored in the related technology mentioned in the above background technology, since the data itself is not bound to the data object, manual intervention is required to achieve the correspondence between the data and the data object when processing the data, which makes it extremely inconvenient to collect CSV data and affects the user's use of CSV data. The present application provides a method for binding CSV data to a data object, in which a target data object can be obtained and metadata of the target data object can be defined; based on the metadata of the target data object, a binding rule of the metadata of the target data object can be constructed; the CSV data to be bound is obtained, and based on the binding rule of the metadata of the target data object, the CSV data to be bound is bound to the target data object. Thus, the problem that the data itself is not bound to the data object in the related technology, which leads to the fact that manual intervention is required to achieve the correspondence between the data and the data object when processing the data, which makes it extremely inconvenient to collect CSV data and affects the user's use of CSV data is solved.

[0036] Specifically, Figure 1 A flowchart of a preferred embodiment of a method for binding CSV data to a data object provided in an embodiment of the present application.

[0037] like Figure 1 As shown, the method of binding CSV data to a data object includes the following steps: In step S101, a target data object is acquired, the target data object is defined, and metadata of the target data object is obtained.

[0038] It can be understood that the embodiments of the present application obtain the attribute information of the target data object based on the target data object, and calculate the encryption value of the target data object based on the attribute information of the target data object; obtain the metadata storage table of the target data object, and traverse the metadata storage table based on the encrypted value to determine whether there is metadata information for the target data object; if there is metadata information in the target data object, obtain the metadata information and define the metadata information as the target data object metadata; if there is no metadata information in the target data object, define the attribute information as the target data object metadata.

[0039] In the embodiment of the present application, during the process of binding CSV data to a data object, it is possible to determine whether the target data object has metadata information through an encrypted value, thereby obtaining the target data object metadata through the existing metadata information, saving power consumption of defining the target data object metadata; at the same time, the target data object that does not have metadata information is also defined to obtain the target data object metadata, thereby ensuring that the target data object can be bound to all CSV data to be bound.

[0040] The data object is a collection of data elements with the same properties, and the attribute information of the target data object includes the same properties contained in the data in the data object. The attribute information of each data object is different, and the corresponding encryption value is also different. That is to say, after obtaining the attribute information of the target data object in the embodiment of the present application, the uniquely determined encryption value of the target data object can be obtained according to the attribute information of the target data object. In one embodiment, the encryption value is an MD5 value obtained by the MD5 algorithm (Message-Digest Algorithm 5).

[0041] It can be understood that after the encryption value of the target data object is calculated, the metadata storage table corresponding to the target data object can be obtained according to the encryption value. If the target data object has metadata information, the metadata information is defined as the target data object metadata; if the target data object does not have metadata information, the attribute information is defined as the target data object metadata. The target data object metadata is used to describe the properties of the data object.

[0042] Furthermore, the target data object is acquired, the target data object is defined, and metadata of the target data object is obtained, and then the step of traversing the metadata of the target data object, profiling the attributes in the metadata of the target data object, obtaining attribute portraits of the metadata of the target data object, and storing the attribute portraits in the metadata storage table, wherein the attributes of the metadata of the target data object include attribute name, attribute description, attribute type, attribute constraint rules, and enumeration type.

[0043] It can be understood that the obtained target data object metadata includes the attributes of the target data object metadata, wherein the attributes of the target data object metadata include attribute name, attribute description, attribute type, attribute constraint rules and enumeration type, etc. In one embodiment, the attribute constraint rules include one or more constraints such as length constraint, precision constraint, non-empty constraint, etc.

[0044] Traverse the target data object metadata to obtain each target data object metadata in the target data object, profile the attributes according to the attributes of the target data object metadata in the target data object (attribute name, attribute description, attribute type, attribute constraint rules, etc.), and then label the target data object, and store the obtained attribute portrait in the metadata storage table. For example, when the attribute of the target data object metadata is a mobile phone attribute field, profile the field and describe it as [number, 11 digits, the first three digits are the operator number segment]. In one embodiment, when storing the attribute portrait in the metadata storage table, the encrypted value of the target data object (such as the MD5 value) can be recorded as Key, the attribute portrait of the target data object metadata can be recorded as Value, and the information [Key, Value] can be stored in the metadata storage table.

[0045] By processing the target data object and obtaining the target data object metadata, the embodiment of the present application can obtain the target data object metadata corresponding to the target data object, and achieve final binding through the metadata.

[0046] In step S102, a binding rule of the target data object metadata is constructed according to the target data object metadata.

[0047] It can be understood that, according to the metadata of the target data object, the process of constructing the binding rules of the metadata of the target data object specifically includes: determining whether the target data object contains a data structure (for each specific data, there is a corresponding data structure, for example, a mobile phone number is 11 digits, and the first three digits are the operator number segment, and an ID card number is 18 digits and the first six digits are the area code); if the target data object contains a data structure, then according to the data structure corresponding to the target data object, define the first recognition rule of the attribute portrait; if the target data object does not contain a data structure (for some data in the data object, there is no data structure, such as name data, there is no corresponding data structure), then establish annotation information for the target data object without a data structure, select or construct an algorithm model according to the annotation information to define the second recognition rule of the attribute portrait; use the first recognition rule and the second recognition rule as the binding rule; store the binding rule in the attribute portrait in the metadata storage table. Among them, the algorithm model is an algorithm model selected by the user that can identify and classify CSV data according to the CSV data type.

[0048] In one embodiment, when the target data object is a mobile phone number, the mobile phone number data standard is described as "11 digits, a legal number of the operator", and the rule is defined as r'^(13[0-9]|14[5-9]|15[0-3,5-9]|16[6]|17[0-8]|18[0-9]|19[8-9])\d{8}$']. Further, when the target data object is a Chinese name, the corresponding Chinese name contains multiple possibilities and there is no data structure. The labeling information is manually established as "Chinese name", and the model algorithm is selected as the SVM algorithm (Support Vector Machine, which can classify Chinese data through training), and the second recognition rule of the attribute portrait is defined according to the SVM algorithm.

[0049] The embodiment of the present application constructs binding rules for target data object metadata for the target data object, and during the binding process, corresponding identification rules are defined through the data architecture for data with a data architecture, and identification rules are defined through an algorithm model after labeling for target data objects without a data architecture. Through the formulation of the above-mentioned identification rules, the target data object can define comprehensive and complete binding rules for the target data object metadata, making the binding process more complete and accurate.

[0050] In step S103, the CSV data to be bound is obtained, and the CSV data to be bound is bound to the target data object according to the binding rule.

[0051] It can be understood that obtaining the CSV data to be bound and binding the CSV data to be bound to the target data object according to the binding rules specifically includes: converting the CSV data to be bound into an array; randomly sampling the array to obtain sampled data, and performing row-column conversion on the sampled data, treating each column of data with the same index as a group of analysis objects; matching the analysis object with the attribute portrait according to the binding rules to obtain a matching result; and binding the CSV data in the column where the analysis object is located to the corresponding target data object according to the matching result.

[0052] In one embodiment, after obtaining the CSV data to be bound, it is first determined whether the CSV file needs to be split based on the computing resources and the size of the CSV file. If necessary, it is dynamically split into small files based on the available computing resources for easy reading. In one embodiment, when the computing resources occupied by the CSV data to be bound exceed half of the current computing resources, the CSV file is split into two files of the same size, and the conversion of the CSV data to be bound into an array and the subsequent processing are performed separately.

[0053] It is understandable that after obtaining the CSV data to be bound, the CSV data is read and split line by line into an array, that is, the CSV text file data is converted into array data in a list. In one implementation, Apache POI (an open source toolkit provided by the Apache organization) is used to stream read and split the CSV file.

[0054] In one implementation, the sampling ratio of the random sampling is pre-set by a user.

[0055] Furthermore, matching the analysis object with the attribute portrait according to the binding rule specifically includes: obtaining an index, matching the analysis object with the corresponding attribute portrait according to the attribute portrait in the metadata storage table to obtain a matching result, and marking the attributes of the target data object metadata through the index according to the matching result; marking the index that does not mark the attributes of the target data object metadata accordingly; recording the marking relationship between the index and the attribute, and matching the CSV data in the column where the analysis object is located with the attributes of the target data object metadata.

[0056] It can be understood that in the present invention, the attributes of the target data object metadata are corresponded by an index. The index is an index preset by the user, wherein a column of data in the CSV data corresponds to a unique index. After the attributes of the target data object metadata are marked by the index, the corresponding data is obtained when the corresponding index is extracted; and when the index query that does not mark the attributes of the target data object metadata is queried, it is converted to a prompt word processing, and preferably, in one embodiment, other indexes corresponding to the CSV data are prompted.

[0057] Furthermore, according to the attribute portrait in the metadata storage table, the analysis object is matched with the corresponding attribute portrait to obtain a matching result, which specifically includes: when the analysis object is matched with the attribute portrait, determining whether the adopted matching rule is the first identification rule or the second identification rule; if the first identification rule is adopted for matching, the match is successful when the data in the analysis object is consistent with the data structure of the target data object, and the match is successful when the matching accuracy between the data in the analysis object and the algorithm model reaches a predetermined threshold.

[0058] It is understandable that when the analysis object is matched with the attribute portrait, it is determined whether the matching rule used is the first identification rule or the second identification rule, that is, whether the matching method of the analysis object and the attribute portrait conforms to the first identification rule or the second identification rule. If the first identification rule is used for matching, a regular match is performed, and the match is successful when the data in the analysis object conforms to the data structure of the corresponding target data object; if the second identification rule is used for matching, the match is successful when the data in the analysis object exceeds the predetermined threshold and conforms to the requirements of the model algorithm. In one embodiment, the accuracy threshold is set to 90%.

[0059] When the analysis object matches the attribute portrait successfully, it is considered that a column of data corresponding to the analysis object matches the target data object corresponding to the attribute portrait, and then the CSV data in the column where the analysis object is located is bound to the corresponding target data object.

[0060] The following further introduces the method of binding CSV data to data objects, such as Figure 2 As shown: Step S21, defining metadata of the target data object, and establishing an attribute profile of the metadata of the target data object; Step S22: construct binding rules to define identification rules or algorithm models for each attribute portrait; Step S23, reading CSV data, that is, performing a reading operation on the acquired CSV data; in one implementation mode, Apache POI streaming reading is adopted; Step S24, sampling and completing the corresponding relationship of each column of data; Step S25: Complete the target data object binding according to the corresponding relationship.

[0061] In summary, the embodiments of the present application can use metadata as an intermediate bridge to bind the CSV data to be bound to the target data object through the binding rules obtained from the metadata, thereby solving the problem that when processing CSV data, the data itself is not bound to the data object, which leads to the need for manual intervention to achieve the correspondence between the data and the data object when processing the data, making the collection of CSV data extremely inconvenient and affecting the user experience. The user can directly bind a large amount of CSV data to the target data object when using it, so that the user can conveniently and quickly perform subsequent storage or other processing processes. Furthermore, in the embodiments of the present application, when binding CSV data to a data object, when specifying the binding rules, for the target data object with a data structure, the binding rules are defined through the data structure, and for the target data object without a data structure, the binding rules are defined through the algorithm model, so that the binding rules corresponding to the data object can cover all the CSV data to be bound.

[0062] Next, a system for planning a vehicle's driving trajectory when passing through an intersection according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0063] Figure 3 It is a block diagram of a system for binding CSV data to a data object according to an embodiment of the present application.

[0064] like Figure 3 As shown, the system for binding CSV data to a data object includes: a metadata definition module 100 , a rule construction module 200 , and a data binding module 300 .

[0065] The metadata definition module 100 is used to obtain a target data object, define the target data object, and obtain metadata of the target data object.

[0066] The rule construction module 200 is used to construct a binding rule of the target data object metadata according to the target data object metadata.

[0067] The data binding module 300 is used to obtain the CSV data to be bound, and bind the CSV data to be bound to the target data object according to the binding rule.

[0068] Optionally, in one embodiment of the present application, the metadata definition module 100 includes: an encryption value calculation unit, an information determination unit, a first definition unit and a second definition unit.

[0069] The encryption value calculation unit is used to obtain the attribute information of the target data object based on the target data object, and calculate the encryption value of the target data object through the attribute information of the target data object.

[0070] The information determination unit is used to traverse the metadata storage table of the target data object based on the encryption value of the target data object to determine whether the target data object has metadata information.

[0071] The first definition unit is used to obtain the metadata information in the data object if the metadata information exists in the data object, and define it as the target data object metadata.

[0072] The second definition unit is used to define target data object metadata for the target data object without metadata information according to attribute information of the target data object if the target data object does not have metadata information.

[0073] Optionally, in one embodiment of the present application, the system for binding CSV data to a data object in the embodiment of the present application further includes: an attribute portrait unit.

[0074] Among them, the attribute portrait unit is used to traverse the target data object metadata, portrait the attributes in the target data object metadata, obtain the attribute portrait of the target data object metadata, and store the attribute portrait in the metadata storage table, wherein the attributes of the target data object metadata include attribute name, attribute description, attribute type, attribute constraint rules, and enumeration type.

[0075] Optionally, in one embodiment of the present application, the rule construction module 200 includes: a first portrait definition unit, a second portrait definition unit, a binding unit and a binding storage unit.

[0076] Among them, the first portrait definition unit is used to traverse the metadata of the target data object based on the data architecture corresponding to the target data object and define the identification rules of each attribute portrait.

[0077] The second portrait definition unit is used to establish labeling information for the target data object without data architecture, and select or construct an algorithm model based on the labeling information to define the recognition rules of the attribute portrait.

[0078] The binding unit is used to use the identification rule of the attribute portrait as the binding rule of the target data object metadata.

[0079] The binding storage unit is used to store the binding rules of the target data object metadata as part of the attribute portrait in the metadata storage table.

[0080] Optionally, in one embodiment of the present application, the data binding module 300 includes: a conversion unit, an analysis object generation unit, an analysis object matching unit and an analysis object binding unit.

[0081] The conversion unit is used to convert the CSV data to be bound into an array.

[0082] The analysis object generating unit is used to randomly sample the array to obtain sampled data, and to perform row-column conversion on the sampled data, and to use each column of data with the same index as a group of analysis objects.

[0083] The analysis object matching unit is used to match the analysis object with the attribute portrait according to the binding rules of the target data object metadata.

[0084] The analysis object binding unit is used to bind the CSV data in the column where the analysis object is located with the corresponding data object based on the matching result of the analysis object.

[0085] Optionally, in an embodiment of the present application, the analysis object matching unit includes: a first marking unit, a second marking unit and a marking binding unit.

[0086] Among them, the first marking unit is used to match the analysis object with the corresponding attribute portrait based on the attribute portrait in the metadata storage table, and based on the matching result, mark the attributes of the target data object metadata through the index.

[0087] The second marking unit is used to mark the indexes that have not marked the attributes of the target data object metadata accordingly.

[0088] A tag binding unit records the tag relationship between the index and the attribute of the target data object metadata, matches the CSV data in the column where the analysis object is located with the attribute of the target data object metadata, and binds the CSV data to be bound to the target data object based on the matching result.

[0089] Optionally, in one embodiment of the present application, the analysis object matching unit further includes: a data architecture matching unit.

[0090] Among them, the data architecture matching unit is used for matching the analysis object with the attribute portrait. If the identification rules defined by the data framework are used for matching, the matching is performed according to the identification rules of the attribute portrait; the algorithm model matching unit is used for matching if the identification rules defined by the algorithm model are used for matching. If the matching accuracy of the data in the analysis object reaches a predetermined threshold, the matching is successful.

[0091] It should be noted that the explanation of the aforementioned method embodiment of binding CSV data to a data object is also applicable to the system of binding CSV data to a data object in this embodiment, and will not be repeated here.

[0092] According to the system for binding CSV data to data objects proposed in the embodiment of the present application, metadata can be used as an intermediate bridge to bind the CSV data to be bound to the target data object through the binding rules obtained from the metadata of the target data object, thereby solving the problem that when processing CSV data, the data itself is not bound to the data object, resulting in manual intervention to achieve the correspondence between the data and the data object when processing the data, thereby making it extremely inconvenient to collect CSV data and affecting the user's use of the CSV data. The user can directly bind a large amount of CSV data to the target data object when using it, so that the user can conveniently and quickly perform subsequent storage or other processing processes.

[0093] This solves the technical problem in the related art that, because the data itself is not bound to the data object, manual intervention is required when processing the data to achieve the correspondence between the data and the data object, which makes it extremely inconvenient to collect CSV data and affects the user's use of the CSV data.

[0094] Figure 4This is a block diagram of the internal structure of the smart terminal provided in the embodiment of the present application. The smart terminal may include: A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0095] When the processor 502 executes the program, the method for binding CSV data to a data object provided in the above embodiment is implemented.

[0096] Furthermore, the intelligent terminal also includes: The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0097] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0098] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0099] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EIS) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0100] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0101] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0102] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method of binding CSV data to a data object as described above is implemented.

[0103] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0104] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0105] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0106] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or N wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary, and then storing it in a computer memory.

[0107] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0108] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0109] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0110] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0111] It should be understood that the application of the present application is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to this application.

Claims

1. A method for binding CSV data to a data object, characterized in that, The method of binding CSV data to a data object includes: Acquire a target data object, define the target data object, and obtain metadata of the target data object; According to the target data object metadata, construct a binding rule for the target data object metadata; The CSV data to be bound is obtained, and according to the binding rule, the CSV data to be bound is bound to the target data object.

2. The method for binding CSV data to a data object according to claim 1, characterized in that: The acquiring of the target data object, defining the target data object, and obtaining metadata of the target data object specifically includes: According to the target data object, acquiring attribute information of the target data object, and calculating an encrypted value of the target data object according to the attribute information of the target data object; Obtaining a metadata storage table of the target data object, and traversing the metadata storage table according to the encrypted value to determine whether metadata information exists for the target data object; If metadata information exists in the target data object, acquiring the metadata information and defining the metadata information as metadata of the target data object; If the target data object does not contain metadata information, the attribute information is defined as the target data object metadata.

3. The method for binding CSV data to a data object according to claim 2, characterized in that: The step of acquiring the target data object, defining the target data object, and obtaining metadata of the target data object further includes: Traverse the target data object metadata, profile the attributes in the target data object metadata, obtain the attribute portrait of the target data object metadata, and store the attribute portrait in the metadata storage table, wherein the attributes of the target data object metadata include attribute name, attribute description, attribute type, attribute constraint rule and enumeration type.

4. The method for binding CSV data to a data object according to claim 3, characterized in that: The step of constructing a binding rule for the target data object metadata according to the target data object metadata specifically includes: Determining whether the target data object includes a data structure; If the target data object includes a data structure, defining a first recognition rule of the attribute portrait according to the data structure corresponding to the target data object; If the target data object does not include a data structure, then establishing annotation information for the target data object without a data structure, and selecting or constructing an algorithm model to define a second recognition rule for the attribute portrait according to the annotation information; using the first identification rule and the second identification rule as the binding rule; The binding rules are stored in the attribute profile in the metadata storage table.

5. The method for binding CSV data to a data object according to claim 4, characterized in that: The acquiring the CSV data to be bound and binding the CSV data to be bound to the target data object according to the binding rule specifically includes: Convert the CSV data to be bound into an array; Randomly sampling the array to obtain sampled data, and performing row-column conversion on the sampled data, taking each column of data with the same index as a group of analysis objects; Matching the analysis object with the attribute portrait according to the binding rule to obtain a matching result; According to the matching result, the CSV data in the column where the analysis object is located is bound to the corresponding target data object.

6. The method of binding CSV data to a data object according to claim 5, characterized in that: The matching the analysis object with the attribute portrait according to the binding rule specifically includes: Obtaining an index, matching the analysis object with the corresponding attribute portrait according to the attribute portrait in the metadata storage table to obtain a matching result, and marking the attribute of the target data object metadata through the index according to the matching result; Marking the indexes that do not mark the attributes of the target data object metadata accordingly; The tag relationship between the index and the attribute is recorded, and the CSV data in the column where the analysis object is located is matched with the attribute of the metadata of the target data object.

7. The method of binding CSV data to a data object according to claim 6, characterized in that: According to the attribute portrait in the metadata storage table, matching the analysis object with the corresponding attribute portrait to obtain a matching result specifically includes: When the analysis object is matched with the attribute portrait, determining whether the adopted matching rule is the first identification rule or the second identification rule; If the first identification rule is used for matching, the match is successful when the data in the analysis object is consistent with the data structure of the target data object; If the second identification rule is used for matching, the matching is successful when the matching accuracy between the data in the analysis object and the algorithm model reaches a predetermined threshold.

8. A system for binding CSV data to a data object, characterized in that, The system for binding CSV data to a data object includes: A metadata definition module is used to obtain a target data object, define the target data object, and obtain metadata of the target data object; A rule building module, used to build a binding rule of the target data object metadata according to the target data object metadata; The data binding module is used to obtain the CSV data to be bound, and bind the CSV data to be bound to the target data object according to the binding rule.

9. An intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and a program for binding CSV data to a data object stored in the memory and executable on the processor. When the program for binding CSV data to a data object is executed by the processor, the steps of the method for binding CSV data to a data object as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for binding CSV data to a data object, and when the program for binding CSV data to a data object is executed by a processor, the steps of the method for binding CSV data to a data object as described in any one of claims 1 to 7 are implemented.

Images