Method for determining unique key of structured data, electronic device and storage medium

Abstract

Embodiments of the present application provide a method for determining a unique key of structured data, an electronic device and a storage medium. In the embodiments of the present application, first, column fields of the structured data are screened to screen out column fields with a greater probability of being a unique key. Then, the screened column fields are combined, and it is determined whether the combined column fields have repeated field values in the structured data. If the combined column fields do not have repeated field values in the structured data, it means that the probability of determining the unique key from the screened column fields is greater. At this time, the unique key of the structured data can be determined based on the screened column fields. Thus, the unique key of the structured data can be automatically and accurately determined, the efficiency of determining the unique key is improved, and a basis for more reliable analysis of the structured data is provided.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for determining the unique key of structured data, an electronic device, and a storage medium. Background Technology

[0002] In the era of big data, structured data is widely found in relational databases, distributed data storage systems, or text files. Typically, structured data in relational databases is assigned a unique key (UK). However, structured data in distributed data storage systems or text files may not have a unique key. A unique key is crucial for identifying the uniqueness of data, preventing data duplication, and is therefore essential for analyzing structured data. Thus, it is necessary to determine the unique key for structured data. Summary of the Invention

[0003] This application provides a method, electronic device, and storage medium for determining the unique key of structured data, thereby automatically and accurately determining the unique key of structured data and improving the efficiency of unique key determination.

[0004] This application provides a method for determining a unique key for structured data, comprising: selecting at least one first column field from the column fields of the structured data; combining the at least one first column field to obtain a second column field; determining whether there are duplicate field values ​​in the field values ​​of the second column field; if not, determining a unique key for the structured data based on the at least one first column field.

[0005] This application also provides a method for determining the unique key of structured data, applied to a cloud server. The method includes: responding to a request from a user device to determine the unique key of structured data; selecting at least one first column field from the column fields of the structured data; combining the at least one first column field to obtain a second column field; determining whether there are duplicate field values ​​in the field values ​​of the second column field; if not, determining the unique key of the structured data based on the at least one first column field; and returning the unique key of the structured data to the user device.

[0006] This application also provides an electronic device, including: a memory and a processor; the memory for storing a computer program; and the processor coupled to the memory for executing the computer program to perform steps in a method for determining unique keys of structured data.

[0007] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the method for determining the unique key of structured data.

[0008] In this embodiment, the column fields of the structured data are first filtered to identify those with a high probability of being used as unique keys. Next, the filtered column fields are combined, and it is determined whether the combined column fields have duplicate values ​​in the structured data. If the combined column fields do not have duplicate values ​​in the structured data, it indicates a high probability of determining a unique key from the filtered column fields. In this case, the unique key of the structured data can be determined based on the filtered column fields. Therefore, the unique key of the structured data can be determined automatically and accurately, improving the efficiency of unique key determination and providing a foundation for more reliable analysis of structured data. Attached Figure Description

[0009] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0010] Figure 1 An exemplary application scenario diagram provided for an embodiment of this application;

[0011] Figure 2 A flowchart illustrating a method for determining the unique key of structured data provided in this application embodiment;

[0012] Figure 3 A flowchart illustrating another method for determining the unique key of structured data provided in this application embodiment;

[0013] Figure 4 A flowchart illustrating another method for determining the unique key of structured data provided in this application embodiment;

[0014] Figure 5 A flowchart illustrating a method for determining a unique key for structured data based on the remaining first column fields, as provided in this application embodiment;

[0015] Figure 6 Signaling interaction diagram of a method for determining the unique key of structured data provided in an embodiment of this application;

[0016] Figure 7 This is a schematic diagram of a unique key determination device for structured data provided in an embodiment of this application;

[0017] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the access relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following associated objects have an "or" relationship. Furthermore, in the embodiments of this application, "first," "second," "third," etc., are only used to distinguish the content of different objects and have no other special meaning.

[0020] The following explains some terms used in the embodiments of this application:

[0021] Structured data (also known as row data) refers to data that can be logically expressed using a two-dimensional table structure, such as, but not limited to, database tables and data files. Structured data consists of rows and columns; typically, the "columns" of structured data are called fields, and the "rows" are called records.

[0022] A field (also known as a column field) refers to a certain characteristic or attribute of a thing or phenomenon; that is, an attribute common to all rows of data in structured data. For example, means of transportation, region, and time are all column fields.

[0023] In the era of big data, structured data is widely found in relational databases, distributed data storage systems, or text files. Typically, structured data in relational databases is assigned a unique key (UK). However, structured data in distributed data storage systems or text files may not have a unique key. A unique key is crucial for identifying the uniqueness of data, preventing data duplication, and is therefore essential for analyzing structured data. Thus, it is necessary to determine the unique key for structured data.

[0024] Based on the above, embodiments of this application provide a method, electronic device, and storage medium for determining unique keys in structured data. In this embodiment, the column fields of the structured data are first filtered to identify those with a high probability of being used as unique keys. Then, the filtered column fields are combined, and it is determined whether the combined column fields have duplicate values ​​in the structured data. If the combined column fields do not have duplicate values ​​in the structured data, it indicates a high probability of determining a unique key from the filtered column fields. In this case, the unique key of the structured data can be determined based on the filtered column fields. Therefore, the unique key of structured data can be automatically and accurately determined, improving the efficiency of unique key determination and providing a foundation for more reliable analysis of structured data.

[0025] Figure 1 This is an exemplary application scenario diagram provided for an embodiment of this application. See also... Figure 1 This application scenario diagram can include the user's device and the cloud server. See also Figure 1 As shown in ①, the user sends a request to the cloud server to determine the unique key for the structured data. Next, see... Figure 1 As shown in ②, the cloud server performs a column field selection operation, that is, it filters the column fields of the structured data to select those columns with a high probability of being used as unique keys. Next, see... Figure 1 As shown in ③, the cloud server performs a column field combination operation, that is, it combines the filtered column fields. Next, see... Figure 1 As shown in ④, the cloud server performs a duplicate field value check, that is, it checks whether the combined column fields have duplicate field values ​​in the structured data. If the combined column fields do not have duplicate field values ​​in the structured data, it means that the combination of one or more of the filtered column fields is likely to be a unique key. If the combined column fields have duplicate field values ​​in the structured data, it means that the combination of one or more of the filtered column fields is unlikely to be a unique key. Next, see... Figure 1 As shown in ⑤, the cloud server performs a unique key determination operation, that is, if the combined column fields do not have duplicate field values ​​in the structured data, the unique key of the structured data is determined based on the selected column fields. Finally, see Figure 1 As shown in ⑥, the cloud server sends a unique key of the structured data to the user device, so that the user can perform various data analysis operations such as data quality assessment or data development based on the unique key of the structured data.

[0026] exist Figure 1 In the scenario shown, cloud servers can automatically and accurately determine the unique key of structured data, improving the efficiency of unique key determination and thus providing a foundation for more reliable analysis of structured data.

[0027] It should be noted that, Figure 1 The application scenario shown is merely an exemplary scenario, and the embodiments of this application do not limit the application scenarios. The embodiments of this application do not... Figure 1 The included equipment is not limited, nor is it restricted. Figure 1 The positional relationships between the devices are defined.

[0028] In this embodiment, the user equipment can interact with the network platform via a wired network or a wireless network. For example, the wired network may include coaxial cable, twisted pair, and optical fiber, while the wireless network may be a 2G (2nd Generation), 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), or Wireless Fidelity (WIFI) network. This application does not limit the specific type or form of interaction, as long as it enables the terminal device to interact with the network platform.

[0029] The user equipment can be either hardware or software. When the user equipment is hardware, it can be, for example, a mobile phone, tablet computer, desktop computer, wearable smart device, smart home device, etc. When the user equipment is software, it can be installed on the aforementioned hardware devices. In this case, the user equipment can be, for example, multiple software modules or a single software module, etc., and this application embodiment is not limited. The cloud server can also be either hardware or software. When the cloud server is hardware, it can be a single server or a distributed server cluster composed of multiple servers. When the cloud server is software, it can be multiple software modules or a single software module, etc., and this application embodiment is not limited.

[0030] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0031] Figure 2 This is a flowchart illustrating a method for determining the unique key of structured data, provided in an embodiment of this application. The method can be executed by a device for determining the unique key of structured data, which can consist of software and / or hardware, and is generally configured in a cloud server. See also... Figure 2 The method may include the following steps:

[0032] 201. Select at least one first column field from the column fields of the structured data.

[0033] 202. Combine at least one of the first column fields to obtain the second column field.

[0034] 203. Determine if there are duplicate values ​​in the second column. If the result is no, proceed to step 204. If the result is yes, proceed to step 205.

[0035] 204. Determine the unique key of structured data based on at least one first column field.

[0036] 205. It is determined that the structured data does not have a unique key.

[0037] In this embodiment, when relevant personnel, such as data developers, require unique key determination for structured data, the unique key determination process for structured data can be triggered. Of course, in practical applications, triggering conditions can be flexibly set, and the unique key determination process for structured data can be automatically triggered when these conditions are met. Triggering conditions include, but are not limited to: the structured data is newly added data, the structured data is critical data, or the structured data is pre-configured data that requires unique key determination, etc.

[0038] In this embodiment, when determining the unique key of structured data, it is first necessary to determine which columns and fields the structured data contains. Taking the order details table shown in Table 1 as an example, the order details table includes 8 columns and fields. The field names of the 8 columns and fields are order number, product name, product code, quantity, unit price, total price, customer name, and customer number, respectively. The column containing the field name stores the field value. For example, the column whose field name is product name stores the field values ​​of desk, computer, mineral water, and pen, respectively.

[0039] Table 1 Order Details

[0040] Order number Product Name Product Code quantity unit price Total price Customer Name Customer Number 1001 desk 3001 10 500 5000 Customer A 6001 1002 computer 3002 10 4000 40000 Customer B 6002 1003 mineral water 3003 20 1 20 Customer C 6003 1004 Pen 3004 20 1 20 Customer D 6004

[0041] In this embodiment, after determining all or some of the column fields of the structured data, at least one first column field is selected from the column fields of the structured data. The first column field refers to the column field selected from the column fields of the structured data to determine the unique key, and there can be one or more first column fields.

[0042] In practical applications, all or some of the columns can be directly used as at least one first column. Alternatively, to balance efficiency and accuracy in determining the unique key, the following are some examples of column selection methods:

[0043] Method 1: Select at least one first column field from the column fields of the structured data whose field values ​​represent classification information.

[0044] It is worth noting that column fields whose field values ​​represent classification information are column fields that can classify records in structured data. These column fields can also be called classification column fields, and the data type of the field values ​​of classification column fields is usually string.

[0045] Taking Table 1 as an example, the fields "Order Number," "Product Name," "Product Code," "Customer Name," and "Customer ID" are category columns. However, the fields "Quantity," "Unit Price," and "Total Price" are not category columns; these are called numeric columns. Numeric columns are characterized by their values ​​being of numeric data types. Numeric types include, but are not limited to, integers and floating-point numbers.

[0046] In practical applications, categorical columns can be identified from all or some columns of structured data based on the semantic information of the field name and / or the data type of the field value, without any restrictions.

[0047] Method 2: Select at least one first column from the column fields of the structured data, excluding the column fields that are prohibited from selection.

[0048] In practical applications, you can flexibly set which columns and fields are not allowed to be selected, and these prohibited columns and fields cannot be used as unique keys. For example, the "Quantity" field, "Unit Price" field, and "Total Price" field are prohibited columns and fields.

[0049] Alternatively, to balance efficiency and accuracy in determining the unique key, at least one first column field whose values ​​represent categorical information can be selected from the columns of the structured data, excluding those that are prohibited from selection. It is understood that this selected first column field is a categorical column field, but not a prohibited column field.

[0050] Method 3: Select at least one of the required columns in the structured data as the first column field.

[0051] In practical applications, you can flexibly set which columns and fields are required as needed, and these required columns and fields can be used as unique keys. For example, the "Order Number" field, the "Product Name" field, the "Product Code" field, the "Customer Name" field, and the "Customer ID" field are required fields.

[0052] Alternatively, from the required columns of the structured data, at least one first column field whose field values ​​represent categorical information can be selected. It is understood that this selected first column field is a categorical column field and is a required column field.

[0053] In this embodiment, at least one first column field can be directly selected from all or some of the column fields of the structured data. Optionally, considering the potentially large volume of structured data, sampling can be performed on the structured data before selecting at least one first column field. It is understood that sampling reduces the volume of structured data, thereby improving the efficiency of determining unique keys. During sampling, the sampling rate can be flexibly set as needed and is not limited. A larger sampling rate extracts more data from the structured data; a smaller sampling rate extracts less data. It should be noted that random sampling is preferred to ensure the unbiasedness of the sampled data. Furthermore, the unique key obtained from the sampled structured data does not fully represent the unique keys of all structured data. If the requirement for non-repetitive data quality is very stringent, unique key verification of all structured data is still necessary. At this point, the unique key of the sampled data (that is, the unique key obtained based on the sampled structured data) can be verified on all the structured data. This can effectively balance the efficiency and accuracy of unique key determination in scenarios with many fields and many rows of records (that is, scenarios with a large amount of structured data).

[0054] Optionally, to balance efficiency and accuracy in determining unique keys, the structured data can be sampled when it meets the sampling criteria. The sampling criteria can be flexibly set according to actual needs. For example, if the amount of structured data is greater than or equal to a preset data volume threshold, it indicates that the amount of structured data is large and meets the sampling criteria, so sampling can be performed. If the amount of structured data is less than the preset data volume threshold, it indicates that the amount of structured data is small and does not meet the sampling criteria, so sampling is unnecessary. The preset data volume threshold can be flexibly set as needed. Similarly, if the amount of structured data falls within a preset data volume range, it indicates that the amount of structured data is large and meets the sampling criteria, so sampling can be performed. If the amount of structured data does not fall within the preset data volume range, it indicates that the amount of structured data is small and does not meet the sampling criteria, so sampling is unnecessary. The preset data volume range can be flexibly set as needed.

[0055] In practical applications, there are no restrictions on the sampling method; for example, non-random sampling can be used. Alternatively, to balance efficiency and accuracy in determining unique keys, structured data can be sampled randomly. Random sampling follows random principles, ensuring that every object in the population has a known, non-zero probability of being selected as a research subject, thus guaranteeing the representativeness of the sample. Random sampling methods include, but are not limited to: simple random sampling, systematic sampling, categorical sampling, and cluster sampling.

[0056] Alternatively, in order to balance the efficiency and accuracy of determining unique keys, when sampling structured data, random sampling can be performed on the structured data if the sampling conditions are met.

[0057] In this embodiment, after selecting at least one first column from the structured data, the at least one first column is combined to obtain a second column. It is important to note that the second column is obtained by combining all the first columns from the at least one first column; that is, the second column includes all the first columns from the at least one first column. This combination can be understood as a full combination. For example, there are five first columns: "Order Number," "Product Name," "Product Code," "Customer Name," and "Customer Number." These five columns are combined to form a combined column, which is the second column. The second column can be denoted as ("Order Number," "Product Name," "Product Code," "Customer Name," "Customer Number").

[0058] In this embodiment, after combining at least one first column field to obtain the second column field, all or some of the field values ​​of the second column field in the structured data are determined. It is understood that since the second column field is obtained by combining at least one first column field, any field value of the second column field is obtained by combining the field values ​​of each of the at least one first column field. Continuing with the example of the second column field being ("Order Number", "Product Name", "Product Code", "Customer Name", "Customer ID"), the second column field includes four field values: (1001, Desk, 3001, Customer A, 6001), (1002, Computer, 3002, Customer B, 6002), (1003, Mineral Water, 3003, Customer C, 6003), and (1004, Pen, 3004, Customer D, 6004).

[0059] In this embodiment, after determining all or part of the field values ​​of the second column in the structured data, it is determined whether there are duplicate field values ​​among the field values ​​of the second column. Duplicate field values ​​are the same field values.

[0060] If there are duplicate values ​​in the field values ​​of the second column, then there will definitely be duplicate values ​​in all the field values ​​corresponding to a single field in the first column, and there will definitely be duplicate values ​​in all the field values ​​corresponding to a combined column field obtained by combining several first columns. In other words, at least one first column field or a combination of multiple first column fields cannot be used as a unique key, which means that the structured data does not have a unique key.

[0061] If there are no duplicate values ​​in the second column, then there are likely no duplicate values ​​in any of the values ​​corresponding to a single first column, and there are also likely no duplicate values ​​in any of the values ​​corresponding to a combined column of several first columns. In other words, at least one first column or a combination of multiple first columns has a very high probability of serving as a unique key. Therefore, if there are no duplicate values ​​in the second column, then the unique key of the structured data is determined based on at least one first column.

[0062] In practical applications, multiple groups of first column fields can be selected from at least one first column field, with each group including one or more first column fields. For each group of first column fields, the first column fields in that group are combined to obtain a combined column field. For all or some of the field values ​​of the combined column field in the structured data, it is determined whether there are duplicate field values. If there are no duplicate field values, the combined column field can be used as a unique key in the structured data. If there are duplicate field values, the combined column field cannot be used as a unique key in the structured data.

[0063] The technical solution provided in this application first filters the column fields of the structured data to identify those with a high probability of serving as unique keys. Next, the filtered column fields are combined, and it is determined whether the combined column fields have duplicate values ​​in the structured data. If the combined column fields do not have duplicate values, it indicates a high probability of determining a unique key from the filtered column fields. In this case, the unique key of the structured data can be determined based on the filtered column fields. Therefore, the unique key of the structured data can be automatically and accurately determined, improving the efficiency of unique key determination and providing a foundation for more reliable analysis of structured data.

[0064] Figure 3A flowchart illustrating another method for determining the unique key of structured data provided in this application embodiment. This method can be executed by a device for determining the unique key of structured data, which can consist of software and / or hardware, and is generally configured in a cloud server. See also... Figure 3 The method may include the following steps:

[0065] 301. Select at least one first column field from the column fields of the structured data.

[0066] 302. Combine at least one of the first column fields to obtain the second column field.

[0067] 303. Determine if there are duplicate values ​​in the second column. If the result is no, proceed to step 304. If the result is yes, proceed to step 307.

[0068] 304. Repeat the following steps until at least one first column field has been processed: For any first column field in at least one first column field, determine whether there are duplicate field values ​​in the first column field. If the result is no, proceed to step 305; if the result is yes, proceed to step 306.

[0069] 305. Determine the first column field as a unique key for the structured data.

[0070] 306. Determine that the first column field is not a unique key in the structured data.

[0071] It is worth noting that step 304 is performed for each of the at least one first column fields, that is, to determine whether each first column field is a unique key of the structured data.

[0072] 307. It is determined that the structured data does not have a unique key.

[0073] The implementation methods of steps 301, 302, 303 and 307 can be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here.

[0074] In this embodiment, if it is determined that there are no duplicate field values ​​among all or some of the field values ​​in the second column, for any first column field in at least one of the first columns, all or some of the field values ​​in the first column field in the structured data are determined, and it is determined whether there are duplicate field values ​​among all or some of the field values ​​in the first column field. If there are duplicate field values ​​among all or some of the field values ​​in the first column field, then the first column field cannot be used as a unique key. If there are no duplicate field values ​​among all or some of the field values ​​in the first column field, then the first column field can be used as a unique key. The above steps are repeated for each first column field in at least one of the first columns to find more unique keys in the structured data.

[0075] The technical solution provided in this application first filters the column fields of the structured data to identify those with a high probability of serving as unique keys. Then, the filtered column fields are combined, and it is determined whether the combined column fields have duplicate values ​​in the structured data. If the combined column fields do not have duplicate values, it indicates a high probability of determining a unique key from the filtered column fields. In this case, the unique key of the structured data can be determined based on the filtered column fields. Therefore, the unique key of the structured data can be determined automatically and accurately, improving the efficiency of unique key determination and providing a foundation for more reliable analysis of structured data. Furthermore, when determining the unique key of the structured data based on the filtered column fields, it can be determined whether each column field can serve as a unique key, thereby finding more unique keys in the structured data and improving the reliability of unique key determination.

[0076] Figure 4 A flowchart illustrating another method for determining the unique key of structured data provided in this application embodiment. This method can be executed by a device for determining the unique key of structured data, which can consist of software and / or hardware, and is generally configured in a cloud server. See also... Figure 4 The method may include the following steps:

[0077] 401. Select at least one first column field from the column fields of the structured data.

[0078] 402. Combine at least one of the first column fields to obtain the second column field.

[0079] 403. Determine if there are duplicate values ​​in the second column. If the result is no, proceed to step 404. If the result is yes, proceed to step 409.

[0080] 404. Repeat the following steps until all at least one first column field has been processed: For any first column field in the at least one first column field, determine whether there are duplicate field values ​​in the field values ​​of the first column field. If the determination result is no, proceed to step 405; if the determination result is yes, proceed to step 406.

[0081] 405. Determine the first column field as a unique key for the structured data.

[0082] 406. Determine that the first column field is not a unique key in the structured data.

[0083] It is worth noting that step 404 is executed for each of the at least one first column fields, that is, determining whether each first column field is a unique key of the structured data. After step 404 has been executed for each of the at least one first column fields, that is, after all at least one first column fields have been processed, the process jumps from step 405 or step 406 to execute step 407.

[0084] 407. Remove the first column field that has been determined as a unique key from at least one first column field to obtain the remaining first column field.

[0085] 408. Determine the unique key of the structured data based on the remaining first column fields.

[0086] 409. It is determined that the structured data does not have a unique key.

[0087] The implementation methods of steps 401, 402, 403, 404, 405, 406, and 409 can be found in the relevant descriptions of the foregoing embodiments, and will not be repeated here.

[0088] In this embodiment, in order to find more unique keys in the structured data and improve the reliability of unique key determination, after determining whether a single first column field is a unique key, the first column field that has been determined to be a unique key is removed from at least one first column field to obtain the remaining first column field, and the unique key of the structured data is determined based on the remaining first column field.

[0089] In practical applications, multiple sets of first column fields can be selected from the remaining first column fields, with each set containing multiple first column fields. For each set of first column fields, the first column fields within that set are combined to obtain a combined column field. For all or some of the field values ​​in the combined column field of the structured data, it is determined whether there are any duplicate field values. If there are no duplicate field values, the combined column field can be used as a unique key in the structured data. If there are duplicate field values, the combined column field cannot be used as a unique key in the structured data.

[0090] Optionally, to identify more unique keys in the structured data and improve the reliability of unique key determination, when determining the unique key of the structured data based on the remaining first column fields, the first column fields can be combined in pairs to obtain at least one third column field. The following steps are repeated until all at least one third column field has been processed: For any third column field among the at least one third column fields, determine if there are any duplicate field values. If there are no duplicate field values, then the third column field is determined to be a unique key of the structured data. If there are duplicate field values, then the third column field is determined not to be a unique key of the structured data.

[0091] It is worth noting that for each of the at least one third column fields, it is determined whether each third column field is a unique key of the structured data.

[0092] It is understandable that any third column field is obtained by combining two different first column fields. In this way, the unique key obtained by combining two first column fields can be found, which enriches the unique key of structured data and improves the reliability of unique key determination.

[0093] For example, the remaining first column is divided into field A, field B, and field C. The third column includes (field A, field B), (field A, field C), and (field B, field C). If there are no duplicate values ​​in all fields of (field A, field B), then (field A, field B) is a unique key; if there are duplicate values ​​in all fields of (field A, field B), then (field A, field B) is not a unique key. This process continues to determine whether (field A, field C) and (field B, field C) are unique keys.

[0094] The technical solution provided in this application first filters the column fields of the structured data to identify those with a high probability of being unique keys. Then, the filtered column fields are combined, and it is determined whether the combined column fields have duplicate values ​​in the structured data. If the combined column fields do not have duplicate values, it indicates a high probability of determining a unique key from the filtered column fields. In this case, the unique key of the structured data can be determined based on the filtered column fields. Therefore, the unique key of the structured data can be determined automatically and accurately, improving the efficiency of unique key determination, identifying more unique keys in the structured data, and increasing the reliability of unique key determination, thus providing a foundation for more reliable analysis of structured data.

[0095] Further optionally, to find more unique keys in the structured data and improve the reliability of unique key determination, after at least one third column field has been processed, the following steps can be repeated until M is a set integer, where M is an integer and its initial value is 3: Determine at least one group of first column fields from the remaining first column fields, each group of first column fields including M first column fields, where any combination of N first column fields in the M first column fields is not a unique key, and N is an integer greater than 1 and less than M; Repeat the following steps until at least one group of first column fields has been processed: For any group of first column fields in the at least one group of first column fields, combine the first column fields in that group of first column fields to obtain a fourth column field; Determine whether there are duplicate field values ​​in the field values ​​of the fourth column field; If there are no duplicate field values ​​in the field values ​​of the fourth column field, then the fourth column field is determined as a unique key of the structured data; If there are duplicate field values ​​in the field values ​​of the fourth column field, then the fourth column field is determined not to be a unique key of the structured data.

[0096] In this embodiment, the fourth column is a combination of M first column fields, where M is greater than or equal to 3, and the maximum value of M is a set integer. The set integer can be flexibly set as needed, for example, 5.

[0097] It is understandable that any fourth column is formed by combining three or more different first column fields. This allows us to find unique keys formed by combining three or more first column fields, which enriches the unique keys in structured data and improves the reliability of unique key determination.

[0098] It is worth noting that when M is any integer between 3 and the set integer, after at least one set of the first column fields has been processed, M is incremented by 1, and the steps of selecting at least one new set of the first column fields and processing the new at least one set of the first column fields are re-executed until M equals the set integer.

[0099] To better understand how unique keys for structured data are determined based on the remaining first column field, combined with Figure 5 Please provide an explanation. See also... Figure 5 , Figure 4 The implementation of "determining the unique key of the structured data based on the remaining first column field", i.e., step 407, can include the following steps:

[0100] 4071. Combine the first column fields of the remaining first column fields in pairs to obtain at least one third column field.

[0101] 4072. Repeat the following steps until at least one third column field has been processed: For any third column field in at least one third column field, determine whether there are duplicate field values ​​in the field values ​​of the third column field; if the result is no, proceed to step 4073; if the result is yes, proceed to step 4074.

[0102] 4073. Determine the third column as a unique key for the structured data.

[0103] 4074. Determine that the third column is not a unique key in the structured data.

[0104] It is worth noting that step 4072 is executed for each of the first columns in at least one third column until each of the at least one third column has been checked for duplicate field values. After step 4072 has been executed for each of the at least one third column, that is, after all at least one third column has been processed, the process jumps from step 4073 or step 4074 to execute step 4075.

[0105] 4075. Repeat the following steps until M is a set integer, where M is an integer and its initial value is 3: Determine at least one set of first column fields from the remaining first column fields. Each set of first column fields includes M first column fields. Any combination of N first column fields among the M first column fields is not a unique key. N is an integer greater than 1 and less than M.

[0106] 4076. Repeat the following steps until at least one set of first column fields has been processed: For any set of first column fields in at least one set of first column fields, combine the first column fields in that set of first column fields to obtain the fourth column field.

[0107] 4077. Determine if there are any duplicate values ​​in the fourth column. If the result is no, proceed to step 4078; if the result is yes, proceed to step 4079.

[0108] 4078. Determine the fourth column as a unique key for the structured data.

[0109] 4079. Determine that the fourth column is not a unique key in the structured data.

[0110] It is worth noting that when M is any integer between 3 and the set integer, after at least one set of the first column fields has been processed, M is incremented by 1 and the process returns to step 4075 until M is the set integer.

[0111] When M is any integer, firstly, at least one set of the first column fields is selected. Then, any set of the first column fields from the at least one set is combined, and it is determined whether there are duplicate values ​​in the combined fourth column field. After all at least one set of the first column fields has been combined and the duplicate values ​​have been determined, the process of determining the unique key of M under the current integer ends, M is incremented by 1, and the process returns to step 4075 until M is the set integer.

[0112] In this embodiment, it is possible to determine whether a single first column field is a unique key, and to determine whether a combined column field consisting of two first column fields is a unique key. It is also possible to determine whether a combined column field consisting of three or more first column fields is a unique key.

[0113] In this embodiment, steps 4075 to 4079 are executed iteratively in ascending order of M. In each iteration, each group of first column fields determined from the remaining first column fields includes M first column fields. Any combination of N first column fields among the M first column fields does not belong to a unique key, where N is an integer greater than 1 and less than M.

[0114] For example, the remaining first column is divided into field A, field B, field C, field D, and field E. If fields A and B are determined to be a unique key (field A, field B), then when M=3, each group of the first column cannot contain both fields A and B simultaneously. Specifically, since the combination of fields A and B constitutes a unique key, it is prohibited for each group of the first column to include fields A, B, and C; however, it is allowed for each group of the first column to include fields A, C, and D.

[0115] For example, if fields A and B are determined to form a unique key (field A, field B), and fields A, C, and D are determined to form a unique key (field A, field C, field D), then when M=4, each group's first column cannot contain fields A and B simultaneously, nor can it contain fields A, C, and D simultaneously. Specifically, each group's first column including fields B, C, D, and E is allowed; since the combination of fields A, C, and D is a unique key, each group's first column including fields A, C, D, and E is prohibited; and since the combination of fields A and B is a unique key, each group's first column including fields A, B, D, and E is also prohibited.

[0116] The technical solution provided in this application first filters the column fields of the structured data to identify those with a high probability of being unique keys. Then, the filtered column fields are combined, and it is determined whether the combined column fields have duplicate values ​​in the structured data. If the combined column fields do not have duplicate values, it indicates a high probability of determining a unique key from the filtered column fields. In this case, the unique key of the structured data can be determined based on the filtered column fields. Therefore, the unique key of the structured data can be determined automatically and accurately, improving the efficiency of unique key determination, identifying more unique keys in the structured data, and increasing the reliability of unique key determination, thus providing a foundation for more reliable analysis of structured data.

[0117] Figure 6 This is a signaling interaction diagram illustrating a method for determining unique keys for structured data provided in an embodiment of this application. See also... Figure 6 The method may include the following steps:

[0118] 601. The user equipment sends a unique key confirmation request for the structured data to the cloud server.

[0119] 602. In response to a unique key determination request for structured data sent by a user device, the cloud server selects at least one first column field from the column fields of the structured data.

[0120] 603. The cloud server combines at least one of the first column fields to obtain the second column field.

[0121] 604. The cloud server determines whether there are duplicate values ​​in the second column. If the result is no, proceed to step 605.

[0122] 605. Determine a unique key for structured data based on at least one first column field.

[0123] 606. The unique key that the cloud server returns to the user device for structured data.

[0124] In practical applications, if there are duplicate values ​​in all or some of the fields in the second column, the cloud server determines that the structured data does not have a unique key and returns a message to the user device indicating that the structured data does not have a unique key.

[0125] For details on the implementation of each step in this embodiment, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0126] The technical solution provided in this application allows users to request a cloud server to determine the unique key of structured data. The cloud server performs the unique key determination operation and returns the unique key of the structured data to the user. Based on this unique key, the user can perform various data analysis operations on the structured data, such as data quality assessment or data development. Because the cloud server can automatically and accurately determine the unique key of structured data, the efficiency of unique key determination is improved, thus providing a foundation for more reliable analysis of structured data.

[0127] It is worth noting that in the above embodiments, when determining whether there are duplicate field values ​​among the field values ​​of various column fields, all or part of the field values ​​of the corresponding column fields can be obtained from the structured data to determine whether there are duplicate field values ​​among all or part of the field values ​​of the column fields, and there is no restriction on this.

[0128] Figure 7 This is a schematic diagram of a unique key determination device for structured data provided in an embodiment of this application. See also... Figure 7 The device may include:

[0129] Selection module 71 is used to select at least one first column field from the column fields of structured data;

[0130] The combination module 72 is used to combine at least one first column field to obtain a second column field;

[0131] The judgment module 73 is used to determine whether there are duplicate field values ​​in the field values ​​of the second column. If not, the unique key determination module 74 is triggered.

[0132] Unique key determination module 74 is used to determine the unique key of structured data based on at least one first column field.

[0133] Further optionally, when the unique key determination module 74 determines the unique key of the structured data based on at least one first column field, it is specifically used to: repeatedly execute the following steps until all at least one first column field has been processed: for any first column field in at least one first column field, determine whether there is a duplicate field value in the field value of the first column field; if not, determine the first column field as a unique key of the structured data.

[0134] Further optionally, after all at least one first column field has been processed, the unique key determination module 74 is also used to: remove the first column field that has been determined as a unique key from at least one first column field to obtain the remaining first column field; and determine the unique key of the structured data based on the remaining first column field.

[0135] Further optionally, when the unique key determination module 74 determines the unique key of the structured data based on the remaining first column fields, it is specifically used to: combine the first column fields in the remaining first column fields in pairs to obtain at least one third column field; repeat the following steps until at least one third column field has been processed: for any third column field in at least one third column field, determine whether there is a duplicate field value in the field value of the third column field; if not, determine a unique key of the structured data for the third column field.

[0136] Optionally, after at least one third column field has been processed, the unique key determination module 74 is further configured to: repeat the following steps until M is a set integer, where M is an integer and its initial value is 3: determine at least one group of first column fields from the remaining first column fields, each group of first column fields including M first column fields, where any combination of N first column fields among the M first column fields is not a unique key, and N is an integer greater than 1 and less than M; for any group of first column fields in the at least one group of first column fields, combine the first column fields in the group of first column fields to obtain a fourth column field; repeat the following steps until the at least one group of first column fields has been processed: determine whether there are duplicate field values ​​in the field values ​​of the fourth column field; if not, determine the fourth column field as a unique key for the structured data.

[0137] Further optionally, when the selection module 71 selects at least one first column field from the column fields of the structured data, it is specifically used to: select at least one first column field whose field value represents classification information from the column fields of the structured data; or, select at least one first column field from the column fields of the structured data excluding the column fields that are prohibited from selection; or, select at least one first column field from the column fields of the structured data that are required to be selected.

[0138] Further optionally, when selecting at least one first column field from the column fields of the structured data excluding the column fields that are prohibited from selection, the selection module 71 is specifically used to: select at least one first column field whose field value represents classification information from the column fields of the structured data excluding the column fields that are prohibited from selection; or,

[0139] When selecting a required column field from the column fields of structured data, module 71 is specifically used to: select at least one first column field whose field value represents classification information from the required column fields of structured data.

[0140] Alternatively, the above apparatus may further include a sampling module for: performing random sampling processing on the structured data when the structured data meets the sampling conditions.

[0141] Figure 7 The device shown can perform Figures 2 to 5 The implementation principle and technical effects of the method shown will not be elaborated further. Regarding the above embodiments... Figure 7 The specific ways in which each module and unit of the device performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0142] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 to 205 can be device A; or the execution subject of steps 201 and 202 can be device A, and the execution subject of steps 203 to 205 can be device B; and so on.

[0143] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 201, 202, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0145] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device includes: a memory 81 and a processor 82;

[0146] Memory 81 is used to store computer programs and can be configured to store various other data to support operation on the computing platform. Examples of this data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.

[0147] The memory 81 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0148] Processor 82, coupled to memory 81, is used to execute a computer program in memory 81 for performing steps in a method for determining the unique key of structured data.

[0149] Furthermore, such as Figure 8 As shown, the electronic device also includes other components such as a communication component 83, a display 84, a power supply component 85, and an audio component 86. Figure 8 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 8 The components shown. Additionally... Figure 8The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the electronic device. The electronic device in this embodiment can be a desktop computer, laptop computer, smartphone, or IoT (Internet of Things) device, or a server-side device such as a conventional server, cloud server, or server array. If the electronic device in this embodiment is a desktop computer, laptop computer, or smartphone, it may include... Figure 8 The components within the dashed box; if the electronic device in this embodiment is implemented as a conventional server, cloud server, or server array, it may be omitted. Figure 8 The component within the dashed box.

[0150] For a detailed description of the implementation process of each action by the processor, please refer to the relevant descriptions in the foregoing method embodiments or device embodiments, which will not be repeated here.

[0151] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by an electronic device in the above method embodiments.

[0152] Accordingly, this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, enables the processor to perform the steps that can be executed by an electronic device in the above method embodiments.

[0153] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as WiFi (Wireless Fidelity), 2G (2nd Generation), 3G (3rd Generation), 4G (4th Generation) / LTE (long Term Evolution), 5G (5th Generation), or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth, and other technologies.

[0154] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0155] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0156] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0157] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0159] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0160] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0161] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0162] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0163] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in this article, computer-readable media do not include transient media, such as modulated data signals and carrier waves.

[0164] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0165] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for determining the unique key of structured data, characterized in that, include: Select at least one first column field from the column fields of the structured data; The at least one first column field is combined to obtain the second column field; Determine if there are any duplicate values ​​in the second column. If not, repeat the following steps until all at least one first column field has been processed: for any first column field in at least one first column field, determine whether there are duplicate field values ​​in the field values ​​of the first column field; If not, then the first column field is determined as a unique key of the structured data; After all the at least one first column fields have been processed, the method further includes: removing the first column field that has been determined as a unique key from the at least one first column field to obtain the remaining first column field; Combine the first column fields of the remaining first column fields in pairs to obtain at least one third column field; Repeat the following steps until all of the at least one third column fields have been processed: For any third column field in the at least one third column field, determine whether there are duplicate field values ​​in the field values ​​of the third column field; if not, determine the third column field as a unique key for the structured data; After all at least one third column field has been processed, the process also includes: Repeat the following steps until M is a set integer, where M is an integer and its initial value is 3: Determine at least one set of first column fields from the remaining first column fields. Each set of first column fields includes M first column fields. Any combination of N first column fields among the M first column fields is not a unique key. N is an integer greater than 1 and less than M. Repeat the following steps until all of the at least one set of the first column fields have been processed: For any group of first column fields in at least one group, combine the first column fields in that group to obtain the fourth column field; Determine if there are any duplicate values ​​in the fourth column. If not, then the fourth column field is determined as a unique key for the structured data.

2. The method according to claim 1, characterized in that, From the column fields of the structured data, select at least one first column field, including: From the column fields of the structured data, select at least one first column field whose field values ​​represent classification information; or, Select the at least one first column field from the column fields of the structured data, excluding the column fields that are prohibited from selection; or, Select the required column field from the column fields of the structured data as at least one first column field.

3. The method according to claim 2, characterized in that, Selecting at least one first column field from the column fields of the structured data, excluding the column fields that are prohibited from selection, includes: From the columns of the structured data, excluding those that are prohibited from selection, select at least one first column whose field value represents classification information; or, Selecting the required column field from the column fields of the structured data as the at least one first column field includes: From the required columns of the structured data, select at least one first column field whose field value represents the classification information.

4. The method according to any one of claims 1 to 3, characterized in that, Before selecting at least one first column field from the column fields of structured data, the process also includes: When the structured data meets the sampling conditions, the structured data is subjected to random sampling.

5. A method for determining the unique key of structured data, characterized in that, Applied to cloud servers, the method includes: In response to a unique key determination request for structured data sent by a user equipment, at least one first column field is selected from the column fields of the structured data; The at least one first column field is combined to obtain the second column field; Determine if there are any duplicate values ​​in the second column. If not, repeat the following steps until all at least one first column field has been processed: for any first column field in at least one first column field, determine whether there are duplicate field values ​​in the field values ​​of the first column field; if not, determine the first column field as a unique key of the structured data; After all at least one first column field has been processed, the method further includes: removing the first column field that has been determined as a unique key from the at least one first column field to obtain the remaining first column field; combining the first column fields in the remaining first column field in pairs to obtain at least one third column field; repeating the following steps until all at least one third column field has been processed: for any third column field in the at least one third column field, determining whether there are duplicate field values ​​in the field values ​​of the third column field; if not, then determining the third column field as a unique key of the structured data; After all at least one third column field has been processed, the method further includes: repeating the following steps until M is a set integer, where M is an integer and its initial value is 3: determining at least one group of first column fields from the remaining first column fields, each group of first column fields including M first column fields, where any combination of N first column fields in the M first column fields is not a unique key, where N is an integer greater than 1 and less than M; repeating the following steps until all at least one group of first column fields has been processed: for any group of first column fields in the at least one group of first column fields, combining the first column fields in that group of first column fields to obtain a fourth column field; determining whether there are duplicate field values ​​in the field values ​​of the fourth column field; if not, determining the fourth column field as a unique key of the structured data; Return the unique key of the structured data to the user equipment.

6. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer programs; The processor is coupled to the memory for executing the computer program to perform the steps of the method according to any one of claims 1-5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-5.

Images