Server and data processing method

Abstract

The invention relates to a server and a data processing method. The server comprises a communication device and a processor, and the processor is configured to match a standard field with each service field to obtain each candidate service field; according to the consanguinity field set of different candidate service fields, evaluating the consanguinity between the candidate service fields to obtain a consanguinity evaluation result corresponding to each candidate service field; taking the candidate service field of which the blood relationship evaluation result is a first evaluation result as a first service field, and taking the candidate service field of which the blood relationship evaluation result is a second evaluation result as a second service field; taking the first business field as a target business field of the standard field; and / or, according to the field value of the second service field, determining a field value evaluation result of the second service field, and taking the second service field of which the field value evaluation result meets a preset condition as a target service field of the standard field. By adopting the server, the data governance efficiency and effect can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a server and a data processing method. Background Technology

[0002] As enterprises grow and their businesses expand, they often build numerous independent business systems covering multiple areas such as marketing, production, and sales. These systems continuously generate massive amounts of data during operation. To achieve standardized data management and ensure the realization of data value, the data governance process requires accurately mapping these data objects (specifically, columns in business database tables) scattered across different business systems to a unified data standard. This operation is known as data labeling.

[0003] Currently, the data labeling work is mainly completed by having technical personnel manually analyze the relationship between data objects and data standards in different business systems.

[0004] However, manual data sorting relies on staff's in-depth understanding of business systems, data structures, and data standards, which requires significant manpower and time costs. Especially when dealing with massive amounts of data, the sorting efficiency is low and it is difficult to meet the timeliness requirements of enterprise data governance. Furthermore, manual sorting is easily affected by factors such as staff's subjective cognitive biases and operational errors, making it difficult to effectively control the accuracy and consistency of data labeling results, which in turn affects the quality of subsequent data governance work. Summary of the Invention

[0005] This application provides a server and a data processing method that can improve the efficiency and effectiveness of data governance.

[0006] In a first aspect, some embodiments provide a server, including:

[0007] A communication device configured to communicate with a terminal;

[0008] and at least one processor, connected to the communication device, and configured to:

[0009] In response to a data processing request for a standard field, the standard field is matched with various business fields in different database tables to obtain candidate business fields that match the standard field.

[0010] Based on the lineage field set of different candidate business fields, the lineage relationship between different candidate business fields is evaluated to obtain the lineage evaluation result corresponding to each candidate business field; wherein, the lineage field set of each candidate business field includes business fields that have a dependency relationship with the candidate business field;

[0011] Among different candidate business fields, the candidate business field with a lineage assessment result of the first assessment result is designated as the first business field, and the candidate business field with a lineage assessment result of the second assessment result is designated as the second business field; wherein, the first assessment result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number, and the number of other candidate business fields that are dependent on the candidate business field according to the second assessment result is less than the preset number;

[0012] Use the first business field as the target business field corresponding to the standard field; and / or,

[0013] Based on the field value of the second business field in its respective database table, the field value evaluation result of the second business field is determined, and the second business field whose field value evaluation result meets the preset conditions is taken as the target business field corresponding to the standard field.

[0014] In the above embodiments, a preliminary matching is performed on the standard field and each business field to obtain candidate business fields corresponding to the standard field. Further, in order to accurately screen out the target business fields that match the standard field, the lineage relationship between different candidate business fields is evaluated. If the number of other candidate business fields that have a dependency relationship with the candidate business field is greater than or equal to a preset number, that is, the lineage assessment result is the first assessment result, indicating that the lineage relationship between this candidate business field and other candidate business fields is strong, then the candidate business field is directly used as the target business field corresponding to the standard field. For the second business field whose lineage assessment result is the second assessment result, it is further evaluated according to the corresponding field value. If the field evaluation result meets the preset conditions, the second business field is used as the target business field. Through the method of preliminary matching + lineage assessment + field value assessment, the target business fields corresponding to the standard field can be accurately screened, improving the accuracy of data labeling and thus improving the data governance effect. Moreover, compared with the manual sorting method, it can effectively improve the data labeling efficiency and ensure the real-time performance of data governance.

[0015] In some embodiments, when the processor performs an evaluation of the lineage relationship between different candidate service fields based on the lineage field set of different candidate service fields, and obtains the lineage evaluation result corresponding to each candidate service field, it is configured to:

[0016] For each candidate business field, if the candidate business field is included in the lineage field set of other candidate business fields, the lineage assessment result of the candidate business field is determined based on the relationship between the lineage field set of the candidate business field and other candidate business fields; if the candidate business field is not included in the lineage field set of other candidate business fields, the lineage assessment result of the candidate business field is determined as the second assessment result.

[0017] In some embodiments, the lineage field set includes forward business fields on which the value of the candidate business field depends, and backward business fields on which the value depends. When the processor executes the process of determining the lineage evaluation result of the candidate business field based on the relationship between the lineage field set of the candidate business field and other candidate business fields, it is configured to:

[0018] If the downstream business field of the candidate business field contains any other candidate business field, the lineage assessment result of the candidate business field is determined as the first assessment result; if the downstream business field of the candidate business field does not contain the other candidate business field, the lineage assessment result of the candidate business field is determined as the second assessment result.

[0019] In some embodiments, when the processor performs the evaluation of the field value of the second business field based on the field value of the second business field in its respective database table, it is configured to:

[0020] For each second business field, determine the frequency of occurrence of each field value of the second business field in the database table to which it belongs; wherein, the frequency of occurrence of each field value is the frequency of occurrence of the field value among the field values ​​of other second business fields; field values ​​with a frequency of occurrence less than a preset number are regarded as rare field values ​​of the second business field; based on the number of rare field values ​​contained in the second business field, determine the field value evaluation result of the second business field.

[0021] In some embodiments, when the processor performs the field value evaluation result of the second business field based on the number of rare field values ​​contained in the second business field, it is configured to:

[0022] The number of common field values ​​contained in the second business field is determined based on the number of rare field values ​​and the total number of field values; the field value evaluation result of the second business field is determined based on the number of common field values ​​and the total number of field values.

[0023] In some embodiments, when the processor performs the field value evaluation result of the second business field based on the number of public field values ​​and the total number of field values, it is configured to:

[0024] If the ratio of the number of public field values ​​to the total number of field values ​​is greater than a preset threshold, the evaluation result of the second business field is determined to be passed; if the ratio is not greater than the preset threshold, the evaluation result of the second business field is determined to be failed.

[0025] In some embodiments, when the processor executes a second business field whose field value evaluation result meets preset conditions as the target business field corresponding to the standard field, it is configured as follows:

[0026] The second business field whose field value evaluation result is passed is used as the target business field corresponding to the standard field.

[0027] In some embodiments, when the processor performs matching of a standard field and various business fields in different database tables to obtain candidate business fields that match the standard field, it is configured as follows:

[0028] Based on the standard fields and various business fields in different database tables, a prompt text is constructed; using the field matching model, the standard fields and each of the business fields are matched according to the prompt text to obtain candidate business fields that match the standard fields.

[0029] In some embodiments, the processor is further configured to:

[0030] The field values ​​of each of the target business fields are associated with the standard fields.

[0031] Secondly, some embodiments also provide a data processing method applied to a server, including:

[0032] In response to a data processing request for a standard field, the standard field is matched with various business fields in different database tables to obtain candidate business fields that match the standard field.

[0033] Based on the lineage field set of different candidate business fields, the lineage relationship between different candidate business fields is evaluated to obtain the lineage evaluation result corresponding to each candidate business field; wherein, the lineage field set of each candidate business field includes business fields that have a dependency relationship with the candidate business field;

[0034] Among different candidate business fields, the candidate business field with a lineage assessment result of the first assessment result is designated as the first business field, and the candidate business field with a lineage assessment result of the second assessment result is designated as the second business field; wherein, the first assessment result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number, and the number of other candidate business fields that are dependent on the candidate business field according to the second assessment result is less than the preset number;

[0035] Use the first business field as the target business field corresponding to the standard field; and / or,

[0036] Based on the field value of the second business field in its respective database table, the field value evaluation result of the second business field is determined, and the second business field whose field value evaluation result meets the preset conditions is taken as the target business field corresponding to the standard field.

[0037] In the above embodiments, a preliminary matching is performed on the standard field and each business field to obtain candidate business fields corresponding to the standard field. Further, in order to accurately screen out the target business fields that match the standard field, the lineage relationship between different candidate business fields is evaluated. If the number of other candidate business fields that have a dependency relationship with the candidate business field is greater than or equal to a preset number, that is, the lineage assessment result is the first assessment result, indicating that the lineage relationship between this candidate business field and other candidate business fields is strong, then the candidate business field is directly used as the target business field corresponding to the standard field. For the second business field whose lineage assessment result is the second assessment result, it is further evaluated according to the corresponding field value. If the field evaluation result meets the preset conditions, the second business field is used as the target business field. Through the method of preliminary matching + lineage assessment + field value assessment, the target business fields corresponding to the standard field can be accurately screened, improving the accuracy of data labeling and thus improving the data governance effect. Moreover, compared with the manual sorting method, it can effectively improve the data labeling efficiency and ensure the real-time performance of data governance.

[0038] Thirdly, some embodiments also provide a data processing apparatus configured in a server, comprising:

[0039] The matching module is used to respond to a data processing request for a standard field, match the standard field with various business fields in different database tables, and obtain candidate business fields that match the standard field.

[0040] The evaluation module is used to evaluate the lineage relationship between different candidate business fields based on the lineage field set of different candidate business fields, and obtain the lineage evaluation result corresponding to each candidate business field; wherein, the lineage field set of each candidate business field includes business fields that have a dependency relationship with the candidate business field;

[0041] The judgment module is used to identify candidate business fields with a lineage assessment result of a first assessment result as first business fields and candidate business fields with a lineage assessment result of a second assessment result as second business fields. The first assessment result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number, and the number of other candidate business fields that are dependent on the candidate business field according to the second assessment result is less than the preset number.

[0042] The determining module is configured to use the first business field as the target business field corresponding to the standard field; and / or,

[0043] Based on the field value of the second business field in its respective database table, the field value evaluation result of the second business field is determined, and the second business field whose field value evaluation result meets the preset conditions is taken as the target business field corresponding to the standard field.

[0044] In the above embodiments, a preliminary matching is performed on the standard field and each business field to obtain candidate business fields corresponding to the standard field. Further, in order to accurately screen out the target business fields that match the standard field, the lineage relationship between different candidate business fields is evaluated. If the number of other candidate business fields that have a dependency relationship with the candidate business field is greater than or equal to a preset number, that is, the lineage assessment result is the first assessment result, indicating that the lineage relationship between this candidate business field and other candidate business fields is strong, then the candidate business field is directly used as the target business field corresponding to the standard field. For the second business field whose lineage assessment result is the second assessment result, it is further evaluated according to the corresponding field value. If the field evaluation result meets the preset conditions, the second business field is used as the target business field. Through the method of preliminary matching + lineage assessment + field value assessment, the target business fields corresponding to the standard field can be accurately screened, improving the accuracy of data labeling and thus improving the data governance effect. Moreover, compared with the manual sorting method, it can effectively improve the data labeling efficiency and ensure the real-time performance of data governance.

[0045] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a data processing apparatus, causes the data processing apparatus to perform the data processing method of the second aspect.

[0046] Fifthly, a computer program product is provided, comprising: a computer program that, when run by a data processing device, causes the data processing device to perform the data processing method of the second aspect. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application;

[0049] Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application;

[0050] Figure 3 This is a schematic diagram of the hardware configuration of the control device provided in some embodiments of this application;

[0051] Figure 4 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application;

[0052] Figure 5 A flowchart illustrating a data processing method provided in some embodiments of this application;

[0053] Figure 6 A flowchart illustrating the field matching steps provided in some embodiments of this application;

[0054] Figure 7 A flowchart illustrating the steps for assessing blood relations provided in some embodiments of this application;

[0055] Figure 8 A flowchart illustrating the process of determining bloodline assessment results provided for some embodiments of this application;

[0056] Figure 9 A flowchart illustrating the process of determining the evaluation results of field values ​​provided in some embodiments of this application;

[0057] Figure 10 A flowchart illustrating the process of determining the evaluation results of field values ​​provided in other embodiments of this application;

[0058] Figure 11 A flowchart illustrating the process of determining the evaluation result of field values ​​provided in some embodiments of this application;

[0059] Figure 12 A flowchart illustrating a data processing method provided for other embodiments of this application;

[0060] Figure 13 This application provides schematic diagrams illustrating data processing between devices during the data processing process in some embodiments.

[0061] Figure 14 This is a structural block diagram of the data processing apparatus in some embodiments;

[0062] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0064] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0065] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0066] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0067] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0068] In some embodiments, the server can be a server 400 with data processing capabilities, which can be a standalone server or a server cluster. Display device 200 generally refers to a device with screen display and data processing capabilities. For example, display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0069] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, users can operate the display device 200 via touch operation, mobile terminal 300, and control device 100. For example, control device 100 can be a remote control, stylus, gamepad, etc.

[0070] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0071] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0072] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support, including but not limited to network television, smart television, Internet Protocol Television (IPTV), etc.

[0073] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0074] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0075] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0076] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0077] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 that includes the international wireless Fidelity (WiFi) standard. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 that includes Bluetooth functionality.

[0078] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0079] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0080] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0081] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0082] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0083] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0084] Figure 3 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of the central control device. (Example) Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0085] The control device 100 is configured to control the display device 200, and to receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0086] In some embodiments, the control device 100 may be an intelligent device. For example, the control device 100 may be equipped with various applications for controlling the display device 200 according to user needs.

[0087] In some embodiments, such as Figure 1 As shown, the mobile terminal 300 or other smart electronic devices can perform similar functions to the control device 100 after installing the application of the control display device 200.

[0088] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0089] Under the control of the controller 110, the communication interface 130 enables communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near-field communication modules such as WiFi chip 131, Bluetooth module 132, and NFC module 133.

[0090] User input / output interface 140, wherein the input interface includes at least one of other input interfaces such as microphone 141, touchpad 142, sensor 143, and button 144.

[0091] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, which can encode user input commands via WiFi, Bluetooth, or NFC protocols and send them to the display device 200.

[0092] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can also store various control signal instructions input by the user.

[0093] The power supply 180 is used to provide operating power support for the various components of the control device 100 under the control of the controller.

[0094] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources in the display device 200. The operating system can (control the display device) provide a user interface, allowing users to interact with the display device 200 and supporting the running of various applications.

[0095] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0096] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0097] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0098] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0099] like Figure 4 As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0100] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0101] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0102] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 4 As shown, hardware drivers can be configured in the kernel layer. The kernel layer can contain at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0103] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in this application embodiment. Depending on the function of the display device, the type of operating system, and other factors, the number of levels and the specific level type of the operating system may be expressed in other forms.

[0104] As enterprises grow and their businesses expand, they often build numerous independent business systems covering multiple areas such as marketing, production, and sales. These systems continuously generate massive amounts of data during operation. To achieve standardized data management and ensure the realization of data value, the data governance process requires accurately mapping these data objects (specifically, columns in business database tables) scattered across different business systems to a unified data standard. This operation is known as data labeling.

[0105] Currently, the data labeling work is mainly completed by having technical personnel manually analyze the relationship between data objects and data standards in different business systems.

[0106] However, manual data sorting relies on staff's in-depth understanding of business systems, data structures, and data standards, which requires significant manpower and time costs. Especially when dealing with massive amounts of data, the sorting efficiency is low and it is difficult to meet the timeliness requirements of enterprise data governance. Furthermore, manual sorting is easily affected by factors such as staff's subjective cognitive biases and operational errors, making it difficult to effectively control the accuracy and consistency of data labeling results, which in turn affects the quality of subsequent data governance work.

[0107] Based on this, in some embodiments, a data processing method is provided. This data processing method can be implemented by a server.

[0108] In one optional embodiment, taking the data processing method applied to a processor in a server as an example, the server includes a communication device and at least one processor; the communication device is communicatively connected to a terminal, which can be a wired communication connection or a wireless communication connection, and the processor is connected to the communication device, such as... Figure 5 As shown, the specific steps include:

[0109] S501, in response to a data processing request for a standard field, matches the standard field with various business fields in different database tables to obtain candidate business fields that match the standard field.

[0110] Standard fields are unified representations of the same data object and can be set according to actual needs. For example, a standard field can be "page id," a character type consisting of lowercase letters and numbers, used to uniquely identify a page in an application. Another standard field can be "resource id," a numeric type consisting of 10 digits, used to uniquely identify a sellable right, including the range of paid content that can be viewed, applicable media asset platforms, and the duration of the right. Users can watch paid content after purchasing it; otherwise, they cannot watch all paid content. Business fields are the data objects contained in the database tables. It's understandable that different business fields may be used to represent the same data object in different database tables. Taking the user identifier as an example, the database table in the membership system might use the business field "user_id," while the database table in the marketing system might use the business field "customer_id." If "user_id" and "customer_id" can be linked through standard fields, comprehensive analysis can be performed by combining data from the membership system and the marketing system, effectively improving the comprehensiveness of data analysis.

[0111] When a user needs to process data, a data processing request can be triggered. This request can contain at least one standard field. This embodiment primarily describes the data targeting process for a single standard field. Different database tables typically contain numerous business fields. Responding to a data processing request involves finding the business field (referred to as the target business field) that corresponds to the same data object as the standard field. Therefore, the standard field and each business field are first matched. Optionally, matching can be performed based on the semantic similarity between the standard field and the business fields. Business fields with a semantic similarity greater than a similarity threshold are considered candidate business fields. The similarity threshold can be set based on experience, multiple trials, and actual needs. Optionally, a large language model can also be used to match the standard field and each business field. Each business field can be input into the large language model along with the standard field, and the model will determine whether each business field matches the standard field. Alternatively, the standard field and each business field can be directly input into the large language model, which will then select candidate business fields that match the standard field. It should be noted that the methods for matching the standard field and each business field are not limited to those listed above; specific settings can be made according to actual needs.

[0112] S502, based on the lineage field set of different candidate business fields, evaluate the lineage relationship between different candidate business fields to obtain the lineage evaluation result corresponding to each candidate business field.

[0113] Each candidate business field's lineage field set includes business fields that have dependencies on the candidate business field. If two business fields have a dependency, it means that the value of one business field can affect the value of the other. For example, if the value of the business field "social security contribution base" depends on the business field "employee salary," then there is a dependency between "social security contribution base" and "employee salary." Optionally, the lineage field set of a candidate business field includes the preceding business fields that the candidate business field's value depends on, and the following business fields that depend on the candidate business field's value. For example, if the value of the business field "employee salary" depends on the business field "attendance days," when the candidate business field is "employee salary," then "attendance days" is the preceding business field of this candidate business field, and "social security contribution base" is the following business field. Optionally, business fields with dependencies on the business fields within a preset number of layers are extracted as lineage fields. The preset number of layers can be set based on experience, multiple trials, and actual needs. For example, if the value of business field 1 depends on business field 2, the value of business field 2 depends on business field 3, the value of business field 3 depends on business field 4, and the value of business field 4 depends on business field 5, and the preset number of layers is 3, then for business field 1, business field 2, business field 3, and business field 4 will all be considered as lineage fields, and for business field 5, business field 2, business field 3, and business field 4 will all be considered as lineage fields.

[0114] It is understandable that evaluating the lineage relationship between different candidate business fields is actually assessing the closeness of the lineage relationship between each candidate business field and other candidate business fields. In other words, after obtaining each candidate business field that matches the standard field, it can be assumed that most of these candidate business fields correspond to the same data object as the standard field, and the lineage relationship between different candidate business fields should be relatively close. Therefore, the target business field can also be selected from the candidate business fields based on the closeness of the lineage relationship between each candidate business field and other candidate business fields. If a candidate business field has dependencies on multiple candidate business fields, it can be considered that the lineage relationship between this candidate business field and each candidate business field is close. Optionally, for a candidate business field, the lineage evaluation result of this candidate business field can be determined based on the number of other candidate business fields that have dependencies on the candidate business field. For example, if the number is greater than or equal to a preset number, the lineage evaluation result is determined as the first evaluation result; if the number is less than the preset number, the lineage evaluation result is determined as the second evaluation result. The preset number can be set based on experience, multiple experiments, and actual needs, and is not specifically limited here.

[0115] S503, among different candidate business fields, the candidate business field whose lineage assessment result is the first assessment result is taken as the first business field, and the candidate business field whose lineage assessment result is the second assessment result is taken as the second business field.

[0116] The first evaluation result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number. The second evaluation result indicates that the number of other candidate business fields that are dependent on the candidate business field is less than the preset number. In other words, based on the lineage evaluation result, each candidate business field is divided into two categories - the first business field and the second business field. The first business field has a close lineage relationship with other candidate business fields, while the second business field has a distant lineage relationship with other candidate business fields. Optionally, the first evaluation result can be represented by "high", and the second evaluation result can be represented by "low".

[0117] S504, the first business field is used as the target business field corresponding to the standard field; and / or, based on the field value of the second business field in the database table, the field value evaluation result of the second business field is determined, and the second business field whose field value evaluation result meets the preset conditions is used as the target business field corresponding to the standard field.

[0118] After determining the first business field and the second business field, determining the target business field corresponding to the standard field can be done in the following three ways:

[0119] (1) Use the first business field as the target business field corresponding to the standard field;

[0120] (2) Based on the field value of the second business field in the database table, determine the field value evaluation result of the second business field, and take the second business field whose field value evaluation result meets the preset conditions as the target business field corresponding to the standard field;

[0121] (3) The first business field is used as the target business field corresponding to the standard field, and the field value evaluation result of the second business field is determined according to the field value of the second business field in the database table to which it belongs, and the second business field whose field value evaluation result meets the preset conditions is used as the target business field corresponding to the standard field.

[0122] Specifically, since the number of other candidate business fields that depend on the first business field is greater than or equal to a preset number, the first business field can be directly used as the target business field corresponding to the target field. For the second business field, because its relationship with other candidate business fields is distant, it is necessary to further determine the field value evaluation result based on the field value of the second business field, and then determine whether to use the second business field as the target business field based on whether the field value evaluation result meets preset conditions. It is understandable that for the second business field, if it corresponds to the same data object as other candidate business fields, then the field value of the second business field should have a high similarity to the field values ​​of other candidate business fields. Therefore, the field value evaluation result (also known as the consistency evaluation result) of the second business field can be determined by comparing the field value of the second business field in its database table with the corresponding field values ​​of other candidate business fields. For example, the similarity between the field value corresponding to the second business field and the corresponding field values ​​of other candidate business fields can be used as the field value evaluation result. It should be noted that the methods for determining the field value evaluation result can be set according to the actual situation, and will not be listed here.

[0123] In the above embodiments, a preliminary matching is performed on the standard field and each business field to obtain candidate business fields corresponding to the standard field. Further, in order to accurately screen out the target business fields that match the standard field, the lineage relationship between different candidate business fields is evaluated. If the number of other candidate business fields that have a dependency relationship with the candidate business field is greater than or equal to a preset number, that is, the lineage assessment result is the first assessment result, indicating that the lineage relationship between this candidate business field and other candidate business fields is strong, then the candidate business field is directly used as the target business field corresponding to the standard field. For the second business field whose lineage assessment result is the second assessment result, it is further evaluated according to the corresponding field value. If the field evaluation result meets the preset conditions, the second business field is used as the target business field. Through the method of preliminary matching + lineage assessment + field value assessment, the target business fields corresponding to the standard field can be accurately screened, improving the accuracy of data labeling and thus improving the data governance effect. Moreover, compared with the manual sorting method, it can effectively improve the data labeling efficiency and ensure the real-time performance of data governance.

[0124] In one embodiment, after determining the target business field corresponding to the standard field, the following can also be performed:

[0125] Establish associations between the field values ​​of each target business field and the standard fields.

[0126] For example, by using the values ​​of target business fields as the values ​​of standard fields, business fields from different database tables are uniformly mapped to standard fields, allowing for unified analysis based on these fields. Taking media asset playback as an example, users access a media asset playback application, searching for media assets of interest through search, navigation, or other entry points, and then watching them. For favorite shows, users can choose to purchase annual or monthly subscriptions. This process generates a large amount of behavioral data, including exposure, browsing, clicks, playback, and purchases, resulting in a large number of business subsystem tables, indicator analysis tables, and algorithm calculation tables, constituting business metadata. If business fields for the same data object in different business subsystem database tables are uniformly mapped to standard fields, various operational metrics such as purchase conversion and trending content analysis can be generated. Furthermore, recommendation algorithms can be introduced to provide users with personalized media asset content, improving user activity and conversion rates.

[0127] In the above embodiments, by associating the field values ​​of each target business field with standard fields, the same data object in different database tables is mapped to the standard field, thereby enabling unified processing of data from different database tables and improving the effectiveness and efficiency of data governance.

[0128] In one embodiment, such as Figure 6 As shown, the steps for matching standard fields and various business fields are further refined, including:

[0129] S601: Construct prompt text based on standard fields and various business fields in different database tables.

[0130] The prompt text includes standard fields and various business fields. Optionally, the prompt text may include the Chinese and English names of each business field (such as page identifier and page id), and may also include the attribution information of the business fields, such as the database name and the database table name. This attribution information helps determine the relationship between the standard fields and the business fields. Optionally, the prompt text may also include mapping examples between standard fields and business fields, output format instructions, and other prompts to more accurately match the standard fields and business fields. For example, the prompt content could be:

[0131] The results of mapping business fields to standard fields are output row by row, with the following format: {"Column ID":"pageid","Column Name":"Page ID","Standard Column ID":"Page ID", "Standard Column Name":"Page ID"}.

[0132] Among them, the column identifier is the identifier of the business field in the database table, the column name is the Chinese name of the business field, the standard column identifier is the identifier of the set standard field, and the standard column name is the Chinese name of the standard field.

[0133] S602 uses a field matching model to match standard fields and various business fields based on the prompt text, and obtains candidate business fields that match the standard fields.

[0134] The field matching model can be a large language model (referred to as the "large model") or a machine learning model specifically designed for field matching; no specific limitation is made here. The prompt text is input into the large model, which matches the standard field with each business field. For each candidate business field that matches the standard field, the large model, by reading each candidate business field and related attribution information, can also summarize new standard fields from each candidate business field. For example, the output of the large model can be shown in Table 1 below:

[0135] Table 1

[0136]

[0137] The phrase "whether it is a standard field actively identified by the large model" indicates whether the standard field was entered by the user through prompt text or actively summarized by the large model. Comparing the data in the rows containing "dev_vipid_endtime_list_ytd" and "dev_vipid_list_ytd" reveals that, since "the set of expiration times for each vipid of a device" and "the set of each vipid of a device" are essentially fields with different meanings, the large model, during the matching process, did not directly map the business field "dev_vipid_endtime_list_ytd" to the standard field "resourceid," but instead mapped it to the actively summarized standard field. The standard field "resourceid_endtime" demonstrates that the large model's matching results do not rigidly map each business field to a standard field, but rather fully consider the correlation between business fields and standard fields. Furthermore, although both "unique identifier sid for TV page" and "card id" correspond to the business field "sourceid," the large model effectively incorporates the Chinese names of these business fields during the matching process, matching them to different standard fields, demonstrating high flexibility. Moreover, for the business field "terminal report - page view id," it is also mapped to the standard field "page id," resolving the word segmentation problem that fuzzy matching cannot address. In summary, the matching results between the standard fields and business fields output by the large model exhibit high flexibility and accuracy.

[0138] In addition, for the standard fields actively summarized by the large model, the data processing method in this application can be directly used to filter out the target business fields corresponding to the standard fields from the candidate business fields that match the standard fields. Alternatively, after the technical personnel determine the standard fields according to the business definition, the data processing method in this application can be used again to filter out the target business fields corresponding to the standard fields from the candidate business fields that match the standard fields.

[0139] In the above embodiments, the standard fields and business fields are matched by a field matching model. Compared with manual matching, this method can effectively improve the efficiency of field matching and has high flexibility and accuracy, providing a solid data foundation for data governance.

[0140] In one embodiment, such as Figure 7 As shown, the steps for assessing kinship are further defined, including:

[0141] S701, for each candidate business field, if the lineage field set of other candidate business fields contains the candidate business field, determine the lineage assessment result of the candidate business field based on the relationship between the lineage field set of the candidate business field and other candidate business fields.

[0142] Understandably, if the set of lineage fields of other candidate business fields contains the candidate business field, it indicates that there is a dependency relationship between the candidate business field and these other candidate business fields, that is, there is a lineage link. Then, we can further determine whether the set of lineage fields of the candidate business field contains other candidate business fields, so as to know the closeness of the lineage relationship between the candidate business field and other candidate business fields, and determine the lineage assessment result of the candidate business field.

[0143] S702, if the lineage field set of other candidate business fields does not contain the candidate business field, the lineage evaluation result of the candidate business field is determined as the second evaluation result.

[0144] If the set of lineage fields for other candidate business fields does not contain the candidate business field, it means that no other candidate business field's value depends on this candidate business field, and the value of this candidate business field does not depend on other candidate business fields. There is no lineage link between the candidate business field and other candidate business fields. In this case, there is no need to perform any other judgments, and the lineage evaluation result of the candidate business field can be directly determined as the second evaluation result.

[0145] In the above embodiments, combining whether the lineage field set of other candidate business fields contains the candidate business field, and whether the lineage field set of the candidate business field contains other candidate business fields, to measure the lineage relationship between the candidate business field and other business fields can improve the accuracy of the lineage assessment results, and thus more accurately determine the target business field corresponding to the standard field.

[0146] In one embodiment, such as Figure 8 As shown, the steps for determining the results of the kinship assessment are further defined, including:

[0147] S801, if the downstream business field of a candidate business field contains any other candidate business field, determine the lineage assessment result of the candidate business field as the first assessment result.

[0148] It is understandable that if the backward business field of a candidate business field contains any other candidate business field, it indicates that the value of other candidate business fields depends on this candidate business field. That is, the candidate business field is not the end point of the lineage link. In this case, the lineage evaluation result of the candidate business field can be directly determined as the first evaluation result.

[0149] S802, if the downstream business field of a candidate business field does not contain other candidate business fields, determine the lineage assessment result of the candidate business field as the second assessment result.

[0150] If the backward business field of a candidate business field does not contain other candidate business fields, it means that no other candidate business field depends on the value of this candidate business field. That is, the candidate business field is the end point of the lineage link. In this case, the lineage evaluation result of the candidate business field can be determined as the second evaluation result.

[0151] In the above embodiments, determining the lineage assessment result of a candidate business field based on whether its backward business field contains other candidate business fields can improve the accuracy of the lineage assessment result and thus more accurately determine the target business field corresponding to the standard field.

[0152] In one embodiment, such as Figure 9 As shown, the evaluation results for the determined field values ​​are further specified, including:

[0153] S901, for each second business field, determine the frequency of occurrence of each field value of the second business field in the database table to which it belongs.

[0154] The second business field may correspond to multiple field values ​​in its respective database table. Optionally, the field values ​​of the second business field in its respective database table are deduplicated, and the frequency of occurrence of each deduplicated field value is determined. For example, after deduplication of the second business field 1, there are 4 field values ​​remaining. The frequency of occurrence of each field value is then calculated. The frequency of occurrence of each field value is the frequency of occurrence of the field value in the field values ​​of other second business fields. Specifically, for a field value, if this field value appears in the field value of another second business field, the frequency of occurrence of this field value is incremented by 1.

[0155] For example, the values ​​of the second business field 1 in its respective database table include: 30001, 30002, 30005, and 120; the values ​​of the second business field 2 in its respective database table include: 120, ac, and 500; the values ​​of the second business field 3 in its respective database table include: 10001, 10002, 120, ac, and 500; and the values ​​of the second business field 4 in its respective database table include: ac and 120. Taking the second business field 1 as an example, for these four values, the frequency of occurrence of field value 1 among the values ​​of the other second business fields is 0; the frequency of occurrence of field value 2 among the values ​​of the other second business fields is 0; the frequency of occurrence of field value 3 among the values ​​of the other second business fields is 3.

[0156] S902, field values ​​that appear less frequently than a preset number will be used as rare field values ​​for the second business field.

[0157] The preset quantity can be set based on experience, multiple trials and actual needs. For example, if the preset quantity is 1, taking the second business field 1 as an example, then field value 1, field value 2 and field value 3 are not rare field values ​​of the second business field 1.

[0158] S903, determine the field value evaluation result of the second business field based on the number of rare field values ​​contained in the second business field.

[0159] Understandably, the more rare field values ​​a second business field contains, the lower the similarity between its field values ​​and those of other second business fields, and the lower the probability that the second business field corresponds to the same data object as the standard field. This can be used to determine the evaluation result of the second business field's field value. Optionally, the number of rare field values ​​contained in the second business field can be directly used as the evaluation result. Alternatively, the evaluation result can be determined by comparing the number of rare field values ​​contained in the second business field with a quantity threshold. For example, if the number of rare field values ​​is greater than the quantity threshold, the evaluation result is "failed," and if the number of rare field values ​​is not greater than the quantity threshold, the evaluation result is "passed." The quantity threshold can be set based on experience, multiple trials, and actual needs.

[0160] In the above embodiments, the field value evaluation result of the second business field is determined based on the frequency of occurrence of each field value of the second business field among the field values ​​of other second business fields. By judging the similarity between field values, the similarity between second business fields can be improved, thereby improving the accuracy of the field evaluation result and more accurately determining the target business field corresponding to the standard field.

[0161] In one embodiment, such as Figure 10 As shown, the steps for determining the evaluation result of field values ​​in S903 are further defined, including:

[0162] S1001, determine the number of common field values ​​contained in the second business field based on the number of rare field values ​​contained in the second business field and the total number of field values.

[0163] The total number of field values ​​in the second business field is the total number of field values ​​contained in the second business field. Taking the second business field 1 as an example, the number of rare field values ​​it contains is 3, the total number of field values ​​is 4, and the number of common field values ​​is 1. Common field values ​​represent the field values ​​that the second business field has in common with other second business fields. For example, field value 4 is the field value that the second business fields 1-4 have in common. Therefore, field value 4 is the common field value contained in the second business field 1.

[0164] S1002, Determine the field value evaluation result of the second business field based on the number of common field values ​​and the total number of field values.

[0165] Optionally, the evaluation result of the second business field can be determined based on the difference between the total number of field values ​​and the number of common field values, or it can be determined based on the ratio between the number of common field values ​​and the total number of field values. These methods are not listed here. Taking the determination of the evaluation result of the second business field based on the difference between the total number of field values ​​and the number of common field values ​​as an example, optionally, if the difference is greater than a difference threshold, the evaluation result of the second business field is determined to be "failed," and if the difference is greater than a difference threshold, the evaluation result of the second business field is determined to be "passed." The difference threshold can be set based on experience, multiple trials, and actual conditions.

[0166] In the above embodiments, the field value evaluation result of the second business field is determined based on the number of common field values ​​contained in the second business field and the total number of field values. By judging the similarity between field values, the similarity between the second business fields can be improved, thereby improving the accuracy of the field evaluation result and more accurately determining the target business field corresponding to the standard field.

[0167] In one embodiment, such as Figure 11 As shown, the steps for determining the evaluation result of field values ​​in S1002 are further defined, including:

[0168] S1101, if the ratio of the number of public field values ​​to the total number of field values ​​is greater than a preset threshold, the field value evaluation result of the second business field is determined to be qualified.

[0169] The preset threshold can be set based on experience, multiple trials and actual conditions. Taking a preset threshold of 0.5 as an example, for the second business field 2 in the above embodiment, the number of common field values ​​is 3, the total number of field values ​​is 3, and the ratio is 1. Then the field value evaluation result of the second business field 2 is that the evaluation is passed.

[0170] S1102, if the ratio is not greater than the preset threshold, determine that the evaluation result of the field value of the second business field is that the evaluation fails.

[0171] In the above embodiment, taking the first business field 1 as an example, it contains 1 common field value, and the total number of field values ​​is 4, with a ratio of 0.25. Therefore, the evaluation result of the field value of the second business field 1 is "failed". Optionally, the evaluation result of the field value that passes the evaluation can be represented by "high", and the evaluation result of the second evaluation that fails can be represented by "low".

[0172] In the above embodiments, the field value evaluation result of the second business field is determined based on the ratio of the number of common field values ​​contained in the second business field to the total number of field values. By judging the similarity between field values, the similarity between the second business fields can be improved, thereby improving the accuracy of the field evaluation result and more accurately determining the target business field corresponding to the standard field.

[0173] Based on determining whether the evaluation result of the field value of the second business field is passed or failed, in one embodiment, the second business field whose field value evaluation result is passed is used as the target business field corresponding to the standard field.

[0174] Understandably, if the field value evaluation result is passed, it indicates that the field value contained in the second business field is highly similar to the field value contained in other second business fields, and this second business field can be directly used as the target business field corresponding to the standard field.

[0175] In the above embodiments, the second business field whose field value evaluation result is passed is used as the target business field corresponding to the standard field. This fully combines the lineage evaluation result and field value evaluation result of the candidate business fields, which can more accurately determine the target business field corresponding to the standard field, thereby improving the efficiency of data governance.

[0176] Based on the technical solutions of the above embodiments, an optional embodiment for data processing is provided, such as... Figure 12 As shown, it includes the following steps:

[0177] S1201 uses a large model to match standard fields with various business fields in different database tables to obtain candidate business fields that match the standard fields.

[0178] S1202, based on the lineage field set of different candidate business fields, evaluate the lineage relationship between different candidate business fields to obtain the lineage evaluation result corresponding to each candidate business field.

[0179] S1203, the candidate business field whose bloodline assessment result is the first assessment result is used as the target business field corresponding to the standard field.

[0180] S1204, Determine the evaluation result of the field value of the second business field based on the field value of the second business field in the database table to which it belongs.

[0181] S1205, the second business field whose field value evaluation result is passed is used as the target business field corresponding to the standard field.

[0182] Based on the technical solutions of the above embodiments, the above data processing method can be executed by a terminal and a server in cooperation, such as... Figure 13 As shown, an optional embodiment is provided, which is a schematic diagram of the data processing process between various devices in the data processing process, including:

[0183] S1301, Sends a data processing request for a standard field to the server.

[0184] S1302, in response to a data processing request for a standard field, construct a prompt text based on the standard field and various business fields in different database tables; and use a field matching model to match the standard field and various business fields based on the prompt text to obtain candidate business fields that match the standard field.

[0185] The lineage field set of the candidate business field includes the forward business field on which the value of the candidate business field depends, and the backward business field on which the value depends on the candidate business field.

[0186] S1303, for each candidate business field, if the set of lineage fields of other candidate business fields contains the candidate business field, and if the backward business field of the candidate business field contains any other candidate business field, then the lineage evaluation result of the candidate business field is determined as the first evaluation result; if the backward business field of the candidate business field does not contain any other candidate business field, then the lineage evaluation result of the candidate business field is determined as the second evaluation result.

[0187] The first evaluation result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number, while the second evaluation result indicates that the number of other candidate business fields that are dependent on the candidate business field is less than a preset number.

[0188] S1304, among different candidate business fields, the candidate business field whose lineage assessment result is the first assessment result is taken as the first business field, and the candidate business field whose lineage assessment result is the second assessment result is taken as the second business field.

[0189] S1305, for each second business field, determine the frequency of occurrence of each field value of the second business field in the database table to which it belongs; field values ​​with a frequency less than a preset number are designated as rare field values ​​of the second business field; based on the number of rare field values ​​contained in the second business field and the total number of field values, determine the number of common field values ​​contained in the second business field; if the ratio of the number of common field values ​​to the total number of field values ​​is greater than a preset threshold, determine that the field value evaluation result of the second business field is passed; if the ratio is not greater than the preset threshold, determine that the field value evaluation result of the second business field is failed.

[0190] S1306, the first business field is taken as the target business field corresponding to the standard field, and the second business field whose field value evaluation result is passed is taken as the target business field corresponding to the standard field.

[0191] S1307 establishes a relationship between the field values ​​of each target business field and the standard field.

[0192] The specific processes of S1301-S1307 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.

[0193] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0194] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0195] In one exemplary embodiment, such as Figure 14 As shown, a data processing apparatus is provided, configured on a server, including: a matching module 1401, an evaluation module 1402, a judgment module 1403, and a determination module 1404, wherein:

[0196] The matching module 1401 is used to respond to data processing requests for standard fields by matching the standard fields with various business fields in different database tables to obtain candidate business fields that match the standard fields.

[0197] The evaluation module 1402 is used to evaluate the lineage relationship between different candidate business fields based on the lineage field set of different candidate business fields, and obtain the lineage evaluation result corresponding to each candidate business field; wherein, the lineage field set of each candidate business field includes business fields that have a dependency relationship with the candidate business field;

[0198] The judgment module 1403 is used to select the candidate business field whose lineage assessment result is the first assessment result as the first business field and the candidate business field whose lineage assessment result is the second assessment result as the second business field; wherein, the first assessment result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number, and the second assessment result indicates that the number of other candidate business fields that are dependent on the candidate business field is less than the preset number.

[0199] The determination module 1404 is used to use the first business field as the target business field corresponding to the standard field; and / or, based on the field value of the second business field in the database table to which it belongs, determine the field value evaluation result of the second business field, and use the second business field whose field value evaluation result meets the preset conditions as the target business field corresponding to the standard field.

[0200] In some embodiments, the evaluation module 1402 includes:

[0201] The first determining unit, for each candidate business field, if the lineage field set of other candidate business fields contains the candidate business field, determines the lineage assessment result of the candidate business field based on the relationship between the lineage field set of the candidate business field and other candidate business fields;

[0202] The second determining unit is used to determine the lineage assessment result of the candidate business field as the second assessment result when the lineage field set of other candidate business fields does not contain the candidate business field.

[0203] In some embodiments, the lineage field set includes forward business fields on which the values ​​of candidate business fields depend, and backward business fields on which the values ​​depend for candidate business fields; the first determining unit is specifically used for:

[0204] If the downstream business field of a candidate business field contains any other candidate business field, the lineage assessment result of the candidate business field is determined as the first assessment result; if the downstream business field of a candidate business field does not contain any other candidate business field, the lineage assessment result of the candidate business field is determined as the second assessment result.

[0205] In some embodiments, the determining module 1404 includes:

[0206] The frequency determination unit is used to determine the frequency of occurrence of each field value of the second business field in the database table to which it belongs, for each second business field; wherein, the frequency of occurrence of each field value is the frequency of occurrence of the field value among the field values ​​of other second business fields;

[0207] The processing unit is used to treat field values ​​that occur less than a preset number as rare field values ​​of the second business field.

[0208] The result determination unit is used to determine the field value evaluation result of the second business field based on the number of rare field values ​​contained in the second business field.

[0209] In some embodiments, the result determination unit is specifically used for:

[0210] The number of common field values ​​contained in the second business field is determined based on the number of rare field values ​​and the total number of field values; the field value evaluation result of the second business field is determined based on the number of common field values ​​and the total number of field values.

[0211] In some embodiments, the result determination unit is specifically used for:

[0212] If the ratio of the number of public field values ​​to the total number of field values ​​is greater than a preset threshold, the evaluation result of the second business field is determined to be passed; if the ratio is not greater than the preset threshold, the evaluation result of the second business field is determined to be failed.

[0213] In some embodiments, the determining module 1404 is specifically used for:

[0214] The second business field whose field value evaluation result is passed is used as the target business field corresponding to the standard field.

[0215] In some embodiments, when the processor performs matching against standard fields and various business fields in different database tables to obtain candidate business fields that match the standard fields, it is configured as follows:

[0216] Based on standard fields and various business fields in different database tables, construct prompt text; using a field matching model, match the standard fields and various business fields according to the prompt text to obtain candidate business fields that match the standard fields.

[0217] In some embodiments, the apparatus further includes an association module for:

[0218] Establish associations between the field values ​​of each target business field and the standard fields.

[0219] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer as software.

[0220] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 15 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores account-related data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0221] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0222] In one alternative embodiment, Figure 15 The computer device shown can be the aforementioned server.

[0223] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0224] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0225] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0226] It should be noted that the data involved in this application (including but not limited to account-related data) is all data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0227] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0228] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0229] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A server, characterized in that, include: A communication device configured to communicate with a terminal; and at least one processor, connected to the communication device, and configured to: In response to a data processing request for a standard field, the standard field is matched with various business fields in different database tables to obtain candidate business fields that match the standard field. Based on the lineage field set of different candidate business fields, the lineage relationship between different candidate business fields is evaluated to obtain the lineage evaluation result corresponding to each candidate business field; wherein, the lineage field set of each candidate business field includes business fields that have a dependency relationship with the candidate business field; Among different candidate business fields, the candidate business field with a lineage assessment result of the first assessment result is designated as the first business field, and the candidate business field with a lineage assessment result of the second assessment result is designated as the second business field; wherein, the first assessment result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number, and the number of other candidate business fields that are dependent on the candidate business field according to the second assessment result is less than the preset number; Use the first business field as the target business field corresponding to the standard field; and / or, Based on the field value of the second business field in its respective database table, the field value evaluation result of the second business field is determined, and the second business field whose field value evaluation result meets the preset conditions is taken as the target business field corresponding to the standard field.

2. The server according to claim 1, characterized in that, When the processor performs an evaluation of the lineage relationships between different candidate service fields based on the lineage field set of different candidate service fields, and obtains the lineage evaluation result corresponding to each candidate service field, it is configured as follows: For each candidate business field, if the lineage field set of other candidate business fields contains the candidate business field, the lineage assessment result of the candidate business field is determined based on the relationship between the lineage field set of the candidate business field and other candidate business fields. If the lineage field set of other candidate business fields does not include the candidate business field, the lineage assessment result of the candidate business field is determined as the second assessment result.

3. The server according to claim 2, characterized in that, The lineage field set includes the forward business fields on which the values ​​of the candidate business fields depend, and the backward business fields on which the values ​​depend for the candidate business fields. When the processor executes the process of determining the lineage assessment result of the candidate business field based on the relationship between the lineage field set of the candidate business field and other candidate business fields, it is configured to: If the downstream business field of the candidate business field contains any other candidate business field, the lineage assessment result of the candidate business field is determined as the first assessment result. If the downstream business field of the candidate business field does not contain the other candidate business fields, the lineage assessment result of the candidate business field is determined as the second assessment result.

4. The server according to claim 1, characterized in that, When the processor executes the process of determining the field value evaluation result of the second business field based on the field value of the second business field in its respective database table, it is configured as follows: For each second business field, determine the frequency of occurrence of each field value of the second business field in the database table to which it belongs; wherein, the frequency of occurrence of each field value is the frequency of occurrence of the field value among the field values ​​of other second business fields; Field values ​​that appear less than a preset number of times will be used as rare field values ​​for the second business field; The field value evaluation result of the second business field is determined based on the number of rare field values ​​contained in the second business field.

5. The server according to claim 4, characterized in that, When the processor determines the field value evaluation result of the second business field based on the number of rare field values ​​contained in the second business field, it is configured as follows: The number of common field values ​​contained in the second business field is determined based on the number of rare field values ​​contained in the second business field and the total number of field values. The evaluation result of the second business field value is determined based on the number of public field values ​​and the total number of field values.

6. The server according to claim 5, characterized in that, When the processor executes the field value evaluation result of the second business field based on the number of common field values ​​and the total number of field values, it is configured as follows: If the ratio of the number of public field values ​​to the total number of field values ​​is greater than a preset threshold, the evaluation result of the second business field is determined to be passed. If the ratio is not greater than the preset threshold, the evaluation result of the field value of the second business field is determined to be "evaluation failed".

7. The server according to claim 6, characterized in that, When the processor executes the second business field whose field value evaluation result meets preset conditions, and uses it as the target business field corresponding to the standard field, it is configured as follows: The second business field whose field value evaluation result is passed is used as the target business field corresponding to the standard field.

8. The server according to any one of claims 1 to 7, characterized in that, When the processor performs matching on the standard field and various business fields in different database tables to obtain candidate business fields that match the standard field, it is configured as follows: Construct prompt text based on standard fields and various business fields in different database tables; Using a field matching model, the standard field and each of the business fields are matched according to the prompt text to obtain candidate business fields that match the standard field.

9. The server according to any one of claims 1 to 7, characterized in that, The processor is also configured to: The field values ​​of each of the target business fields are associated with the standard fields.

10. A data processing method, characterized in that, The method includes: In response to a data processing request for a standard field, the standard field is matched with various business fields in different database tables to obtain candidate business fields that match the standard field. Based on the lineage field set of different candidate business fields, the lineage relationship between different candidate business fields is evaluated to obtain the lineage evaluation result corresponding to each candidate business field; wherein, the lineage field set of each candidate business field includes business fields that have a dependency relationship with the candidate business field; Among different candidate business fields, the candidate business field with a lineage assessment result of the first assessment result is designated as the first business field, and the candidate business field with a lineage assessment result of the second assessment result is designated as the second business field; wherein, the first assessment result indicates that the number of other candidate business fields that are dependent on the candidate business field is greater than or equal to a preset number, and the number of other candidate business fields that are dependent on the candidate business field according to the second assessment result is less than the preset number; Use the first business field as the target business field corresponding to the standard field; and / or, Based on the field value of the second business field in its respective database table, the field value evaluation result of the second business field is determined, and the second business field whose field value evaluation result meets the preset conditions is taken as the target business field corresponding to the standard field.

Images