Connector-based data exchange method, device and medium

Abstract

This application discloses a data exchange method, device, and medium based on a Connector. Belonging to the field of data processing technology, it addresses the problem of insufficient privacy and integrity guarantees during data transmission in data exchange methods. The method includes: acquiring required data through a Connector based on meta-component configuration or ODBC-based data access configuration; preprocessing the acquired data and masking specific fields within the preprocessed data; obtaining the user's identity credentials; determining the user's access permission to the Connector if the identity credentials pass verification; determining the data access scope corresponding to the user with access permission to the Connector based on preset access rules; and querying the masked data based on the data access scope to display the matching data to the user.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data exchange method, device and medium based on a Connector. Background Technology

[0002] In the digital age, data has become a core asset for enterprise operations and decision-making, and its efficient and secure exchange and sharing are crucial for driving business development and maintaining market competitiveness.

[0003] However, existing traditional data exchange methods, such as API interfaces, file transfer protocols, WebSockets, data exchange platforms, remote procedure calls, and ETL (Extract-Transform-Load) tools, while facilitating data flow, also expose numerous security vulnerabilities. For example, many traditional data exchange methods lack sufficient encryption protection during data transmission, making them highly susceptible to interception, parsing, and even tampering by malicious actors. This not only leads to the leakage of corporate secrets but may also pose a serious threat to customer privacy, thus severely compromising the privacy and integrity of data. Summary of the Invention

[0004] This application provides a data exchange method, device, and medium based on a Connector to solve the following technical problem: many traditional data exchange methods lack sufficient encryption protection during data transmission, which poses a serious threat to the privacy and integrity of the data.

[0005] The embodiments of this application adopt the following technical solutions:

[0006] This application provides a data exchange method based on a Connector. The method includes: acquiring required data through a Connector based on meta-component configuration or ODBC-based data access configuration; preprocessing the acquired data and masking specific fields in the preprocessed data; acquiring the user's identity credentials, and determining the user's access permission to the Connector if the identity credentials pass verification; determining the data access scope corresponding to the user with access permission to the Connector based on preset access rules; wherein the preset access rules are related to the user's identity, user role, and user permission level; and querying the data after specific field masking based on the data access scope to display the data that meets the requirements to the user.

[0007] This application's embodiments achieve non-intrusive data acquisition and retrieval through the communication and collaboration of Connectors, avoiding direct operation and modification of the original system and reducing secondary development. Secondly, data acquisition and retrieval based on meta-components and data access based on ODBC enable the integration of multi-source heterogeneous data. Furthermore, this application's embodiments introduce security authentication and access control mechanisms, supporting multiple authentication methods to ensure security and data protection during data exchange. The Connector-based non-intrusive data exchange method provides key components such as efficient data collection, access control, security authentication, and data processing, ensuring seamless data flow and interaction, enabling the system to efficiently utilize this data while maintaining efficiency and controllability.

[0008] In one implementation of this application, the required data is obtained through a Connector based on meta-component configuration or ODBC-based data access configuration. Specifically, this includes: when obtaining the required data through meta-component configuration, determining the corresponding meta-component in the component pool based on the obtained metadata; and determining the corresponding parsing method based on the data structure and data format stored in the data source.

[0009] Based on the meta-components and corresponding parsing methods, the required data is obtained from the data storage system through the Connector.

[0010] In one implementation of this application, the required data is acquired through a Connector based on meta-component configuration or ODBC-based data access configuration. Specifically, this includes: when acquiring the required data through ODBC-based data access configuration, establishing a connection with the data source by configuring ODBC data source information; performing query type matching based on the acquired query content, and sending an SQL query request to the data source through the Connector; determining the query data in the data source based on the query type; and optimizing the queried information to use the processed data as the required data. The optimization process includes at least one of information selection, information filtering, and information arrangement.

[0011] In one implementation of this application, the acquired required data is preprocessed, and the preprocessed data is then subjected to specific field masking. Specifically, this includes: preprocessing the required data; wherein the preprocessing includes at least one of date formatting and data cleaning; determining the data source system and target system corresponding to the preprocessed data, and performing format conversion on the preprocessed data based on the data formats corresponding to the data source system and target system respectively, so as to map the preprocessed data to the target system; determining the data type corresponding to the preprocessed data, and determining the corresponding masking field in a preset masking content table based on the data type; and performing specific field masking on the preprocessed data based on the masking field.

[0012] In one implementation of this application, the preprocessed data is subjected to specific field masking processing based on the masked field. Specifically, this includes: determining the masking type corresponding to the masked field based on the field type corresponding to the masked field; wherein the masking type includes at least one of fixed character replacement, generating random characters, retaining part of the characters, and character encryption; determining the encryption level corresponding to the field type, and superimposing the masking type based on the encryption level; and masking the masked field based on the superimposed masking type.

[0013] In one implementation of this application, obtaining the user's corresponding identity credential and determining the user's permission to access the Connector if the identity credential verification passes, specifically includes: obtaining the user's corresponding identity credential; wherein the identity credential has been encrypted using a preset encryption algorithm; decrypting the identity credential using a decryption algorithm corresponding to the preset encryption algorithm to obtain a reference identity credential and a reference timestamp; calculating a hash value for the reference identity credential and comparing the calculated hash value with a pre-stored hash value; and performing an expiration check on the reference timestamp; if the comparison results are consistent and the expiration check passes, allocating data query permissions to the user based on the reference identity credential, and determining the user's permission to access the Connector based on the data query permissions.

[0014] In one implementation of this application, the data access range corresponding to a user with access rights to the Connector is determined based on preset access rules. Specifically, this includes: determining a reference data access range corresponding to the user based on preset access rules, user identity, and user role; determining abnormal access information and normal access information based on the user's historical access information, and determining the initial confidence level corresponding to the user based on the numerical ratio between abnormal access information and normal access information; obtaining reference users that are associated with the user, and obtaining the reference confidence level corresponding to each reference user; determining the final confidence level corresponding to the user based on the initial confidence level and each reference confidence level, and adjusting the reference data access range based on the final confidence level to obtain the data access range.

[0015] In one implementation of this application, based on the data access scope, the required data after specific field masking is queried to display the compliant data to the user. Specifically, this includes: obtaining query information sent by the user; querying within the data access scope based on the query information to obtain the data to be displayed; determining the viewable fields based on user permissions; matching the viewable fields with the masked specific fields; and displaying the masked specific fields in the data to be displayed based on the matching results; obtaining the user's corresponding historical query information and sorting the historical query information based on the number of queries; matching the processed data with the historical query information to determine the display order of the processed data; and displaying the processed data to the user based on the display order.

[0016] This application provides a Connector-based data exchange device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: acquire required data through a Connector based on meta-component configuration or ODBC-based data access configuration; preprocess the acquired required data and perform specific field masking on the preprocessed data; acquire the user's identity credentials, and if the identity credentials pass the verification, determine that the user has permission to access the Connector; determine the data access scope corresponding to the user with access to the Connector based on preset access rules; wherein the preset access rules are related to the user's identity, user role, and user permission level; and query the required data after specific field masking based on the data access scope, so as to display the data that meets the requirements to the user.

[0017] This application provides a non-volatile computer storage medium storing computer-executable instructions. These instructions are configured to: acquire required data via a Connector based on meta-component configuration or ODBC-based data access configuration; preprocess the acquired data and mask specific fields in the preprocessed data; acquire the user's identity credentials, and if the identity credentials pass verification, determine that the user has permission to access the Connector; determine the data access scope corresponding to the user with access permissions based on preset access rules; wherein the preset access rules are related to the user's identity, user role, and user permission level; and query the data after specific field masking based on the data access scope to display the compliant data to the user.

[0018] The at least one technical solution adopted in this application embodiment achieves the following beneficial effects: Through the communication and collaboration of Connectors, this application embodiment realizes non-intrusive data acquisition and retrieval, avoiding direct operation and modification of the original system and reducing secondary development. Secondly, data acquisition and retrieval based on meta-components and data access based on ODBC achieve the integration of multi-source heterogeneous data. Furthermore, this application embodiment introduces security authentication and access control mechanisms, supporting multiple authentication methods to ensure security and data protection during data exchange. The Connector-based non-intrusive data exchange method provides key components such as efficient data collection, access control, security authentication, and data processing, ensuring seamless data flow and interaction, enabling the system to efficiently utilize this data while possessing both efficiency and controllability. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0020] Figure 1 A flowchart of a data exchange method based on a Connector provided in this application embodiment;

[0021] Figure 2 A schematic diagram illustrating the relationship between a component pool, meta-components, and data elements is provided for an embodiment of this application.

[0022] Figure 3This is a schematic diagram of the structure of a Connector-based data exchange device provided in an embodiment of this application. Detailed Implementation

[0023] This application provides a data exchange method, device, and medium based on a Connector.

[0024] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0025] The technical solutions proposed in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a flowchart of a data exchange method based on a Connector, provided as an embodiment of this application.

[0027] like Figure 1 As shown, the data exchange method includes the following steps:

[0028] S101. Based on meta-component configuration or ODBC-based data access configuration, the required data is obtained through the Connector.

[0029] In one embodiment of this application, when acquiring required data through meta-component configuration, the corresponding meta-component is determined from the component pool based on the acquired metadata. Based on the data structure and format stored in the data source, a corresponding parsing method is determined. Based on the meta-component and the corresponding parsing method, the required data is acquired from the data storage system via a Connector.

[0030] Specifically, data acquisition and retrieval based on meta-components obtains data from various data storage methods through Connectors, including warehouse management systems, supplier databases, transportation management systems, files, and file systems, reflecting the diversity and complexity of data sources. During data acquisition and retrieval, the data structure and format of the data sources are considered to ensure correct parsing and processing. A deep understanding of the characteristics of different data sources is crucial, and appropriate data processing and parsing methods are adopted. Data sources mainly refer to databases, files, web services, and user input. For relational databases, SQL (Structured Query Language) is used for data querying, insertion, updating, and deletion. For different text files, file operation functions or libraries in programming languages ​​can be used to read and parse file content. For websites that do not provide API interfaces, UIPath is used to crawl web page content and extract the required data. Data entered by users through forms is validated to ensure its accuracy and integrity.

[0031] Furthermore, data acquisition and retrieval based on meta-components do not rely solely on single data elements, but rather on the reorganization and representation of data elements through higher-level meta-components and metadata. The core idea is to access data by reorganizing and utilizing meta-components, rather than directly accessing the original data elements. Meta-components are supported by various data elements and typically contain richer information and relationships between data elements, thus providing more comprehensive data representation and processing capabilities.

[0032] Furthermore, data elements are the basic building blocks of meta-components; they represent the smallest units of data and can be of different types, such as numbers, text, and dates. Within a meta-component, one or more data elements are interconnected, collectively forming a larger data entity. During data access, the system utilizes the meta-components corresponding to the data elements in the component pool. The component pool is a central storage area for storing and managing meta-components, enabling rapid response to data requests and improving the efficiency and accuracy of data access. Figure 2 This application provides a schematic diagram illustrating the relationship between a component pool, meta-components, and data elements, as shown in the embodiments of this application. Figure 2 As shown, metadata is used to retrieve meta-components. Multiple different meta-components are stored in a component pool. Each meta-component contains different data elements. Data is extracted based on the meta-components, for example, data extraction can be performed in a logistics information system.

[0033] In one embodiment of this application, when acquiring required data through ODBC-based data access configuration, a connection to the data source is established by configuring ODBC data source information. Based on the acquired query content, query type matching is performed, and an SQL query request is sent to the data source via a Connector. The query data is then determined from the data source based on the query type. The retrieved information is optimized to use the processed data as the required data; wherein the optimization process includes at least one of information selection, information filtering, and information arrangement.

[0034] Specifically, ODBC-based data access is another data collection method. The Connector establishes a connection with the data source by configuring ODBC data source information. The Connector can automatically identify the type, structure, and parameters of the data source and generate corresponding connection configurations, thus achieving intelligent configuration. This allows the Connector to intelligently identify the type and parameters of the data source and automatically complete the connection configuration. Users do not need to manually enter tedious connection information; instead, the Connector automatically completes the connection settings based on the characteristics of the data source, saving time and effort.

[0035] Furthermore, once a connection is established with the data source, the Connector can send SQL query requests to retrieve the required data. The Connector in this embodiment can be optimized according to the user's query needs, including operations such as selection, filtering, and sorting, to ensure that only the necessary data is retrieved. This improves data retrieval efficiency, reduces unnecessary data transmission and processing, and thus accelerates data acquisition.

[0036] Furthermore, the acquired data is processed by the Connector and returned in a structured format. This application also provides data processing functions to further process and transform the data to meet the data standards and format requirements of the target system. This includes operations such as data mapping, data cleaning, and data transformation to ensure data quality and consistency. This data processing function allows users to directly integrate data into the target system without additional data processing work.

[0037] S102. The obtained required data is preprocessed, and specific fields of the preprocessed data are masked.

[0038] In one embodiment of this application, the required data is preprocessed; wherein, the preprocessing includes at least one of date formatting and data cleaning. The source system and target system corresponding to the preprocessed data are determined, and the format of the preprocessed data is converted based on the data formats corresponding to the source system and target system, respectively, to map the preprocessed data to the target system. The data type corresponding to the preprocessed data is determined, and the corresponding masking field is determined in a preset masking content table based on the data type. Based on the masking field, specific field masking processing is performed on the preprocessed data.

[0039] Specifically, the preprocessing in this application refers to a series of important steps involving standardization, cleaning, and transformation of the collected data. In logistics data processing, various operations are required to ensure that the data is formatted uniformly, its content is clear, and it meets the needs of the target system. First, the date format needs to be standardized so that date data from different sources can be correctly identified and processed by the system.

[0040] Secondly, it is necessary to remove duplicate or erroneous data to ensure that analysis and decision-making are based on accurate information.

[0041] Furthermore, data transformation maps data to the data standards adopted by the target system, enabling seamless data transfer and application across different systems. This ensures data accuracy and consistency, providing a reliable foundation for subsequent data analysis and business applications.

[0042] Furthermore, to ensure data security and privacy, pre-processed data also needs to be masked. This is particularly important in the logistics sector, where data processing is crucial because logistics data may involve information that access is restricted. Specifically, data masking is a type of obfuscation used to hide portions of the data to prevent information leakage. Based on the type of pre-processed data, corresponding masking fields are determined in a pre-defined masking content table. This table includes various data types and their respective masking fields. In summary, data processing plays a vital role in the logistics industry, not only protecting data security but also improving data usability and applicability.

[0043] In one embodiment of this application, a masking type is determined based on the field type corresponding to the masked field; wherein the masking type includes at least one of fixed character replacement, generating random characters, retaining part of the characters, and character encryption. An encryption level corresponding to the field type is determined, and masking types are superimposed based on the encryption level. The masked field is then masked based on the superimposed masking type.

[0044] Specifically, common sensitive field types include personal identification information (such as name, ID number, passport number, etc.), financial information (such as bank card number, transaction amount, etc.), and contact information (such as phone number, email address, etc.). Each field type requires different masking methods due to the varying sensitivity and format of the information it contains. For example, methods might include replacing all or part of the characters in the field with specific characters (such as asterisks), replacing sensitive information with a randomly generated character sequence, retaining only some characters (such as the first and last characters) and replacing the rest with specific characters, or using encryption algorithms to encrypt the field, turning sensitive information into unreadable ciphertext.

[0045] Furthermore, the encryption level corresponding to each field is determined. The encryption level reflects the sensitivity of the data and the degree of protection required. Different field types may correspond to different encryption levels. For example, highly sensitive information such as ID card numbers and bank card numbers may require a higher encryption level, while some relatively less sensitive fields (such as phone numbers) may use a lower encryption level. In some cases, a single masking type may not be sufficient to meet data protection needs. In such cases, masking types can be layered based on the encryption level. For example, for highly sensitive fields, character encryption can be performed first, and then fixed character replacement or partial character retention can be applied to the encrypted result to increase the layers and difficulty of data protection. Based on the determined masking type and encryption level, the sensitive fields are then actually masked.

[0046] Furthermore, this application employs RPA technology to automatically mask sensitive information, displaying only essential information. For example, UiPath can automatically mask sensitive information, displaying only necessary details. UiPath can automatically mask specific fields in PDF, Excel, and other files, ensuring that sensitive information is only visible to personnel with specific permissions. UiPath's automated process can acquire information from multiple data sources and ensure data consistency and accuracy through a series of preprocessing steps. After data collection and processing, users with access permissions can retrieve the processed data through the Connector. Before accessing the data, users need to undergo authentication and access control through security mechanisms such as UiPath Orchestrator to ensure data security.

[0047] Specifically, users with different permissions will retrieve different information corresponding to their roles and permissions based on the set access rules. For example, a warehouse manager can access inventory data but cannot view the actual information. Data masking is a widely used technique in data sharing and display. Its main purpose is to hide or obscure restricted access portions of data to limit the scope of information access. In the logistics field, data masking is often applied to the processing of logistics orders, cargo tracking, and other information. Through data masking, only authorized users can view the complete data, while others can only see partial or obscured information, thereby protecting the security and privacy of sensitive data.

[0048] For example, suppose a logistics company needs to share cargo tracking information with customers but doesn't want customers to see the specific location of the goods. In this case, data masking can be used to show customers only vague location information or hide some sensitive cargo information. This way, even if the data is accessed by unauthorized individuals, important information will not be leaked, ensuring data security and privacy.

[0049] S103. Obtain the user's identity credentials. If the identity credentials pass the verification, determine that the user has permission to access the Connector.

[0050] In one embodiment of this application, a user's identity credential is obtained; wherein the identity credential has been encrypted using a preset encryption algorithm. The identity credential is decrypted using a decryption algorithm corresponding to the preset encryption algorithm to obtain a reference identity credential and a reference timestamp. A hash value is calculated on the reference identity credential, and the calculated hash value is compared with a pre-stored hash value. The reference timestamp is then checked for validity. If the comparison results are consistent and the validity check passes, data query permissions are assigned to the user based on the reference identity credential, and the user's access permissions to the Connector are determined based on these data query permissions.

[0051] Specifically, user identity credentials are obtained. In this embodiment, the identity credentials are identification information held by a user for accessing the system or performing certain operations, such as a username, password, or token. The identity credentials are encrypted using a pre-defined encryption algorithm to protect data security during transmission or storage. A decryption algorithm corresponding to the pre-defined encryption algorithm is used to restore the encrypted identity credentials to their original state. The decrypted information typically includes the user's identity credentials (e.g., username) and a timestamp. A hash value is calculated on the decrypted reference identity credentials, and the calculated hash value is compared with a pre-stored hash value to verify the authenticity and validity of the user-submitted identity credentials. If the comparison results match, the user-submitted identity credentials are considered legitimate.

[0052] Furthermore, by comparing the reference timestamp with the current time, the system determines whether the encryption operation was completed within a certain valid timeframe. If the time difference exceeds a preset threshold (e.g., minutes, hours), the identity credential is considered expired and no longer valid. If the comparison results are consistent and the validity check passes, the system assigns corresponding data query permissions based on the reference identity credential (actually the user's identity). These permissions may include the range of data the user can query and the level of detail required for the query.

[0053] Furthermore, security authentication is the first line of defense for ensuring Connector security and data protection. Only authenticated users or components can access the system, thereby reducing unauthorized access and potential security vulnerabilities. It is the process within the Connector that verifies the identity of a user or the legitimacy of a component, typically performed before access control is implemented. In this embodiment, security authentication is supported by UiPath Orchestrator, used to manage access permissions for users, bots, or other components connecting to the Connector. UiPath Orchestrator also supports various authentication methods, including username and password, single signature authentication, and multi-factor authentication. Users or components can authenticate by providing identity credentials supported by UiPath Orchestrator. During the security authentication process, UiPath Orchestrator can issue temporary access tokens to indicate that the user or component has been authenticated and has permission to access the Connector. Secondly, the centralized identity management capabilities of UiPath Orchestrator can be used to assign appropriate permissions to users or components. In the data exchange method of non-intrusive data acquisition and retrieval based on Connector, security authentication can also prevent unauthorized data access and ensure the security of its connection to different systems or data sources, thus jointly building a secure Connector architecture.

[0054] S104. Based on the preset access rules, determine the data access scope corresponding to the user with the access to the Connector.

[0055] In one embodiment of this application, a reference data access range for a user is determined based on preset access rules, user identity, and user role. Based on the user's historical access information, abnormal access information and normal access information are identified. An initial confidence level for the user is determined based on the ratio between the abnormal and normal access information. Reference users associated with the user are obtained, and a reference confidence level for each reference user is acquired. The initial confidence level and each reference confidence level are used to determine the final confidence level for the user. The reference data access range is then adjusted based on the final confidence level to obtain the final data access range.

[0056] Specifically, this application embodiment pre-defines a series of access rules, which define the data scope that different identities (such as employees, management) and roles (such as department managers, project managers) can access. Based on the user's identity information (such as employee number, name) and role information (such as position, responsibilities), the data access permissions that the user should enjoy can be preliminarily determined, i.e., the reference data access scope. This application embodiment records the user's past data access behavior, including the data type, time, frequency, etc., of the accessed data. By analyzing the historical access information, abnormal access behavior of the user can be identified, such as frequent access to data outside the scope of responsibilities, access to sensitive data at unusual times, and normal access behavior. By calculating the ratio between abnormal access information and normal access information, the compliance and risk of the user's access behavior can be assessed, thereby determining the user's initial confidence level.

[0057] Furthermore, reference users with direct or indirect relationships with the user are identified. These relationships may be based on work relationships (such as colleagues, superiors and subordinates), social network relationships, etc. For each reference user, a reference confidence score is calculated in a similar manner (based on their identity, role, historical access behavior, etc.), representing the credibility of the user's access behavior. Combining the user's initial confidence score and the reference confidence scores of each reference user, a comprehensive evaluation is performed using a certain algorithm (such as weighted average, machine learning model, etc.) to obtain the user's final confidence score. Based on the user's final confidence score, the system dynamically adjusts the user's data access scope. If the final confidence score is high, it indicates that the user's access behavior is relatively compliant and credible, and their data access permissions may be expanded; conversely, if the final confidence score is low, the system may restrict or reduce their data access permissions to reduce potential security risks.

[0058] Furthermore, in the Connector-based non-intrusive data acquisition and retrieval data exchange method, access control is a crucial component in ensuring system data and functional security, and also serves as the second line of defense for the Connector. In this embodiment, the goal of access control is to clearly define who can access the system, and the scope of data and functions that can be accessed. Achieving this goal relies on pre-defined rules and security authentication passed by users or components. Simultaneously, a semantic-based access control reasoning method is employed to ensure the accuracy and efficiency of access control. The formulation of access control rules involves consideration of multiple factors, all of which are determined by the data holder.

[0059] Specifically, user identity, role, and permission level are all important factors influencing access control rule settings. When formulating rules, they can be flexibly adjusted according to different needs and scenarios to ensure that only authorized users can access the system. For example, rules can define specific types of data a user can access or specific operations they can perform, dynamically determining their access permissions based on specific user or system attributes. Rules can infer from semantic information, ensuring effective implementation of access control in various contexts. For instance, if a user is the administrator of warehouse A, they have the permission to view information about goods in warehouse A and manage those goods. In this case, the administrator has the permission to view information about fragile goods, dangerous goods, etc., which is inferred through semantic settings. Specifically, these rules can combine the user's role and current context information. For example, the system can dynamically determine whether a user has the right to perform a certain operation or access specific data based on the access information obtained and subsequently inferred information or the user's attributes. This means that data holders can update rules to clarify data ownership based on new business needs or security considerations without modifying application code. This flexible access control mechanism allows the system to better adapt to different application scenarios. For example, in a logistics scenario, warehouse staff can access inventory data based on their administrator attributes, but cannot see order or transportation data. Carriers can view cargo transportation data based on their attributes as operators of transportation vehicles. This improves both system security and user convenience.

[0060] S105. Based on the data access scope, query the required data after specific field masking processing, so as to display the data that meets the requirements to the user.

[0061] In one embodiment of this application, query information sent by a user is obtained. Based on the query information, a query is performed within the data access scope to obtain the data to be displayed. Viewable fields are determined based on user permissions. These viewable fields are matched with specific obscured fields, and the obscured fields in the data to be displayed are displayed based on the matching results. Historical query information corresponding to the user is obtained and sorted based on the number of queries. The processed data is matched with the historical query information to determine the display order of the processed data. Based on the display order, the processed data is displayed to the user.

[0062] Specifically, the system first needs to receive user-input query information, typically through web forms, API requests, or other user interface elements. Query information may include specific search terms, filtering criteria, and data ranges. Based on the user's query information, the system executes the query operation within the permitted data access scope. The system checks the user's permissions to determine which data fields the user is authorized to view. User permissions may be based on their role, position, or other security policies. All specific fields that need to be masked (such as sensitive information, private data, etc.) are identified, and these fields are then matched against the fields the user is authorized to view. For fields the user is not authorized to view, the system may mask, hide, or replace them with placeholders to ensure that the data displayed to the user conforms to their authorized scope while protecting sensitive information from being leaked.

[0063] Furthermore, the system records and maintains users' historical query information, including query content and timestamps. This historical query information is then sorted based on query frequency or other sorting criteria (such as chronological order). The processed data is matched against the users' historical query information to determine the optimal display order. Following this determined display order, the processed data is presented to the user in a user-friendly manner.

[0064] Figure 3 This is a schematic diagram of a connector-based data exchange device provided in an embodiment of this application. Figure 3As shown, a Connector-based data exchange device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: acquire required data through a Connector based on a meta-component configuration or an ODBC-based data access configuration; preprocess the acquired required data and perform specific field masking on the preprocessed data; acquire the user's identity credentials, and, if the identity credentials pass the verification, determine that the user has permission to access the Connector; determine the data access scope corresponding to the user with access permissions to the Connector based on preset access rules; wherein the preset access rules are related to the user's identity, user role, and user permission level; and query the required data after specific field masking based on the data access scope, so as to display the data that meets the requirements to the user.

[0065] This application provides a non-volatile computer storage medium storing computer-executable instructions. These instructions are configured to: acquire required data via a Connector based on meta-component configuration or ODBC-based data access configuration; preprocess the acquired data and mask specific fields in the preprocessed data; acquire the user's identity credentials, and if the identity credentials pass verification, determine that the user has permission to access the Connector; determine the data access scope corresponding to the user with access permissions based on preset access rules; wherein the preset access rules are related to the user's identity, user role, and user permission level; and query the data after specific field masking based on the data access scope to display the compliant data to the user.

[0066] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0067] The above descriptions are merely embodiments of this application and are not intended to limit the scope of this application. For those skilled in the art, various modifications and variations can be made to the embodiments of this application. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions in the embodiments of this application.

Claims

1. A data exchange method based on a Connector, characterized in that, The method includes: Based on meta-component configuration or ODBC-based data access configuration, the required data is obtained through a Connector. Specifically, when obtaining the required data through the meta-component configuration, the corresponding meta-component is determined in the component pool based on the obtained metadata; the corresponding parsing method is determined based on the data structure and data format stored in the data source; and the required data is obtained in the data storage system through the Connector based on the meta-component and the corresponding parsing method. The obtained required data is preprocessed, and specific fields of the preprocessed data are masked. Obtaining the user's corresponding identity credential, and if the identity credential verification passes, determining that the user has permission to access the Connector, specifically includes: obtaining the user's corresponding identity credential; wherein the identity credential has been encrypted using a preset encryption algorithm; decrypting the identity credential using a decryption algorithm corresponding to the preset encryption algorithm to obtain a reference identity credential and a reference timestamp; calculating a hash value for the reference identity credential and comparing the calculated hash value with a pre-stored hash value; and performing an expiration check on the reference timestamp; if the comparison results are consistent and the expiration check passes, allocating data query permissions to the user based on the reference identity credential, and determining the user's permission to access the Connector based on the data query permissions; Based on preset access rules, the data access range corresponding to the user with the access to the Connector is determined. Specifically, this includes: determining a reference data access range for the user based on the preset access rules, the user's identity, and the user's role; determining abnormal access information and normal access information based on the user's historical access information, and determining an initial confidence level for the user based on the ratio between the abnormal access information and the normal access information; obtaining reference users that are associated with the user, and obtaining a reference confidence level for each reference user; determining a final confidence level for the user based on the initial confidence level and each reference confidence level, and adjusting the reference data access range based on the final confidence level to obtain the data access range; wherein, the preset access rules are related to the user's identity, user role, and user permission level. Based on the data access scope, the required data after specific field masking is queried to display the data that meets the requirements to the user.

2. The data exchange method based on a Connector according to claim 1, characterized in that, The aforementioned component-based configuration or ODBC-based data access configuration obtains the required data through a Connector, specifically including: When acquiring the required data through the ODBC-based data access configuration, a connection with the data source is established by configuring the ODBC data source information; Based on the obtained query content, query type matching is performed, and an SQL query request is sent to the data source through the Connector; Based on the query type, the query data is determined in the data source; The retrieved information is optimized to use the optimized data as the required data; wherein the optimization includes at least one of information selection, information filtering, and information arrangement.

3. The data exchange method based on a Connector according to claim 1, characterized in that, The process of preprocessing the acquired required data and then masking specific fields in the preprocessed data specifically includes: The required data is preprocessed; wherein the preprocessing includes at least one of date formatting and data cleaning. The source system and target system corresponding to the preprocessed data are determined, and the format of the preprocessed data is converted based on the data format corresponding to the source system and target system respectively, so as to map the preprocessed data to the target system; The data type corresponding to the preprocessed data is determined, and the corresponding blocking field is determined in the preset blocking content table based on the data type; Based on the masking field, specific field masking processing is performed on the preprocessed data.

4. The data exchange method based on a Connector according to claim 3, characterized in that, The specific field masking process performed on the preprocessed data based on the masking field includes: Based on the field type corresponding to the masked field, the masking type corresponding to the masked field is determined; wherein, the masking type includes at least one of fixed character replacement, generating random characters, retaining part of the characters, and character encryption; Determine the encryption level corresponding to the field type, and then apply the blocking type based on the encryption level; Based on the overlay masking type, the masking field is masked.

5. The data exchange method based on a Connector according to claim 1, characterized in that, The process of querying the required data after specific field masking based on the data access range, in order to display the data that meets the requirements to the user, specifically includes: Obtain the query information sent by the user, and based on the query information, perform a query within the data access range to obtain the data to be displayed; Based on user permissions, the viewable fields are determined, the viewable fields are matched with the specific fields that are covered, and the specific fields that are covered in the data to be displayed are displayed based on the matching results. Obtain the historical query information corresponding to the user, and sort the historical query information based on the number of queries; The processed data is matched with the historical query information to determine the display order of the processed data. Based on the aforementioned display order, the processed data is presented to the user.

6. A data exchange device based on a Connector, characterized in that, The device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to perform the method described in any one of claims 1-5.

7. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of performing the method described in any one of claims 1-5.

Images