Data fusion method and device for platform analysis, electronic equipment and medium

Abstract

The embodiment of the invention provides a data fusion method and device for platform analysis, electronic equipment and a medium, relates to the technical field of data processing, and is suitable for the technical field of financial science and technology. The method comprises the following steps: acquiring data fusion task information, wherein the data fusion task information comprises at least one business platform identification code, a data fusion rule and a target data format; and obtaining at least two target data sources according to each service platform identification code. And then obtaining initial business data of each business platform through each target data source. Performing data format conversion on the initial service data according to the target data format to obtain candidate service data; and finally performing data fusion on the at least two candidate service data according to a data fusion rule to obtain target service data. According to the embodiment of the invention, the dependence on customized development during data fusion can be reduced, the defects of low data fusion flexibility, difficulty in expansion and high resource demand in the prior art are effectively overcome, and finally, the data fusion efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, applicable to the field of financial technology technology, and in particular to a data fusion method and device, electronic equipment and medium for platform analysis. Background Art

[0002] In the field of financial technology, financial companies provide financial services to users through different service platforms, thereby generating a large amount of financial business data. By integrating and centrally analyzing these business data from different data sources, financial companies can better optimize and select platform business strategies. In related technologies, customized development is required for the microservice architecture of different business platforms. When processing data fusion tasks, exclusive microservices of different business units can be called, and each microservice processes its specific data and business logic. However, this solution has low flexibility, high expansion difficulty, and requires more development resources, which affects the efficiency of data fusion. Therefore, how to improve the efficiency of data fusion has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a data fusion method and device, electronic device and medium for platform analysis, aiming to improve the data fusion efficiency.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a data fusion method for platform analysis, the method comprising:

[0005] Acquire data fusion task information; wherein the data fusion task information includes: at least one business platform identification code, data fusion rules and target data format;

[0006] Acquire at least two target data sources according to each of the business platform identification codes;

[0007] Acquiring initial business data of each business platform through each of the target data sources;

[0008] Convert the initial service data into a data format according to the target data format to obtain candidate service data;

[0009] At least two of the candidate business data are fused according to the data fusion rule to obtain target business data.

[0010] In some embodiments, the candidate business data includes at least two candidate business sub-data, and the data fusion rule includes: a target data type and a data fusion sub-rule matching the target data type; fusing the at least two candidate business data according to the data fusion rule to obtain the target business data includes:

[0011] Acquire the data type of each candidate business sub-data to obtain the business data type;

[0012] Filtering a target data fusion sub-rule from the data fusion sub-rules according to the business data type and the target data type;

[0013] The candidate service sub-data are data-fused according to the target data fusion sub-rule to obtain the target service data.

[0014] In some embodiments, after fusing at least two of the candidate service data according to the data fusion rule to obtain the target service data, the method further includes:

[0015] Obtaining operation abnormality information of the target data source;

[0016] Perform abnormal transaction behavior detection on the business platform according to the target business data to obtain abnormal transaction behavior detection data; wherein the abnormal transaction behavior detection data includes the number of abnormal transactions of the business platform in a preset period;

[0017] The data fusion rule is updated according to the abnormal transaction behavior detection data and the operation abnormality information to obtain an updated fusion rule;

[0018] Extract data from the candidate business data according to the update fusion rule to obtain updated business data; wherein the updated business data includes business data matched by the operation abnormality information;

[0019] The target service data is updated according to the update fusion rule and the update service data.

[0020] In some embodiments, converting the data format of the initial service data according to the target data format to obtain candidate service data includes:

[0021] Performing format verification on the initial business data to obtain format verification information;

[0022] extracting initial unit data from the initial service data according to the format check information;

[0023] Filtering a target template from preset candidate templates according to the target data format;

[0024] Fill the initial unit data into the target template to obtain the candidate service data.

[0025] In some embodiments, the step of filling the initial unit data into the target template to obtain the candidate service data includes:

[0026] Acquire the data type of the initial unit data to obtain the unit data type;

[0027] Extracting sensitive data from the initial unit data according to the unit data type;

[0028] Filter the initial unit data according to the sensitive data to obtain target unit data;

[0029] Fill the target unit data into the target template to obtain the candidate service data.

[0030] In some embodiments, the acquiring at least two target data sources according to each of the service platform identification codes includes:

[0031] Acquire at least two candidate data sources according to the business platform identification code;

[0032] Obtaining access permission level information for each of the candidate data sources;

[0033] Performing permission verification on each of the candidate data sources according to preset request permission level information and the access permission level information to obtain permission verification information; wherein the request permission level information is the permission level information of the data fusion task information;

[0034] At least two of the target data sources are acquired according to the authority verification information.

[0035] In some embodiments, after fusing at least two of the candidate service data according to the data fusion rule to obtain the target service data, the method further includes:

[0036] Performing data anomaly detection according to the initial business data, the candidate business data, and the target business data to obtain data anomaly detection information;

[0037] If the data anomaly detection information meets the preset anomaly triggering condition, data anomaly prompt information is generated;

[0038] The data abnormality prompt information is displayed through a preset data analysis platform.

[0039] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a data fusion device for platform analysis, the device comprising:

[0040] A task acquisition module, used to acquire data fusion task information; wherein the data fusion task information includes: at least one business platform identification code, data fusion rules and target data format;

[0041] A data source acquisition module, used to acquire at least two target data sources according to each of the business platform identification codes;

[0042] A business data acquisition module, used to acquire the initial business data of each business platform through each of the target data sources;

[0043] A format conversion module, used for performing data format conversion on the initial service data according to the target data format to obtain candidate service data;

[0044] The data fusion module is used to fuse at least two of the candidate business data according to the data fusion rule to obtain target business data.

[0045] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0046] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0047] The data fusion method and device, electronic device and medium for platform analysis proposed in the present application first obtain data fusion task information to clarify the business platform and fusion rules that need to be fused. Then, by identifying the business platform identification code in the data fusion task information, accurate access to the target data source of each business platform is achieved to obtain the initial business data. Subsequently, the data format of these initial business data is uniformly converted into the target data format to obtain candidate business data, which provides standardized data input for data fusion and reduces the complexity and error rate in subsequent processing. Finally, the candidate business data is efficiently merged according to the data fusion rules to obtain the target business data. The method provided in the embodiment of the present application realizes flexible acquisition of the data source by flexibly setting the business platform identification code in the data fusion task information, and completes the business data fusion of the data source in combination with the data fusion rules. There is no need to set a customized microservice to complete the data fusion operation, which reduces the dependence on customized development and effectively overcomes the shortcomings of low flexibility, difficulty in expansion and high resource demand in data fusion in the prior art. In addition, compared with the existing solution of synchronizing data to a third-party centralized storage engine, the method provided in the embodiment of the present application does not rely on a third-party centralized storage engine, reduces the risk of query performance degradation due to the increase in data volume, shortens the data link, and ultimately improves the efficiency of data fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a data fusion method for platform analysis provided in an embodiment of the present application;

[0049] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0050] Figure 3 yes Figure 1 Flow chart of step S104 in FIG.

[0051] Figure 4 yes Figure 3 Flow chart of step S304 in FIG.

[0052] Figure 5 yes Figure 1 Flow chart of step S105 in FIG.

[0053] Figure 6 is another flow chart of the data fusion method for platform analysis provided by an embodiment of the present application;

[0054] Figure 7 is another flow chart of the data fusion method for platform analysis provided by an embodiment of the present application;

[0055] Figure 8 is a structural schematic diagram of a data fusion device for platform analysis provided in an embodiment of the present application;

[0056] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0060] First, some nouns involved in this application are analyzed:

[0061] Platform analysis: refers to the process of using data processing related technologies to conduct in-depth analysis of the massive data generated by financial trading platforms in order to extract valuable information from the data to guide decision-making, optimize services, manage risks, and promote business innovation. Platform analysis can help institutions more accurately grasp market trends and understand customer needs through the analysis of transaction data and user behavior, so as to make more scientific decisions in product design, marketing strategy formulation, etc. In addition, platform analysis can also reveal efficiency bottlenecks in the operation process, help financial institutions optimize business processes, reduce costs, and improve overall operational efficiency.

[0062] In the field of financial technology, financial companies will design different service systems or platforms to provide financial services to users, thereby generating a large amount of financial business data, including transaction data, customer information and market analysis reports, etc. These data usually come from different financial business platforms, and each financial business platform is connected to different ports, and the data formats of data from different ports are also different. By integrating and centrally analyzing these business data from different data sources, financial companies can better optimize and select platform business strategies.

[0063] In related technologies, customized development is required for the microservice architecture of different business platforms. When processing data fusion tasks, the exclusive microservices of different business units can be called, and each microservice processes its specific data and business logic. However, this solution has low flexibility, high expansion difficulty, and requires more development resources.

[0064] In another related technology, data integration technology is used to extract, transform and load multi-source heterogeneous data (Extract, Transform, Load, ETL), and then synchronize the data in a unified format to a third-party centralized storage engine. After that, the query task for the required business data is converted into a query on a single data source. However, this solution requires additional third-party development manpower and storage resources, which lengthens the data link. And as business data grows, the storage resources required also continue to grow, resulting in a decrease in the query performance of the centralized storage engine.

[0065] Therefore, how to improve the efficiency of data fusion has become an urgent problem to be solved.

[0066] Based on this, the embodiments of the present application provide a data fusion method and device, electronic device and medium for platform analysis, aiming to improve the data fusion efficiency.

[0067] The data fusion method and device, electronic device and storage medium for platform analysis provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the data fusion method for platform analysis in the embodiments of the present application is described.

[0068] The data fusion method for platform analysis provided in the embodiments of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the data fusion method for platform analysis, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0071] Figure 1is an optional flow chart of a data fusion method for platform analysis provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0072] Step S101, obtaining data fusion task information.

[0073] Step S102: acquiring at least two target data sources according to each service platform identification code.

[0074] Step S103: acquiring initial business data of each business platform through each target data source.

[0075] Step S104: convert the data format of the initial business data according to the target data format to obtain candidate business data.

[0076] Step S105 , fusing at least two candidate business data according to a data fusion rule to obtain target business data.

[0077] Steps S101 to S105 shown in the embodiment of the present application first obtain data fusion task information to clarify the business platform that needs to be fused and the rules of fusion. Then, by identifying the business platform identification code in the data fusion task information, accurate access to the target data source of each business platform is achieved to obtain the initial business data. Subsequently, the data format of these initial business data is uniformly converted into the target data format to obtain candidate business data, which provides standardized data input for data fusion and reduces the complexity and error rate in subsequent processing. Finally, the candidate business data is efficiently merged according to the data fusion rules to obtain the target business data. The method provided in the embodiment of the present application realizes flexible acquisition of data sources by flexibly setting the business platform identification code in the data fusion task information, and completes the business data fusion of the data source in combination with the data fusion rules. There is no need to set a customized microservice to complete the data fusion operation, which reduces the dependence on customized development, effectively overcomes the shortcomings of low flexibility, difficulty in expansion and high resource demand in data fusion in the prior art, and improves the efficiency of data fusion. In addition, compared with the existing solution of synchronizing data to a third-party centralized storage engine, the method provided in the embodiment of the present application does not rely on a third-party centralized storage engine, reducing the risk of query performance degradation due to the increase in data volume, while shortening the data link and improving the efficiency of data fusion.

[0078] In step S101 of some embodiments, data fusion task information refers to configuration information defined before data fusion, which is used to guide the process and behavior of data fusion. In this embodiment, the data fusion task information includes: at least one business platform identification code, data fusion rules and target data format. Specifically, the business platform identification code is a unique identification code that identifies the business platform, which is used to distinguish different business units or platforms in a large organization. Its form can be a digital sequence, a string code and a hash value, but is not limited to this. The data fusion rule is a specific step that defines how to combine data from different sources, including the type of data that needs to be fused, the logic of merging different types of data, and how to handle differences and anomalies in the data. The target data format is the structure and form that the fused data should present, including the encoding method of the data, the layout of the field and the data type, etc. The target format should support subsequent data analysis and business operation requirements, while ensuring that the data is easy to store, query and process, preferably in tabular form.

[0079] In some embodiments, data fusion task information can be configured and defined by using components that can be decoupled from each other, thereby achieving flexible configuration without the need for customized development for each business platform, thereby saving a lot of development costs.

[0080] In step S102 of some embodiments, the target data source is a port of a business platform for providing original business data, such as a web page, an application, and a business terminal, but not limited thereto. The business platform identification code in the data fusion task information is obtained to determine the business platform for which data needs to be obtained in this task, and then all ports of the business platform are used as target data sources.

[0081] It is understandable that different subsystems and business teams are often involved in the process of cross-data source data acquisition or processing. Directly granting the object executing the data fusion task the authority to access the business data of all platforms will cause problems of excessive authority and data sensitivity. In order to ensure the security of business data, before acquiring business data, it is necessary to verify the authority of the object executing the data fusion task information to confirm whether the object executing the data fusion is an authorized object of the current business platform. Relevant settings of authority information can be made in the data fusion task information. For example, when the development team of business platform A creates data fusion task information, and the data fusion task information indicates that only format conversion or data tracking needs to be performed on the initial business data provided by a specific port, the team will be authorized to access the data of all ports of business platform A, but will not have the right to access the data of business platform B.

[0082] In other embodiments, the target data source is further restricted by setting the level of permission to determine whether the object has permission to access. Figure 2Step S102 may include but is not limited to steps S201 to S204:

[0083] Step S201: acquiring at least two candidate data sources according to the service platform identification code.

[0084] Step S202: Obtain access permission level information of each candidate data source.

[0085] Step S203: Perform permission verification on each candidate data source according to preset request permission level information and access permission level information to obtain permission verification information.

[0086] Step S204: Acquire at least two target data sources according to the authority verification information.

[0087] In step S201 of some embodiments, the candidate data source is the data source of a specific business platform identified by the business platform identification code, which may be a web page, an application, a business, etc. In some embodiments, if the data fusion task information includes two or more business platform identification codes, it means that the business data of two or more business platforms need to be obtained when executing this data fusion task, and all ports of each business platform can be used as candidate data sources.

[0088] In step S202 of some embodiments, access permission information refers to the permission control settings of each candidate data source, which is used to specify the access permissions of different users or tasks to the data source. Access permission level information can be access level, access control list and role-based access control settings. Among them, the access control list sets a set of rules that define which users or tasks can access the data source and the types of operations allowed, such as reading and writing. Role-based access control settings assign permissions by role, for example, assigning different users or tasks to roles such as "administrator" and "read-only user", and the role determines their access permissions.

[0089] In step S203 of some embodiments, the request authority level information is the authority level information of the data fusion task information.

[0090] It is understandable that the request permission level information and the access permission level information are compared to obtain permission verification information, and the permission verification information indicates whether the object currently executing the data fusion task has the permission to access the candidate data source. Exemplarily, the pre-settings are represented by 10 to 1 from the highest permission to the lowest permission. The access permission level information obtained is level 5, and the request permission level information obtained is level 7. Through permission verification, it can be determined that the permission verification information indicates that the object currently executing the data fusion task has the permission to access. If the request permission level information obtained is level 2, it can be determined that the permission verification information indicates that the object currently executing the data fusion task does not have the permission to access the data source.

[0091] In some embodiments, the access permission level information of candidate data sources belonging to the same service platform is the same. In other embodiments, the permission level can be set separately for different ports of the same service platform according to the actual scenario.

[0092] In step S204 of some embodiments, when the authority verification information indicates that the object currently executing the data fusion task has the authority to access the candidate data source, the corresponding candidate data source is used as the target data source. When the authority verification information indicates that the object currently executing the data fusion task does not have the authority to access the candidate data source, the candidate data source is not accessed.

[0093] In the steps S201 to S204 shown in the embodiment of the present application, this method effectively manages data access rights by setting controllable data access points between different business platforms and implementing hierarchical permission control on these data sources, reduces the risk of data leakage and illegal access, and ensures that data fusion activities comply with business and security policies. In addition, the flexible configuration of permissions can adapt to different business needs and changes, making data management more flexible and improving data processing efficiency.

[0094] In step S103 of some embodiments, the initial business data is the original data provided by each port of the business platform. The initial business data may include customer information (such as customer ID, customer IP and customer age), transaction information (such as transaction amount, transaction business and transaction time), and product information (such as product type, product ID, price, configuration), etc., but is not limited thereto.

[0095] In step S104 of some embodiments, the candidate business data is business data after the initial business data is converted into a data format. Due to different data sources, the formats of the obtained initial business data are also different. In order to achieve effective data fusion, the formats of the initial business data need to be unified.

[0096] See also Figure 3In some embodiments, step S104 may include but is not limited to steps S301 to S304:

[0097] Step S301, perform format verification on initial business data to obtain format verification information.

[0098] Step S302: extracting initial unit data from initial service data according to format verification information.

[0099] Step S303: Filter out a target template from preset candidate templates according to the target data format.

[0100] Step S304, fill the initial unit data into the target template to obtain candidate business data.

[0101] In step S301 of some embodiments, format verification is performed on the initial business data to determine whether the initial business data is structured data. The format verification information includes whether the initial business data is structured data, and whether the initial business data is unstructured data.

[0102] In step S302 of some embodiments, when the format verification information indicates that the initial business data is formatted data, the data format of the initial business data is identified, and the definition of the initial business data is determined according to the format, and the initial business data is decomposed, and the initial unit data is each unit data in the decomposition result of the initial business data. Exemplarily, the initial business data is confirmed to be JSON data, and the initial business data at this time can be a simple set of key-value pairs, and may also contain nested objects or arrays. The structure definition of the initial business data obtained is {"Customer Name": "XXX", "Customer Age": "25". "Customer IP Address": "City A"}, where the initial unit data can be {"Customer Name": "XXX"}, {"Customer Age": "25"}, and {"Customer IP": "City A"}, etc.

[0103] When the format verification information indicates that the initial business data is unstructured data, the initial business data is not acquired and an error is reported.

[0104] In step S303 of some embodiments, the candidate templates are preset templates of multiple data formats, each of which defines a specific data organization method. The target template is a file template that matches the target data format.

[0105] In step S304 of some embodiments, since the initial business data includes a large amount of customer information, in order to ensure the security of customer information and the sensitivity of business data, it is necessary to perform desensitization processing on the sensitive information in the initial business data. Figure 4 Step S304 may include but is not limited to steps S401 to S404:

[0106] Step S401, obtaining the data type of the initial unit data to obtain the unit data type.

[0107] Step S402: extract sensitive data from the initial unit data according to the unit data type.

[0108] Step S403, filtering the initial unit data according to the sensitive data to obtain target unit data.

[0109] Step S404, fill the target unit data into the target template to obtain candidate business data.

[0110] In step S401 of some embodiments, the unit data type may be, but is not limited to, customer name, customer ID number, customer age, bank card number and bank card password used by the customer, customer IP location, transaction time, transaction amount and product information.

[0111] In step S402 of some embodiments, it is determined whether the corresponding initial unit data is data that needs to be processed for allergies based on the unit data type of each unit data. Sensitive data refers to data involving customer personal privacy, corporate secrets or other data that needs to be handled confidentially. From the perspective of individual users, sensitive data may include customer name, customer ID number, bank card number and bank card password used by the customer, etc. From the perspective of corporate users, sensitive data may include relevant sensitive information of business activities with external companies. The specific types of these relevant sensitive information can be adaptively set according to the actual business contract agreement, and the embodiments of the present application do not strictly limit this.

[0112] In step S403 of some embodiments, the target unit data is sensitive data that has been desensitized, or non-sensitive data. The sensitive data is desensitized, and the desensitization method may include: data masking, pseudo-anonymization, hashing and encryption. If the initial unit data does not belong to sensitive information, desensitization does not need to be performed, and data filling is performed directly according to step S404.

[0113] In step S404 of some embodiments, the target unit data is filled into the target template according to the data organization method defined by the target template. Exemplarily, the target template is a CSV file, the data format of the initial business data is a JSON file, the key of each initial unit data (customer name, customer age and customer IP location) will become the column title of the CSV, and the value of each object will become the row in the CSV.

[0114] Steps S401 to S404 shown in the embodiment of the present application determine which data needs to be desensitized by identifying the initial unit data type, ensuring that data processing complies with the provisions of privacy protection and data security. Subsequently, depending on the data type, a systematic desensitization operation is performed on sensitive information. Ultimately, the processed data and non-sensitive initial unit data are filled into a pre-defined template to complete the format conversion of the data. The method of this embodiment not only achieves efficient processing and conversion of data, but also strengthens the security and compliance of the data.

[0115] Steps S301 to S304 shown in the embodiment of the present application determine whether the initial business data is structured data through format verification to ensure that only data that meets expectations enters the subsequent process. Once the data is confirmed to be structured, the initial unit data will be extracted from it. These data elements will be organized according to the defined templates to ensure that the data output meets the expected format and is clean. Finally, these screened and formatted initial unit data will be filled into the selected template to generate candidate business data, which provides a standardized and unified data format for subsequent data processing and ensures the smooth progress of data fusion.

[0116] See also Figure 5 In some embodiments, step S105 may also include but is not limited to steps S501 to S503:

[0117] Step S501, obtaining the data type of each candidate business sub-data to obtain the business data type.

[0118] Step S502: Filter out target data fusion sub-rules from the data fusion sub-rules according to the business data type and the target data type.

[0119] Step S503: fusing the candidate business sub-data according to the target data fusing sub-rule to obtain the target business data.

[0120] In step S501 of some embodiments, the candidate business data includes at least two candidate business sub-data, and the candidate business sub-data is the smallest unit constituting the candidate business data. The business data type of the candidate business sub-data can be the customer name, the customer's ID number, the customer's age, the bank card number and bank card password used by the customer, the customer's IP location, the transaction time, the transaction amount, and the product information, but is not limited thereto.

[0121] In step S502 of some embodiments, it can be known from the description of the above embodiments that each business data may include a large amount of different types of information, but in the actual platform analysis scenario, if all the information is presented to the data analyst, it will cause information redundancy, which is not conducive to data insight. Therefore, it is necessary to clarify the data types required in this fusion operation through data fusion rules, and perform targeted operations on these required data types to filter out key data, so that data analysts can quickly focus on information related to the current analysis needs and improve the efficiency and insight of the analysis.

[0122] Specifically, the data fusion rule includes a target data type and a data fusion sub-rule that matches the target data type. In other words, the data fusion rule is a set of different calculation methods specified for different types of data, and the data fusion sub-rule is any calculation method in the set. The target data type is the required data type corresponding to the data fusion task information. When it is identified that the acquired business data type matches the target data type, the data fusion sub-rule corresponding to the target data type, namely the target data fusion sub-rule, is obtained. If the business data type does not match the target data type, the corresponding candidate business sub-data is eliminated. The data fusion rule may be a summation, average or other mathematical calculation for digital data, and may involve splicing or selecting a specific format for output for text data.

[0123] For example, in order to analyze the sales amount of products launched by business platform A and business platform B in different cities, the business platform identification code in the data fusion task information can be set to include: the identification code of business platform A and the identification code of business platform B, and the data fusion rule is: the target data type includes customer IP, the business platform to which the customer's transaction product belongs, and the transaction amount. The data fusion sub-rules are: use the customer IP field information as the row title and splice it; if the business platform to which the customer's transaction product belongs is the same, the corresponding transaction amount is calculated in total; generate a column title ("business platform A / B......product sales amount") according to the business platform to which the customer's transaction product belongs and splice it.

[0124] In step S503 of some embodiments, the candidate sub-data are fused according to the target data fusion sub-rule, and the obtained data result is the target business data. In this embodiment, the target business data is preferably a data visualization image or table, so that subsequent data analysts can observe it intuitively and effectively.

[0125] For example, according to the example description of the data fusion rule in the embodiment of step S502, after executing step S503, a table as shown in Table 1 may be generated:

[0126]

[0127] Table 1

[0128] Steps S501 to S503 shown in the embodiment of the present application set corresponding fusion rules for different types of candidate business sub-data, and finally integrate them into unified target business data. This not only improves the efficiency and accuracy of data fusion, but also enhances the adaptability and scalability of the method, so that it can flexibly respond to diverse data sources and business needs.

[0129] After step S106 in some embodiments, the data fusion rules may be adjusted according to the specific status of the transaction behavior occurring in real time on the business platform.

[0130] See also Figure 6 The data fusion method for platform analysis provided in the embodiment of the present application may also include but is not limited to steps S601 to S605:

[0131] Step S601, obtaining operation abnormality information of the target data source.

[0132] Step S602: Perform abnormal transaction behavior detection on the business platform according to the target business data to obtain abnormal transaction behavior detection data.

[0133] Step S603, updating the data fusion rules according to the abnormal transaction behavior detection data and the operation abnormality information to obtain updated fusion rules.

[0134] Step S604: extracting data from the candidate business data according to the update fusion rule to obtain updated business data.

[0135] Step S605: Update the target business data according to the update fusion rule and the update business data.

[0136] In step S601 of some embodiments, the abnormal operation information refers to abnormal events or data that occur in the daily operation of the business platform to which the target data source belongs, which is usually obtained by users of the business platform providing feedback on the system. For example, the business platform receives feedback complaints from a user of an APP terminal, reflecting that the account frequently receives reminders of remote login, or that the payment service often freezes when making payments.

[0137] In step S602 of some embodiments, the abnormal transaction behavior detection data includes the number of abnormal transactions of the business platform in a preset period. Abnormal transaction behaviors may include high-frequency transactions, frequent changes of user IP in a short period of time, and transaction behaviors that are inconsistent with the user's historical transaction habits, etc., but are not limited thereto.

[0138] In step S603 of some embodiments, it can be understood that in order to facilitate data analysts to quickly gain insight into specific characteristics of the data, the data types in the target business data are screened through data fusion rules. In some embodiments, the initial target business data can focus on reflecting specific abnormal behaviors. If more operational abnormality information is received in the near future, the data types in the target business data can be adjusted in a targeted manner based on the content fed back by the operational abnormality information. The adjusted data fusion rules are the updated fusion rules. In other embodiments, the current abnormal transaction behavior detection data reflects a small number of abnormal transactions within a preset period. The data type in the target business data can be adjusted so that the subsequently generated target business data can reflect other possible potential abnormal behaviors.

[0139] For example, the current data fusion task information reflects the sales situation of a certain business platform, and it is hoped that the abnormal behavior of high-frequency transactions can be effectively reflected. The corresponding data fusion rule can set the target data type as user ID, transaction amount and transaction time, and splice all categories of data to obtain a table, as shown in Table 2. In Table 2, it can be reflected that the user with ID ABC12345 conducted transactions at 10:30:34, 10:30:35 and 10:30:36, respectively, which belongs to high-frequency transaction abnormal behavior. If the transaction amount is the same each time, it may be that the payment service is stuck during payment. At this time, the abnormal transaction behavior detection data is updated as follows: the number of payment stuck phenomena is 1. If high-frequency transactions occur and the transaction amount is different each time, it indicates that the account is at risk of being used for non-compliant transactions. At this time, the abnormal transaction behavior detection data is updated as follows: the number of abnormal risk phenomena in the account is 1.

[0140] When the number of occurrences of the same type of abnormal transaction behavior exceeds the preset number within the preset period, or the business platform receives a large number of similar operational abnormality information, the data fusion rules will be adjusted. For example, in the past week, the number of abnormal risk phenomena in the account exceeds 50 times, and the data fusion rules are adjusted. On the basis of the original table, the user credit score is used as the new row title. It should be noted that the impact of operational abnormality information on the modification of data fusion rules has a higher priority. When receiving a large number of operational abnormality information, even if the number of occurrences of the same type of abnormal transaction behavior does not exceed the preset number, the data fusion rules need to be adjusted.

[0141] User ID Transaction amount (yuan) Transaction time (hours: minutes: seconds) ABC12345 XXX 10:30:34 ABC12345 XXX 10:30:35 ABC12345 XXX 10:30:36 ... ... ... EFG22345 XXX

[0142] Table 2

[0143] In other embodiments, as shown in Table 2, the target business data has the same type of abnormal transaction behavior within the preset period, and the number of occurrences does not exceed the preset number, and no excessive operational abnormality information is received. It is possible that the current business platform has a low risk of high-frequency transactions, and the current target business data has no way to understand whether the user's account has been logged in from another place. For example, the same user logs in in different countries in a short period of time and realizes large-amount transactions, which may involve non-compliant transactions. At this time, the data fusion rules are adjusted, and the user IP information is added as a new row list based on Table 2, so that the target business data shown in Table 3 can be generated later. Through the above adjustment, the type of abnormal behavior that the target business data can reflect is changed.

[0144] User ID User IP Transaction amount Trading Hours

[0145] Table 3

[0146] In step S604 of some embodiments, the updated business data includes business data matched with the operational abnormality information, and a newly added data type can be determined according to the update fusion rule and acquired together with the original target data type to obtain the updated business data.

[0147] In step S605 of some embodiments, a data processing operation specified by the update fusion rule is performed on each data type in the updated business data to obtain new business data and new target business data, thereby updating the target business data.

[0148] Steps S601 to S605 shown in the embodiment of the present application improve the flexibility and accuracy of data fusion through dynamic detection and rule updating of abnormal operation information and abnormal transaction behavior. The method can dynamically adjust the data fusion rules according to operational feedback and actual detection data, so that the target business data is more in line with the current business analysis needs, and can help data analysts quickly understand and respond to the abnormal operation status and transaction behavior of the platform, and improve the effectiveness and security of subsequent platform data analysis.

[0149] After step S106 in some other embodiments, please refer to Figure 7 In some embodiments, the data fusion method for platform analysis provided in the embodiments of the present application may also include but is not limited to steps S701 to S703:

[0150] Step S701: Perform data anomaly detection based on initial business data, candidate business data and target business data to obtain data anomaly detection information.

[0151] Step S702: if the data anomaly detection information meets the preset anomaly triggering condition, data anomaly prompt information is generated.

[0152] Step S703: displaying data abnormality prompt information through a preset data analysis platform.

[0153] In step S701 of some embodiments, the data anomaly detection information represents whether an anomaly occurs in the initial business data, the candidate business data, or the target business data, and the corresponding specific anomaly information when an anomaly occurs.

[0154] In step S702 of some embodiments, the abnormal triggering conditions include: the initial business data is unstructured data, the sensitive data in the candidate business data is not desensitized, and the target business data represents an abnormal transaction, but are not limited thereto. The data abnormality prompt information is a detailed description of the abnormal situation, which may include: the abnormal type, the specific business module, field or data source where the abnormal data is located, the abnormal time, and the recommended solution.

[0155] In some embodiments, data synchronization is performed on the target business data at intervals of a preset period, and the target business data is synchronized from the upstream data source to the downstream data source. The preset period may be 24 hours. It is necessary to check and verify the incremental data every 24 hours, and to detect whether the upstream and downstream data volumes are consistent. If the upstream and downstream data volumes are different, it is determined that an abnormality has occurred in the target business data. It is also necessary to detect whether the data upload and download rates have reached a preset ratio. If the preset ratio has not been reached, it is determined that an abnormality has occurred in the target business data.

[0156] In step S703 of some embodiments, the data analysis platform is an interactive interface with data visualization and reporting functions, which can display abnormal prompt information in the form of charts, tables, etc., so that data analysts can quickly obtain and analyze abnormal information.

[0157] Steps S701 to S703 shown in the embodiment of the present application automatically identify abnormal situations in the data set, generate specific abnormal prompt information, and present the abnormal prompt information on the data analysis platform, so that data analysts and business operations personnel can quickly learn about potential problems and respond. The method of this embodiment integrates data tracking and data abnormality detection on the basis of data fusion, thereby reducing the need for manual work and improving the timeliness of abnormality processing.

[0158] See also Figure 8 The embodiment of the present application further provides a data fusion device for platform analysis, which can implement the above-mentioned data fusion method for platform analysis, and the device includes:

[0159] The task acquisition module is used to acquire data fusion task information; wherein the data fusion task information includes: at least one business platform identification code, data fusion rules and target data format.

[0160] The data source acquisition module is used to acquire at least two target data sources according to each business platform identification code.

[0161] The business data acquisition module is used to acquire the initial business data of each business platform through each target data source.

[0162] The format conversion module is used to convert the data format of the initial business data according to the target data format to obtain candidate business data.

[0163] The data fusion module is used to fuse at least two candidate business data according to the data fusion rule to obtain the target business data.

[0164] The specific implementation of the data fusion device for platform analysis is basically the same as the specific implementation of the data fusion method for platform analysis mentioned above, and will not be repeated here.

[0165] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned data fusion method for platform analysis when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0166] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0167] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0168] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the data fusion method for platform analysis in the embodiment of this application;

[0169] Input / output interface 903, used to implement information input and output;

[0170] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0171] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0172] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0173] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the data fusion method for platform analysis is implemented.

[0174] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The data fusion method for platform analysis, the data fusion device for platform analysis, the electronic device and the storage medium provided in the embodiment of the present application first obtain the data fusion task information to clarify the business platform and the fusion rules that need to be fused. Then, by identifying the business platform identification code in the data fusion task information, accurate access to the target data source of each business platform is achieved to obtain the initial business data. Subsequently, the data format of these initial business data is uniformly converted into the target data format to obtain candidate business data, which provides standardized data input for data fusion and reduces the complexity and error rate in subsequent processing. Finally, the candidate business data is efficiently merged according to the data fusion rules to obtain the target business data. The method provided in the embodiment of the present application realizes flexible acquisition of the data source by flexibly setting the business platform identification code in the data fusion task information, and completes the business data fusion of the data source in combination with the data fusion rules. There is no need to set a customized microservice to complete the data fusion operation, which reduces the dependence on customized development, effectively overcomes the shortcomings of low flexibility, difficulty in expansion and high resource demand in data fusion in the prior art, and improves the efficiency of data fusion. In addition, compared with the existing solution of synchronizing data to a third-party centralized storage engine, the method provided in the embodiment of the present application does not rely on a third-party centralized storage engine, reducing the risk of query performance degradation due to the increase in data volume, while shortening the data link and improving the efficiency of data fusion.

[0176] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0177] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0178] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0180] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0181] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0182] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0186] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

[0187] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A data fusion method for platform analysis, characterized in that: The method comprises: Acquire data fusion task information; wherein the data fusion task information includes: at least one business platform identification code, data fusion rules and target data format; Acquire at least two target data sources according to each of the business platform identification codes; Acquiring initial business data of each business platform through each of the target data sources; Convert the initial service data into a data format according to the target data format to obtain candidate service data; At least two of the candidate business data are fused according to the data fusion rule to obtain target business data.

2. The method according to claim 1, characterized in that The candidate business data includes at least two candidate business sub-data, and the data fusion rule includes: a target data type and a data fusion sub-rule matching the target data type; fusing at least two of the candidate business data according to the data fusion rule to obtain the target business data includes: Acquire the data type of each candidate business sub-data to obtain the business data type; Filtering a target data fusion sub-rule from the data fusion sub-rules according to the business data type and the target data type; The candidate service sub-data are data-fused according to the target data fusion sub-rule to obtain the target service data.

3. The method according to claim 1, characterized in that After fusing at least two of the candidate service data according to the data fusion rule to obtain target service data, the method further includes: Obtaining operation abnormality information of the target data source; Perform abnormal transaction behavior detection on the business platform according to the target business data to obtain abnormal transaction behavior detection data; wherein the abnormal transaction behavior detection data includes the number of abnormal transactions of the business platform in a preset period; The data fusion rule is updated according to the abnormal transaction behavior detection data and the operation abnormality information to obtain an updated fusion rule; Extract data from the candidate business data according to the update fusion rule to obtain updated business data; wherein the updated business data includes business data matched by the operation abnormality information; The target service data is updated according to the update fusion rule and the update service data.

4. The method according to claim 1, characterized in that The converting the data format of the initial service data according to the target data format to obtain candidate service data includes: Performing format verification on the initial business data to obtain format verification information; extracting initial unit data from the initial service data according to the format check information; Filtering a target template from preset candidate templates according to the target data format; Fill the initial unit data into the target template to obtain the candidate service data.

5. The method according to claim 4, characterized in that The step of filling the initial unit data into the target template to obtain the candidate service data includes: Acquire the data type of the initial unit data to obtain the unit data type; Extracting sensitive data from the initial unit data according to the unit data type; Filter the initial unit data according to the sensitive data to obtain target unit data; Fill the target unit data into the target template to obtain the candidate service data.

6. The method according to any one of claims 1 to 5, characterized in that: The acquiring at least two target data sources according to each of the service platform identification codes includes: Acquire at least two candidate data sources according to the business platform identification code; Obtaining access permission level information for each of the candidate data sources; Performing permission verification on each of the candidate data sources according to preset request permission level information and the access permission level information to obtain permission verification information; wherein the request permission level information is the permission level information of the data fusion task information; At least two of the target data sources are acquired according to the authority verification information.

7. The method according to any one of claims 1 to 5, characterized in that: After fusing at least two of the candidate service data according to the data fusion rule to obtain target service data, the method further includes: Performing data anomaly detection according to the initial business data, the candidate business data, and the target business data to obtain data anomaly detection information; If the data anomaly detection information meets the preset anomaly triggering condition, data anomaly prompt information is generated; The data abnormality prompt information is displayed through a preset data analysis platform.

8. A data processing device for platform analysis, characterized in that: The device comprises: A task acquisition module, used to acquire data fusion task information; wherein the data fusion task information includes: at least one business platform identification code, data fusion rules and target data format; A data source acquisition module, used to acquire at least two target data sources according to each of the business platform identification codes; A business data acquisition module, used to acquire the initial business data of each business platform through each of the target data sources; A format conversion module, used for performing data format conversion on the initial service data according to the target data format to obtain candidate service data; The data fusion module is used to fuse at least two of the candidate business data according to the data fusion rule to obtain target business data.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images