A data integration processing method and device, electronic equipment and storage medium

Abstract

The application relates to a data integration processing method and device, electronic equipment and a storage medium, and the method comprises the following steps: obtaining feedback information of a customer on product data, wherein the feedback information at least comprises customer evaluation, transaction behavior data, hot-selling products and anchor point click volume; after synonym and similarity feature extraction is performed on the feedback information, aggregated feature data is obtained through aggregation, and the aggregated feature data at least comprises a customer evaluation set, a transaction behavior set and a user access behavior set; the emphasis of a research and development direction is determined from the aggregated feature data; and data corresponding to the emphasis of the research and development direction is collected, cleaned, processed and applied in combination with a business scene, product decision, product definition and research and development management to construct a digital integration main line; wherein the digital integration main line integrates data of a product whole life cycle. In the application, the digital integration main line pushes information to corresponding research and development personnel, production personnel and the like, and provides data support for research and development, production and the like of a product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data integration, and in particular to a data integration processing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Data integration is to logically or physically organically concentrate data of different sources, formats, characteristics and properties, thereby providing comprehensive data sharing for enterprises. In the field of enterprise data integration, there are many mature frameworks that can be used. A federated system is usually constructed by using methods such as middleware model and data warehouse. These technologies solve data sharing and provide decision support for enterprises in different focuses and applications. In an enterprise, due to different development times or development departments, multiple heterogeneous information systems running on different software and hardware platforms often run simultaneously. The data sources of these systems are independent of and closed to each other, making it difficult for data to be exchanged, shared and integrated between systems, thereby forming an "information island". With the continuous deepening of informatization applications, the demand for internal and external information interaction in enterprises is increasingly strong, and there is an urgent need to integrate existing information, connect "information islands" and share information. SUMMARY

[0003] Based on the above problems, the present application provides a data integration processing method, device, electronic equipment and storage medium.

[0004] In a first aspect, an embodiment of the present application provides a data integration processing method, comprising:

[0005] Obtaining feedback information of customers on product data, the feedback information at least including customer evaluation, transaction behavior data, hot-selling products and anchor click volume;

[0006] After synonym extraction and similarity feature extraction on the feedback information, aggregated feature data is obtained by aggregation, the aggregated feature data at least including a customer evaluation set, a transaction behavior set and a user access behavior set;

[0007] Determining the focus of the research and development direction from the aggregated feature data;

[0008] Collecting, cleaning, processing and applying data corresponding to the focus of the research and development direction, and combining with business scenarios, product decision-making, product definition and research and development management to construct a digital integration main line;

[0009] In the digital integration main line, data of the whole life cycle of a product is integrated.

[0010] Further, in the above data integration processing method, the feedback information of customers on product data is obtained through a Jaeger tracker,

[0011] The Jaeger tracker includes a Jaeger client program, and the Jaeger client program sends the instance sampling data of the plurality of microservices to the Jaeger Agent through the API of OpenTracing;

[0012] The source of the feedback information at least includes: server-owned data, sampling data, and third-party data aggregation.

[0013] Further, in the above-mentioned data integration processing method, after the synonym and similarity features of the feedback information are extracted, the aggregated feature data is obtained by aggregation, comprising:

[0014] Extracting a field phrase from the feedback information by a keyword or a phrase;

[0015] Statistical index features of the field phrase;

[0016] Mining synonyms according to the index features;

[0017] Calculating the semantic similarity of the mined synonyms and the corresponding keywords or phrases to determine whether the mined synonyms and the corresponding keywords or phrases are synonyms;

[0018] Aggregating the determined synonyms to obtain the aggregated feature data.

[0019] Further, in the above-mentioned data integration processing method, calculating the semantic similarity of the mined synonyms and the corresponding keywords or phrases comprises:

[0020] Obtaining the word vectors of the mined synonyms and the corresponding keywords or phrases by using the Glove model;

[0021] Calculating the cosine value of the included angle between the two vectors of the mined synonyms and the corresponding keywords or phrases by using the Euclidean dot product formula to measure the similarity between the mined synonyms and the corresponding keywords or phrases.

[0022] Further, in the above-mentioned data integration processing method, determining the focus of the research and development direction from the aggregated feature data comprises: analyzing the set of keywords or phrases that appear most frequently in the aggregated feature data set to determine the focus of the research and development direction.

[0023] The collection, cleaning, processing and application of the data corresponding to the focus of the research and development direction are combined with business scenarios, product decision-making, product definition and research and development management to construct a digital integration main line in the digital main line module in the digital data platform.

[0024] Further, in the above-mentioned data integration processing method, the digital integration main line link at least includes: multi-source heterogeneous data, selection of computing power nodes, data integration of the product full life cycle, and operator composition operation chain of each link.

[0025] Further, in the above data integration processing method, the multi-source heterogeneous data includes: deploying a real-time data warehouse based on an iceberg data lake technology in the central server to collect underlying data;

[0026] The computing node selection includes: selecting a path with sufficient idle computing power and the shortest path from the iceberg table format layer to the network computing node network as the optimal computing node; and performing the whole process of data AI analysis by the Flink data engine and the whole process data flow processing and point embedding of Redis according to the weight distribution of the computing node.

[0027] The data integration of the product whole life cycle and the operator composition operation chain of each link include:

[0028] A data stream is mapped as a process for transmitting data to the center, and a plurality of threads are included in the process; the data generated by the data integration of the product whole life cycle in each link is mapped as an operator, and the operators in each link are composed into an operation chain.

[0029] In a second aspect, the embodiments of the present application further provide a data integration processing device, which includes:

[0030] The acquisition module is configured to acquire feedback information of the product data of the customer, and the feedback information at least includes: customer evaluation, transaction behavior data, hot-selling product and anchor click volume;

[0031] The aggregation module is configured to aggregate the aggregated feature data after synonym extraction and similarity feature extraction of the feedback information, and the aggregated feature data at least includes: customer evaluation set, transaction behavior set and user access behavior set;

[0032] The determination module is configured to determine the emphasis of the research and development direction from the aggregated feature data;

[0033] The construction module is configured to combine the collection, cleaning, processing and application of the data corresponding to the emphasis of the research and development direction with the business scene, product decision, product definition and research and development management to construct a digital integration main line.

[0034] The data of the product whole life cycle is integrated in the digital integration main line.

[0035] In a third aspect, the embodiments of the present application further provide an electronic device, which includes: a processor and a memory;

[0036] The processor is configured to execute any one of the above data integration processing methods by calling the program or instruction stored in the memory.

[0037] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a program or instructions that cause a computer to perform any of the data integration processing methods described above.

[0038] The advantages of this invention are as follows: This invention obtains customer feedback on product data, including at least customer reviews, transaction behavior data, best-selling products, and anchor click volumes. After extracting synonyms and similarity features from the feedback information, aggregated feature data is obtained, including at least a set of customer reviews, a set of transaction behaviors, and a set of user access behaviors. The focus of R&D direction is determined from the aggregated feature data. The collection, cleaning, processing, and application of data corresponding to the focus of R&D direction are combined with business scenarios, product decisions, product definitions, and R&D management to construct a digital integration thread. This digital integration thread integrates data from the entire product lifecycle. The digital integration thread in this invention delivers appropriate information to the right people at the right time, in the right context, and in the right form, thereby providing data support for product decision-making, production, and R&D. This achieves a high degree of data sharing among all team members throughout the product lifecycle, improving data utilization. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A schematic diagram of a data integration processing method provided in an embodiment of the present invention. Figure 1 ;

[0041] Figure 2 A schematic diagram of a data integration processing method provided in an embodiment of the present invention. Figure 2 ;

[0042] Figure 3 A schematic diagram of a data integration processing method provided in an embodiment of the present invention. Figure 3 ;

[0043] Figure 4 A schematic diagram of a data integration and processing device provided in an embodiment of the present invention;

[0044] Figure 5 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0045] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below with reference to the drawings. In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the spirit of the present application, so the present application is not limited to the specific implementations disclosed below.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0047] Figure 1 A data integration processing method provided by an embodiment of the present application Figure 1 .

[0048] In a first aspect, an embodiment of the present application provides a data integration processing method, which is combined with Figure 1 , comprising:

[0049] S101: Obtain feedback information of customers on product data, the feedback information at least including customer evaluation, transaction behavior data, hot-selling products and anchor click volume.

[0050] Specifically, in the embodiment of the present application, the feedback information of customers on product data is obtained through a Jaeger tracker, and the obtaining method is described in detail below.

[0051] S102: After synonym and similarity feature extraction, the feedback information is aggregated to obtain aggregated feature data, the aggregated feature data at least including a customer evaluation set, a transaction behavior set and a user access behavior set.

[0052] Specifically, in the embodiment of the present application, after the feedback information is aggregated to obtain the aggregated feature data, some fuzzy and synonymous data are aggregated to improve the effectiveness of data analysis.

[0053] S103: Determine the emphasis of the research and development direction from the aggregated feature data.

[0054] Specifically, in the embodiment of the present application, the keyword with the most occurrences in the aggregated feature data set is analyzed to guide the research and development as the emphasis of the research and development direction.

[0055] S104: The data collection, cleaning, processing and application corresponding to the focus of the research and development direction are combined with the business scene, product decision, product definition and research and development management to build a digital integrated main line;

[0056] The data of the product full life cycle is integrated in the digital integrated main line.

[0057] Specifically, in the embodiment of the present application, the data collection, cleaning, processing and application corresponding to the focus of the research and development direction are combined with the business scene, product decision, product definition and research and development management to build a digital integrated main line, the data of the product full life cycle is integrated in the digital integrated main line, and the digital integrated main line pushes the appropriate information to the appropriate person at the appropriate time point, in the appropriate occasion and form, thereby providing data support for the product decision, production and research and development, and improving the data utilization rate.

[0058] Further, in the above-mentioned data integration processing method, the feedback information of the customer to the product data is obtained through a Jaeger tracker,

[0059] The Jaeger tracker includes a Jaeger client program, and the Jaeger client program sends the instance sampling buried point data of a plurality of microservices to the Jaeger Agent through the API of OpenTracing;

[0060] The source of the feedback information at least includes: server own data, buried point data, and third-party data summary.

[0061] Figure 2 A data integration processing method provided by the embodiment of the present application is shown in Figure 2 .

[0062] Further, in the above-mentioned data integration processing method, the feedback information is subjected to synonym extraction, similarity feature extraction and aggregation to obtain aggregated feature data, combined with Figure 3 , including:

[0063] S201: Extracting information groups from the feedback information through keywords or phrases to form domain phrases;

[0064] S202: Statistics of index features of the domain phrases;

[0065] S203: Mining synonyms according to the index features;

[0066] S204: Calculating the semantic similarity of the mined synonyms and the corresponding keywords or phrases to determine whether the mined synonyms and the corresponding keywords or phrases are synonyms;

[0067] S205: Aggregating the determined synonyms to obtain aggregated feature data.

[0068] Specifically, in the embodiments of the present application, the field phrases are extracted from the feedback information, the index features of the field phrases are counted, and the synonymous words are mined by using the statistical index features of the phrases. Meanwhile, the semantic similarity between the mined synonymous words and the corresponding keywords or phrases is calculated to determine whether the mined synonymous words and the corresponding keywords or phrases are synonymous words, the synonymous words are accurately determined, and the determined synonymous words are aggregated to obtain the aggregated feature data.

[0069] In some embodiments, the data keywords or phrases can also be refined to filter out waste data and repeatedly used training data.

[0070] Figure 3 A data integration processing method provided by the embodiments of the present application is shown in the following Figure 4 .

[0071] Further, in the above-mentioned data integration processing method, the semantic similarity between the mined synonymous words and the corresponding keywords or phrases is calculated, and the Figure 4 , includes two steps of S301 to S302:

[0072] S301: obtaining the word vector of the mined synonymous words and the corresponding keywords or phrases by using the Glove model;

[0073] S302: calculating the cosine value of the angle between the two vectors of the mined synonymous words and the corresponding keywords or phrases by using the Euclidean dot product formula to measure the similarity between the mined synonymous words and the corresponding keywords or phrases.

[0074] Specifically, in the embodiments of the present application, the word vector of the mined synonymous words and the corresponding keywords or phrases is obtained by using the Glove model, and then the similarity between the mined synonymous words and the corresponding keywords or phrases can be obtained by using different similarity calculation methods. In the present application, the cosine value of the angle between the two vectors of the mined synonymous words and the corresponding keywords or phrases is calculated by using the Euclidean dot product formula to measure the similarity between the mined synonymous words and the corresponding keywords or phrases.

[0075] The cosine value is used to measure the similarity between the mined synonymous words and the corresponding keywords or phrases by measuring the cosine value of the angle between the two vectors.

[0076] The Euclidean dot product formula is as follows:

[0077]

[0078] Here, A and B represent the components of vectors A and B, respectively.

[0079] Further, in the data integration processing method, the focus of the research and development direction is determined from the aggregated feature data, including: analyzing the set in which the keyword or phrase appears most frequently in the aggregated feature data set as the focus of the research and development direction.

[0080] The collection, cleaning, processing and application of data corresponding to the focus of the research and development direction are combined with business scenarios, product decision-making, product definition and research and development management to construct a digital integrated main line in the digital main line module in the digital data platform.

[0081] Specifically, in the embodiment of the present application, by constructing a digital integrated main line in the digital main line module in the digital data platform, the data related to research and development is highly shared among all team members in the whole life cycle of the product.

[0082] Further, in the data integration processing method, the digital integrated main line includes at least: multi-source heterogeneous data, selection of computing power nodes, data integration of the product whole life cycle and operator composition operation chain of each link.

[0083] Further, in the data integration processing method, the multi-source heterogeneous data includes: deploying a real-time data warehouse based on iceberg data lake technology in the central server to collect underlying data.

[0084] Specifically, in the embodiment of the present application, iceberg is a table format in form, that is, an intermediate layer based on a computing layer (flink, spark, Hive, Presto) and a storage layer (ORC, Parquet, AVRO), which provides a basic environment for performance adaptation and optimization of big data operation through multi-source heterogeneous data.

[0085] The selection of computing power nodes includes: selecting the optimal computing power node with sufficient idle computing power and the shortest path from the iceberg table format layer to the network computing power node network path; and performing the whole process of data AI analysis by integrating the whole process data flow processing and point embedding of the Flink data engine and Redis according to the weight distribution of the computing power node.

[0086] The data integration of the product whole life cycle and the operator composition operation chain of each link include:

[0087] A data stream is mapped to a process for transmitting data to the center, and a process includes multiple threads; the data generated by the data integration of the product whole life cycle in each link is mapped to an operator, and the operators in each link are composed into an operation chain.

[0088] Figure 5 A data integration processing device provided by the embodiment of the present application is shown in the figure.

[0089] In a second aspect, the embodiment of the present application further provides a data integration processing device, which is combined with Figure 5 , comprising:

[0090] The acquisition module 401 is configured to acquire feedback information of the product data from the customers, and the feedback information at least includes customer evaluation, transaction behavior data, hot-selling products and anchor click volume.

[0091] Specifically, in the embodiment of the present application, the feedback information of the product data from the customers is acquired by the Jaeger tracker, and the acquisition method is described in detail below.

[0092] The aggregation module 402 is configured to aggregate the feedback information after synonym and similarity feature extraction to obtain aggregated feature data, and the aggregated feature data at least includes a customer evaluation set, a transaction behavior set and a user access behavior set.

[0093] Specifically, in the embodiment of the present application, the aggregated feature data is obtained by the aggregation module 402 after the feedback information is aggregated after synonym and similarity feature extraction, which aggregates some fuzzy and synonymous data to improve the effectiveness of data analysis.

[0094] The determination module 403 is configured to determine the emphasis of the research and development direction from the aggregated feature data.

[0095] Specifically, in the embodiment of the present application, the determination module 403 guides the research and development by analyzing the keyword with the most occurrences in the aggregated feature data set as the emphasis of the research and development direction.

[0096] The construction module 404 is configured to combine the collection, cleaning, processing and application of the data corresponding to the emphasis of the research and development direction with business scenarios, product decisions, product definitions and research and development management to construct a digital integration main line.

[0097] The digital integration main line integrates the data of the whole life cycle of the product.

[0098] Specifically, in the embodiment of the present application, the construction module 404 combines the collection, cleaning, processing and application of the data corresponding to the emphasis of the research and development direction with business scenarios, product decisions, product definitions and research and development management to construct a digital integration main line, and the digital integration main line integrates the data of the whole life cycle of the product. The digital integration main line pushes the appropriate information to the appropriate person at the appropriate time point, in the appropriate occasion and form, thereby providing data support for the decision, production and research and development of the product and improving the data utilization rate.

[0099] In a third aspect, the embodiment of the present application further provides an electronic device, which comprises a processor and a memory.

[0100] The processor is configured to execute any one of the data integration processing methods described above by invoking programs or instructions stored in the memory.

[0101] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium storing programs or instructions, which cause a computer to execute any one of the data integration processing methods described above.

[0102] Figure 5 is a schematic block diagram of an electronic device provided by the embodiments of the present application.

[0103] As shown in ​ , the electronic device includes at least one processor 501, at least one memory 502 and at least one communication interface 503. The various components in the electronic device are coupled together by a bus system 504. The communication interface 503 is configured to perform information transmission between the electronic device and an external device. It can be understood that the bus system 504 is configured to realize the connection communication between the components. In addition to the data bus, the bus system 504 also includes a power bus, a control bus and a status signal bus. However, in order to clearly illustrate the present application, all the buses are marked as the bus system 504 in the ​ .

[0104] It can be understood that the memory 502 in the embodiments can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0105] In some embodiments, the memory 502 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system and an application program.

[0106] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program includes various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The programs for implementing any one of the data integration processing methods provided by the embodiments of the present application can be included in the application program.

[0107] In the embodiments of the present application, the processor 501 is configured to execute the steps of the embodiments of the data integration processing method provided by the embodiments of the present application by invoking the programs or instructions stored in the memory 502, specifically, the programs or instructions stored in the application program.

[0108] Obtaining feedback information of the product data from the customer, the feedback information at least including: customer evaluation, transaction behavior data, hot-selling product and anchor click volume;

[0109] The synonymy, similarity feature extraction of the feedback information is aggregated to obtain aggregated feature data, and the aggregated feature data at least includes a customer evaluation set, a transaction behavior set and a user access behavior set.

[0110] The focus of the research and development direction is determined from the aggregated feature data.

[0111] The data collection, cleaning, processing and application corresponding to the focus of the research and development direction are combined with business scenarios, product decision-making, product definition and research and development management to construct a digital integrated main line.

[0112] The data of the product full life cycle is integrated in the digital integrated main line.

[0113] Any of the data integration processing methods provided by the embodiments of the application can be applied to the processor 501 or implemented by the processor 501. The processor 501 can be an integrated circuit chip with signal capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 501 or the instruction in the form of software. The above processor 501 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a ready programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general processor can be a microprocessor or the processor can also be any conventional processor or the like.

[0114] The steps of any of the data integration processing methods provided by the embodiments of the application can be directly embodied as hardware decoding processor execution completion or combined execution completion by hardware and software units in the decoding processor. The software unit can be located in a random memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register and other mature storage media in the art. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502, and combines the hardware to complete the steps of a data integration processing method.

[0115] Those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means that it is within the scope of the application and forms different embodiments.

[0116] Those skilled in the art will appreciate that the description of various embodiments has been for illustrative purposes only, and that many changes and modifications can be made by those skilled in the art without departing from the scope of the application.

[0117] The above merely illustrates the technical solutions of the present application, but the protection scope of the present application is not limited to this. Any modification or replacement within the technical scope disclosed by the present application can be easily conceived by those skilled in the art, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data integration processing method characterized by, The method comprises the following steps: Obtaining customer feedback information on product data, the feedback information at least including customer evaluation, transaction behavior data, hot-selling products and anchor click volume; After synonymy and similarity feature extraction, the feedback information is aggregated to obtain aggregated feature data, the aggregated feature data at least including a customer evaluation set, a transaction behavior set and a user access behavior set; Determining the focus of the research and development direction from the aggregated feature data; The data corresponding to the focus of the research and development direction is collected, cleaned, processed and applied to build a digital integrated main line in combination with business scenarios, product decision-making, product definition and research and development management; The digital integrated main line integrates data of the whole product life cycle; After synonymy and similarity feature extraction, the feedback information is aggregated to obtain aggregated feature data, comprising: Extracting information from the feedback information to form a domain phrase through a keyword or a phrase; Statistical index features of the domain phrase are extracted; Synonyms are mined according to the index features; The semantic similarity of the mined synonyms and the corresponding keywords or phrases is calculated to determine whether the mined synonyms and the corresponding keywords or phrases are synonyms; The determined synonyms are aggregated to obtain aggregated feature data; The digital integrated main line includes multi-source heterogeneous data, selection of computing power nodes, data integration of the whole product life cycle and an operation chain composed of operators at each link; The multi-source heterogeneous data includes a real-time data warehouse based on iceberg data lake technology deployed on a central server to collect underlying data; The selection of computing power nodes includes selecting the optimal computing power node with sufficient idle computing power and the shortest path from the iceberg table format layer to the network computing power node network path; The data integration of the whole product life cycle and the operation chain composed of operators at each link include: Mapping a data stream to a process of transmitting data to the center from a local, and a process contains multiple threads; Mapping the data generated by the data integration of the whole product life cycle at each link to an operator, and composing an operation chain with operators at each link.

2. The data integration processing method of claim 1, wherein, The customer feedback information on product data is obtained through a Jaeger tracker, The Jaeger tracker includes a Jaeger client program, which sends sampled and embedded point data of multiple microservice instances to a Jaeger Agent through the API of OpenTracing; The sources of the feedback information at least include server-owned data, embedded point data and third-party data summary.

3. The data integration processing method of claim 1, wherein, The calculation of the semantic similarity of the mined synonyms and the corresponding keywords or phrases includes: Using the Glove model to obtain the word vectors of the mined synonyms and the corresponding keywords or phrases; Using the Euclidean dot product formula to calculate the cosine value of the angle between the two vectors of the mined synonyms and the corresponding keywords or phrases to measure the similarity between the mined synonyms and the corresponding keywords or phrases.

4. The data integration processing method of claim 1, wherein determining the focus of the R&D direction from the aggregated feature data comprises: analyzing a set in which a keyword or phrase appears the most in the aggregated feature data set as the focus of the R&D direction; and the collection, cleaning, processing, and application of data corresponding to the focus of the R&D direction are combined with business scenarios, product decisions, product definitions, and R&D management to build a digital integrated main line in a digital main line module in a digital data platform. comprises: an acquisition module configured to acquire feedback information of a customer on product data, the feedback information comprising at least customer evaluation, transaction behavior data, hot-selling products, and anchor click volume; 5. A data integration processing apparatus characterized by comprising: an aggregation module configured to aggregate the feedback information after synonym and similarity feature extraction to obtain aggregated feature data, the aggregated feature data comprising at least a customer evaluation set, a transaction behavior set, and a user access behavior set; a determination module configured to determine the focus of the R&D direction from the aggregated feature data; a construction module configured to combine the collection, cleaning, processing, and application of data corresponding to the focus of the R&D direction with business scenarios, product decisions, product definitions, and R&D management to build a digital integrated main line; wherein the digital integrated main line integrates data of a product full life cycle; the aggregation of the feedback information after synonym and similarity feature extraction comprises: extracting information from the feedback information by keywords or phrases to form domain phrases; statistically analyzing index features of the domain phrases; mining synonyms according to the index features; calculating semantic similarity of the mined synonyms and corresponding keywords or phrases to determine whether the mined synonyms and corresponding keywords or phrases are synonyms; aggregating the determined synonyms to obtain aggregated feature data; the digital integrated main line comprises: multi-source heterogeneous data, selection of computing power nodes, data integration of a product full life cycle, and an operation chain composed of operators at each link; the multi-source heterogeneous data comprises: a real-time data warehouse based on iceberg data lake technology deployed on a central server to collect underlying data; the selection of computing power nodes comprises: selecting a path with sufficient idle computing power and the shortest path from an iceberg table format layer to a network computing power node network as an optimal computing power node; and performing a full-process data stream processing and a power node distribution algorithm based on weight distribution through a Flink data engine and integration of Redis for a full process of data AI analysis; the data integration of a product full life cycle and the operation chain composed of operators at each link comprises: mapping a data stream to a process of transmitting data from a local to a central, and a process containing multiple threads; mapping data generated at each link of the data integration of a product full life cycle to an operator, and composing an operation chain with operators at each link. comprises: a processor and a memory; 6. An electronic device, comprising: the processor is configured to execute the data integration processing method according to any one of claims 1 to 4 by calling programs or instructions stored in the memory. ​ ​ 7. A computer readable storage medium characterized in that, The computer readable storage medium stores programs or instructions, which make the computer execute the data integration processing method as claimed in any one of claims 1 to 4.

Images