A unified multi-platform portal system based on Kafka technology

Abstract

The application relates to a unified multi-platform portal system based on Kafka technology, wherein a front end of the portal system comprises front end pages of a plurality of third party systems for display and interaction, a back end of the portal system comprises a proxy server, a portal service platform and corresponding message queues, the portal service platform acquires a service request from the front end pages of the third party systems through the proxy server and returns data, and the message queues based on Kafka technology serve registration and deregistration services from the third party systems; and the synchronization of user information of the third party systems is realized based on incremental synchronization. The application supports a plurality of third party systems as the front end of the portal system and realizes unification through the back end of the portal system; the Kafka technology is adopted to unify users, realize intelligent registration and deregistration, and support the display and arrangement of the front end of the portal system by the third party systems; and the incremental synchronization is adopted to realize the transmission of large files in slices, so that the effect and rate of file transmission are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital information transmission, and in particular to a multi-platform portal system based on Kafka technology. BACKGROUND

[0002] Portal originally refers to the entrance of a house, and now is used for the portal website of the Internet. The portal website refers to an application system that leads to a certain comprehensive Internet information resource and provides relevant information services. With the development of technology, industry portal websites, enterprise information portals and other concepts have also emerged.

[0003] With the development of society, on the basis of government department informatization construction, a cross-departmental and comprehensive business application system, i.e. a government portal website, is established, which enables the public, enterprises and office workers to quickly and conveniently understand the business applications, organizational content and information of relevant government departments, and obtain personalized services.

[0004] The government portal website serves the public, but there is a part of special roles, i.e. staff, who need to work together and obtain different operation permissions according to the differences in level and affiliation. For example, some provincial departments with special needs need to access each municipal subsystem through an interface to realize information viewing, and need to be managed and controlled by the backend of the portal website through intelligent registration and cancellation without the need of third-party system operation and management.

[0005] However, the current government portal website does not directly unify multiple third-party systems, but only unifies several sub-modules in the same system, and the current government portal website does not have unified registration, cancellation and user association functions, and does not have an expandable function of portal display arrangement, which cannot meet the existing application requirements.

[0006] Further, for a special portal website such as a government portal website, the interface of its user synchronization mechanism is not open to the outside, so it can only use devices for synchronization, and the synchronization form can only use encrypted files for synchronization. However, due to the limitations of device on file size and transmission rate, it leads to the inconvenience of data migration. SUMMARY

[0007] The present application solves the problems in the prior art and provides a multi-platform portal system based on Kafka technology.

[0008] The technical solution adopted by the present application is a multi-platform portal system based on Kafka technology, which comprises:

[0009] The portal system front end comprises front end pages of a plurality of third-party systems for display and interaction;

[0010] The portal system backend includes a proxy server, a portal service platform, and a corresponding message queue. The portal service platform obtains a business request of a front-end page from a third-party system through the proxy server and returns data. The message queue service registers and deregisters the business third-party system based on Kafka technology;

[0011] Incremental synchronization of user information of the third-party system to the portal service platform is implemented.

[0012] Preferably, the synchronization of the user information of the third-party system includes the following steps:

[0013] Step 1.1: An incremental file index table is created to record uploaded files of each business module in any third-party system.

[0014] Step 1.2: User information and attachment data of each business module in the project are collected and processed.

[0015] Step 1.3: A timing task incremental synchronization strategy is set. On the premise of evaluating the network environment, the file increment is pushed to the target server by the data source party based on the preset data segmentation rule. The evaluation of the network environment can be based on the judgment of the network bandwidth. The higher the bandwidth, the larger each volume will be.

[0016] Step 1.4: A timing task file loading strategy is set. When the target server detects the file pushing behavior, the integrity of the file is judged, and the file is loaded based on the judgment result until the complete file is obtained.

[0017] Step 1.5: The file is decompressed into the corresponding directory according to the path in the file name.

[0018] Preferably, in step 1.2, the return value parameters of the user added or modified interface or method are obtained, the type of the return value is judged, different strategies are adopted for analysis according to different return value types, unified user file information is obtained, and the corresponding path URL is generated. The URL is processed based on the module rule strategy to generate the complete directory and the corresponding file path in the server. The incremental file is generated according to the file path, and the incremental file information is recorded.

[0019] Preferably, the step 1.3 includes the following steps:

[0020] Step 1.3.1: The file synchronization task is obtained, and the last file synchronization completion time is obtained.

[0021] Step 1.3.2: The uploaded file record created after the last file synchronization completion time is obtained, and the data backup is obtained. Generally, the backup data can be deleted after the file synchronization is completed.

[0022] Step 1.3.3: Asymmetrically encrypt the original file with the public key to obtain an encrypted file and compress it;

[0023] Step 1.3.4: If the compressed encrypted file exceeds the threshold, determine the total number of volumes to be divided according to the file size modulo the threshold, divide the compressed encrypted file, and proceed to the next step, otherwise, directly push the compressed encrypted file to the target server and return;

[0024] Step 1.3.5: Create a temporary directory for the volume, divide the compressed encrypted file and output to obtain a set of volume files;

[0025] Step 1.3.6: Finally, push the volume file set to the target server one by one and record the push result information.

[0026] Preferably, the step 1.4 includes the following steps:

[0027] Step 1.4.1: The target server listens to the file synchronization message, if the file synchronization message is listened to, the synchronization file is loaded, all file volumes are obtained according to the file name prefix matching, otherwise step 1.4.1 is repeated;

[0028] Step 1.4.2: Create a temporary directory to generate an original compressed file with all file volumes;

[0029] Step 1.4.3: Decompress the compressed file to obtain an encrypted file, decrypt it using the private key of the asymmetric algorithm, and place the file in the corresponding directory;

[0030] Step 1.4.4: The target server parses the user file to generate corresponding JavaBean information and stores it in the database.

[0031] Preferably, any of the third-party systems registers with the portal system backend through a registration code, and any of the third-party systems logs out of the portal system backend through the registration code.

[0032] Preferably, any third-party system carries authentication parameters registration code, Kafka username and password, and request parameters to request authentication from Kafka, Kafka authenticates the username and password, returns the authentication result after authentication, and stores the request in Kafka;

[0033] The portal service of the portal system backend listens to Kafka, when there is a registration message, it obtains the registration code for registration code verification, and if the verification is passed, it obtains the request parameters, generates public and private keys for user information synchronization authentication, the portal system backend stores the public and private keys, and asynchronously returns the public key information to the third-party system through the public key callback interface URL, and the third-party system saves and stores it.

[0034] Preferably, during user synchronization, the third-party system queries local user information, records the query time as the parameter for the next user query time, and encrypts the user information with a public key. The encrypted user information is then sent to the Kafka service along with Kafka authentication. Kafka performs login authentication, and stores the encrypted user information after successful authentication.

[0035] The portal system backend listens for messages on the Kafka service. If a message is received, the encrypted user information is decrypted using the private key to obtain the user information. Based on the unique identifier, a new user is added. For users who are not bound to the portal system backend, the account is automatically bound. The user's unique identifier is associated with the ID assigned by the portal system backend.

[0036] Preferably, any third-party system carries CorpId, AppKey, AppSecret, portal access URL, public key callback interface URL, and Logo as request parameters.

[0037] Preferably, invoking the entire chain through the portal system frontend includes the following steps:

[0038] Step 2.1: Perform login authentication for the current portal system frontend;

[0039] Step 2.2: After the current portal system's front-end authentication is successful, the portal system retrieves the portal list information from the back-end and renders and displays it on the front end.

[0040] Step 2.3: Click on any third-party system and obtain the authentication code based on the CorpId; the front end of the third-party system initiates authentication to the back end of the third-party system based on the authentication code. Upon receiving the request, the back end of the third-party system first initiates a request to the third-party system based on the AppKey and AppSecret to obtain the authentication accessToken, and then obtains the user information based on the accessToken and the authentication code.

[0041] Step 2.4: Obtain the unique identifier corresponding to the user information of the third-party system backend based on the user information, and obtain the user information of the third-party system based on the mobile phone. If the user information of the third-party system can be found, the authentication is successful, and finally enter the corresponding portal system to complete the portal unification.

[0042] The application relates to a Kafka technology-based unified multi-platform portal system, wherein a front end of the portal system comprises front end pages of a plurality of third-party systems for display and interaction, a back end of the portal system comprises a proxy server, a portal service platform and corresponding message queues, the portal service platform acquires a service request from the front end pages of the third-party systems through the proxy server and returns data, and a message queue based on the Kafka technology is used to serve registration and deregistration services from the third-party systems; and user information of the third-party systems is synchronized based on incremental synchronization.

[0043] The application has the following advantages:

[0044] (1) A plurality of third-party systems are supported as the front end of the portal system, and unification is realized through the back end of the portal system.

[0045] (2) Kafka technology is used to unify users, realize intelligent registration and deregistration, and support the third-party systems in display and arrangement of the front end of the portal system.

[0046] (3) Incremental synchronization is used to realize the transmission of large files in slices, and the effect and rate of file transmission are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 FIG. 1 is a system structure diagram of the application;

[0048] Figure 2 FIG. 4 is a user information synchronization method flowchart of the third-party system in the application;

[0049] Figure 3 FIG. 5 is a method flowchart for calling the entire link through the front end of the portal system in the application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0051] The application relates to a Kafka technology-based unified multi-platform portal system, which unifies the application entrances of platforms of a plurality of third-party systems, and connects the platforms so that they have access permissions to each other, uses Kafka technology to connect the user information of the platforms, completes portal registration and portal deregistration, and supports portal display position arrangement.

[0052] In the application, Kafka is applied to the system as a high-throughput distributed publish-subscribe message system, can process all action flow data of consumers in the website, including but not limited to webpage browsing, search and other user actions, unified online and offline message processing through the parallel loading mechanism of Hadoop, and provides real-time messages through clustering.

[0053] The system comprises:

[0054] The portal system front end comprises front end pages of a plurality of third party systems for display and interaction;

[0055] The portal system back end comprises a proxy server nginx, a portal service platform and a corresponding message queue, the portal service platform acquires a business request from the front end page of the third party system through the proxy server and returns data, and registers and deregisters a business third party system using a message queue service based on Kafka technology;

[0056] The application realizes incremental synchronization of user information of the third party system to the portal service platform.

[0057] In the application, in order to ensure the applicability of the third party system, a timing heartbeat technology is generally used to monitor whether each portal system front end is online.

[0058] In the application, in order to guarantee the consistency of data and the real-time performance and reliability of synchronization, an incremental mode is used for user information synchronization, including file synchronization, and when the file is a large file, the large file is fragmented and volumeed, the fragments and volumes are combined into an original file at the receiving end, the data is finally parsed and stored, after synchronization to the target server, the volumes are combined to form the original file, and are placed in the corresponding directory.

[0059] In the implementation of the application, the synchronization of user information of the third party system comprises the following steps:

[0060] Step 1.1: creating an incremental file index table for recording uploaded files of each business module in any third party system;

[0061] Step 1.2: collecting and processing user information and attachment data of each business module in the project;

[0062] Here, a combination of an aspect (UploadFileAspect) and an annotation (UploadFileRecord) is used to collect the attachment data uploaded by each business module, wherein the annotation only acts on the uploaded file method, and the UploadFileAspect is the specific execution process of the annotation.

[0063] Specifically, in step 1.2, the return value parameter of the newly added or modified interface or method of the user is obtained, the type of the return value is judged, different strategies are taken for analysis according to different return value types such as String, javabean type, etc., unified user file information is obtained, and a corresponding path URL is generated, the URL is processed based on the module rule strategy, and the complete directory and corresponding file path in the server are generated;

[0064] For example, when the return value is of the String type, the file path of the return value is directly returned, that is, the return value is directly used as the file path; when the return value is only an object, the object attributes are id, file url, file name, etc., and the strategy for obtaining the file path is to parse the return value, generate the original object through the java reflection mechanism, and finally obtain the file url according to the object as the file path.

[0065] An incremental file is generated according to the file path, and the incremental file is stored in the form of a compressed file. Generally, the compressed file name is the file storage path, so that the target server can conveniently load the file and place it in the corresponding directory. The incremental file information includes but is not limited to the business module, the file name, the file storage path path, the compressed file path, the file upload time, the creator information, etc., so as to facilitate subsequent file synchronization.

[0066] Step 1.3: Set the incremental synchronization strategy of the timing task. Under the premise of evaluating the network environment, the file increment is pushed to the target server by the data source party based on the preset data segmentation rule.

[0067] The step 1.3 includes the following steps:

[0068] Step 1.3.1: Obtain the file synchronization task and obtain the last file synchronization completion time;

[0069] Step 1.3.2: Obtain the uploaded file record created after the last file synchronization completion time (the creation time needs to be later than the last synchronization completion time) from the t_upload_file_record table, and perform data backup to prevent loss;

[0070] Step 1.3.3: Perform asymmetric encryption on the original file with a public key to obtain an encrypted file and compress it to obtain a zip file package ASSET-fileName.zip;

[0071] Step 1.3.4: If the compressed encrypted file exceeds the threshold, determine the total number of volumes to be divided according to the file size and the threshold modulo, perform volume division processing on the compressed encrypted file, and proceed to the next step, otherwise, directly push the compressed encrypted file to the target server and return;

[0072] Step 1.3.5: A temporary directory ASSET-fileName-X is created for the split volume, the compressed encrypted file is split and output, and a split file set, ASSET-fileName.z1, ASSET-fileName.z2, etc., is obtained;

[0073] Step 1.3.6: Finally, the split file set is pushed to the target server one by one, and the pushing result information is recorded; generally, the retry number of pushing failure is 3 times.

[0074] Step 1.4: A file loading strategy is set by a timing task; when the target server detects a file pushing behavior, the integrity of the file is judged, and the file is loaded based on the judgment result until the complete file is obtained; that is, it is judged whether the file is a split and split volume file, if yes, the original file is generated after file split and split volume processing and then loaded, otherwise, it is directly loaded, and finally, it is decompressed according to the path in the file name and loaded into the corresponding directory.

[0075] The step 1.4 includes the following steps:

[0076] Step 1.4.1: The target server listens to the file synchronization message, if the file synchronization message is listened to, the synchronization file is loaded, all file volumes such as ASSET-fileName.z1, ASSET-fileName.z2, etc., are obtained according to the file name prefix matching, otherwise, step 1.4.1 is repeated;

[0077] Step 1.4.2: A temporary directory ASSET-fileName is created, and a zip split volume command is used to generate an original compressed file from all file volumes; in actual application, when the file synchronization is completed, the user record can be marked as completed file synchronization in the incremental file index table, at this time, the incremental file index table is synchronized to the third party system side, the third party system side only needs to process the marked user file which has been synchronized, until based on the index table, the third party system side has received all the split volumes, that is, all the file volumes, and does not need to be determined again;

[0078] Step 1.4.3: The compressed file is decompressed to obtain an encrypted file, the private key of the asymmetric algorithm is used for decryption, and the file is placed in the corresponding directory;

[0079] Step 1.4.4: The target server parses the user file to generate corresponding JavaBean information and stores it in the database.

[0080] Step 1.5: The file is decompressed to the corresponding directory according to the path in the file name.

[0081] In the application, the portal service platform of the portal system backend provides the following services:

[0082] Assign a portal registration code, that is, generate a portal registration code for the third-party system registration before the portal registration, and give the username and key of Kafka at the same time;

[0083] Portal registration, the third-party system carries the registration code to register the portal, and needs to upload CorpId, AppKey, AppSecret and save during registration, and can also upload the display logo, system name, etc.

[0084] Portal deregistration, the registered portal is deregistered, and the registration code needs to be carried for authentication and deregistration during deregistration;

[0085] User synchronization, integrate all user information of the third-party system in the portal service platform;

[0086] Portal management, manage the registered portal, such as modification, deletion, etc.

[0087] Portal arrangement, manage the portal system display position, style, logo, etc.

[0088] It needs to be pointed out in advance that in the system of the application, any third-party system carries CorpId, AppKey, AppSecret, portal access URL, public key callback interface URL and logo as request parameters, wherein the logo is in binary format or base64 string form.

[0089] Any of the third-party systems is registered to the portal system backend through the registration code, and any of the third-party systems is deregistered to the portal system backend through the registration code.

[0090] Specifically, any third-party system carries authentication parameter registration code, Kafka's username and password and request parameters to request authentication from Kafka, Kafka authenticates the username and password, returns the authentication result after authentication, and stores the request in Kafka for other services to consume; here, the authentication parameter registration code of each third-party system is independently assigned and different from each other;

[0091] The portal service of the portal system backend listens to Kafka. When there is a registration message, the message is processed in real time to obtain the registration code for registration code verification. If the verification is passed, the request parameters, which are CorpId, AppKey, AppSecret, and Logo, are obtained. CorpId, AppKey, and AppSecret are saved. The binary format or base64 string of Logo is converted into text, a file path is generated for storage, and the public key and private key for user information synchronization authentication are generated. The portal system backend stores the public key and private key, and asynchronously returns the public key information to the third-party system through the public key callback interface URL. The third-party system saves and stores it.

[0092] Obviously, the logout process of the third-party system is the same as the portal registration. The registration code and public key are carried to initiate a logout request. The private key can be obtained according to the registration code and public key, and the portal deletion logout operation is performed.

[0093] When synchronizing users, the third-party system queries local user information, which is an incremental query. The query time is recorded as the next user query time parameter. The public key is encrypted after obtaining the user information. The encrypted user information is sent to the Kafka service with Kafka authentication. Kafka performs login authentication, and stores the encrypted user information after authentication.

[0094] The portal system backend listens to Kafka service for messages. If there is a message, it is processed in real time. The encrypted user information is decrypted with the private key to obtain the user information. If the decryption is successful, the unique identification number such as the mobile phone number is used for addition judgment. If the user's mobile phone number exists, no further processing is performed. If it does not exist, the addition processing is performed. The user who has not bound the portal system backend is automatically bound to the account. The AppKey and AppSecret information are used to initiate a token request to the portal system backend. If it is successful, the token is returned. Then, the token and mobile phone number parameters are used to request the user interface of the portal system backend to obtain user information. If the authentication is passed, the user information is returned, which includes the ID of the portal system backend. The user's unique identification number is associated with the ID allocated by the portal system backend, so that the user has access to all portals, and the account can be used in all systems.

[0095] Users can arrange the interface of the portal system front end. According to the sub-portal system AppKey, the portal position is associated and bound. Finally, it is stored in the database. When rendering the portal system front end, the AppKey and portal position are configured and displayed to achieve the arrangement effect.

[0096] The entire link is called through the portal system front end, including the following steps:

[0097] Step 2.1: login authentication of the current portal system front end is performed;

[0098] Specifically, the portal system front end acquires an authentication code authCode from the portal system back end according to the CorpId, the portal system front end initiates authentication to the portal system back end according to the authCode, the portal system back end receives the request, first initiates a request to the third-party system according to the Appkey and Appsecret to acquire an authentication AccessToken, then acquires user information according to the AccessToken and the authCode, including the portal system back end ID, and finally acquires portal system back end user information such as a mobile phone number according to the portal system back end ID, and obtains portal user information according to the mobile phone number, if the portal user information can be queried, the authentication is passed, and finally enters the corresponding portal system.

[0099] Step 2.2: after the current portal system front end authentication is passed, the portal system back end is acquired to obtain portal list information, including but not limited to CorpId, portal name, portal URL, portal logo URL, front end rendering display is performed;

[0100] Step 2.3: click any third-party system, acquire an authentication code according to the CorpId; the third-party system front end initiates authentication to the third-party system back end according to the authentication code, the third-party system back end receives the request, first initiates a request to the third-party system according to the AppKey and Appsecret to acquire an authentication accessToken, then acquires user information according to the accessToken and the authentication code;

[0101] Step 2.4: according to the user information, a unique identifier corresponding to the third-party system back end user information is acquired, and the third-party system user information is obtained according to the mobile phone number, if the third-party system user information can be queried, the authentication is passed, and finally enters the corresponding portal system, and the portal unification is completed.

[0102] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) having computer-usable program code embodied therein.

[0103] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0104] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0106] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those of skill in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, the attached claims are intended to embrace all such variations and modifications as fall within the scope of the present application.

[0107] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A unified multi-platform portal system based on Kafka technology, characterized in that: The system comprises: A portal system front end comprising a plurality of front end pages of third party systems for display and interaction; A portal system back end comprising a proxy server, a portal service platform and a corresponding message queue, the portal service platform obtaining business requests from the front end pages of the third party systems through the proxy server and returning data, and the message queue service registering and deregistering business third party systems based on Kafka technology; Any of the third party systems registers with the portal system back end through a registration code, and any of the third party systems deregisters with the portal system back end through the registration code; Any third party system carries authentication parameters, a Kafka username and password and request parameters to request authentication from Kafka, Kafka authenticates the username and password, returns an authentication result after authentication is passed, and stores the request in Kafka; The portal service platform of the portal system back end listens to Kafka, obtains a registration code for registration code verification when there is a registration message, obtains request parameters after verification is passed, generates a public key and a private key for user information synchronization authentication of the third party system, the portal system back end stores the public key and the private key, and asynchronously returns the public key information to the third party system through a public key callback interface URL, and the third party system saves and stores it; When synchronizing users, the third party system queries local user information, records the query time as the next user query time parameter, and encrypts the public key after obtaining the user information; the encrypted user information is sent to the Kafka service with Kafka authentication, Kafka performs login authentication, and stores the encrypted user information after authentication is passed; The portal system back end listens to whether there is a message in the Kafka service, decrypts the encrypted user information with the private key if there is a message, obtains the user information, judges whether to add based on a unique identification number, and automatically binds the account of the user who has not bound the portal system back end by associating the user unique identification number with the ID allocated by the portal system back end; Incremental synchronization of user information of the third party system to the portal service platform is realized. 2.The unified multi-platform portal system based on Kafka technology according to claim 1, characterized in that: The synchronization of the user information of the third party system comprises the following steps: Step 1.1: Create an incremental file index table for recording uploaded files of each business module in any third party system; Step 1.2: Collect and process user information and attachment data of each business module in the project; Step 1.3: Set a timing task incremental synchronization strategy, under the premise of evaluating the network environment, based on the preset data segmentation rule, push the file increment to the target server by the data source party; Step 1.4: Set a timing task file loading strategy, when the target server detects file pushing behavior, perform integrity judgment on the file, load the file based on the judgment result, and obtain the complete file; Step 1.5: Decompress the file to the corresponding directory according to the path in the file name. 3.The unified multi-platform portal system based on Kafka technology according to claim 2, characterized in that: In step 1.2, the return value parameters of the user-added or modified interface or method are obtained, the type of the return value is judged, different strategies are adopted for analysis according to different return value types, unified user file information is obtained, and the corresponding path URL is generated, the URL is processed based on the module rule strategy, the complete directory and the corresponding file path in the server are generated; and the incremental file is generated according to the file path, and the incremental file information is recorded.

4. The unified multi-platform portal system based on Kafka technology according to claim 2, characterized in that: The step 1.3 includes the following steps: Step 1.3.1: Obtain the file synchronization task, and obtain the last file synchronization completion time; Step 1.3.2: Obtain the upload file record created after the last file synchronization completion time and need to be pushed, and backup the data; Step 1.3.3: Perform asymmetric encryption on the original file with a public key to obtain an encrypted file and compression; Step 1.3.4: If the compressed encrypted file exceeds the threshold, determine the total number of volumes to be divided according to the modulus of the file size and the threshold, perform volume division processing on the compressed encrypted file, and proceed to the next step, otherwise, directly push the compressed encrypted file to the target server and return; Step 1.3.5: Create a temporary directory for the volume, divide the compressed encrypted file and output to obtain a volume file set; Step 1.3.6: Finally, push the volume file set to the target server one by one, and record the push result information. 5.The unified multi-platform portal system based on Kafka technology according to claim 2, characterized in that: The step 1.4 includes the following steps: Step 1.4.1: The target server listens to the file synchronization message, if the file synchronization message is listened to, the synchronization file is loaded, all file volumes are obtained according to the file name prefix matching, otherwise step 1.4.1 is repeated; Step 1.4.2: Create a temporary directory to generate an original compressed file from all file volumes; Step 1.4.3: Decompress the compressed file to obtain an encrypted file, use the asymmetric algorithm private key to decrypt, and place the file in the corresponding directory; Step 1.4.4: The target server parses the user file to generate corresponding JavaBean information and stores it in the database. 6.The unified multi-platform portal system based on Kafka technology according to claim 1, characterized in that: Any third-party system carries CorpId, AppKey, AppSecret, portal access URL, public key callback interface URL and Logo as request parameters. 7.The unified multi-platform portal system based on Kafka technology according to claim 6, characterized in that: The entire link is called through the portal system front end, including the following steps: Step 2.1: Perform login authentication of the current portal system front end; Step 2.2: After the current portal system front end authentication passes, obtain the portal list information from the portal system back end, and perform front-end rendering and display; Step 2.3: Click any third-party system, obtain an authentication code according to CorpId; the portal system front end initiates authentication to the portal system back end according to the authentication code, the portal system back end receives the request, first initiates a request to the third-party system according to AppKey and AppSecret, obtains an authentication accessToken, and then obtains user information according to accessToken and authentication code; Step 2.4: Obtain the unique identification number corresponding to the user information of the portal system backend according to the user information, and obtain the user information of the third-party system according to the unique identification number, if the user information of the third-party system can be queried, then the authentication is passed, and finally enters the corresponding portal system to complete the portal unification.

Images