Data distribution method, data reception method and apparatus

Abstract

The application provides a data distribution method and device and a data receiving method, and relates to the technical field of big data. The method comprises the following steps: reading data required by at least one client from a database according to a preset data preprocessing timing task, preprocessing the data required by each client, and obtaining the processed data of each client after preprocessing. The data preprocessing timing task is used to instruct the server to extract data and perform preprocessing at regular intervals. According to a preset encryption algorithm, the processed data of each client is encrypted to obtain encrypted data of each client, and the encrypted data is cached in a local cache space. In response to a data download request initiated by any one or more clients, the corresponding encrypted data is extracted from the local cache space, and the encrypted data is distributed to the client. The method can effectively solve the problem of low transaction efficiency or slow application response caused by the client pulling data from the server multiple times.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a data distribution method, a data receiving method, and an apparatus. Background Technology

[0002] In a client / server (C / S) architecture, the client typically needs to initiate transactions to obtain the required data (such as configuration data, business data, etc.) from the server. Currently, the main approach is to synchronize data with the server through multiple transactions and pagination.

[0003] With the widespread adoption of cloud-native technologies, the number of clients has exploded. Upon initial client startup, a large amount of business data typically needs to be fetched from the server. Using methods such as multiple transactions or paginated data retrieval results in small amounts of data being fetched at once, leading to low transaction efficiency. Furthermore, in containerized scenarios, there is a possibility of thousands or even tens of thousands of clients starting simultaneously. Since clients need to wait for multiple data retrievals to complete, application startup time increases significantly, which has a significant negative impact on user experience, especially in application scenarios requiring rapid response. Summary of the Invention

[0004] This application provides a data distribution method, a data receiving method, and an apparatus to at least solve one of the above-mentioned technical problems.

[0005] Firstly, this application provides a data distribution method applied to a server, the method comprising:

[0006] According to the preset data preprocessing scheduled task, at least one client's required data is read from the database, and the data required by each client is preprocessed to obtain the preprocessed data for each client; wherein, the data preprocessing scheduled task is used to instruct the server to extract data and preprocess it at regular intervals, and the data includes business data and / or configuration data;

[0007] According to the preset encryption algorithm, the data processed by each client is encrypted to obtain the encrypted data of each client, and the encrypted data is cached in the local cache space;

[0008] In response to a data download request initiated by any one or more clients, the corresponding encrypted data is extracted from the local cache space and distributed to the client.

[0009] In one implementation, the data required by each client is preprocessed to obtain preprocessed data for each client, including:

[0010] For the data required by each client, the data is arranged and processed according to the data version characteristics to obtain the preprocessed data for each client;

[0011] The data version feature is determined by the server based on the data extraction batch when extracting the corresponding data from the database.

[0012] In one embodiment, the preset encryption algorithm includes the Chinese national cryptographic algorithm SM2 and the Chinese national cryptographic algorithm SM4; the method further includes:

[0013] For each client's processed data, a random SM4 encryption key is generated for the processed data;

[0014] The step of encrypting the processed data of each client according to a preset encryption algorithm to obtain the encrypted data of each client includes:

[0015] Based on the SM4 encryption key, the processed data of each client is encrypted to obtain the ciphertext data;

[0016] The SM4 encryption key is encrypted using the pre-acquired SM2 public key of the client to obtain the key ciphertext;

[0017] The encrypted data is obtained by assembling the encrypted data and the encrypted key into a digital envelope file.

[0018] In one implementation, the data download request carries the client's SM2 private key signature;

[0019] The step of retrieving the corresponding encrypted data from the local cache space in response to a data download request initiated by any one or more clients includes:

[0020] In response to a data download request initiated by any one or more clients, the identity information of the client is verified by signing with the SM2 private key, and the corresponding encrypted data is extracted from the local cache space when the client's authentication is successful.

[0021] In one implementation, the step of retrieving corresponding encrypted data from the local cache space and distributing the encrypted data to the client in response to a data download request initiated by any one or more clients includes:

[0022] When responding to data download requests initiated by multiple clients, if the multiple clients in the data download request belong to the same client cluster, then the encrypted data corresponding to any one of the multiple clients is extracted from the local cache space;

[0023] The encrypted data is synchronously distributed to the multiple clients.

[0024] In one embodiment, the method further includes:

[0025] The encrypted data is processed based on the national cryptographic algorithm SM3 to generate a hash value for the encrypted data, which is used to indicate the consistency of the encrypted data during transmission.

[0026] The encrypted data and the hash value are encapsulated to obtain the encapsulated encrypted data;

[0027] Distributing the encrypted data to the client includes: distributing the encapsulated encrypted data to the client.

[0028] Secondly, this application provides a data receiving method applied to a client, the method comprising:

[0029] Send a data download request to the server. The data download request carries the client's SM2 private key signature, which is used to instruct the server to verify the client's identity.

[0030] Receive encrypted data about the client distributed by the server in response to the data download request;

[0031] The encrypted data is decrypted to obtain processing data about the client, and relevant business processing is performed based on the processing data.

[0032] In one embodiment, the encrypted data includes data ciphertext and key ciphertext. The data ciphertext is obtained by encrypting the processed data using the SM4 encryption key, a Chinese national cryptographic algorithm, and the key ciphertext is obtained by encrypting the SM4 encryption key using the SM2 public key.

[0033] The process of decrypting the encrypted data to obtain processing data about the client includes:

[0034] The decryption key is obtained by decrypting the ciphertext of the key based on the pre-stored SM2 private key;

[0035] The encrypted data is decrypted using the SM4 encryption key to obtain processing data about the client.

[0036] Thirdly, this application provides a data distribution device applied to a server, the device comprising:

[0037] The reading and processing module is configured to read data required by at least one client from the database according to a preset data preprocessing timer task, and preprocess the data required by each client to obtain preprocessed processed data for each client; wherein, the data preprocessing timer task is used to instruct the server to extract data and preprocess it at regular intervals, and the data includes business data and / or configuration data;

[0038] The encryption module is configured to encrypt the processed data of each client according to a preset encryption algorithm, obtain the encrypted data of each client, and cache the encrypted data in the local cache space.

[0039] The extraction and distribution module is configured to extract corresponding encrypted data from the local cache space and distribute the encrypted data to the client in response to a data download request initiated by any one or more clients.

[0040] In one embodiment, the reading and processing module includes:

[0041] An orchestration unit is configured to orchestrate the data required by each client according to the data version characteristics, so as to obtain preprocessed processed data for each client.

[0042] The data version feature is determined by the server based on the data extraction batch when extracting the corresponding data from the database.

[0043] In one embodiment, the preset encryption algorithm includes the Chinese national cryptographic algorithm SM2 and the Chinese national cryptographic algorithm SM4; the device further includes:

[0044] The key generation module is configured to randomly generate an SM4 encryption key for the processed data for each client.

[0045] The encryption module includes:

[0046] The data encryption unit is configured to encrypt the processed data of each client according to the SM4 encryption key to obtain ciphertext data;

[0047] A key encryption unit is configured to encrypt the SM4 encryption key based on the pre-acquired SM2 public key of the client to obtain key ciphertext;

[0048] An encrypted data generation unit is configured to assemble the data ciphertext and the key ciphertext into a digital envelope file to obtain the encrypted data.

[0049] In one implementation, the data download request carries the client's SM2 private key signature;

[0050] The extraction and distribution module includes:

[0051] The data extraction unit is configured to respond to a data download request initiated by any one or more clients, verify the identity information of the client by signing the SM2 private key, and extract the corresponding encrypted data from the local cache space when the client's authentication is successful.

[0052] In one implementation, the extraction and distribution module is specifically configured as follows:

[0053] When responding to data download requests initiated by multiple clients, if the multiple clients in the data download request belong to the same client cluster, then the encrypted data corresponding to any one of the multiple clients is extracted from the local cache space;

[0054] The encrypted data is synchronously distributed to the multiple clients.

[0055] In one embodiment, the device further includes:

[0056] A hash value generation module is configured to process the encrypted data based on the national cryptographic algorithm SM3 to generate a hash value for the encrypted data, the hash value being used to indicate the consistency of the encrypted data during transmission;

[0057] The encapsulation module is configured to encapsulate the encrypted data and the hash value to obtain encapsulated encrypted data;

[0058] The extraction and distribution module includes a data distribution unit, which is configured to distribute the encapsulated encrypted data to the client.

[0059] Fourthly, this application provides a data receiving device for use on a client, the device comprising:

[0060] The request module is configured to send a data download request to the server. The data download request carries the client's SM2 private key signature, which is used to instruct the server to verify the client's identity.

[0061] A receiving module is configured to receive encrypted data about the client distributed by the server in response to the data download request;

[0062] The decryption module is configured to decrypt the encrypted data to obtain processing data about the client, and perform relevant business processing based on the processing data.

[0063] In one embodiment, the encrypted data includes data ciphertext and key ciphertext. The data ciphertext is obtained by encrypting the processed data using the SM4 encryption key, a Chinese national cryptographic algorithm, and the key ciphertext is obtained by encrypting the SM4 encryption key using the SM2 public key.

[0064] The decryption module is specifically configured to: decrypt the ciphertext of the key based on the pre-stored SM2 private key to obtain the decryption key; and decrypt the ciphertext of the data based on the SM4 encryption key to obtain the processing data about the client.

[0065] Fifthly, this application provides a server, including: a memory and a processor;

[0066] The memory stores computer-executed instructions;

[0067] The processor executes computer execution instructions stored in the memory, causing the server to execute the data distribution method provided in any of the first aspects above.

[0068] Sixthly, this application provides a client, comprising: a memory and a processor;

[0069] The memory stores computer-executed instructions;

[0070] The processor executes computer execution instructions stored in the memory, causing the client to perform the data receiving method provided in any of the second aspects above.

[0071] In a seventh aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement a data distribution method as provided in any of the first aspects above, or to implement a data receiving method as provided in any of the second aspects above.

[0072] Eighthly, this application provides a computer program product comprising computer program code, which, when executed on a computer, causes the computer to perform a data distribution method as provided in any of the first aspects above, or to implement a data receiving method as provided in any of the second aspects above.

[0073] The data distribution method, data receiving method, and apparatus provided in this application involve the server reading data (including business data and / or configuration data) required by at least one client from a database according to a pre-set data preprocessing timer task. The server preprocesses the data required by each client to obtain preprocessed data for each client. The data preprocessing timer task instructs the server to periodically extract and preprocess data, and encrypts the processed data for each client according to a preset encryption algorithm, obtaining encrypted data for each client. This encrypted data is then cached in a local cache space. In response to a data download request initiated by any one or more clients, the server extracts the corresponding encrypted data from the local cache space and distributes it to the client. During this process, the server pre-queries and processes the corresponding data files for the client. The client can pull and assemble a large amount of data in a single transaction, greatly reducing the number of transactions and data readiness time required for the client's official startup. Furthermore, data encryption ensures data security and integrity during distribution among multiple clients. This effectively solves the problems of low transaction efficiency or slow application response caused by clients repeatedly pulling data from the server. Attached Figure Description

[0074] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0075] Figure 1 This is a schematic diagram of a possible scenario provided for an embodiment of this application;

[0076] Figure 2 A flowchart illustrating a data distribution method provided in an embodiment of this application;

[0077] Figure 3 This is a flowchart illustrating the encrypted data generation process in an embodiment of this application.

[0078] Figure 4 This is a flowchart illustrating the encrypted data decryption process in an embodiment of this application.

[0079] Figure 5 for Figure 2 A flowchart illustrating step S203;

[0080] Figure 6 A flowchart illustrating another data distribution method provided in an embodiment of this application;

[0081] Figure 7 A flowchart illustrating a data receiving method provided in an embodiment of this application;

[0082] Figure 8This is a schematic diagram of the structure of a data distribution device provided in an embodiment of this application;

[0083] Figure 9 This is a schematic diagram of the structure of a data receiving device provided in an embodiment of this application;

[0084] Figure 10 This is a schematic diagram of a server structure provided in an embodiment of this application;

[0085] Figure 11 This is a schematic diagram of the structure of a client provided in an embodiment of this application.

[0086] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0087] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0088] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals. It should be noted that certain software, components, models, and other existing solutions in the industry may be mentioned in the embodiments of this application. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but does not mean that the applicant has used or necessarily used such solutions.

[0089] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0090] Figure 1 The diagram illustrates an application scenario of the technical solution of this application embodiment. For example... Figure 1As shown, this application scenario includes: a server 110, a database 120 electrically connected to the server 110, and multiple clients 130. Optionally, the database can be a relational database management system (RDBMS), which can be used to store structured business data and configuration data, and supports complex queries and transaction processing. In practical applications, the specific database selection can be determined based on the application requirements and related technology stack. Optionally, the server 110 can pre-determine the data required by each client 130 based on historical request data from each client 130, or pre-requests from clients 130 (carrying information such as the required data type). It can then use a data preprocessing scheduled task to read the required data from the database 120 using structured query statements, perform corresponding processing such as duplicate and invalid data removal, and data arrangement, etc., and encrypt the processed data using national cryptographic algorithms. When a client 130 has a data download request, the server can provide the required data to the client 130 all at once. The client can include, but is not limited to, computers, smartphones, tablets, e-book readers, laptops, in-vehicle computers, wearable devices, desktop computers, set-top boxes, smart TVs, etc. Optionally, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0091] In contrast to related technologies where clients download data from the server through multiple transactions and pagination when they have download needs, this embodiment of the application involves the server reading data (including business data and / or configuration data) required by at least one client from the database according to a pre-set data preprocessing timer task. The server then preprocesses the data required by each client to obtain preprocessed data for each client. This data preprocessing timer task instructs the server to periodically extract and preprocess data, and encrypts the processed data for each client according to a preset encryption algorithm, obtaining encrypted data for each client. This encrypted data is cached in a local cache space. In response to data download requests initiated by any one or more clients, the server extracts the corresponding encrypted data from the local cache space and distributes the encrypted data to the client. During this process, the server pre-queries and processes the corresponding data files for the client, allowing the client to download and assemble a large amount of data in a single transaction. This significantly reduces the number of transactions and data readiness time required for the client's official startup. Furthermore, the data encryption ensures data security and integrity during distribution among multiple clients. This effectively solves the problem of low transaction efficiency or slow application response caused by the client repeatedly fetching data from the server.

[0092] based on Figure 1 As shown in the application scenario, this application provides a data distribution method. Applied to the server side, Figure 2 This is a flowchart illustrating the data distribution method provided in an embodiment of this application. Figure 2 As shown, the method may include steps S201-S203:

[0093] Step S201: According to the preset data preprocessing timed task, read the data required by at least one client from the database, and preprocess the data required by each client to obtain the preprocessed data of each client; wherein, the data preprocessing timed task is used to instruct the server to extract data and preprocess it at regular intervals, and the data includes business data and / or configuration data.

[0094] In this embodiment, a data preprocessing scheduled task is set on the server side. This scheduled task can be set to different time intervals according to business needs, such as once per hour, once per day, or once per week. The setting of the scheduled task can be implemented through programming, and a scheduled task scheduler (such as Quartz, Cron, etc.) is usually used to manage the execution time and frequency of the task.

[0095] When the scheduled task is triggered, the server begins the data extraction operation. Specifically, the server connects to the database and can use a structured query language to read the relevant data, such as client-related business data and / or configuration data, depending on the client's needs. During the data extraction process, the server can use different query conditions or filtering conditions according to the different client requirements to extract the corresponding data from the database.

[0096] After retrieving the relevant data, the server can preprocess it. Preprocessing methods can include one or more of the following: data cleaning to remove noise, duplicate data, or incomplete data, ensuring data quality; data transformation to convert the data into a unified format or structure for easier subsequent processing and analysis. For example, standardizing date formats or converting data from different units to the same unit; and data aggregation to summarize or group data according to business needs. For example, summarizing sales data by day, week, or month, or grouping statistical data by region or product category. Furthermore, data can be formatted based on version characteristics so that clients can retrieve the corresponding version (e.g., the latest version) of data in sequence.

[0097] For example, the preprocessing of the data required by each client in the above steps to obtain the preprocessed processing data for each client can be carried out in the following way: for the data required by each client, the data is arranged and processed according to the data version characteristics to obtain the preprocessed processing data for each client; wherein, the data version characteristics are determined by the server based on the data extraction batch when extracting the corresponding data from the database.

[0098] Optionally, during the data retrieval process from the database, the server records the data retrieval batches. This batch information serves as a data version identifier, with each batch having a unique version characteristic. Based on each client's needs and the data version characteristics, the retrieved data is organized and transformed into a format and content suitable for each client. This organization process generates preprocessed data. This data is customized for each client, ensuring that each client receives data that meets its specific requirements.

[0099] In this way, the server regularly collects and generates necessary configuration information and business data for the client, and arranges them reasonably according to data version characteristics, which can effectively ensure data order and integrity.

[0100] Step S202: According to the preset encryption algorithm, encrypt the processing data of each client to obtain the encrypted data of each client, and cache the encrypted data in the local cache space.

[0101] In this embodiment, the preset encryption algorithm can be a highly secure national cryptographic algorithm, including SM1-SM4 algorithms. The SM4 algorithm can be used to encrypt the processed data. SM4 is a block symmetric cipher algorithm with plaintext, ciphertext, and key lengths all being 128 bits. In some embodiments, besides using national cryptographic algorithms for data encryption, other encryption algorithms can also be used; this embodiment does not impose any particular limitations on this.

[0102] In one example, SM2 and SM4 can be used simultaneously to encrypt data to improve data security. Specifically, the preset encryption algorithm can be the Chinese national cryptographic algorithm SM2 and the Chinese national cryptographic algorithm SM4; the method may also include the following steps: for each client's processed data, randomly generate an SM4 encryption key for the processed data.

[0103] In this example, a random number generator or other secure algorithm can be used to generate a random SM4 encryption key (a symmetric key; SM4 is a symmetric encryption algorithm, meaning the same key is used for both encryption and decryption of data). That is, the generated encryption key is also the decryption key.

[0104] Next, in step S202 above, the processed data of each client is encrypted according to a preset encryption algorithm, specifically as follows: the processed data of each client is encrypted according to the SM4 encryption key to obtain data ciphertext; the SM4 encryption key is encrypted according to the pre-acquired SM2 public key of the client to obtain key ciphertext; the data ciphertext and the key ciphertext are assembled into a digital envelope file to obtain the encrypted data.

[0105] As is understandable, the SM2 algorithm is a public-key cryptography algorithm based on elliptic curve cryptography, and is an asymmetric encryption algorithm within the national cryptographic algorithms. It includes a public key and a private key, where the public key is used for encryption and the private key for decryption. The SM2 public and private keys can be generated using an encryption machine; the specific generation process can be found in existing technologies and will not be elaborated here. The encrypted data generation process is as follows: Figure 3 As shown, the server can encrypt the data using the national cryptographic SM4 algorithm through data interaction with the encryption machine, then encrypt the symmetric key using the client's SM2 public key, and finally assemble the data ciphertext and key ciphertext into a digital envelope file and store it on disk.

[0106] The above technical solution combines digital envelope files corresponding to symmetric and asymmetric encryption. These digital envelope files contain encrypted data and encryption key information, ensuring that only authorized recipients can decrypt and access the data. This effectively guarantees data security. Even if the encrypted data is intercepted during transmission, a client with the corresponding SM2 private key is required to decrypt the key ciphertext and thus decrypt the encrypted data.

[0107] In some embodiments, when caching encrypted data to the local cache space, each client's encrypted data can be identified to facilitate a quick response to the client's data download request and efficiently provide the corresponding data to the client.

[0108] See also Figure 2 Step S203: In response to a data download request initiated by any one or more clients, extract the corresponding encrypted data from the local cache space and distribute the encrypted data to the client.

[0109] When a client starts up or in any other state where it needs to request data from the server, it sends a data download request to the server. The server can then retrieve the previously prepared and cached encrypted data from its local cache based on the client's identifier or other client information, and distribute it to one or more clients at once. This allows clients to download and assemble large amounts of data in a single transaction, significantly reducing the number of transactions and data readiness time required for the client's official startup. In some scenarios, for the same group of clients, encrypted data can also be distributed to all of these client clusters at once to further improve data distribution efficiency.

[0110] Understandably, in response to, is used to indicate the conditions or states on which the operation being performed depends. When the conditions or states on which it depends are met, one or more operations being performed can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations being performed are executed.

[0111] In one example, to improve data transmission security, the data download request in this embodiment carries the client's SM2 private key signature. Step S203 above, responding to a data download request initiated by any one or more clients, involves retrieving the corresponding encrypted data from the local cache space, which may include the following steps:

[0112] In response to a data download request initiated by any one or more clients, the identity information of the client is verified by signing with the SM2 private key, and the corresponding encrypted data is extracted from the local cache space when the client's authentication is successful.

[0113] Specifically, the client can use its SM2 private key to sign its identity information. The calculated signature is compared with the signature provided by the client's SM2 private key to verify their consistency. If they match, the client's authentication is successful. The server retrieves the corresponding encrypted data from its local cache and distributes it to the corresponding client.

[0114] As a further description, the process by which the server provides data to the client in response to a data download request is as follows: Figure 4 As shown, when a client initiates a download request, the server first reads data from the cache. If the cache is empty, it reads data from the disk file and caches it, then signs the data and sends the response to the client. Upon receiving the response, the client first verifies the server's signature. If the verification is successful, it decrypts the data using its private key to obtain the symmetric key, then uses the symmetric key to decrypt the data and assemble it. Similarly, a cryptographic device can be used for private key signing during the data decryption process.

[0115] In this way, by signing messages and / or data, the client's identity can be verified, thereby providing access-level security conditions for data distribution.

[0116] In another example, step S203 above, in response to a data download request initiated by any one or more clients, retrieves the corresponding encrypted data from the local cache space and distributes the encrypted data to the client, such as... Figure 5 As shown, it may include the following steps:

[0117] Step S2031: When responding to data download requests initiated by multiple clients, if the multiple clients in the data download request belong to the same client cluster, then extract the encrypted data corresponding to any one of the multiple clients from the local cache space.

[0118] In scenarios involving real-time data updates or data configuration, the server requests the same data from clients within the same cluster. Current data distribution methods involve the server temporarily preparing and distributing data for each client upon initiating a download request. Even for clients within the same cluster, the server prepares and distributes data separately, significantly impacting distribution efficiency. This embodiment addresses this by identifying clients within the same cluster and retrieving the corresponding data from the cache, achieving one-time data distribution and dramatically improving efficiency.

[0119] For the same client cluster, these clients typically carry the same cluster ID. This embodiment can use this same cluster ID to identify whether clients belong to the same client cluster. Alternatively, it can also identify whether clients belong to the same client cluster based on network topology, IP address, etc.

[0120] Step S2032: The encrypted data is synchronously distributed to the multiple clients.

[0121] In some examples, the data required by the same cluster clients can be read and processed uniformly during the data reading phase to improve the efficiency of server-side data processing.

[0122] The above technical solution enables unified data distribution among clients in the same client cluster, improving data distribution efficiency.

[0123] Figure 6 This is a flowchart illustrating another data distribution method provided in this application embodiment. Based on the above embodiment, considering potential data integrity issues during data transmission, this embodiment combines the national cryptographic SM3 algorithm to generate hash values ​​to further improve data security. Specifically, in addition to the above steps S201-S203, the method provided in this embodiment may also include the following steps S601 and S602, and step S203 is further divided into step S2033.

[0124] Step S601: Process the encrypted data based on the national cryptographic algorithm SM3 to generate a hash value for the encrypted data, which is used to indicate the consistency of the encrypted data during transmission.

[0125] The SM3 algorithm, a Chinese national cryptographic standard, is a cryptographic hash function that accepts input data of arbitrary length and generates a fixed-length hash value of 256 bits (32 bytes). The hash value is unique; any tiny change in the input will result in a completely different output, similar to SHA-256. It is primarily used to generate fixed-length hash values ​​to ensure data integrity and security. This embodiment uses the SM3 algorithm to calculate the corresponding hash value for encrypted data. This hash value indicates the original data state on the server side before data transmission. If no data loss occurs during transmission, the hash value calculated for the encrypted data received by the client will match the hash value calculated on the server side.

[0126] Step S602: Encapsulate the encrypted data and the hash value to obtain encapsulated encrypted data, and replace the encrypted data in the cache space with the encapsulated encrypted data.

[0127] Encapsulation can be understood as a simple splicing or combination of data, that is, carrying the corresponding hash value within encrypted data. It can also be the formation of a new data structure from data; the encapsulated data can contain a combination of encrypted data and hash values ​​to facilitate subsequent transmission or storage.

[0128] In some embodiments, after generating encrypted data, the server may not store the encrypted data, but instead directly calculate the hash value, encapsulate it, and store the encapsulated data in response, thus eliminating the need for data replacement.

[0129] Step S603: In response to a data download request initiated by any one or more clients, extract the corresponding encapsulated encrypted data from the local cache space and distribute the encapsulated encrypted data to the client.

[0130] After receiving the encapsulated encrypted data, the client calculates the hash value of the received encrypted data and compares it with the received hash value. If they match, it means that the encrypted data has not been attacked or has not been incompletely transmitted. The client can then continue to decrypt the encrypted data and perform subsequent processing, effectively improving data security and integrity.

[0131] Figure 7 This is a flowchart illustrating a data receiving method provided in an embodiment of this application, applied to a client. The method includes steps S701-S703:

[0132] Step S701: Send a data download request to the server. The data download request carries the client's SM2 private key signature, which is used to instruct the server to verify the client's identity.

[0133] Step S702: Receive encrypted data about the client distributed by the server in response to the data download request;

[0134] Step S703: Decrypt the encrypted data to obtain processing data about the client, and perform relevant business processing based on the processing data.

[0135] In one embodiment, the encrypted data includes data ciphertext and key ciphertext. The data ciphertext is obtained by encrypting the processed data using the SM4 encryption key, a Chinese national cryptographic algorithm, and the key ciphertext is obtained by encrypting the SM4 encryption key using the SM2 public key.

[0136] Step S703 above decrypts the encrypted data to obtain processing data about the client, and may include the following steps:

[0137] The decryption key is obtained by decrypting the ciphertext of the key based on the pre-stored SM2 private key;

[0138] The encrypted data is decrypted using the SM4 encryption key to obtain processing data about the client.

[0139] It should be noted that the method provided in the embodiments of this application is a peer execution method of the server, and its implementation principle and technical effect are similar to the data distribution method described above, and will not be repeated here.

[0140] Figure 8 This is a schematic diagram of a data distribution device provided in an embodiment of this application, applied to a server, such as... Figure 8 As shown, the device 800 includes a reading and processing module 801, an encryption module 802, and an extraction and distribution module 803; wherein,

[0141] The reading and processing module 801 is configured to read data required by at least one client from the database according to a preset data preprocessing timer task, and preprocess the data required by each client to obtain preprocessed processed data for each client; wherein, the data preprocessing timer task is used to instruct the server to extract data and preprocess it at regular intervals, and the data includes business data and / or configuration data;

[0142] The encryption module 802 is configured to encrypt the processed data of each client according to a preset encryption algorithm, obtain the encrypted data of each client, and cache the encrypted data in the local cache space.

[0143] The extraction and distribution module 803 is configured to extract corresponding encrypted data from the local cache space and distribute the encrypted data to the client in response to a data download request initiated by any one or more clients.

[0144] In one embodiment, the reading and processing module 801 includes:

[0145] An orchestration unit is configured to orchestrate the data required by each client according to the data version characteristics, so as to obtain preprocessed processed data for each client.

[0146] The data version feature is determined by the server based on the data extraction batch when extracting the corresponding data from the database.

[0147] In one embodiment, the preset encryption algorithm includes the Chinese national cryptographic algorithm SM2 and the Chinese national cryptographic algorithm SM4; the device further includes:

[0148] The key generation module is configured to randomly generate an SM4 encryption key for the processed data for each client.

[0149] The encryption module 802 includes:

[0150] The data encryption unit is configured to encrypt the processed data of each client according to the SM4 encryption key to obtain ciphertext data;

[0151] A key encryption unit is configured to encrypt the SM4 encryption key based on the pre-acquired SM2 public key of the client to obtain key ciphertext;

[0152] An encrypted data generation unit is configured to assemble the data ciphertext and the key ciphertext into a digital envelope file to obtain the encrypted data.

[0153] In one implementation, the data download request carries the client's SM2 private key signature;

[0154] The extraction and distribution module 803 includes:

[0155] The data extraction unit is configured to respond to a data download request initiated by any one or more clients, verify the identity information of the client by signing the SM2 private key, and extract the corresponding encrypted data from the local cache space when the client's authentication is successful.

[0156] In one embodiment, the extraction and distribution module 803 is specifically configured as follows:

[0157] When responding to data download requests initiated by multiple clients, if the multiple clients in the data download request belong to the same client cluster, then the encrypted data corresponding to any one of the multiple clients is extracted from the local cache space;

[0158] The encrypted data is synchronously distributed to the multiple clients.

[0159] In one embodiment, the device further includes:

[0160] A hash value generation module is configured to process the encrypted data based on the national cryptographic algorithm SM3 to generate a hash value for the encrypted data, the hash value being used to indicate the consistency of the encrypted data during transmission;

[0161] The encapsulation module is configured to encapsulate the encrypted data and the hash value to obtain encapsulated encrypted data;

[0162] The extraction and distribution module 803 includes a data distribution unit, which is configured to distribute the encapsulated encrypted data to the client.

[0163] It should be noted that the apparatus provided in the embodiments of this application can be used to execute the technical solution of the data distribution method in the above embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0164] Figure 9 This application provides a data receiving device, which is applied to a client, such as... Figure 9 As shown, the device includes a request module 901, a receiving module 902, and a decryption module 903, wherein...

[0165] The request module 901 is configured to send a data download request to the server. The data download request carries the client's SM2 private key signature, which is used to instruct the server to verify the client's identity.

[0166] The receiving module 902 is configured to receive encrypted data about the client distributed by the server in response to the data download request;

[0167] The decryption module 903 is configured to decrypt the encrypted data to obtain processing data about the client, and perform relevant business processing based on the processing data.

[0168] In one embodiment, the encrypted data includes data ciphertext and key ciphertext. The data ciphertext is obtained by encrypting the processed data using the SM4 encryption key, a Chinese national cryptographic algorithm, and the key ciphertext is obtained by encrypting the SM4 encryption key using the SM2 public key.

[0169] The decryption module 903 is specifically configured to: decrypt the key ciphertext based on the pre-stored SM2 private key to obtain the decryption key; and decrypt the data ciphertext based on the SM4 encryption key to obtain processing data about the client.

[0170] It should be noted that the apparatus provided in the embodiments of this application can be used to execute the technical solution of the data receiving method in the above embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0171] It should be understood that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented in software via processing element calls, while others are implemented in hardware. For example, the decryption module 903 can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its function can be called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0172] Figure 10 This is a schematic diagram of the server structure provided in an embodiment of this application. Figure 10 As shown, the server may include: transceiver 101, processor 102, and memory 103.

[0173] Processor 102 executes computer execution instructions stored in memory, causing processor 102 to execute the data distribution scheme in the above embodiments. Processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0174] The memory 103 is connected to the processor 102 via the system bus and completes communication between them. The memory 103 is used to store computer program instructions.

[0175] Transceiver 101 can be used to transmit data with external devices.

[0176] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0177] Figure 11 This is a schematic diagram of the server structure provided in an embodiment of this application. Figure 11 As shown, the server may include: transceiver 111, processor 112, and memory 113.

[0178] Processor 112 executes computer execution instructions stored in memory, causing processor 112 to execute the data receiving scheme in the above embodiments. Processor 112 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0179] The memory 113 is connected to the processor 112 via the system bus and completes communication between them. The memory 113 is used to store computer program instructions.

[0180] Transceiver 111 can be used to transmit data with external devices.

[0181] An embodiment of this application provides a client, including: a memory and a processor;

[0182] The memory stores computer-executed instructions;

[0183] The processor executes computer execution instructions stored in the memory, causing the client to perform the data receiving method provided in any of the second aspects above.

[0184] This application also provides a chip for executing instructions, which is used to execute the technical solutions of the data distribution method or data receiving method in the above embodiments.

[0185] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer causes the computer to perform the data distribution method or data reception technical solution described in the above embodiments.

[0186] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solutions of the data distribution method or data receiving method in the above embodiments.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0188] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0189] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0190] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0191] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0192] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0193] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0194] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0195] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0196] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A data distribution method, characterized in that, Applied to the server side, the method includes: According to the preset data preprocessing scheduled task, at least one client's required data is read from the database, and the data required by each client is preprocessed to obtain the preprocessed data for each client; wherein, the data preprocessing scheduled task is used to instruct the server to extract data and preprocess it at regular intervals, and the data includes business data and / or configuration data; According to the preset encryption algorithm, the data processed by each client is encrypted to obtain the encrypted data of each client, and the encrypted data is cached in the local cache space; In response to a data download request initiated by any one or more clients, the corresponding encrypted data is extracted from the local cache space and distributed to the client; The process of preprocessing the data required by each client to obtain preprocessed data for each client includes: For the data required by each client, the data is arranged and processed according to the data version characteristics and the client's needs to obtain the preprocessed data for each client; The data version feature is determined by the server based on the data extraction batch when extracting the corresponding data from the database; The step of responding to a data download request initiated by any one or more clients, retrieving the corresponding encrypted data from the local cache space, and distributing the encrypted data to the client includes: When responding to data download requests initiated by multiple clients, if the multiple clients in the data download request belong to the same client cluster, then the encrypted data corresponding to any one of the multiple clients is extracted from the local cache space; The encrypted data is synchronously distributed to the multiple clients.

2. The method according to claim 1, characterized in that, The preset encryption algorithm includes the Chinese national cryptographic algorithm SM2 and the Chinese national cryptographic algorithm SM4; the method further includes: For each client's processed data, a random SM4 encryption key is generated for the processed data; The step of encrypting the processed data of each client according to a preset encryption algorithm to obtain the encrypted data of each client includes: Based on the SM4 encryption key, the processed data of each client is encrypted to obtain the ciphertext data; The SM4 encryption key is encrypted using the pre-acquired SM2 public key of the client to obtain the key ciphertext; The encrypted data is obtained by assembling the encrypted data and the encrypted key into a digital envelope file.

3. The method according to claim 2, characterized in that, The data download request carries the client's SM2 private key signature; The step of retrieving the corresponding encrypted data from the local cache space in response to a data download request initiated by any one or more clients includes: In response to a data download request initiated by any one or more clients, the identity information of the client is verified by signing with the SM2 private key, and the corresponding encrypted data is extracted from the local cache space when the client's authentication is successful.

4. The method according to any one of claims 1-3, characterized in that, Also includes: The encrypted data is processed based on the national cryptographic algorithm SM3 to generate a hash value for the encrypted data, which is used to indicate the consistency of the encrypted data during transmission. The encrypted data and the hash value are encapsulated to obtain the encapsulated encrypted data; The step of distributing the encrypted data to the client includes: distributing the encapsulated encrypted data to the client.

5. A data receiving method, characterized in that, Applied to a client, the method includes: A data download request is sent to the server, enabling the server to respond to data download requests initiated by any one or more clients. The server retrieves the corresponding encrypted data from its local cache space. The data download request carries the client's SM2 private key signature, which instructs the server to verify the client's identity. Specifically, when the server responds to data download requests from multiple clients, if these clients belong to the same client cluster, the server retrieves the encrypted data corresponding to any one of the multiple clients from its local cache space and synchronously distributes the encrypted data to all clients. The server receives encrypted data about the client in response to the data download request. The encrypted data is decrypted to obtain processing data about the client, and relevant business processing is performed based on the processing data. The processing data is obtained by the server reading data required by at least one client from the database according to a preset data preprocessing timer task, and arranging the data according to the data version characteristics and client requirements for each client's required data. The data version characteristics are determined by the server based on the data extraction batch when extracting corresponding data from the database. The data preprocessing timer task is used to instruct the server to extract data and perform preprocessing at regular intervals.

6. The method according to claim 5, characterized in that, The encrypted data includes data ciphertext and key ciphertext. The data ciphertext is obtained by encrypting the processed data using the SM4 encryption key, a national cryptographic algorithm, and the key ciphertext is obtained by encrypting the SM4 encryption key using the SM2 public key. The process of decrypting the encrypted data to obtain processing data about the client includes: The decryption key is obtained by decrypting the ciphertext of the key based on the pre-stored SM2 private key; The encrypted data is decrypted using the SM4 encryption key to obtain processing data about the client.

7. A data distribution device, characterized in that, Applied to the server side, the device includes: The reading and processing module is configured to read data required by at least one client from the database according to a preset data preprocessing timer task, and preprocess the data required by each client to obtain preprocessed processed data for each client; wherein, the data preprocessing timer task is used to instruct the server to extract data and preprocess it at regular intervals, and the data includes business data and / or configuration data; The encryption module is configured to encrypt the processed data of each client according to a preset encryption algorithm, obtain the encrypted data of each client, and cache the encrypted data in the local cache space. The extraction and distribution module is configured to extract corresponding encrypted data from the local cache space and distribute the encrypted data to the client in response to a data download request initiated by any one or more clients. The reading and processing module includes: An orchestration unit is configured to orchestrate the data required by each client based on data version characteristics and client needs, thereby obtaining preprocessed processed data for each client; wherein, the data version characteristics are determined by the server based on the data extraction batch when retrieving corresponding data from the database; The extraction and distribution module is specifically configured to: when responding to data download requests initiated by multiple clients, if the multiple clients in the data download request belong to the same client cluster, extract the encrypted data corresponding to any one of the multiple clients from the local cache space; The encrypted data is synchronously distributed to the multiple clients.

8. A data receiving device, characterized in that, Applied to a client, the device includes: The request module is configured to send a data download request to the server, enabling the server to respond to data download requests initiated by any one or more clients by retrieving corresponding encrypted data from its local cache. The data download request carries the client's SM2 private key signature, which instructs the server to verify the client's identity. Specifically, when the server responds to data download requests from multiple clients, if these clients belong to the same client cluster, the server retrieves encrypted data corresponding to any one of the clients from its local cache and synchronously distributes the encrypted data to all clients. A receiving module is configured to receive encrypted data about the client distributed by the server in response to the data download request; The decryption module is configured to decrypt the encrypted data to obtain processing data about the client, and perform relevant business processing based on the processing data. The processing data is obtained by the server reading data required by at least one client from the database according to a preset data preprocessing timer task, and arranging the data according to the data version characteristics and client requirements for each client's required data. The data version characteristics are determined by the server based on the data extraction batch when extracting corresponding data from the database. The data preprocessing timer task is used to instruct the server to extract data and perform preprocessing at regular intervals.

9. A server-side component, characterized in that, include: Memory and processor; The memory stores computer-executed instructions; The processor executes the computer execution instructions stored in the memory, causing the server to perform the data distribution method according to any one of claims 1-4.

10. A client, characterized in that, include: Memory and processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the client to perform the data receiving method according to any one of claims 5 or 6.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data distribution method as described in any one of claims 1-4, or to implement the data receiving method as described in any one of claims 5 or 6.

12. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the data distribution method as described in any one of claims 1-4, or to implement the data receiving method as described in any one of claims 5 or 6.

Images