Data processing method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a data processing method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining temporary path information sent by a first client; the temporary path information is path information corresponding to a temporary path for temporarily storing a target data file uploaded by the first client; determining a target path for storing the target data file, and moving the target data file stored in a temporary path corresponding to the temporary path information to the target path; target metadata information of the target data file is determined, and a target metadata file including the target metadata information is generated; the target metadata information comprises path information of the target path; and processing a subsequent data access request according to the target metadata file. According to the invention, the server does not need to analyze the data file, the newly added data can be stored, and the resource consumption of the server can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a data processing method, apparatus, computer device, and storage medium. Background Art

[0002] In existing BS (Browser / Server) and CS (Client / Server) architectures, during the communication and interaction process between the client (a browser is also a type of client) and the server, it is inevitable to perform format conversion and adaptation on the interaction data. For example, in the way of interacting and communicating data through HTTP (Hyper Text Transfer Protocol), usually the format of the data submitted by the client cannot be directly used by the server database and needs to go through certain parsing, conversion, and adaptation to be recognized by the server and then stored using database technology; conversely, when the client obtains data from the server, it also needs to query the data on the server through the database engine and then convert and adapt it into a format that the client can recognize.

[0003] In this communication and interaction mode, obvious delays will occur in scenarios with large amounts of data and complex conversion calculations, and resource consumption will also increase. Summary of the Invention

[0004] In view of this, the present invention provides a data processing method, apparatus, computer device, and storage medium to solve the problem of large communication resource consumption of the server.

[0005] In a first aspect, the present invention provides a data processing method applied to a server, including:

[0006] Obtaining temporary path information sent by a first client; the temporary path information is the path information corresponding to a temporary path for temporarily storing a target data file uploaded by the first client, and the format of the target data file is a preset format uniformly used between the client and the server;

[0007] Determining a target path for storing the target data file, and moving the target data file stored in the temporary path corresponding to the temporary path information to the target path;

[0008] Determining target metadata information of the target data file, and generating a target metadata file including the target metadata information; the target metadata information includes the path information of the target path;

[0009] Processing subsequent data access requests according to the target metadata file.

[0010] In some alternative embodiments, determining the target path for storing the target data file includes:

[0011] Obtaining the file information of the target data file sent by the first client;

[0012] Calculating the target path for storing the target data file according to the file information of the target data file.

[0013] In some alternative embodiments, generating the target metadata file including the target metadata information includes:

[0014] Generating the target metadata file corresponding to the target data object while retaining the historical metadata file corresponding to the target data object; the target data object is the data object corresponding to the target path, and the target metadata file includes the target metadata information and the historical metadata information of the historical data file in the historical metadata file;

[0015] After the target data file is moved to the target path and the target metadata file is successfully generated, switching the metadata file of the target data object from the historical metadata file to the target metadata file.

[0016] In some alternative embodiments, switching the metadata file of the target data object from the historical metadata file to the target metadata file includes:

[0017] Determining the file path of the target metadata file;

[0018] Updating the metadata pointer of the target data object to the file path of the target metadata file.

[0019] In some alternative embodiments, generating the target metadata file corresponding to the target data object while retaining the historical metadata file corresponding to the target data object includes:

[0020] Generating the target list file of the target data file; the target list file includes the target metadata information;

[0021] Generating a target list file; the target list file includes the target list file and each historical list file corresponding to the latest historical list file in the historical metadata file;

[0022] While retaining the historical metadata file, adding the target list file to the historical metadata file to obtain the target metadata file corresponding to the target data object.

[0023] In some alternative embodiments, after the target data file is moved to the target path and the target metadata file is successfully generated, switching the metadata file of the target data object from the historical metadata file to the target metadata file includes:

[0024] When the first client uploads multiple target data files, after each target data file is moved to the corresponding target path and the target metadata file corresponding to each target data file is successfully generated, synchronously switch the metadata file of the target data object corresponding to each target data file from the corresponding historical metadata file to the target metadata file.

[0025] In some alternative embodiments, processing subsequent data access requests according to the target metadata file includes:

[0026] When the data access request is for the second client to access the target data file, determine the target path where the target data file is stored according to the target metadata file;

[0027] Return the path information of the target path to the second client, instructing the second client to obtain the target data file from the target path according to the path information of the target path.

[0028] In a second aspect, the present invention provides a data processing method applied to a client. The method includes:

[0029] Convert the data to be transmitted into a target data file in a preset format; the preset format is a format uniformly used between the client and the server;

[0030] Upload the target data file and store the target data file in a temporary path;

[0031] Send the path information corresponding to the temporary path to the server, instructing the server to move the target data file from the temporary path to the corresponding target path and generate a target metadata file including target metadata information; the target metadata information includes the path information of the target path.

[0032] In some alternative embodiments, the method further includes:

[0033] Initiate a data access request for the data file to be accessed to the server;

[0034] Obtain the path information of the storage path determined by the server according to the metadata file for storing the data file to be accessed;

[0035] Obtain the data file to be accessed from the storage path according to the path information of the storage path;

[0036] Perform parsing processing on the data file to be accessed.

[0037] In a third aspect, the present invention provides a data processing device, which is applied to a server, and the device includes:

[0038] Obtain the temporary path information sent by a first client; the temporary path information is the path information corresponding to the temporary path for temporarily storing the target data file uploaded by the first client, and the format of the target data file is a preset format uniformly used between the client and the server;

[0039] Determine the target path for storing the target data file, and move the target data file stored in the temporary path corresponding to the temporary path information to the target path;

[0040] Determine the target metadata information of the target data file, and generate a target metadata file including the target metadata information; the target metadata information includes the path information of the target path;

[0041] Process subsequent data access requests according to the target metadata file.

[0042] In a fourth aspect, the present invention provides a data processing device, which is applied to a client, and the device includes:

[0043] Convert the data to be transmitted into a target data file in a preset format; the preset format is a format uniformly used between the client and the server;

[0044] Upload the target data file and store the target data file in a temporary path;

[0045] Send the temporary path information corresponding to the temporary path to the server, instruct the server to move the target data file from the temporary path to the corresponding target path, and generate a target metadata file including target metadata information; the target metadata information includes the path information of the target path.

[0046] In a fifth aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method according to the first aspect, the second aspect or any corresponding embodiment thereof by executing the computer instructions.

[0047] In a sixth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, and the computer instructions are used to cause a computer to execute the data processing method according to the first aspect, the second aspect or any corresponding embodiment thereof described above.

[0048] In a seventh aspect, the present invention provides a computer program product, including computer instructions, and the computer instructions are used to cause a computer to execute the data processing method according to the first aspect, the second aspect or any corresponding embodiment thereof described above.

[0049] When the present invention submits data at the client side, it does not directly submit the data to the server side, but uses a data file in the same format of the client and the cloud to realize data transmission, and uploads the data file to a temporary path; the server side moves the data file to a suitable target path and updates the metadata file corresponding to the data file, then the new data can be stored in the database. The server side does not need to parse the data file, avoiding the performance overhead of data adaptation and conversion. In the case of a large amount of data and complex calculations, it can better balance the computing resources of the client side and the server side, thus avoiding the occurrence of situations such as return timeout and server side exception, and is particularly suitable for application platforms with a large amount of data storage and complex conversion calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related art, the following will briefly introduce the drawings required to be used in the description of the specific embodiments or the related art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0051] Figure 1 is a schematic diagram of the overall architecture of the client side and the server side according to an embodiment of the present invention;

[0052] Figure 2 is a schematic flowchart of the data processing method according to an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a data submission processing flow according to an embodiment of the present invention;

[0054] Figure 4 is a schematic flowchart of another data processing method according to an embodiment of the present invention;

[0055] Figure 5 is a schematic diagram of data weaving according to an embodiment of the present invention;

[0056] Figure 6 is a schematic diagram of the principle of multi-table transaction guarantee according to an embodiment of the present invention;

[0057] Figure 7 is a schematic diagram of the data acquisition and processing flow according to an embodiment of the present invention;

[0058] Figure 8 is a structural block diagram of the server-side data processing device according to an embodiment of the present invention

[0059] Figure 9 is a structural block diagram of the client-side data processing device according to an embodiment of the present invention;

[0060] Figure 10 is a schematic diagram of the hardware structure of the computer device according to an embodiment of the present invention. Detailed implementation manners

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0062] In the currently existing platform software architectures, they can generally be divided into two categories: BS and CS. Among them, in the BS architecture, the client is responsible for the display of the interface, the reception of user input, and then sends the data to the server by means of a network request. The server is mainly responsible for processing business logics, including data processing, calculation, storage, etc., and returns the results to the client for display. The BS architecture usually simplifies the processing logic of the client, making the client lightweight. Common applications of the BS architecture include websites accessed by browsers, Web-based management systems, etc.

[0063] In the CS architecture, the client is not only responsible for displaying the interface and user interaction, but also processes part of the business logic on the client side, that is, both the client and the server process a part of the business logic, and the client and the server communicate with each other through the network. Common applications of the CS architecture include desktop applications, mobile applications, game clients, etc.

[0064] However, whether it is the above CS architecture or BS architecture, they are all inseparable from data calculation, storage, and communication interaction. Generally speaking, the data calculation on the client side depends on the logical processing of the program, and the calculation on the server side depends on the service program and the database engine; for data storage, the client of the CS architecture generally does a small amount of data storage, and technically usually uses files in a specific format or a lighter database engine, while in the BS architecture, the client rarely does data storage; generally speaking, both CS and BS will save business data on the server side, and the server side uses common database services or big data storage services to store data.

[0065] In the current scenarios of BS and CS architectures, the communication interaction between the client and the server can be well adapted to requests with small data volume and relatively simple conversion and adaptation processing. However, once dealing with scenarios with large data volume and relatively complex conversion processing, problems such as significantly longer return time and request timeout will obviously occur in the interaction requests. Even worse, it may lead to abnormal situations such as insufficient computing resources on the server side.

[0066] Specifically, in scenarios with large data volume and complex conversion calculations, this communication interaction mode has the following disadvantages:

[0067] (1) Deteriorated user experience:

[0068] In scenarios with large data volume and relatively complex conversion processing, the server side needs to perform a large amount of data calculation and format conversion. At this time, problems such as significantly longer return time and increased user waiting time will obviously occur in the communication interaction requests between the client and the server side, greatly affecting the user experience;

[0069] (2) Risk of service crash:

[0070] With the further increase in the pressure of data volume and conversion processing, the communication interaction requests between the client and the server side will have the request waiting time exceeding the set maximum threshold, and the timeout will cause the service processing to not be completed normally; in a more serious situation, it will exhaust the CPU (Central Processing Unit) and memory resources of the server side, affecting the processing of other services, thus triggering the crash of the overall service;

[0071] (3) Increased operation and maintenance costs:

[0072] To deal with the existing situation of request timeout or resource shortage, usually, methods such as increasing server resources and increasing distributed deployment instances of the server side are adopted to alleviate the occurrence of abnormal situations. However, the subsequent problems are the increase in service resource costs, the increase in the management complexity of service instances, etc., thus leading to the increase in operation and maintenance costs.

[0073] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0074] The data processing method provided by the embodiment of the present invention is implemented based on a dual-homologous fusion architecture of a client and a server, that is, the client and the server use a computing engine that supports the same data format to calculate and store data files. In this embodiment, the client and the server use the same table format (Table Format) as the data format they adopt. As Figure 1 shown, the overall architecture of the client and the server can be divided into five levels from bottom to top: the object storage layer, the file layer, the table layer, the data directory layer, and the computing engine layer.

[0075] Object storage layer: That is, the location where the actual data files are stored. General extensible object storage technology is used. Optional technical solutions include object storage services provided by various cloud providers, open-source object storage, etc.; it is also possible to choose the storage of a general file system according to the actual deployment situation. Among them, object storage (Object Storage) is a storage architecture that processes, stores, and retrieves data in units of data objects. Object storage manages data as objects, and each data object contains the data itself, metadata, and a globally unique identifier. This unique storage method makes object storage have obvious advantages when processing large amounts of unstructured data.

[0076] File layer: The file storage format used to store data files. In order to obtain better file compression ratios and reading efficiencies, a file format that supports column-oriented storage is generally used, such as: Parquet format, ORC (Optimized Row Columnar) format, etc.

[0077] Table layer: The way data is organized according to management logic. Generally, a database engine technology that supports the corresponding format of the file layer is used to define the table layer in the form of creating a database table (Table).

[0078] Data directory layer: Each database table defined by the table layer is managed by specific metadata. The data directory layer provides the ability to manage the metadata that defines the relationship between the tables in the table layer and the data files in the corresponding file layer; for the storage of a large amount of data, according to the capabilities of the computing engine, it generally supports managing the directory storage of each partition separately according to the partition fields defined when the table is defined;

[0079] Computing Engine Layer: The client and the server use the same computing engine. Both the client and the server can directly parse and compute the files stored in the file layer. This computing engine can be, for example, an embedded computing engine, a batch processing computing engine, an MPP (Massively Parallel Processing) computing engine, a streaming computing engine, etc.; In the architecture provided in this embodiment, both the client and the server can directly identify and pull the stored files in the file layer, that is, the data format in the file layer can be parsed and recognized by the computing engines of all parties.

[0080] For example, if the file layer selects a Parquet format file with a columnar storage structure as the underlying storage structure, the client selects a desktop computing engine or a Web (web page) computing engine that can directly operate on Parquet format files, and the server also uses the same engine or alternatively selects a computing engine that can recognize the same file format.

[0081] In this embodiment, a data processing method is provided, which can be used for the above-mentioned server, such as a cloud server; Figure 2 It is a flowchart of the data processing method according to an embodiment of the present invention, as Figure 2 shown, this process includes the following steps S201 to step S204.

[0082] Step S201, obtain the temporary path information sent by the first client; The temporary path information is the path information corresponding to the temporary path for temporarily storing the target data file uploaded by the first client, and the format of the target data file is a preset format uniformly used between the client and the server.

[0083] In this embodiment, when the client needs to submit data to the server, it needs to submit a data file in a preset format uniformly used between the client and the server. For the convenience of description, this client is referred to as the first client.

[0084] Specifically, when the first client needs to submit data to the server, its data processing process includes the following steps A1 to step A3.

[0085] Step A1, convert the data to be transmitted into a target data file in a preset format.

[0086] In this embodiment, for the data to be transmitted, the first client converts the data into a data file (Data files) in the preset format based on the preset format pre-unified between the client and the server, that is, the target data file. For example, if the preset format is a table format, the target data file is a data file in table format.

[0087] Among them, to ensure the reliability of data transmission, the first client also needs to save the target data file. Specifically, if the first client is a desktop client, since the desktop client supports file writing, the target data file generated after corresponding processing can be saved to the local disk; if the first client is a Web client, since it has no disk read and write permissions, the target data file can be cached in memory at this time.

[0088] Step A2, upload the target data file and store the target data file in a temporary path.

[0089] The first client can directly upload the target data file and store the target data file in the free path of the server; generally, the target data file can be temporarily stored in the file layer. Among them, the file layer is only used to temporarily store the target data file at this time, so the path where the target data file is stored at this time is called the temporary path.

[0090] The temporary path can be an authorized directory determined in any way, as long as it is ensured that there is no conflict. Moreover, when the client submits data files with the same file identifier multiple times, or submits multiple data files simultaneously, it is also necessary to ensure that the temporary paths corresponding to each data file do not conflict.

[0091] Step A3, send the temporary path information corresponding to the temporary path to the server.

[0092] In this embodiment, after the first client completes the upload of the target data file, it sends the path information corresponding to the temporary path to the server, so that the server can perform subsequent processing on the target data file temporarily stored. For the convenience of description, the path information corresponding to the temporary path is called the temporary path information.

[0093] Figure 3 Shows a schematic flow diagram of data submission processing. As Figure 3 shown, the server includes a backend management service, a computing engine, a file layer for implementing file storage, etc. When the client (such as the first client) needs to submit data, it uses the local computing engine to process the data into a columnar target data file and uploads it to the temporary directory, and the path of this temporary directory is the temporary path. The client calls the data submission interface of the backend management service, and transmits the temporary path information of the temporary storage of the target data file, the file information of the target data file, etc. After the backend management service obtains the temporary path information, etc., it can execute the subsequent steps.

[0094] Step S202, determine the target path for storing the target data file, and move the target data file stored in the temporary path corresponding to the temporary path information to the target path.

[0095] In this embodiment, the server re-determines the path for storing the target data file, that is, the target path; after determining the target path, the server can move the target data file from the temporary path to the newly determined target path.

[0096] Optionally, as Figure 3 shown, the information submitted by the first client to the server may include, in addition to the temporary path information, the file information of the target data file, and the server can determine a suitable target path based on the file information. Specifically, step S202 "determine the target path for storing the target data file" may include the following steps B1 to B2.

[0097] Step B1, obtain the file information of the target data file sent by the first client.

[0098] Step B2, calculate the target path for storing the target data file according to the file information of the target data file.

[0099] In this embodiment, the first client may submit file information such as the file identifier (file ID) of the target data file; taking the target data file as a table file as an example, the file identifier may specifically be the table name. After the server obtains the file information, the backend management service calculates the target directory corresponding to the current target data file according to the submitted file information and the data file path rules agreed upon by the data directory layer and the table layer, and stores the target data file in the target directory, and the path of the target directory is the target path.

[0100] Among them, the data file is stored in the corresponding data object, and the data object may be, for example, a table (Table) for storing data. The backend management service can determine which data object the target data file should be stored in, that is, the target data object, according to the file information of the target data file, and then determine the corresponding target path based on the predefined rules. For example, it can be segmented based on the directory of the partition field, or segmented by date, or other rules can also be used, and this embodiment does not limit this.

[0101] Step S203, determine the target metadata information of the target data file, and generate a target metadata file including the target metadata information; the target metadata information includes the path information of the target path.

[0102] In this embodiment, the server determines the metadata information of the target data file, that is, the target metadata information. Specifically, the target metadata information at least includes the path information of the target path, which can represent the address of the target data file; the target metadata information may also include information such as the file format and record count of the target data file.

[0103] Generate a corresponding metadata file, i.e., a target data file, based on the target metadata information. The target metadata file includes at least the target metadata information.

[0104] Generally, each data object has a corresponding metadata file. Therefore, after adding the target data file for the target data object, adaptively update the metadata file of the target data object, and add the target metadata information of the target data file to the metadata file as well, then the target data file can be generated.

[0105] Step S204: Process subsequent data access requests according to the target metadata file.

[0106] In this embodiment, if the subsequent server receives a data access request from the client, it can respond based on the updated metadata file, that is, respond based on the target data file, so as to enable the client to access the data of the server.

[0107] Traditional data storage operations are generally implemented based on database insert operations. This requires parsing the data in the request or file first, organizing it into an insert statement of SQL (Structured Query Language), and then executing the SQL statement to complete the new data storage. In this embodiment, the client-server data formats are the same, that is, only data files with the same defined format are transmitted between the client and the server. When the client uploads data, the server only needs to move the data file (from the temporary path to the target path) and generate the corresponding target metadata file, so that the processing of new data storage can be completed.

[0108] In the data processing method provided in this embodiment, when the client submits data, it does not directly submit the data to the server, but uses the client-server homologous format data file to achieve data transfer, and uploads the data file to the temporary path; the server moves the data file to the appropriate target path and updates the metadata file corresponding to the data file, then the new data can be stored in the database. The server does not need to parse the data file, avoiding the performance overhead of data adaptation and conversion. In the case of a large amount of data and complex calculations, it can better balance the computing resources of the client and the server, thus avoiding situations such as return timeouts and server-side exceptions, and is particularly suitable for application platforms with large data storage and complex conversion calculations.

[0109] In this embodiment, another data processing method is provided, which can be used for the above-mentioned server, such as a cloud server; Figure 4 It is a flowchart of the data processing method according to the embodiment of the present invention, as Figure 4 shown. The process includes the following steps S401 to S404.

[0110] Step S401: Obtain the temporary path information sent by the first client; the temporary path information is the path information corresponding to the temporary path for temporarily storing the target data file uploaded by the first client, and the format of the target data file is the preset format uniformly used between the client and the server.

[0111] For details, please refer to Figure 2 Step S201 of the embodiment shown, which will not be elaborated here.

[0112] Step S402: Determine the target path for storing the target data file, and move the target data file stored in the temporary path corresponding to the temporary path information to the target path.

[0113] For details, please refer to Figure 2 Step S202 of the embodiment shown, which will not be elaborated here.

[0114] Step S403: Determine the target metadata information of the target data file, and generate a target metadata file including the target metadata information; the target metadata information includes the path information of the target path.

[0115] Specifically, the above step S403 "generate a target metadata file including the target metadata information" includes steps S4031 to S4032.

[0116] Step S4031: Generate the target metadata file corresponding to the target data object while retaining the historical metadata file corresponding to the target data object; the target data object is the data object corresponding to the target path, and the target metadata file includes the target metadata information and the historical metadata information of the historical data file in the historical metadata file.

[0117] In this embodiment, as described above, the target data file is stored in the corresponding target data object, and the path where the target data file is stored in the target data object is the target path, that is, the data object corresponding to the target path is the target data object.

[0118] Among them, before the client uploads the target data file, the target data object itself has a metadata file (Metadata File), that is, the historical metadata file, which records the metadata information of each data file in the target data object. For the convenience of description, the existing data files in the target data object are called historical data files, and the metadata information of the historical data file is called historical metadata information. It can be understood that similar to the target metadata information, the historical metadata information includes the path information for storing the corresponding historical data file.

[0119] When generating the metadata file of the target data object, based on the historical metadata information of each historical data file, the target metadata information of the target data file is added to generate the corresponding target data file. That is, the target data file includes not only the newly added target metadata information but also the previous metadata information, namely the historical metadata information of each historical data file.

[0120] Moreover, during the process of generating the target metadata file, the existing historical metadata files are also retained. In other words, after generating the target metadata file, the target data object corresponds to two metadata files, namely the historical metadata file and the target metadata file.

[0121] Step S4032: After the target data file is moved to the target path and the target metadata file is successfully generated, switch the metadata file of the target data object from the historical metadata file to the target metadata file.

[0122] In this embodiment, for the target data object, before switching the metadata file, the historical metadata file is valid, that is, the data access request is responded based on the historical metadata file. After the server obtains the target data file, the backend management service uses the metadata management capabilities provided by the data catalog layer middleware (for example, the data lake middleware, based on which tables and metadata can be organized and managed) to start data weaving and generate a new metadata file, that is, the target data file. Note that at this time, the switching of the metadata file is not processed, that is, the metadata file used by the target data object is still the historical metadata file. When the target data object is accessed, only the old version of the content, that is, the historical data file, can be obtained.

[0123] Among them, the data weaving technique refers to when a data file is newly added, using the characteristics of the data catalog layer and table layer middleware to manage tables and metadata files, only passing data files with the same definition format, and using the metadata file operation interface provided by the middleware to only perform file movement and adjustment of the metadata file, thus completing the processing of data addition and storage in the database.

[0124] If the target data file is moved to the target path and the target metadata file is successfully generated, that is, after data weaving is completed, then switch the metadata file of the target data object to the newly generated target metadata file. The data obtained by subsequent data access requests can be the latest data including the target data file.

[0125] If the target data file is not moved to the target path or the target metadata file is not generated, that is, an error occurs in the data file migration or metadata file generation, at this time, the metadata file switching is not performed, which is equivalent to achieving the effect of failure rollback, and continue to process the access request based on the historical metadata file to ensure the accuracy of transaction processing.

[0126] Optionally, the above step S4032, "switch the metadata file of the target data object from the historical metadata file to the target metadata file", specifically includes: determining the file path of the target metadata file; updating the metadata pointer of the target data object to the file path of the target metadata file.

[0127] In this embodiment, each data object has a corresponding metadata pointer, which points to the file path of the metadata file. By updating this metadata pointer, the switching of the metadata file can be achieved. Specifically, by updating the metadata pointer of the target data object from the file path of the historical metadata file to the file path of the target metadata file, the switching of the metadata file can be completed.

[0128] In some optional implementation manners, the above step S4031, "generate the target metadata file corresponding to the target data object while retaining the historical metadata file corresponding to the target data object", includes steps C1 to C3.

[0129] Step C1, generate the target list file of the target data file; the target list file includes the target metadata information.

[0130] Step C2, generate the target list file; the target list file includes the target list file and each historical list file corresponding to the latest historical list file in the historical metadata file.

[0131] Step C3, while retaining the historical metadata file, add the target list file to the historical metadata file to obtain the target metadata file corresponding to the target data object.

[0132] In this embodiment, the metadata file organization structure mainly includes three parts: the metadata file, the list file, and the manifest file. The metadata file stores the basic information about the data object (such as a table), such as the schema, partition information, and snapshot records of the data object. Each snapshot corresponds to a list file. The list file can also be called the manifest list, which contains one or more manifest files that record the metadata information corresponding to the data generated by the current operation, such as the address of the data file, the file format, the record count, and other information.

[0133] Specifically, the metadata of the data object is tracked based on this metadata file. Each change to the data object (such as adding a new data file) will generate a new metadata file, which can be specifically managed by the middleware service at the data directory layer for the metadata file corresponding to the data object. By utilizing the organizational structure characteristics and management characteristics of the metadata file in this embodiment, the new data can be added to the repository without operating and parsing the detailed data of the data file.

[0134] Taking the addition of a target data file as an example, a manifest file for recording the target metadata information can be generated, that is, the target manifest file, which records the path information, file format, etc. of the target data file. Then, the corresponding target list file and target metadata file are gradually generated, and then the metadata file is switched.

[0135] Taking the data object as a table as an example, Figure 5 shows a schematic diagram of data weaving, as Figure 5 shown. In the process of processing data submission, the process of data weaving includes Step.1 to Step.3.

[0136] Step.1: File movement. The target data file uploaded by the client to the temporary directory has the same format definition as the table table001, and this table table001 is the corresponding target data object. Then, the target data file is moved to the target directory of the data layer (such as the file layer) of table001 for storage, realizing the migration of the data file.

[0137] Step.2: Metadata file generation. Using the interface for metadata management provided by the middleware, new table metadata is generated, that is, the target metadata file; this step weaves the following three types of files:

[0138] Manifest file: A new manifest file is generated, that is, the target manifest file, and the file path, file format, record count, etc. of the newly moved target data file in the file layer directory of table001 are saved in this target manifest file.

[0139] List file: A new manifest list file is generated, that is, the target list file, which is a new snapshot, Figure 5 and S2 is used to represent this snapshot in it. This target list file combines the list information of the historical manifest file of the latest snapshot (i.e., snapshot S1) before the current submission of table001 and the newly generated target manifest file and adds them to the newly generated list file to obtain the target list file.

[0140] Metadata file: Generate a new metadata file, i.e., the target metadata file. Specifically, on the basis of the historical metadata file before submission, add a new snapshot record, i.e., snapshot S2, record the path of the detailed target list file of S2, and mark S2 as the latest snapshot; as Figure 5 shown, the historical metadata file contains snapshot S0 and snapshot S1, where snapshot S1 is the latest snapshot. After adding the new snapshot S2 to generate the target metadata file, the latest snapshot in the target metadata file is snapshot S2.

[0141] When generating the target metadata file, that is, when the processing in Step.2 is completed, the metadata pointer of table table001 in the data directory layer does not switch to the newly generated target metadata file. Therefore, for other requests at this time, the latest data of table table001 obtained is still the content of snapshot S1.

[0142] Step.3: Switch the metadata file. After the target data file is moved and the target metadata file is generated successfully, based on the data directory management, switch the metadata of table table001 to the newly generated target metadata file; at this time, the latest data corresponding to table table001 takes effect, and the data obtained by other requests during access is the latest data, that is, the data of snapshot S2.

[0143] The data weaving technology provided in this embodiment utilizes the organizational structure characteristics and management characteristics of the table metadata to complete the new data warehousing without operating and parsing the detailed data of the data file, which can effectively avoid the overhead of the server parsing and converting the data submitted by the client. Thus, the overhead of data submission processing is basically independent of the size of the data volume processed, so as to achieve the effect of rapid warehousing with a small amount of computing processing. Moreover, after the data file migration and the metadata file generation are completed, switching the metadata file can ensure the accuracy of transaction processing.

[0144] Optionally, if the first client uploads multiple target data files simultaneously, it is necessary to switch the metadata files of these target data files simultaneously. Specifically, the above step S4032 "After the target data file is moved to the target path and the target metadata file is successfully generated, switch the metadata file of the target data object from the historical metadata file to the target metadata file" specifically includes: in the case where the first client uploads multiple target data files, after each target data file is moved to the corresponding target path and the target metadata file corresponding to each target data file is successfully generated, synchronously switch the metadata files of the target data objects corresponding to each target data file from the corresponding historical metadata files to the target metadata files.

[0145] In this embodiment, when a client submits multiple data files (such as tables) at one time, it is necessary to ensure that all processing of the multiple data files is either completely successful or completely fails and rolls back. A scenario where some are successful and some are failed is not allowed, otherwise it will lead to data chaos. To achieve this effect, when processing the storage of a single data file, only control the computing engine to generate a new metadata file, but do not switch to the latest metadata file. Only after all data files have been migrated and new metadata files have been generated, switch the latest metadata files corresponding to all data files at one time to ensure the success of all data file processing and achieve the effect of all being successful. Similarly, when an error occurs during the file migration or new metadata file generation of any one data file, no metadata switching is performed on all data files to achieve the effect of all failing and rolling back.

[0146] Taking the data object as a table as an example, if the client submits the data files of table table001 and table table002 at one time, Figure 6 shows a schematic diagram of the principle of multi-table transaction guarantee, which is similar to the above Figure 5 process, and this process also includes Step.1 to Step.3.

[0147] Specifically, based on steps Step.1 and Step.2, first complete the movement of the data files (target data file 1 and target data file 2) of each table (table001, table002) and the generation of new metadata files (target metadata file 1 and target metadata file 2). Before all these two-step processes for all tables are successfully completed, the data directory management does not switch the metadata files of any table; at this time, the latest snapshots in the original version of the metadata are still used for reading each table, that is, snapshot S1 and snapshot S11 in the figure.

[0148] After all tables have successfully completed the generation of new metadata files, switch the metadata pointers of all corresponding tables to their newly generated metadata files at one time; this processing needs to make different adaptations according to the storage technology used for metadata management; for example, if a relational database is used for metadata pointer management, a transaction of the relational database needs to be started here to update the path fields of the current metadata files of multiple tables. When all table records are updated, commit the transaction to ensure all updates are successful; if the update of the path field of a current metadata file fails, all updates are also rolled back and this submission fails as a whole; when the switching of the metadata pointers of all tables is completed, the new metadata files take effect, and at this time, when other requests read table data, they will read the information of the latest snapshot, that is, Figure 6 snapshot S2 and snapshot S12 in

[0149] Step S404, process subsequent data access requests according to the target metadata file.

[0150] For details, please refer to Figure 2 Step S204 of the illustrated embodiment, which will not be elaborated here.

[0151] In some alternative embodiments, step S404 of "processing subsequent data access requests according to the target metadata file" may specifically include steps D1 to D2.

[0152] Step D1, when the data access request is for the second client to access the target data file, determine the target path where the target data file is stored according to the target metadata file.

[0153] Step D2, return the path information of the target path to the second client, instructing the second client to obtain the target data file from the target path according to the path information of the target path.

[0154] In this embodiment, the client can initiate a data access request for corresponding data to the server, that is, the data access request. For ease of description, this client is referred to as the second client.

[0155] Specifically, the second client does not transfer data in a single request interaction when obtaining data from the server, but obtains the path of the corresponding data file, and the second client completes it by the method of obtaining the data file through the path and then processing it.

[0156] It can be understood that the second client and the first client can be different clients or the same client. In other words, the first client can also perform data access in the above manner.

[0157] Figure 7 Shows a schematic diagram of the data acquisition processing flow, as Figure 7 As shown, the client submits a data access request to the backend management service. The data access request can be the file information of the data file to be accessed, such as the table name of the data file to be accessed, etc.; and, the data access request can also include data retrieval information such as filtering conditions to be able to quickly access the required data.

[0158] After receiving information such as the table name and filtering conditions, the backend management service uses the interface provided by the middleware of data directory management and table management to obtain the path information of the location where the corresponding data file is located from the metadata query using the table name and filtering conditions, and returns it to the client. Among them, the server can perform access authorization on this path, and after authorization, return the data file path information and authorization information to the client together; this process only searches through the metadata file and does not require the server to parse the data file for query.

[0159] For example, if the client accesses the target data file, it can submit the file information of the target data file (such as the table name, etc.), so that the server can query the target path where the target data file is stored and return the corresponding path information to the client.

[0160] After the client obtains the data file path information and authorization information, it can obtain the corresponding data file (such as the data file to be accessed) from the file layer based on this path information to the local client, and then the computing engine on the local client can be used to parse and process this data file.

[0161] In this way of data acquisition, the server will not need to read and convert the data, but only calculate the location where the data file is located. After the client receives the file path, it loads the data file to the local. Since the computing engines of the client and the server are of the same origin, the client can directly perform subsequent parsing and processing on the read data file using the local computing engine. For any data file acquisition request, the server only calculates the storage location of the data file, and the resource consumption is basically not affected by the size of the data volume and the complexity of data conversion.

[0162] In this embodiment, another data processing method is also provided, which can be used for the above-mentioned client. This method may specifically include:

[0163] Convert the data to be transmitted into a target data file in a preset format; the preset format is the format uniformly used between the client and the server;

[0164] Upload the target data file and store the target data file in a temporary path;

[0165] Send the temporary path information corresponding to the temporary path to the server, instruct the server to move the target data file from the temporary path to the corresponding target path, and generate a target metadata file including the target metadata information; the target metadata information includes the path information of the target path.

[0166] Among them, for the process of the client submitting the target data file to implement data processing, reference can be specifically made to the relevant descriptions in steps A1 to A3 above, and details will not be elaborated here.

[0167] Optionally, the client can also obtain data from the server. This process specifically includes the following steps:

[0168] Initiate a data access request for the data file to be accessed to the server;

[0169] Obtain the path information of the storage path used to store the data file to be accessed determined by the server according to the metadata file;

[0170] Obtain the data file to be accessed from the storage path according to the path information of the storage path;

[0171] Parse and process the accessed data file.

[0172] Among them, for the process of the client obtaining data from the server, refer to steps D1 to D2 and Figure 7 the relevant descriptions, which will not be elaborated here.

[0173] For the data processing method provided in this embodiment, the client and the server adopt a homologous architecture with the same computing engine to process the same data format, use unique data weaving and transaction guarantee technologies, and based on the transfer of files with the same data format, only move or copy the data files and operate the database metadata to achieve the effect of newly adding data to the database. When the client submits data to the server, the performance overhead of data adaptation and conversion is avoided. In addition, when the client obtains data from the server, the server obtains the location where the data file is located based on a small amount of computing overhead. The client obtains the data file through the file location and uses the engine homologous to the server to process the conversion and adaptation of the data, thereby further reducing the data adaptation and conversion overhead of the server. This homologous fusion architecture can improve the interaction efficiency between the client and the server, enhance the stability of the server system, and thus enable users to have a good experience. Moreover, it can greatly reduce the service resource cost, simplify the deployment architecture, and reduce the operation and maintenance cost.

[0174] In this embodiment, a data processing device is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be elaborated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0175] This embodiment provides a data processing device, which is applied to the server, as Figure 8 shown, and this device includes:

[0176] An acquisition module 801, configured to acquire temporary path information sent by a first client; the temporary path information is the path information corresponding to the temporary path for temporarily storing the target data file uploaded by the first client, and the format of the target data file is a preset format uniformly used between the client and the server;

[0177] A data migration module 802, configured to determine a target path for storing the target data file, and move the target data file stored in the temporary path corresponding to the temporary path information to the target path;

[0178] A processing module 803, configured to determine target metadata information of the target data file and generate a target metadata file including the target metadata information; the target metadata information includes path information of the target path.

[0179] A request response module 804, configured to process subsequent data access requests according to the target metadata file.

[0180] In some alternative embodiments, the data migration module 802 determines a target path for storing the target data file, including:

[0181] Obtaining file information of the target data file sent by the first client;

[0182] Calculating a target path for storing the target data file according to the file information of the target data file.

[0183] In some alternative embodiments, the processing module 803 generates a target metadata file including the target metadata information, including:

[0184] Generating a target metadata file corresponding to the target data object while retaining a historical metadata file corresponding to the target data object; the target data object is a data object corresponding to the target path, and the target metadata file includes the target metadata information and historical metadata information of historical data files in the historical metadata file.

[0185] After the target data file is moved to the target path and the target metadata file is successfully generated, switching the metadata file of the target data object from the historical metadata file to the target metadata file.

[0186] In some alternative embodiments, the processing module 803 switches the metadata file of the target data object from the historical metadata file to the target metadata file, including:

[0187] Determining a file path of the target metadata file;

[0188] Updating a metadata pointer of the target data object to the file path of the target metadata file.

[0189] In some alternative embodiments, the processing module 803 generates a target metadata file corresponding to the target data object while retaining a historical metadata file corresponding to the target data object, including:

[0190] Generating a target list file of the target data file; the target list file includes the target metadata information.

[0191] Generate a target list file; the target list file includes the target inventory file and each historical inventory file corresponding to the latest historical list file in the historical metadata file;

[0192] While retaining the historical metadata file, add the target list file to the historical metadata file to obtain the target metadata file corresponding to the target data object.

[0193] In some alternative embodiments, after the target data file is moved to the target path and the target metadata file is successfully generated, the processing module 803 switches the metadata file of the target data object from the historical metadata file to the target metadata file, including:

[0194] When multiple target data files are uploaded by the first client, after each target data file is moved to the corresponding target path and the target metadata file corresponding to each target data file is successfully generated, synchronously switch the metadata file of the target data object corresponding to each target data file from the corresponding historical metadata file to the target metadata file.

[0195] In some alternative embodiments, the request response module 804 processes subsequent data access requests according to the target metadata file, including:

[0196] When the data access request is for the second client to access the target data file, determine the target path storing the target data file according to the target metadata file;

[0197] Return the path information of the target path to the second client, instructing the second client to obtain the target data file from the target path according to the path information of the target path.

[0198] This embodiment provides a data processing device, which is applied to a client, as Figure 9 shown, the device includes:

[0199] A format conversion module 901, configured to convert the data to be transmitted into a target data file in a preset format; the preset format is a format uniformly used between the client and the server;

[0200] An upload module 902, configured to upload the target data file and store the target data file in a temporary path;

[0201] A sending module 903, configured to send the temporary path information corresponding to the temporary path to a server, instruct the server to move the target data file from the temporary path to a corresponding target path, and generate a target metadata file including target metadata information; the target metadata information includes the path information of the target path.

[0202] In some alternative embodiments, the apparatus further includes: a data acquisition module, configured to:

[0203] Initiate a data access request for a data file to be accessed to the server;

[0204] Obtain the path information of the storage path for storing the data file to be accessed determined by the server according to the metadata file;

[0205] Obtain the data file to be accessed from the storage path according to the path information of the storage path;

[0206] Perform parsing processing on the data file to be accessed.

[0207] The further function descriptions of the above-mentioned various modules and units are the same as those in the corresponding foregoing embodiments, and will not be elaborated herein.

[0208] The data processing apparatus in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, including a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0209] An embodiment of the present invention further provides a computer device having the above Figure 8 or Figure 9 shown data processing apparatus.

[0210] Please refer to Figure 10 , Figure 10 which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. As shown in Figure 10As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 10 A processor 10 is taken as an example.

[0211] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0212] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0213] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0214] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0215] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0216] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0217] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0218] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations should all be included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: Applied to the server, the method includes: Acquire temporary path information sent by the first client; the temporary path information is path information corresponding to a temporary path for temporarily storing a target data file uploaded by the first client, and the format of the target data file is a preset format uniformly used between the client and the server; Determine a target path for storing the target data file, and move the target data file stored in the temporary path corresponding to the temporary path information to the target path; Determine target metadata information of the target data file, and generate a target metadata file including the target metadata information; the target metadata information includes path information of the target path; Subsequent data access requests are processed according to the target metadata file.

2. The method according to claim 1, characterized in that The determining of a target path for storing the target data file comprises: Acquire file information of the target data file sent by the first client; A target path for storing the target data file is calculated according to the file information of the target data file.

3. The method according to claim 1, characterized in that The generating of the target metadata file including the target metadata information comprises: In the case of retaining the historical metadata file corresponding to the target data object, generating a target metadata file corresponding to the target data object; the target data object is a data object corresponding to the target path, and the target metadata file includes the target metadata information and the historical metadata information of the historical data file in the historical metadata file; After the target data file is moved to the target path and the target metadata file is successfully generated, the metadata file of the target data object is switched from the historical metadata file to the target metadata file.

4. The method according to claim 3, characterized in that The step of switching the metadata file of the target data object from the historical metadata file to the target metadata file includes: Determine the file path of the target metadata file; The metadata pointer of the target data object is updated to the file path of the target metadata file.

5. The method according to claim 3, characterized in that: The step of generating a target metadata file corresponding to the target data object while retaining the historical metadata file corresponding to the target data object comprises: Generate a target list file for the target data file; the target list file includes the target metadata information; Generate a target list file; the target list file includes the target list file and each history list file corresponding to the latest history list file in the history metadata file; In the case of retaining the historical metadata file, the target list file is added on the basis of the historical metadata file to obtain the target metadata file corresponding to the target data object.

6. The method according to any one of claims 3 to 5, characterized in that After the target data file is moved to the target path and the target metadata file is successfully generated, switching the metadata file of the target data object from the historical metadata file to the target metadata file comprises: In the case where the first client uploads multiple target data files, after each of the target data files is moved to the corresponding target path and the target metadata file corresponding to each of the target data files is successfully generated, the metadata file of the target data object corresponding to each of the target data files is synchronously switched from the corresponding historical metadata file to the target metadata file.

7. The method according to claim 1, characterized in that The processing of subsequent data access requests according to the target metadata file includes: In a case where the data access request is for a second client to access the target data file, determining a target path for storing the target data file according to the target metadata file; The path information of the target path is returned to the second client, instructing the second client to obtain the target data file from the target path according to the path information of the target path.

8. A data processing method, characterized in that: Applied to a client, the method comprises: Convert the data to be transmitted into a target data file in a preset format; the preset format is a format uniformly used between the client and the server; Upload the target data file and store the target data file in a temporary path; The temporary path information corresponding to the temporary path is sent to the server, instructing the server to move the target data file from the temporary path to the corresponding target path, and generate a target metadata file including target metadata information; the target metadata information includes the path information of the target path.

9. The method according to claim 8, characterized in that Also includes: Initiate a data access request to the server for the data file to be accessed; Acquire path information of a storage path for storing the data file to be accessed, determined by the server according to the metadata file; Acquire the to-be-accessed data file from the storage path according to the path information of the storage path; The data file to be accessed is parsed.

10. A data processing device, characterized in that: Applied to the server, the device comprises: Acquire temporary path information sent by the first client; the temporary path information is path information corresponding to a temporary path for temporarily storing a target data file uploaded by the first client, and the format of the target data file is a preset format uniformly used between the client and the server; Determine a target path for storing the target data file, and move the target data file stored in the temporary path corresponding to the temporary path information to the target path; Determine target metadata information of the target data file, and generate a target metadata file including the target metadata information; the target metadata information includes path information of the target path; Subsequent data access requests are processed according to the target metadata file.

11. A data processing device, characterized in that: Applied to a client, the device comprises: Convert the data to be transmitted into a target data file in a preset format; the preset format is a format uniformly used between the client and the server; Upload the target data file and store the target data file in a temporary path; The temporary path information corresponding to the temporary path is sent to the server, instructing the server to move the target data file from the temporary path to the corresponding target path, and generate a target metadata file including target metadata information; the target metadata information includes the path information of the target path.

12. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method according to any one of claims 1 to 9 by executing the computer instructions.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data processing method according to any one of claims 1 to 9.

Images