Data synchronization method and device

Abstract

The embodiment of the present application provides a data synchronization method and device, which can reduce the overall time consumption of the data synchronization process and improve the system performance of the device. The method comprises the following steps: a first device receives a request message from a second device, wherein the request message is used for requesting to obtain target data. If the capacity of the target data is less than or equal to a first threshold, the first device obtains the target data and transmits the target data to the second device. If the capacity of the target data is greater than the first threshold, the first device determines the capacity of each batch data in a plurality of batch data which the target data needs to be divided into based on a batch coefficient, the first device obtains each batch data according to the capacity of each batch data, and transmits each batch data to the second device. Wherein, y batch data in the plurality of batch data are obtained in the process of transmitting x batch data in the plurality of batch data, and x and y are positive integers.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a data synchronization method and device. Background Technology

[0002] Cross-device data synchronization can be understood as the process of one device acquiring data from another device. Currently, the cross-device data synchronization process generally consists of four stages, such as... Figure 1 As shown, the data synchronization process consists of four stages: S101, Data Request: The requesting end sends a data request to the data source end. S102, Data Preparation: The data source end performs data preparation, which includes, but is not limited to, data querying, data calculation, and data reading. S103, Data Transmission: The data source end transmits data to the requesting end. S104, Data Processing: The requesting end performs data processing, which includes, but is not limited to, data insertion, data calculation, and data writing. The overall time consumption of this data synchronization process mainly includes the time consumed in these four stages: data request, data preparation, data transmission, and data processing. Reducing the overall time consumption of the data synchronization process is crucial for improving the system performance of the device. Summary of the Invention

[0003] This application provides a data synchronization method and device that can reduce the overall time spent in the data synchronization process and improve the system performance of the device.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, embodiments of this application provide a data synchronization method, the method comprising: a first device receiving a request message from a second device, the request message being used to request the acquisition of target data. If the capacity of the target data is less than or equal to a first threshold, the first device acquires the target data and transmits the target data to the second device. If the capacity of the target data is greater than the first threshold, the first device determines the capacity of each batch of data among multiple batches into which the target data needs to be divided based on a batching coefficient, the first device acquires each batch of data according to the capacity of each batch of data, and transmits each batch of data to the second device. Wherein, y batches of data among the multiple batches of data are acquired during the transmission of the x-th batch of data among the multiple batches of data, where x and y are positive integers.

[0006] Based on the above technical solution, this application adopts a dynamic batching strategy. Depending on the amount of data requested by the data requesting end, if the data amount is less than or equal to a certain threshold, it is not processed in batches. If the data amount exceeds the certain threshold, it is processed in batches, and the capacity of each batch can be determined according to a batching coefficient. That is, different strategies are used to batch the data. The yth batch of data in multiple batches can be acquired during the transmission of the xth batch. In other words, data preparation can be performed simultaneously with data transmission, effectively reducing the overall time consumption of the data synchronization phase and improving system performance.

[0007] In one possible design, if the size of the target data is greater than a first threshold and less than or equal to a second threshold, the target data will be divided into two batches. The second threshold is greater than the first threshold.

[0008] In one possible design, there are two batches of data, one of which has a capacity of S*k and the other has a capacity of S*(1-k), where S is the capacity of the target data and k is the batching coefficient, and both S and k are positive real numbers.

[0009] In one possible design, if the size of the target data exceeds a second threshold, the target data will be divided into 2a1 batches and one batch of data with a size of m. In the 2a1 batches, the size of each batch is determined by the second threshold and a batching coefficient, where a1 is a positive integer, m is greater than 0 and less than or equal to the first threshold, and the second threshold is greater than the first threshold. Alternatively, the target data will be divided into 2a2 batches. In the 2a2 batches, the size of each batch is determined by the second threshold and a batching coefficient, where a2 is a positive integer greater than or equal to 2. Alternatively, the target data will be divided into 2a3 batches and two additional batches. In the 2a3 batches, the size of each batch is determined by the second threshold and a batching coefficient, where a3 is a positive integer, and the sum of the sizes of the other two batches is m. The size of each of these additional batches is determined by m and a batching coefficient, where m is greater than the first threshold and less than the second threshold.

[0010] In one possible design, there are 2a1 batches of data, where the capacity of a1 batches is U*k and the capacity of the other a1 batches is U*(1-k), where U is the second threshold and k is the batching coefficient, and both U and k are positive real numbers.

[0011] In one possible design, there are 2a2 batches of data, where the capacity of a2 batches is U*k and the capacity of the other a2 batches is U*(1-k), where U is the second threshold and k is the batching coefficient, and both U and k are positive real numbers.

[0012] In one possible design, there are 2a3 batches of data, where the capacity of a3 batches is U*k and the capacity of the other a3 batches is U*(1-k), where U is the second threshold and k is the batching coefficient, and both U and k are positive real numbers.

[0013] In one possible design, among the other two batches of data, one batch has a capacity of m*k and the other batch has a capacity of m*(1-k), where k is the batching coefficient and k is a positive real number.

[0014] In one possible design, the batching factor is determined by the data transmission time and the data processing time. The data transmission time is the duration taken for the first device to transmit a unit of data to the second device, and this time is related to the performance of network input / output I / O. The data processing time is the duration taken for the second device to process a unit of data, and this time is related to the performance of the second device's write I / O.

[0015] Based on this design, this application uses data transmission time and data processing time to determine the batching coefficient. It also considers that data transmission time is related to network I / O performance and data processing time is related to the write I / O performance of the second device. Therefore, the determined batching coefficient can be applied to scenarios with different network I / O performance and / or different write I / O performance. That is, it can effectively reduce the overall time of data synchronization process for scenarios with different network I / O performance and / or different write I / O performance.

[0016] In one possible design, the batching factor satisfies the following condition: Where k is the batching coefficient, α is a real number greater than 0 and less than or equal to 1, B is the data transmission time, and W is the data processing time. k, B, and W are all positive real numbers.

[0017] In one possible design, the first threshold is determined by the capacity of a single target data entry, the batching coefficient, the time gain, the data preparation time, and the data processing time. The data preparation time is the time taken for the first device to acquire a unit of data, and this time is related to the read I / O performance of the first device. The data processing time is the time taken for the second device to process a unit of data, and this time is related to the write I / O performance of the second device. The time gain is the difference between the first time and the second time, where the first time is the time required to batch process the target data, and the second time is the time required to process the target data without batching. When the target data is stored as a file, a single target data entry is a single file, and / or, when the target data is stored as a data record, a single target data entry is a single data record.

[0018] Based on this design, the first threshold in this application is determined by the capacity of a single target data item, the batching coefficient, the time gain, the data preparation time, and the data processing time. Therefore, the determination of the first threshold takes into account the magnitude of the time gain, ensuring that the time gain is guaranteed when processing data in batches, thus effectively reducing the overall time consumption of the data synchronization process. Furthermore, considering that the data preparation time is related to the read I / O performance of the first device, and the data processing time is related to the write I / O performance of the second device, the determined first threshold can be applied to scenarios with different read / write I / O performances, effectively reducing the overall time consumption of the data synchronization process for scenarios with different read / write I / O performances.

[0019] In one possible design, the first threshold satisfies the following condition: Where L is the first threshold, k is the batching coefficient, L1 is the capacity of a single target data, R is the data preparation time, W is the data processing time, G is the time gain, and L, L1, k, R, W, and G are all positive real numbers.

[0020] In one possible design, the second threshold is determined by the first threshold, the batching coefficient, data preparation time, data transmission time, data processing time, and time requirement. Specifically, data transmission time is the duration taken for the first device to transmit a unit of data to the second device, and this time is related to network I / O performance. Data processing time is the duration taken for the second device to process a unit of data, and this time is related to the write I / O performance of the second device. Data preparation time is the duration taken for the first device to acquire a unit of data, and this time is related to the read I / O performance of the first device. The time requirement is the user's time requirement for a batch of data to arrive at the second device. Based on this design, the determination of the second threshold in this application takes into account that the data preparation time is related to the performance of the first device's read I / O, the data transmission time is related to the performance of the network I / O, and the data processing time is related to the performance of the second device's write I / O. Therefore, the determined second threshold can be applied to scenarios with different performance levels of read / write I / O and network I / O. That is, it can effectively reduce the overall time consumption of the data synchronization process in scenarios with different performance levels of read / write I / O and network I / O. Time requirements are also considered, ensuring that while reducing the overall data synchronization time, data can be promptly responded to the user, improving the efficiency of human-computer interaction and enhancing the user experience.

[0021] In one possible design, the second threshold satisfies the following condition:

[0022]

[0023] Where U is the second threshold, k is the batching coefficient, L is the first threshold, R is the data preparation time, W is the data processing time, B is the data transmission time, RT is the time requirement, and U, k, L, R, W, B, and RT are all positive real numbers.

[0024] In one possible design, the first device transmits each batch of data to the second device, including: the first device transmitting each batch of data to the second device using a data packet in a target format. The data packet in the target format carries the following identifiers: a first identifier indicating that the batches of data carried in the data packet belong to the same batch; and / or a second identifier indicating the number of batches into which the target data is divided and the order of the current batch of data within the target data; or, the second identifier indicating the order of the batches into which the target data is divided within the target data. Based on this design, transmitting data using data packets in the target format facilitates the differentiation and / or identification of batches of data.

[0025] In one possible design, the target format data packet also carries a third identifier, which is used to indicate the capacity of the current batch of data.

[0026] Secondly, embodiments of this application provide a data synchronization method, the method comprising: a second device sending a request message to a first device, the request message being used to request the acquisition of target data. If the capacity of the target data is less than or equal to a first threshold, the second device receives the target data from the first device and processes the target data. If the capacity of the target data is greater than the first threshold, the second device receives multiple batches of data from the first device and processes the multiple batches of data. The multiple batches of data are obtained by dividing the target data, and the capacity of the batches of data is determined by a batching coefficient. Wherein, c batches of data are received during the processing of the e-th batch of data, where c and e are positive integers.

[0027] Based on the above technical solution, this application adopts a dynamic batching strategy. Depending on the amount of data requested by the data requesting end, if the data amount is less than or equal to a certain threshold, it is not processed in batches. If the data amount exceeds the certain threshold, it is processed in batches. The capacity of each batch can be determined according to a batching coefficient, meaning different strategies are used for batch processing. c batches of data can be received during the processing of the e-th batch, allowing data transmission to occur simultaneously with data processing. This effectively reduces the overall time consumption during the data synchronization phase and improves system performance.

[0028] In one possible design, if the target data volume is greater than a first threshold and less than or equal to a second threshold, the target data will be divided into two batches. The second threshold is greater than the first threshold.

[0029] In one possible design, there are two batches of data, one of which has a capacity of S*k and the other has a capacity of S*(1-k), where S is the capacity of the target data and k is the batching coefficient, and both S and k are positive real numbers.

[0030] In one possible design, if the size of the target data exceeds a first threshold, the second device receives multiple batches of data from the first device. The design further includes: if the size of the target data exceeds a second threshold, the second device receives 2a1 batches of data and one batch of data with a size of m from the first device; wherein, in the 2a1 batches of data, the size of each batch is determined by the second threshold and a batching coefficient, a1 is a positive integer, m is greater than 0 and less than or equal to the first threshold, and the second threshold is greater than the first threshold. Alternatively, the second device receives 2a2 batches of data from the first device, wherein the size of each batch is determined by the second threshold and a batching coefficient, and a2 is a positive integer greater than or equal to 2. Alternatively, the second device receives 2a3 batches of data from the first device, wherein the size of each batch is determined by the second threshold and a batching coefficient, and a3 is a positive integer. The second device also receives two more batches of data from the first device, the sum of the sizes of the two batches being m, and the size of each batch is determined by m and a batching coefficient.

[0031] In one possible design, if the size of the target data exceeds a first threshold, the second device receives multiple batches of data from the first device. The design further includes: if the size of the target data exceeds a second threshold, the second device receives 2a1 batches of data and one batch of data with a size of m from the first device. In the 2a1 batches, the size of each batch is determined by the second threshold and a batching coefficient, where a1 is a positive integer, m is greater than 0 and less than or equal to the first threshold, and the second threshold is greater than the first threshold. Alternatively, the second device receives 2a2 batches of data from the first device. In the 2a2 batches, the size of each batch is determined by the second threshold and a batching coefficient, where a2 is a positive integer greater than or equal to 2. Alternatively, the second device receives 2a3 batches of data from the first device. In the 2a3 batches, the size of each batch is determined by the second threshold and a batching coefficient; a3 is a positive integer. The second device also receives two batches of data from the first device. The sum of the capacities of the two batches of data is m. The capacity of each batch of data is determined by m and the batching coefficient. m is greater than the first threshold and less than the second threshold.

[0032] In one possible design, there are 2a1 batches of data, where the capacity of a1 batches is U*k and the capacity of the other a1 batches is U*(1-k), where U is the second threshold and k is the batching coefficient, and both U and k are positive real numbers.

[0033] In one possible design, there are 2a2 batches of data, where the capacity of a2 batches is U*k and the capacity of the other a2 batches is U*(1-k), where U is the second threshold and k is the batching coefficient, and both U and k are positive real numbers.

[0034] In one possible design, there are 2a3 batches of data, where the capacity of a3 batches is U*k and the capacity of the other a3 batches is U*(1-k), where U is the second threshold and k is the batching coefficient, and both U and k are positive real numbers.

[0035] In one possible design, the second device also receives two batches of data from the first device, one of which has a capacity of m*k and the other has a capacity of m*(1-k), where k is the batching coefficient and k is a positive real number.

[0036] In one possible design, the batching factor is determined by the data transmission time and the data processing time. The data transmission time is the duration taken for the first device to transmit a unit of data to the second device, and this time is related to the performance of network input / output I / O. The data processing time is the duration taken for the second device to process a unit of data, and this time is related to the performance of the second device's write I / O.

[0037] In one possible design, the batching factor satisfies the following condition: Where k is the batching coefficient, α is a real number greater than 0 and less than or equal to 1, B is the data transmission time, and W is the data processing time. k, B, and W are all positive real numbers.

[0038] In one possible design, the first threshold is determined by the capacity of a single target data entry, the batching coefficient, the time gain, the data preparation time, and the data processing time. The data preparation time is the time taken for the first device to acquire a unit of data, and this time is related to the read I / O performance of the first device. The data processing time is the time taken for the second device to process a unit of data, and this time is related to the write I / O performance of the second device. The time gain is the difference between the first time and the second time, where the first time is the time required to batch process the target data, and the second time is the time required to process the target data without batching. When the target data is stored as a file, a single target data entry is a single file, and / or, when the target data is stored as a data record, a single target data entry is a single data record.

[0039] In one possible design, the first threshold satisfies the following condition: Where L is the first threshold, k is the batching coefficient, L1 is the capacity of a single target data, R is the data preparation time, W is the data processing time, G is the time gain, and L, L1, k, R, W, and G are all positive real numbers.

[0040] In one possible design, the second threshold is determined by the first threshold, the batching coefficient, data preparation time, data transmission time, data processing time, and time requirement. Specifically, the data transmission time is the duration taken for the first device to transmit a unit of data to the second device, and this time is related to network I / O performance. The data processing time is the duration taken for the second device to process a unit of data, and this time is related to the write I / O performance of the second device. The data preparation time is the duration taken for the first device to acquire a unit of data, and this time is related to the read I / O performance of the first device. The time requirement is the user's time requirement for a batch of data to arrive at the second device.

[0041] In one possible design, the second threshold satisfies the following condition:

[0042]

[0043] Where U is the second threshold, k is the batching coefficient, L is the first threshold, R is the data preparation time, W is the data processing time, B is the data transmission time, RT is the time requirement, and U, k, L, R, W, B, and RT are all positive real numbers.

[0044] In one possible design, the second device receives multiple batches of data from the first device, including: the second device receiving multiple batches of data transmitted using a target format data packet from the first device; wherein the target format data packet carries the following identifiers: a first identifier, used to indicate that the batches of data carried in the data packet belong to the same batch; and / or, a second identifier, used to indicate the number of batches into which the target data is divided and the order of the current batch of data in the target data; or, the second identifier used to indicate the order of the current batch of data in the target data.

[0045] In one possible design, the target format data packet also carries a third identifier, which is used to indicate the capacity of the current batch of data.

[0046] Thirdly, embodiments of this application provide a data synchronization device, which is a first device. The first device includes a processor and a memory coupled to the processor. The memory is used to store computer program code, which includes computer instructions. The processor reads the computer instructions from the memory, causing the first device to execute the method described in the first aspect and any of the designs described above.

[0047] Fourthly, embodiments of this application provide a data synchronization device, which is a second device. The second device includes a processor and a memory coupled to the processor. The memory is used to store computer program code, which includes computer instructions. The processor reads the computer instructions from the memory, causing the second device to execute the method described in the second aspect and any of the designs therein.

[0048] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the method described in either the first or second aspect described above.

[0049] In a sixth aspect, embodiments of this application provide a computer program product, which includes a computer program or instructions that, when executed on a computer, cause the computer to perform the method described in either the first or second aspect described above.

[0050] In a seventh aspect, embodiments of this application provide a chip system including at least one processor and at least one interface circuit. The at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor. When the at least one processor executes the instructions, the at least one processor performs the method described in either the first or second aspect above.

[0051] Eighthly, embodiments of this application provide a data synchronization system, which includes the first device and the second device described above. The first device and the second device can interact to execute the method described in any of the designs in the first to second aspects.

[0052] It should be noted that the technical effects of any of the designs in the second to eighth aspects mentioned above can be found in the technical effects of the corresponding designs in the first aspect, and will not be repeated here. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of an existing data synchronization process;

[0054] Figure 2 A schematic diagram of a data synchronization communication system provided in an embodiment of this application;

[0055] Figure 3 A schematic diagram of the hardware structure of a first device and / or a second device provided in an embodiment of this application;

[0056] Figure 4 A schematic diagram of the software system architecture of a first device provided in an embodiment of this application;

[0057] Figure 5 This is a schematic flowchart of a data synchronization method provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram illustrating how target data is not processed in batches when the target data is less than or equal to a first threshold, as provided in an embodiment of this application.

[0059] Figure 7a This application provides a schematic diagram illustrating how target data is divided into two batches for processing, as shown in the embodiments of this application.

[0060] Figure 7b This is a schematic diagram illustrating another method of dividing target data into two batches for processing, as provided in an embodiment of this application.

[0061] Figure 8a This application provides a schematic diagram illustrating how target data is divided into four batches for processing, as shown in this embodiment.

[0062] Figure 8b This is a schematic diagram illustrating another method of dividing target data into four batches for processing, as provided in an embodiment of this application.

[0063] Figure 8c This is a schematic diagram illustrating another method of dividing target data into four batches for processing, as provided in an embodiment of this application.

[0064] Figure 8d This is a schematic diagram illustrating another method of dividing target data into four batches for processing, as provided in an embodiment of this application.

[0065] Figure 9 A schematic diagram illustrating the format of a data packet carrying target data, provided for an embodiment of this application;

[0066] Figure 10 This is a schematic diagram of the structure of a data synchronization device provided in an embodiment of this application;

[0067] Figure 11 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0068] The data synchronization method and device provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0069] The terms “comprising” and “having”, and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0070] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0071] In the description of this application, unless otherwise stated, "multiple" means two or more. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The terms "first," "second," etc., do not limit the quantity or order of execution, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0072] First, for ease of understanding, the relevant terms and concepts that may be involved in the embodiments of this application will be introduced below.

[0073] 1. Input / output (I / O)

[0074] Files in a computer exist as streams. Computers can manipulate data by operating on these streams; this operation is called I / O (Input / Output) operation. For example, a computer system can read data from a stream by calling the `read` function and write data to it by calling the `write` function. These streams are uniquely identified by file descriptors (FDs). Depending on the object being operated on, I / O can be categorized into memory I / O, disk I / O, network I / O, etc. Depending on the type of I / O operation, it can also be divided into read I / O (or I / O read), write I / O (or I / O write), etc.

[0075] The performance of network I / O determines the data transmission rate, therefore, Figure 1 The data transfer time shown is primarily related to network I / O performance. The performance of read I / O and write I / O determines the data processing rate, which is mainly related to the performance of the device configuration (e.g., disk, memory). Figure 1 The time required for data preparation and processing is mainly related to the performance of read I / O and write I / O.

[0076] 2. Threads

[0077] A thread is the smallest unit of computation that can be invoked in an operating system. A thread refers to a single, sequential flow of control within a process. Multiple threads can run concurrently within a single process, each executing different tasks in parallel. Threads from different processes can also execute in parallel.

[0078] This application provides a data synchronization method that reduces the overall time required for data synchronization and improves the system performance of the device. The data synchronization method provided in this application can be applied to various data synchronization scenarios. For example, synchronizing contact information between a mobile phone and a watch, such as the watch synchronizing contact information from the phone after establishing a communication connection between the two devices. Another example is synchronizing media files from a mobile phone using a tablet, such as the tablet synchronizing media files from the phone after establishing a communication connection between the two devices. Yet another example is the fast-forward function during media playback, which also falls under the data synchronization scenario, where the client can synchronize media files from the server.

[0079] For example, such as Figure 2 The diagram shown is a schematic representation of a data synchronization communication system provided in an embodiment of this application. The communication system includes a first device and a second device.

[0080] Optionally, the first device can act as a data source or a data requester. Specifically, the first device can be a mobile phone, tablet computer, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), artificial intelligence (AI) device, wearable device, etc. Wearable devices can be smartwatches, smart bracelets, etc. When the first device acts as a data source, it can also be a server, such as a streaming media server. This application does not impose any limitations on the specific type of the first device.

[0081] Optionally, in this communication system, when the first device acts as the data source, the second device can act as the data requester; conversely, when the first device acts as the data requester, the second device can act as the data source. Specifically, the second device can be a mobile phone, tablet computer, in-vehicle device, AR / VR device, laptop computer, UMPC, netbook, PDA, artificial intelligence (AI) device, wearable device, or other terminal device. Wearable devices can be smartwatches, smart bracelets, etc. When the second device acts as the data source, it can also be a server, such as a streaming media server. This application embodiment does not impose any limitations on the specific type of the second device.

[0082] Optionally, the first device and / or the second device may adopt a distributed system architecture, or other architectures, which are not limited in this application. It should be understood that... Figure 2 This is a simplified schematic diagram provided merely for the purpose of understanding this application. Other devices may also be included in this communication system. Figure 2 It was not shown in the text.

[0083] For example, Figure 3 A schematic diagram of a first device and / or a second device is shown. The first device and / or the second device may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, antenna 1, antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc.

[0084] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the first device and / or the second device. In other embodiments of this application, the first device and / or the second device may include more or fewer components than illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0085] Processor 310 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0086] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0087] The processor 310 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. This memory can store instructions or data that the processor 310 has just used or that are used repeatedly. If the processor 310 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0088] The charging management module 340 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 receives charging input from the wired charger via a USB interface 330. In some wireless charging embodiments, the charging management module 340 receives wireless charging input via the wireless charging coil of a first device and / or a second device. While charging the battery 342, the charging management module 340 can also supply power to the first device and / or the second device via the power management module 341.

[0089] The power management module 341 connects the battery 342, the charging management module 340, and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, providing power to the processor 310, internal memory 321, display screen 394, camera 393, and wireless communication module 360. The power management module 341 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 341 may be located within the processor 310. In other embodiments, the power management module 341 and the charging management module 340 may be housed in the same device.

[0090] The wireless communication function of the first device and / or the second device can be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor, and baseband processor, etc.

[0091] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the first and / or second device can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0092] The mobile communication module 350 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to a first device and / or a second device. The mobile communication module 350 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 350 may be housed in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 and at least some modules of the processor 310 may be housed in the same device.

[0093] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device or displays an image or video through the display screen 394. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 310 and may be housed in the same device as the mobile communication module 350 or other functional modules.

[0094] The wireless communication module 360 ​​can provide solutions for wireless communication applications on the first and / or second devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 360 ​​can be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 310. The wireless communication module 360 ​​can also receive signals to be transmitted from processor 310, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0095] In some embodiments, antenna 1 of the first device and / or the second device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 360, enabling the first device and / or the second device to communicate with the network and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0096] The first and / or second device implements display functions through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 394 and the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0097] Display screen 394 is used to display images, videos, etc. Display screen 394 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized display, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the first device and / or the second device may include one or N displays 394, where N is a positive integer greater than 1.

[0098] The first and / or second devices can achieve shooting functions through ISP, camera 393, video codec, GPU, display 394, and application processor, etc.

[0099] The ISP (Image Signal Processor) is used to process data fed back from the camera 393. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 393.

[0100] Camera 393 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the first device and / or the second device may include one or N cameras 393, where N is a positive integer greater than 1.

[0101] The external storage interface 320 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the first device and / or the second device. The external storage card communicates with the processor 310 through the external storage interface 320 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0102] Internal memory 321 can be used to store executable program code, including instructions. Internal memory 321 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the first device and / or the second device (such as audio data, phonebook, etc.). Furthermore, internal memory 321 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 310 executes various functional applications and data processing of the first device and / or the second device by running instructions stored in internal memory 121 and / or instructions stored in memory located within the processor.

[0103] The first and / or second devices can implement audio functions, such as music playback and recording, through the audio module 370 and application processor.

[0104] The audio module 370 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. The audio module 370 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 370 may be located in the processor 310, or some functional modules of the audio module 370 may be located in the processor 310.

[0105] The sensor module 380 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0106] Buttons 390 include a power button, volume buttons, etc. Buttons 390 can be mechanical buttons or touch-sensitive buttons. The first and / or second devices can receive button input and generate key signal inputs related to user settings and function control of the first and / or second devices.

[0107] Motor 381 can generate vibration alerts. Motor 391 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can be corresponding to touch operations applied to different applications (such as taking photos, playing audio, etc.). Motor 391 can also correspond to different vibration feedback effects for touch operations applied to different areas of the display screen 394. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0108] Indicator 392 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0109] The SIM card interface 395 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 395 to achieve contact and separation with the first device and / or the second device. The first device and / or the second device can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 395 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 395 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 395 is also compatible with different types of SIM cards. The SIM card interface 395 is also compatible with external memory cards. The first device and / or the second device interact with the network through the SIM card to achieve functions such as calls and data communication. In some embodiments, the first device and / or the second device use an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the first device and / or the second device and cannot be separated from it.

[0110] Optionally, the first device and the second device can use heterogeneous hardware. For example, the first device can use the Kirin 990 chip, and the second device can use the Kirin 880 chip. Alternatively, the first device and the second device can use homogeneous hardware. For example, both the first device and the second device can use the Kirin 990 chip. This application does not limit this.

[0111] The software system of the first device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc. For example, taking the first device as the data source, such as... Figure 4The diagram shown is a software system architecture diagram of a first device provided in an embodiment of this application. The system includes a data synchronization component 400, a data storage module 401, a data transmission management module 402, a business data model analysis module 403, a platform I / O capability configuration module 404, a network environment analysis module 405, and a dynamic batching strategy decision module 406. Optionally, when the second device acts as the data source, the second device can also adopt... Figure 4 The software system architecture shown.

[0112] The data synchronization component 400 receives data requests from other devices, indicating the data type and capacity requested by the other device. In this embodiment, the data request may originate from a second device. Optionally, the data request from the second device may be sent by the data synchronization component of the second device to the data synchronization component 400 of the first device. The data request from the data synchronization component of the second device may originate from an application (e.g., gallery, contacts) on the second device. Based on the data request, the data synchronization component 400 calls the dynamic batching strategy module 406 to obtain the corresponding batching strategy. According to the batching strategy, it reads the corresponding data from the data storage module 401 and sends the corresponding data to the data transmission management module 402. The batching strategy will not be described in detail here but will be explained in more detail later.

[0113] The data storage module 401 is used for data storage. For example, the data storage module can be a file system or a database system (e.g., a non-relational database KV).

[0114] The data transmission management module 402 is used to manage different communication connection methods between devices, such as Bluetooth connection, WiFi connection, distributed soft bus connection, etc., and is also used to manage the sending and receiving of data between devices.

[0115] The business data model analysis module 403 is used to obtain basic parameters of data requests sent by other devices, such as total data volume and single data record size. Optionally, assuming that the data requested in the data request sent by the second device (e.g., images, thumbnails, etc.) is stored in file format, then the single data record size is the size of a single file, and the total data volume is the sum of the sizes of all requested files. For example, assuming that the data request sent by the second device requests 10 images, and each of these 10 images is stored in a single file, then the total data volume is the size of the 10 images, and the single data record size is the size of one image. Optionally, if the size of each image is different, the single data record size can be the size of the image with the largest size, or it can be the size of other images. Optionally, assuming that the data requested in the data request sent by the second device (e.g., contact information) is stored in the form of data records, then the single data record size is the size of a single data record, and the total data volume is the sum of the sizes of all requested data records. The business data model analysis module 403 is also used to analyze the complexity of data preparation and / or processing.

[0116] The platform I / O capability configuration module 404 is used to locally store the device's I / O (e.g., read I / O, write I / O, etc.) performance metrics. After the first device and the second device establish a connection, they can exchange their I / O performance metrics.

[0117] The network environment analysis module 405 is used to provide the dynamic synchronization strategy module 406 with the communication connections currently available between the first device and the second device, such as Bluetooth connection, WiFi connection, distributed soft bus connection, etc. It can also provide the performance of the current communication connection, that is, the performance of network I / O.

[0118] The dynamic batching strategy decision module 406 uses relevant information provided by the integrated business data model analysis module 403, platform I / O capability configuration module 404, and network environment analysis module 405 to generate a corresponding batching strategy. The first device, such as its data synchronization component 400, can prepare and transmit data according to this batching strategy. The batching strategy includes a first threshold and a batching coefficient. The first threshold determines whether to process the data in batches, and the batching coefficient determines the size of each batch of data. Optionally, the size of each batch of data can be the same or different. Batch processing means that data preparation and data transmission are performed in batches; similarly, data processing is also performed in batches. This is explained uniformly here and will not be elaborated further below.

[0119] Optionally, the dynamic batching strategy decision module 406 is also used to update the batching strategy based on feedback information. For example, updating the first threshold and / or batching coefficient. Optionally, the feedback information includes data preparation time, data transmission time, and data processing time. Data preparation time refers to the time taken for the first device to acquire a unit of data (e.g., reading data from the disk into memory). The unit of data is a portion of the data requested by the second device. For example, assuming the second device requests 10 megabytes (MB) of data, the unit of data could be 1 MB or 1 kilobyte (KB), meaning the unit of data can be MB, KB, B, etc. This application does not specifically limit its size; it can be 1 MB, 1 KB, etc. This is a unified explanation and will not be repeated later.

[0120] It should be noted that unit data is different from single data as mentioned above, and the capacity of single data and unit data are not necessarily the same.

[0121] Data transmission time refers to the time it takes for the first device to transmit a unit of data to the second device. Data processing time refers to the time it takes for the second device to process a unit of data (e.g., writing data from memory to disk).

[0122] Optionally, the units for data preparation time, data transmission time, and data processing time can be milliseconds (ms), seconds (s), etc. This application does not make specific limitations on these, and will be uniformly stated here and will not be repeated later.

[0123] Among them, the dynamic batching strategy decision module 406 obtains feedback information in several ways.

[0124] In one possible implementation, the feedback information can be sent from the second device to the first device. Optionally, in this implementation, the data preparation time and / or data transmission time in the feedback information can be determined by the first device itself; that is, the second device does not need to send the data preparation time and / or data transmission time to the first device, but only needs to send the data processing time to the first device.

[0125] In another possible implementation, feedback information can be automatically synchronized to the first device. For example, if the first and second devices share a database, both devices' data read and write operations are operations on that database. Both devices can know the other's operation status on the database, such as the data processing time of the second device and the data preparation time of the first device. There is no need for the second device to send this feedback information to the first device. This application does not limit the method by which the first device obtains the feedback information.

[0126] It should be noted that when the second device acts as the data requesting end, its system architecture can be the same as or different from that of the first device. In the same case, the business data model analysis module, and / or dynamic batching strategy decision module, and / or network environment analysis module in the second device will not function. In different cases, the second device may lack these modules. This application does not specifically limit the system architecture of the second device.

[0127] It should be noted that the names of the above modules do not constitute a limitation on their functions, and other names are also permitted; this application does not impose any restrictions on this. The above modules can be implemented separately, or one or more modules can be integrated together; this application also does not impose any restrictions on this.

[0128] For example, such as Figure 5 As shown, this application provides a data synchronization method, taking a second device as the data requesting end and a first device as the data source end as an example. The method includes the following steps:

[0129] S501, the second device sends a request message to the first device. Correspondingly, the first device receives the request message from the second device.

[0130] The request message is used to request target data. Optionally, the target data can be multimedia data, such as images, videos, text, etc., or other forms of data. Optionally, the target data can be stored in the form of a file or in the form of a data record; this application does not limit this.

[0131] Optionally, the second device also adopts... Figure 4 In the software system architecture shown, the second device can send the request message to the data synchronization component of the first device through the data synchronization component.

[0132] S502, the first device determines whether the capacity of the target data is less than or equal to the first threshold.

[0133] Upon receiving the request message, the first device compares the capacity of the target data requested by the second device with a first threshold. If the target data capacity (i.e., the total target data volume) is less than or equal to the first threshold, steps S503, S504, and S505 are executed; otherwise, if the target data capacity is greater than the first threshold, steps S506, S507, S508, and S509 are executed. The first threshold can be used to determine whether the target data should be processed in batches.

[0134] S503, The first device acquires the target data.

[0135] Specifically, if the size of the target data requested by the second device is less than or equal to the first threshold, it indicates that there is no need to process the target data in batches, and the first device can directly obtain the target data, meaning that the target data will not be obtained in batches. The process by which the first device obtains the target data is the data preparation process.

[0136] Optionally, in this step, the first device may start a thread to acquire the target data.

[0137] S504. The first device transmits target data to the second device. Correspondingly, the second device receives the target data transmitted by the first device.

[0138] S505, The second device processes the target data.

[0139] Optionally, in this step, the second device may start a thread to process the target data.

[0140] When the target data is less than or equal to the first threshold, for example, such as Figure 6 The diagram illustrates a process where target data is not processed in batches. The first device starts a thread to directly acquire the target data; the time taken for this process is the total data preparation time. After acquisition, the thread terminates. The first device then directly transmits the acquired target data to the second device; the time taken for this process is the total data transmission time. The second device receives the target data, starts a thread to process it; the time taken for this process is the total data processing time. After processing, the thread terminates. Without considering the time spent in the data request phase, the total data synchronization time is the sum of the total data preparation time, the total data transmission time, the total data processing time, and the total data preparation time. In this example, both the data preparation and data transmission phases can be single-threaded.

[0141] S506. The first device determines the capacity of each batch of data in the multiple batches into which the target data needs to be divided based on the batching coefficient.

[0142] If the size of the target data requested by the second device exceeds a first threshold, it indicates that the target data needs to be processed in batches. The first device can determine the size of each batch of data among the multiple batches into which the target data needs to be divided based on a batching coefficient. Optionally, the size of each batch of data can be the same or different.

[0143] Optionally, the first device determines the capacity of each batch of data in the multiple batches into which the target data needs to be divided based on the batching coefficient, specifically including the following implementation methods:

[0144] In one possible implementation, if the size of the target data is greater than a first threshold and less than or equal to a second threshold, the target data will be divided into two batches. That is, the first device determines to divide the target data into two batches for processing. The second threshold is greater than the first threshold. For example, in this implementation, assuming the size of the target data is S and the batching coefficient is k, then in the two batches of the target data, one batch has a size of S*k, and the other batch can have a size of S*(1-k), where S and k are both positive real numbers.

[0145] In another possible implementation, if the target data size exceeds a second threshold, the target data will be divided into 2a1 batches and one batch of size m. The size of each of the 2a1 batches is determined by the second threshold and a batching coefficient. For example, assuming the second threshold is U and the batching coefficient is k, the size of the a1 batches is k*U, and the size of the other a1 batches is (1-k)*U. a1 is a positive integer, and m is greater than 0 and less than or equal to the first threshold. The second threshold is greater than the first threshold. That is, if m is less than the first threshold, the data with size m will not be batched.

[0146] Alternatively, the target data can be divided into 2a2 batches. The size of each batch is determined by a second threshold and a batching coefficient. For example, assuming the second threshold is U and the batching coefficient is k, the size of the a2 batches is k*U, and the size of the other a2 batches is (1-k)*U. a2 is a positive integer greater than or equal to 2.

[0147] Alternatively, the target data can be divided into 2a3 batches and two additional batches. The capacity of each of the 2a3 batches is determined by a second threshold and a batching coefficient. For example, assuming the second threshold is U and the batching coefficient is k, the capacity of the a3 batches is k*U, and the capacity of the other a3 batches is (1-k)*U, where a3 is a positive integer. The sum of the other two batches is m, and the capacity of each batch is determined by m and the batching coefficient, where m is greater than the first threshold and less than the second threshold. Optionally, one batch can have a capacity of k*m, and the other batch can have a capacity of (1-k)*m. That is, if m is greater than the first threshold and less than the second threshold, the data with a capacity of m can be divided into two batches for processing.

[0148] The values ​​of a1, a2, a3, and m can be determined according to a preset algorithm. This preset algorithm satisfies the following condition: S = a * U + m, where S is the size of the target data, U is the second threshold, k is the batching coefficient, a is a positive integer greater than or equal to 1, representing the number of first target data items with size U, which is determined by the quotient of S and U, and m is the size of the data that is greater than or equal to 0 and less than the second threshold, which is determined by the remainder of S and U. The values ​​of a1, a2, and a3 are the corresponding values ​​of a.

[0149] As exemplary, Table 1 shows the pseudocode of the algorithm provided in the embodiments of this application.

[0150] Table 1

[0151]

[0152] S507. The first device obtains each batch of data based on the capacity of each batch of data.

[0153] Optionally, in this step, the first device can start one or more threads to acquire multiple batches of data; that is, the process can be either single-threaded or multi-threaded. One thread is used to acquire one or more batches of data. Optionally, when the first device starts multiple threads to acquire multiple batches of data, these multiple threads can be started simultaneously or sequentially.

[0154] Optionally, the first device's acquisition of batch data can be an asynchronous operation. It should be understood that asynchronous operation means that after the first device starts a thread to acquire a batch of data, it can perform other operations without waiting for that process to complete, such as restarting another thread to acquire another batch of data.

[0155] Optionally, if the target data is arranged in a certain order, the first device can acquire each batch of data sequentially based on its size. For example, if the target data is video (i.e., data composed of multiple frames of images), which requires a specific order, the first device can acquire it sequentially. Of course, if the target data does not require a specific order, the first device can acquire each batch of data out of order, such as contact information.

[0156] For example, such as Figure 7a , Figure 7bThe diagram illustrates the process of dividing target data into two batches for processing. Assuming the target data capacity is S and the batching coefficient is k, one batch will have a capacity of S*k, and the other batch can have a capacity of S*(1-k). After batching, the total preparation time R for the batch with capacity S*k may overlap with the total preparation time R for the batch with capacity S*(1-k). That is, while acquiring the batch with capacity S*k, the batch with capacity S*(1-k) can also be acquired. Therefore, the total data preparation time (i.e., the total time for acquiring the batch with capacity S*k and the total time for acquiring the batch with capacity S*(1-k)) is less than (e.g.,...). Figure 7b ) or equal to (e.g. Figure 7a The total time R spent on data preparation before batching.

[0157] It should be noted that the process of the first device acquiring these multiple batches of data is the data preparation process, and the time consumed by the first device to acquire two batches of data (e.g., Figure 7a As shown, the total data preparation time R for batch data with capacity S*k may be the same as or different from the total data preparation time R for batch data with capacity S*(1-k). Similarly, the data transmission process and data processing process are similar and will be explained uniformly here.

[0158] exist Figure 7a In the example shown, the first device can start a thread (e.g., thread 1) to retrieve the two batches of data. For example, the first device starts thread 1 to retrieve one batch of data (e.g., a batch of data with a capacity of k*S), and after retrieval, it uses the same thread 1 to retrieve the other batch of data (e.g., a batch of data with a capacity of (1-k)*S). After retrieval, thread 1 ends. Optionally, the first device can also retrieve one batch of data, end the thread, and then start a new thread to retrieve the other batch of data. However, for the first device, regardless of the method, only one thread can run simultaneously. This situation also applies to the second device, and will not be elaborated further below.

[0159] Of course, in Figure 7b In the example shown, the first device can also start two threads to acquire these two batches of data. For example, the first device starts one thread (e.g., thread 1) to acquire a batch of data with a capacity of k*S, and then while thread 1 is running, it starts another thread (e.g., thread 2) to acquire a batch of data with a capacity of (1-k)*S. At this time, Figure 7b The total data preparation time R corresponding to the two batches of data shown can partially overlap, or they can completely overlap (not shown in the figure).

[0160] For example, such as Figure 8a , Figure 8b , Figure 8c , Figure 8d The diagram illustrates dividing the target data into four batches for processing. Assuming the target data capacity is S, the batching coefficient is k, the second threshold is U, and m is greater than the first threshold and less than the second threshold, then S = a*U + m, a = 1. The capacities of the four batches are k*U, (1-k)*U, k*m, and (1-k)*m, respectively. The first device can start one or more threads to acquire these four batches. After batching, while acquiring one batch, one or more other batches can also be acquired. For example, while acquiring a batch of data with a capacity of k*U, one or more batches with capacities of (1-k)*U, k*m, and (1-k)*m can also be acquired. Therefore, the overall time spent on data preparation (i.e., the overall time spent acquiring the aforementioned four batches) is less than (e.g., ...). Figure 8b , Figure 8c , Figure 8d ) or equal to (e.g. Figure 8a The total time R spent on data preparation before batching.

[0161] like Figure 8a The diagram illustrates how a first device starts a thread (e.g., thread 1) to acquire these four batches of data. The first device starts a thread to acquire one of the batches of data (e.g., a batch of data with a capacity of k*U). After acquiring the data, the same thread is used to acquire another batch of data (e.g., a batch of data with a capacity of (1-k)*U), and so on, until all four batches of data have been acquired, at which point the thread terminates.

[0162] like Figure 8b The diagram illustrates how the first device starts two threads to acquire these four batches of data. The first device starts two threads (e.g., thread 1 and thread 2) to acquire two of the batches of data (e.g., a batch of data with a capacity of k*U and a batch of data with a capacity of (1-k)*U). After acquiring the data, the two threads (e.g., thread 1 and thread 2) are used to acquire the other two batches of data (e.g., a batch of data with a capacity of (1-k)*m and a batch of data with a capacity of k*m).

[0163] like Figure 8cThe diagram illustrates how a first device starts three threads to acquire four batches of data. The first device starts one thread (e.g., thread 1) to acquire one batch of data (e.g., a batch with a capacity of k*U), and then starts two other threads (e.g., thread 2 and thread 3) to acquire two other batches of data (e.g., a batch with a capacity of (1-k)*U and a batch with a capacity of k*m). Then, if one of the three batches of data (e.g., a batch with a capacity of k*m) is acquired first, the thread for that batch (e.g., thread 3) can be used to acquire the last batch of data (e.g., a batch with a capacity of (1-k)*m).

[0164] like Figure 8d The diagram shows the first device starting four threads (e.g., threads 1 to 4) to acquire these four batches of data. The first device runs four threads simultaneously to acquire these four batches of data respectively.

[0165] Optionally, the number of threads that can run simultaneously on the first device may be less than a certain threshold, which can be set by the developers according to actual needs, and this application does not limit it.

[0166] S508. The first device transmits each batch of data to the second device. Correspondingly, the second device receives each batch of data transmitted by the first device.

[0167] In this context, y batches of data from multiple data batches are acquired during the transmission of the x-th batch of data, where x and y are both positive integers. That is, while transmitting a certain batch of data to the second device, the first device can acquire one or more other batches of data, including batches that the first device has not yet acquired. For example, as shown... Figure 7a , Figure 7b As shown, when the first device transmits batch data of capacity k*S to the second device, the first device can acquire batch data of capacity (1-k)*S. For example... Figure 8a , Figure 8b As shown, when the first device transmits batch data of capacity k*S to the second device, the first device can acquire one or more of the following: batch data of capacity (1-k)*U, batch data of capacity k*m, and batch data of capacity (1-k)*m.

[0168] If the size of the target data is greater than the second threshold, the second device receives 2a1 batches of data and one batch of data with a size of m from the first device. The size of each batch of data in the 2a1 batches is determined by the second threshold and the batching coefficient. For example, assuming the second threshold is U and the batching coefficient is k, the size of the a1 batches of data is k*U, and the size of the other a1 batches of data is (1-k)*U. a1 is a positive integer, and m is greater than 0 and less than or equal to the first threshold.

[0169] Alternatively, the second device receives 2a2 batches of data from the first device; wherein, the capacity of each batch of data is determined by a second threshold and a batching coefficient, for example: assuming the second threshold is U and the batching coefficient is k, the capacity of the a2 batches of data is k*U, and the capacity of the other a2 batches of data is (1-k)*U; a2 is a positive integer greater than or equal to 2;

[0170] Alternatively, the second device receives 2a3 batches of data from the first device. The capacity of each batch of data is determined by a second threshold and a batching coefficient. For example, assuming the second threshold is U and the batching coefficient is k, the capacity of the a3 batches is k*U, and the capacity of the other a3 batches is (1-k)*U; a3 is a positive integer. The second device also receives two batches of data from the first device, the sum of the capacities of the two batches being m. The capacity of each batch is determined by m and the batching coefficient, where m is greater than the first threshold and less than the second threshold. Optionally, one batch can have a capacity of k*m, and the other batch can have a capacity of (1-k)*m. For details on how to determine a1, a2, a3, and m, please refer to the above description; it will not be repeated here.

[0171] Optionally, the batched data can be transmitted serially, with each batch transmitted through the same transmission channel. The next batch can only be transmitted after the previous batch has finished transmitting. Alternatively, the transmission of batched data from the first device to the second device can be a synchronous operation. It should be understood that synchronous operations cannot start new threads; after transmitting one batch of data, the first device must wait for that process to complete before performing other operations, such as transmitting another batch of data. For example,... Figures 7a to 8d As shown, the next batch of data can only be transmitted after one batch of data transmission is completed. In this example, as... Figure 7a , Figure 7b As shown, the total time for data transmission after batching is the sum of the total time B for data transmission of batches with a capacity of S*k and the total time B for data transmission of batches with a capacity of S*(1-k), which is... Figure 7a , Figure 7bThe figure shows the total time B for data transmission before batching. Similarly, as shown... Figures 8a to 8d As shown, the total time for data transmission after batching is the sum of the total time B for data transmission in batches of capacity S*k, B for data transmission in batches of capacity S*(1-k), B for data transmission in batches of capacity k*m, and B for data transmission in batches of capacity (1-k)*m. Figures 8a to 8d The figure shows the total time B for data transmission before batching.

[0172] Optionally, to facilitate the differentiation and / or identification of batched data, the first device may transmit the batched data to the second device using data packets in the target format. Correspondingly, the second device may receive the batched data transmitted using data packets in the target format. The data packets in the target format carry the following identifiers:

[0173] The first identifier indicates that the batches of data carried in the data packet belong to the same batch. For example... Figure 9 As shown, the first identifier can be a batch identifier (ID). For example, if there are n data packets that belong to the same batch of data, then the batch ID carried in these n data packets can all be "1" or all be "2" to indicate that these n data packets belong to the same batch of data.

[0174] It should be noted that this application does not limit the batch IDs carried by data packets belonging to the same batch of data to be the same; they can also be different. For example, some data packets may carry a batch ID of "1," while another group of data packets may carry a batch ID of "2," but these two groups of data packets still belong to the same batch of data. In this example, the protocol may specify, or the first device may notify the second device in advance, which batches of data carrying different batch IDs (e.g., those carrying batch ID "1" and those carrying batch ID "2") belong to the same batch of data.

[0175] Optionally, the first identifier can also indicate the order of the batches of data carried in at least one data packet within the same batch. This order can be the transmission order. For example, assuming the target data is divided into 4 batches, and n data packets belong to the 3rd batch, then the batch ID carried in these n data packets can all be "3" to indicate that these n data packets belong to the same batch, and also to indicate that these n data packets belong to the 3rd batch of the target data.

[0176] And / or, a second identifier, used to indicate the number of batches into which the target data is divided and the order of the current batches. Alternatively, the second identifier is used to indicate the order of the current batches. This order may refer to the transmission order.

[0177] like Figure 9 As shown, the second identifier can be a batch ID. When the second identifier is used to indicate the number of batches the target data is divided into and the current order of the batches, assuming the target data is divided into 4 batches, the batch ID that can be carried in the data packet belonging to the 3rd batch can be "4,3", where 4 represents the number of batches the target data is divided into, i.e., how many batches the target data is divided into, and 3 represents the order of the batch to which this data packet belongs among all the batches.

[0178] Taking the example above, if the second identifier is used to indicate the order of the current batch of data, the batch ID that can be carried in the data packet is "3".

[0179] Optionally, the target format data packet may also include a third identifier, which indicates the capacity of the current batch of data. Optionally, this capacity can be the size of the batch of data, the number of files in the batch, or the number of data records. Figure 9 As shown, the third identifier can be the number of batch files. For example, if a batch of data contains 5 images, and each image is stored as a file, then the number of batch files can be "5". Another example: if a batch of data contains 6 data records, then the number of batch files can be "6".

[0180] Optionally, in this embodiment, if data security is involved, such as the need to encrypt target data, the encryption can be completed when the first device acquires the batch data. Subsequent transmission of the batch data does not require further encryption. That is, after acquiring the batch data, the first device encrypts it. Then, when transmitting the target data, it adds a header (e.g., batch ID, sub-batch ID, etc.) to the encrypted batch data and encapsulates it into a data packet of the target format before transmission. No further encryption is performed on the data packet of the target format encapsulated with the added header.

[0181] S509, The second device processes each batch of data.

[0182] In this context, c batches of data are received during the processing of the e-th batch, where c and e are both positive integers. That is, during the processing of a particular batch of data, the second device can receive one or more batches of data from the first device that are not currently being received by the second device from the first device. For example, as shown... Figure 7a , Figure 7b As shown, when the second device processes batch data of capacity k*S, the first device can transmit batch data of capacity (1-k)*S to the second device, such as... Figure 8b As shown, when the second device processes batch data with a capacity of k*S, the first device can transmit batch data with a capacity of (1-k)*S and batch data with a capacity of k*m to the second device.

[0183] Optionally, the second device can also start one or more threads to process multiple batches of data; that is, the process can be single-threaded or multi-threaded. One thread is used to process one or more batches of data. Optionally, the second device's operation of processing batches of data can be asynchronous, and asynchronous operations can start new threads. For example, after starting a thread to process one batch of data, the second device can perform other operations without waiting for that process to complete, such as restarting another thread to process another batch of data.

[0184] For example: Suppose that the end time of the second device processing the previous batch of data (i.e., the start time of processing the next batch of data) is the same as the end time of the first device transmitting the next batch of data. That is, the second device finishes processing the previous batch of data and just as it receives the next batch of data from the first device, the second device can start a thread to process these multiple batches of data. For example... Figure 8a As shown, the second device starts a thread to process one of the batches of data (e.g., a batch of data with a capacity of k*U). After the processing is completed, the next batch of data (e.g., a batch of data with a capacity of (1-k)*U) is transmitted to the second device. The second device continues to use the same thread to process the next batch of data, and so on, until all the batches of data are processed and the thread ends.

[0185] For example, suppose that while the second device is processing the previous batch of data, the first device has already transmitted the next batch of data to the second device. In this case, the second device can start another thread to process the next batch of data; that is, the first device can start multiple threads to process these multiple batches of data. Figure 8bAs shown, the second device starts a thread to process one of the batches of data (e.g., a batch of data with a capacity of k*U). During the processing, it receives the next batch of data transmitted by the first device (e.g., a batch of data with a capacity of (1-k)*U). The second device can then start another thread to process the next batch of data.

[0186] For example, suppose the second device has finished processing the previous batch of data, but the next batch of data has not yet been transmitted to the second device. In this case, the second device can either use the existing thread to process the next batch of data, or it can start a new thread to process the next batch of data. Figure 8c As shown, the second device starts a thread to process one of the batches of data (e.g., a batch of data with a capacity of k*U). After the processing is completed, the next batch of data (e.g., a batch of data with a capacity of (1-k)*U) changes but is not transmitted to the second device. At this time, the second device can use the original thread or terminate the original thread and restart a thread to process the next batch of data.

[0187] After batching, while processing one batch of data, it is also possible to process one or more other batches of data. For example... Figure 8b As shown, in the process of processing batches of data with a capacity of k*U, it is also possible to process batches of data with a capacity of (1-k)*U. Therefore, as Figures 7a to 8d As shown, after batch processing, the total processing time (i.e., the sum of the total processing time W for each batch of data) is less than (e.g., ... Figure 8b ) or equal to (e.g. Figure 8a The total time W for data processing before batch processing.

[0188] Optionally, the number of threads that the second device can start simultaneously may be less than a certain threshold, which can be set by the developers according to actual needs. This application does not limit this.

[0189] It should be understood that, Figures 8a to 8d In this case, since the target data was divided into four batches, it indicates that the size of the target data exceeds the second threshold. Therefore, the overall data synchronization time before batching is greater than [a certain threshold]. Figure 7a , Figure 7b The figure shows the total time taken before batching. However, after batching, the total time for data synchronization may be greater than, less than, or equal to... Figure 7a , Figure 7b The total time taken before batching is shown.

[0190] In summary, as Figures 7a to 8dAs shown, based on the above analysis of the total time spent on data preparation, data transmission, and data processing after batching, since the total time spent on data preparation after batching is less than or equal to the time spent on data preparation before batching, the total time spent on data transmission after batching is equal to the total time spent on data transmission before batching, and the total time spent on data processing after batching is less than or equal to the time spent on data processing before batching, the total time spent on data synchronization after batching (which, without considering the time spent in the data request phase, i.e., the sum of the aforementioned three) is also less than or equal to the total time spent on data synchronization before batching.

[0191] And, as Figures 7a to 8d As shown, since data preparation can be performed simultaneously with data transmission, and data preparation can also be performed simultaneously with processing, therefore, as Figures 7a to 8d As shown, the total time for data synchronization after batching is close to the total time for data transmission before batching. Therefore, the total time for data synchronization after batching is less than the total time for data synchronization before batching.

[0192] Based on the above technical solution, this application adopts a dynamic batching strategy. Different strategies are used to process the data in batches according to the different amounts of data requested by the data requesting end. The capacity of each batch of data can be determined according to the batching coefficient. Furthermore, data preparation can be carried out while data is being transmitted, and data transmission can be carried out while data processing is being performed. This can effectively reduce the overall time consumption of the data synchronization stage and improve the performance of the system.

[0193] In some embodiments, the first threshold mentioned in step S506 above is related to time gain. Here, time gain is the difference between a first duration and a second duration, where the first duration is the time required to process the target data in batches, and the second duration is the time required to process the target data without batching. That is, time gain refers to the difference between the total time spent on data synchronization when processing the target data in batches using dynamic batching and the total time spent on data synchronization when processing the target data without batching. Based on this design, this application considers the issue of time gain when determining the first threshold. Developers can set an appropriate time gain according to actual needs, and thus determine the corresponding first threshold for batch processing of data, ensuring time gain.

[0194] The second threshold mentioned in step S505 above is related to time requirements, which are the user's expectations regarding the time it takes for a batch of data to arrive at the second device. Here, time requirements can also be understood as human factors, which refer to a statistical result for different users. Different users may have different time requirements for the arrival of a batch of data at the second device. For example, when browsing short videos, some users may expect a video to start playing within 1 second or less, while others may not have such high expectations and consider playing a video within 5 seconds to be sufficient. Therefore, human factors are a statistical result of different user requirements. Based on this design, this application considers the timeliness requirements of human factors when determining time requirements. Developers can set time requirements according to actual needs, and then determine the corresponding second threshold based on these time requirements for batch data processing. This effectively reduces the overall data synchronization time while ensuring timely data response to users, improving the user experience.

[0195] The batching coefficient is determined by the data transmission time and the data processing time.

[0196] Here, data transmission time refers to the duration taken for the first device to transmit a unit of data to the second device, and this data transmission time is related to the performance of network I / O. For example, the data transmission time can be determined based on the performance of network I / O.

[0197] In some examples, the data processing time is the duration taken by the second device to process a unit of data, and this time is related to the performance of the second device's write I / O. For example, the data processing time can be determined based on the performance of the second device's write I / O. In other examples, since the data processing phase may only involve write I / O operations by the second device, the data processing time may be related only to the performance of the second device's write I / O. Of course, if the data processing phase also involves read I / O operations by the second device, the data processing time may also be related to the performance of the second device's read I / O. For example, the data processing time can be determined based on the performance of the second device's write I / O and read I / O.

[0198] Based on this design, this application uses data transmission time and data processing time to determine the batching coefficient. It also considers that data transmission time is related to network I / O performance and data processing time is related to the write I / O performance of the second device. Therefore, the determined batching coefficient can be applied to scenarios with different network I / O performance and / or different write I / O performance. That is, it can effectively reduce the overall time of data synchronization process for scenarios with different network I / O performance and / or different write I / O performance.

[0199] In other embodiments, the first threshold can be determined by the size of a single target data entry, the batching coefficient, the time gain, the data preparation time, and the data processing time. Specifically, when the target data is stored as a file, a single target data entry is a single file; and / or, when the target data is stored as a data record, a single target data entry is a single data record. Of course, when the target data is stored in other formats, a single target data entry can also be a single file in other formats.

[0200] For a detailed introduction and distinction between single target data and unit data, please refer to the above introduction on unit data and single data.

[0201] Based on this design, the first threshold in this application is determined by the capacity of a single target data item, the batching coefficient, the time gain, the data preparation time, and the data processing time. Therefore, the determination of the first threshold takes into account the magnitude of the time gain, ensuring that the time gain is guaranteed when processing data in batches, thus effectively reducing the overall time consumption of the data synchronization process. Furthermore, considering that the data preparation time is related to the read I / O performance of the first device, and the data processing time is related to the write I / O performance of the second device, the determined first threshold can be applied to scenarios with different read / write I / O performances, effectively reducing the overall time consumption of the data synchronization process for scenarios with different read / write I / O performances.

[0202] The second threshold can be determined by the first threshold, the batching coefficient, the data preparation time, the data transmission time, the data processing time, and the time requirement.

[0203] Data preparation time is the time taken for the first device to acquire a unit of data, and it is related to the read I / O performance of the first device. Since the data preparation phase may only involve read I / O operations of the second device, the data preparation time may only be related to the read I / O performance of the first device. Of course, if the data preparation phase also involves write I / O operations of the second device, the data preparation time may also be related to the write I / O performance of the first device.

[0204] Based on this design, the determination of the second threshold in this application takes into account that the data preparation time is related to the performance of the first device's read I / O, the data transmission time is related to the performance of the network I / O, and the data processing time is related to the performance of the second device's write I / O. Therefore, the determined second threshold can be applied to scenarios with different performance levels of read / write I / O and network I / O. That is, it can effectively reduce the overall time consumption of the data synchronization process in scenarios with different performance levels of read / write I / O and network I / O. Time requirements are also considered, ensuring that while reducing the overall data synchronization time, data can be promptly responded to the user, improving the efficiency of human-computer interaction and enhancing the user experience.

[0205] The following examples will provide a detailed explanation of how to determine the batching coefficient, the first threshold, and the second threshold.

[0206] When determining the batching coefficients, since the time spent in the data request phase is basically fixed, while the time spent in the data preparation, data transmission, and data processing phases may change dynamically, in order to minimize the overall time spent in the data synchronization process, the data preparation, data transmission, and data processing phases can be maximized to run in parallel, such as... Figure 8a As shown, this allows the data preparation process for the next batch of data to run in parallel with the data transmission process for the previous batch, and the data transmission process for the next batch of data to run in parallel with the data processing process for the previous batch. This makes the total network transmission time for the next batch of data approach the total data processing time for the previous batch, and consequently, makes the overall time of the entire data synchronization process approach the total time of the network transmission phase. For example, suppose that when the second device finishes processing the i-th batch of data, the (i+1)-th batch of data is just being transmitted to the second device. Therefore, the second device can immediately start processing the (i+1)-th batch of data, minimizing the overall time of the three phases of data preparation, data transmission, and data processing.

[0207] In most scenarios, the total time spent on data preparation is relatively small compared to the total time spent on data transmission and data processing. Therefore, in the embodiments of this application, Equations 1.1 to 1.2 can be used to calculate the batching coefficient.

[0208] α*k*S*W=(1-k)*S*B 1.1

[0209] In Equation 1.1, k is the batching coefficient, S is the target data capacity, α is the flexibility coefficient (a real number greater than 0 and less than or equal to 1, with a default value of 1), B is the data transmission time, and W is the data processing time. k, S, B, and W are all positive real numbers.

[0210] In this process, the data requested by the second device is divided into two batches. In Equation 1.1, α*k*S*W is the time taken for the second device to process the first batch of data, and (1-k)*S*B is the time taken for the first device to transmit the second batch of data to the second device. Since the two are equal, the data transmission and data processing stages can be maximized to run in parallel.

[0211] Simplifying 1.1, we get:

[0212]

[0213] Optionally, the batching factor can be a real number greater than or equal to 0.2 and less than or equal to 0.8.

[0214] When determining the first threshold, it is considered that the first threshold is related to time gain and also to the capacity of a single data entry. Therefore, in the embodiments of this application, the first threshold can be calculated according to Equations 1.3 to 1.5.

[0215] L g *(R+B+W)-(k*L g *R+(1-k)*L g *W+L g *B)=G 1.3

[0216] In Equation 1.3, k is the batching coefficient, G is the time gain (optional, G is greater than or equal to 30ms), R is the data preparation time, B is the data transmission time, and W is the data processing time.

[0217] Simplifying 1.3, we get:

[0218]

[0219] Finally, the first threshold can be obtained as:

[0220] L = max(L1, L g 1.5

[0221] In Equation 1.5, L is the first threshold, and L1 is the size of a single data entry. Optionally, the unit of the first threshold can be MB, KB, etc., and this application does not specifically limit it. Optionally, the first threshold is greater than 0 MB and less than or equal to 1 MB.

[0222] When determining the second threshold, considering that the second threshold is related to the revenue and that the second threshold should be greater than or equal to the first threshold, the second threshold can be calculated according to Equations 1.6 to 1.10 in the embodiments of this application.

[0223] First, calculate U according to Equation 1.6. x .

[0224] k*U x *(R+W+B)=RT 1.6

[0225] In Equation 1.6, RT represents the time requirement, i.e., the time expected by the user for a batch of data to arrive at the user's end. Optionally, the unit of RT can be milliseconds, seconds, etc., and this application does not specifically limit it. Optionally, RT is greater than or equal to 500ms and less than or equal to 700ms. k, R, B, and W are described above and will not be repeated here.

[0226] Simplifying equation 1.6, we get:

[0227]

[0228] Then calculate U according to Equation 1.8 y .

[0229] (1-k)*U y *(R+W+B)=RT 1.8 Simplifying equation 1.8, we get:

[0230]

[0231] Finally, according to U x And U y Determine the second threshold.

[0232] U = max(min(U) x U y ),L) 1.10

[0233] In Equation 1.10, U is the second threshold and L is the first threshold. Optionally, the unit of the second threshold can be MB, KB, etc., and this application does not specifically limit it. Optionally, the second threshold is greater than or equal to 2MB and less than or equal to 6MB.

[0234] It should be noted that, as mentioned above, the time spent on data preparation, data transmission, and data processing is related to the performance of read / write I / O and network I / O. Different devices, due to varying hardware configurations, will exhibit different read / write I / O and network I / O performance. Furthermore, different communication connection methods between devices will result in varying network I / O performance.

[0235] Therefore, the aforementioned data preparation time, data transmission time, and data processing time can be obtained through training a model. This model can be a mathematical statistical model or other types of models, and this application does not limit this. During the model training process, the data capacity is used as input, and the output values ​​of data preparation time, data transmission time, and data processing time are tested under different read / write I / O performance and different network I / O performance scenarios. Curve fitting is then performed to obtain the final result, namely, the relationship between data capacity and data preparation time, data transmission time, and data processing time.

[0236] For example, the final obtained curve can be expressed by functions in the form of Equations 1.11 to 1.13 respectively.

[0237] R = γ * log(S + c) R )+Δ R 1.11

[0238] W = θ * log(S + c) W )+Δ W 1.12

[0239]

[0240] In equations 1.11 to 1.13, S represents the input data capacity, R represents the data preparation time, W represents the data processing time, and B represents the data transmission time. γ, θ, c R c W c B Δ R Δ W Δ B The coefficient can be calculated based on the input value S and the output values ​​R, W, and B.

[0241] After a communication connection is established between devices, the performance of network I / O and read / write I / O will dynamically change due to their instability. Consequently, the data preparation time, data transmission time, and data processing time will also change dynamically. Therefore, the batching coefficient, the first threshold, and the second threshold mentioned above may also change dynamically.

[0242] For example, after the first device and the second device establish a communication connection, the second device determines the initial data preparation time, data transmission time, and data processing time based on the capacity of the target data requested by the first device and the performance of current read / write I / O and network I / O. Then, based on the initial data preparation time, data transmission time, and data processing time, the second device calculates the initial batching coefficient, the first threshold, the second threshold, etc. Finally, the target data is processed in batches based on the initial batching coefficient, the first threshold, the second threshold, etc.

[0243] Subsequently, due to the instability of read / write I / O and network I / O, the initial data preparation time, data transmission time, and data processing latency may change. In this case, the first device can recalculate the corresponding batching coefficients, first threshold, and second threshold based on the actual data preparation time, network transmission time, and data processing time, and update these coefficients. Then, the data is processed in batches according to the updated batching coefficients, first threshold, and second threshold. For details on how the first device obtains the actual data preparation time, data transmission time, and data processing time, please refer to the section above on how the first device obtains feedback information; this will not be repeated here. Optionally, this update operation can be performed periodically or actively triggered by the first device and / or the second device; this application does not limit this.

[0244] Based on this design, this application updates the values ​​of the corresponding batching coefficient, first threshold, and second threshold according to the actual data transmission time, network transmission time, and data processing time, so that the batching coefficient, first threshold, and second threshold can adapt to the current operating environment in real time, such as the current performance of network I / O and read / write I / O, further reducing the overall data synchronization time and improving the system performance of the device.

[0245] It should be noted that the method provided in the embodiments of this application is also applicable to scenarios with more or fewer than three stages (data preparation, data transmission, and data processing), and this application does not limit it in this regard.

[0246] The above primarily describes the solutions provided by the embodiments of this application from a methodological perspective. It is understood that, in order to achieve the above functions, the device includes corresponding hardware structures and / or software modules for executing each function. By combining the units and algorithm steps of the various examples described in the embodiments disclosed in this application, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by a computer driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solutions of the embodiments of this application.

[0247] This application provides embodiments that can divide the device into functional modules based on the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0248] like Figure 10 The diagram shown is a structural schematic of a data synchronization device provided in an embodiment of this application. This data synchronization device 1000 can be used to implement the methods executed by the first or second device described in the above method embodiments. For example, the data synchronization device 1000 specifically includes a processing unit 1001 and a communication unit 1002.

[0249] In one possible example, taking data synchronization device 1000 as the first device, the processing unit 1001 is used to support the execution of the first device. Figure 5 Steps S502, S503, S506, and S507, and / or other processing operations performed by the first device in this embodiment of the application. The communication unit 1002 is used to support the first device in performing these steps. Figure 5Steps S501, S504, and S508, and / or other communication operations performed by the first device in the embodiments of this application.

[0250] In another possible example, taking data synchronization device 1000 as the second device, the processing unit 2401 is used to support the second device in performing... Figure 5 Steps S505 and S509, and / or other processing operations performed by the second device in the embodiments of this application. Communication unit 2402 is used to support the second device in performing these operations. Figure 5 Steps S501, S504, and S508, and / or other communication operations performed by the second device in the embodiments of this application.

[0251] Optional, Figure 10 The data synchronization device 1000 shown may also include a storage unit ( Figure 10 (not shown in the image), this storage unit stores a program or instruction. When the processing unit executes the program or instruction, it causes... Figure 10 The data synchronization device shown can perform Figure 5 The data synchronization method performed by the first or second device shown.

[0252] Optional, Figure 10 The data synchronization device shown may also include a display unit ( Figure 10 (Not shown in the image), the display unit may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a microelectromechanical system (MEMS) display, or an electronic paper display. It is used to display content that can be viewed by the user (e.g., text, images, videos, icons, symbols, etc.). In embodiments of this application, the display unit may be implemented as a display screen.

[0253] Figure 10 The technical performance of the data synchronization device 1000 shown can be used as a reference. Figure 5 The technical effects of the data synchronization method shown will not be elaborated here. Figure 10 The processing unit 1001 involved in the data synchronization device 1000 shown can be implemented by a processor or processor-related circuit components, and can be a processor or a processing module. The communication unit 1002 can be implemented by a transceiver or transceiver-related circuit components, and can be a transceiver or a transceiver module.

[0254] This application also provides a chip system, such as... Figure 11As shown, the chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 and the interface circuit 1102 are interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices. As another example, the interface circuit 1102 can be used to send signals to other devices (e.g., the processor 1101). Exemplarily, the interface circuit 1102 can read instructions stored in memory and send those instructions to the processor 1101. When the instructions are executed by the processor 1101, the data synchronization device can perform the steps executed by the first or second device in the above embodiments. Of course, the chip system may also include other discrete components, which are not specifically limited in this application embodiment.

[0255] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0256] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application does not specifically limit the type of memory or the arrangement of the memory and processor.

[0257] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0258] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0259] This application provides a computer-readable storage medium storing a computer program or instructions. When the computer program or instructions are run on a computer, the computer performs the method described in the above-described method embodiments.

[0260] This application provides a computer program product, which includes a computer program or instructions that, when run on a computer, cause the computer to perform the methods described in the above-described method embodiments.

[0261] In addition, embodiments of this application also provide an apparatus, which may specifically be a component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the apparatus to perform the methods in the above-described method embodiments.

[0262] In this application, the device, computer-readable storage medium, computer program product or chip provided in the embodiments are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0263] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0264] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The embodiments can be combined with or referenced to each other without conflict. The device embodiments described above are merely illustrative; for example, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of modules or units may be electrical, mechanical, or other forms.

[0265] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0266] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0267] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program instructions, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0268] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data synchronization method, characterized in that, The method includes: The first device receives a request message from the second device, the request message being used to request the acquisition of target data; If the capacity of the target data is less than or equal to a first threshold, the first device acquires the target data and transmits the target data to the second device. If the size of the target data is greater than the first threshold, the first device determines the size of each batch of data in the multiple batches into which the target data needs to be divided based on a batching coefficient; the batching coefficient is determined by the data transmission time and the data processing time; the data transmission time is the time taken by the first device to transmit a unit of data to the second device, and the data transmission time is related to the performance of network input / output I / O; the data processing time is the time taken by the second device to process the unit of data, and the data processing time is related to the performance of the second device's write I / O. The batching coefficients satisfy the following conditions: Among them, the The batching coefficient is the number of batches. The real number is greater than 0 and less than or equal to 1. For the data transmission time, the The data processing time is [time]. The above The above All are positive real numbers; The first device acquires each batch of data according to the capacity of each batch of data, and transmits each batch of data to the second device; Among them, y batches of data in the plurality of batches are obtained during the transmission of the xth batch of data in the plurality of batches of data, where x and y are positive integers.

2. The method according to claim 1, characterized in that, If the size of the target data is greater than the first threshold and less than or equal to the second threshold, the target data will be divided into two batches of data; wherein the second threshold is greater than the first threshold.

3. The method according to claim 2, characterized in that, Of the two batches of data, one batch has a capacity of [missing information]. The capacity of another batch of data is Wherein, S is the capacity of the target data, k is the batching coefficient, and both S and k are positive real numbers.

4. The method according to any one of claims 1-3, characterized in that, If the size of the target data is greater than the second threshold, the target data will be divided into 2a1 batches and one batch of data with a size of m; wherein, in the 2a1 batches, the size of each batch is determined by the second threshold and the batching coefficient, a1 is a positive integer, m is greater than 0 and less than or equal to the first threshold; the second threshold is greater than the first threshold; Alternatively, the target data may be divided into 2a2 batches; wherein, the capacity of each batch is determined by the second threshold and the batching coefficient, and a2 is a positive integer greater than or equal to 2. Alternatively, the target data may be divided into 2a3 batches and another 2 batches; wherein, in the 2a3 batches, the capacity of each batch is determined by the second threshold and the batching coefficient, and a3 is a positive integer; the sum of the capacities of the other 2 batches is m, and the capacity of each batch is determined by m and the batching coefficient, where m is greater than the first threshold and less than the second threshold.

5. The method according to claim 4, characterized in that, The first threshold is determined by the capacity of a single target data item, the batching coefficient, the time gain, the data preparation time, and the data processing time. The data preparation time is the time it takes for the first device to acquire a unit of data, and the data preparation time is related to the read I / O performance of the first device. The data processing time is the time taken for the second device to process the unit of data, and the data processing time is related to the write I / O performance of the second device. The time gain is the difference between a first duration and a second duration, where the first duration is the time required to process the target data in batches, and the second duration is the time required to process the target data without batches. When the target data is stored in the form of a file, the single target data is a single file, and / or, when the target data is stored in the form of a data record, the single target data is a single data record.

6. The method according to claim 5, characterized in that, The first threshold satisfies the following condition: Among them, the As the first threshold, the The batching coefficient, the For the capacity of a single target data item, the The time required for data preparation, the For data processing time, the For time gain, the stated The above The above The above The above The above All are positive real numbers.

7. The method according to claim 6, characterized in that, The second threshold is determined by the first threshold, the batching coefficient, the data preparation time, the data transmission time, the data processing time, and the time requirement; The data transmission time is the time taken for the first device to transmit a unit of data to the second device, where the unit of data belongs to the target data, and the data transmission time is related to the performance of network I / O. The data processing time is the time taken for the second device to process the unit of data, and the data processing time is related to the write I / O performance of the second device. The data preparation time is the time it takes for the first device to acquire the unit data, and the data preparation time is related to the read I / O performance of the first device. The time requirement refers to the time required for a batch of data to arrive at the second device.

8. The method according to claim 7, characterized in that, The second threshold satisfies the following condition: Among them, the The second threshold, the The batching coefficient, the As the first threshold, the The time required for data preparation, the For data processing time, the For data transmission time, the Due to time constraints, the aforementioned The above The above The above The above The above The above All are positive real numbers.

9. The method according to claim 8, characterized in that, The first device transmits each batch of data to the second device, including: The first device transmits each batch of data to the second device using data packets in the target format; The target format data packet carries the following identifier: The first identifier is used to indicate that the batches of data carried in the data packet belong to the same batch of data; And / or, a second identifier, the second identifier being used to indicate the number of batches into which the target data is divided and the current order of the batches in the target data; or, the second identifier being used to indicate the order of the batches into which the target data is divided in the target data.

10. The method according to claim 9, characterized in that, The target format data packet also carries a third identifier, which is used to indicate the capacity of the current batch of data.

11. A data synchronization method, characterized in that, The method includes: The second device sends a request message to the first device, the request message being used to request the acquisition of target data; If the size of the target data is less than or equal to a first threshold, the second device receives the target data from the first device and processes the target data. If the capacity of the target data is greater than the first threshold, the second device receives multiple batches of data from the first device and processes the multiple batches of data; the multiple batches of data are obtained by dividing the target data, and the capacity of the batches of data is determined by a batching coefficient; the batching coefficient is determined by the data transmission time and the data processing time; the data transmission time is the time taken by the first device to transmit a unit of data to the second device, and the data transmission time is related to the performance of network input / output I / O; the data processing time is the time taken by the second device to process the unit of data, and the data processing time is related to the performance of the second device's write I / O. The batching coefficients satisfy the following conditions: Among them, the The batching coefficient is the number of batches. The real number is greater than 0 and less than or equal to 1. For the data transmission time, the The data processing time is [time]. The above The above All are positive real numbers; Among them, c batches of data in the plurality of batches are received during the processing of the e-th batch of data in the plurality of batches of data, where c and e are positive integers.

12. The method according to claim 11, characterized in that, If the size of the target data is greater than the first threshold and less than or equal to the second threshold, the target data will be divided into two batches of data; wherein the second threshold is greater than the first threshold.

13. The method according to claim 12, characterized in that, Of the two batches of data, one batch has a capacity of [missing information]. The capacity of another batch of data is Wherein, S is the capacity of the target data, k is the batching coefficient, and both S and k are positive real numbers.

14. The method according to any one of claims 11-13, characterized in that, If the size of the target data exceeds the first threshold, the second device receives multiple batches of data from the first device, further including: If the capacity of the target data is greater than the second threshold, the second device receives 2a1 batches of data and one batch of data with a capacity of m from the first device; wherein, in the 2a1 batches of data, the capacity of each batch of data is determined by the second threshold and the batching coefficient; a1 is a positive integer, m is greater than 0 and less than or equal to the first threshold; the second threshold is greater than the first threshold; Alternatively, the second device receives 2a2 batches of data from the first device; wherein, the capacity of each batch of data is determined by the second threshold and the batching coefficient; and a2 is a positive integer greater than or equal to 2. Alternatively, the second device receives 2a3 batches of data from the first device; wherein, the capacity of each batch of data is determined by the second threshold and the batching coefficient, and a3 is a positive integer; The second device also receives two batches of data from the first device. The sum of the capacities of the two batches of data is m. The capacity of each batch of data is determined by m and a batching coefficient. The m is greater than the first threshold and less than the second threshold.

15. The method according to claim 14, characterized in that, The second device receives multiple batches of data from the first device, including: The second device receives multiple batches of data transmitted using target format data packets from the first device; The target format data packet carries the following identifier: The first identifier is used to indicate that the batches of data carried in the data packet belong to the same batch of data; And / or, a second identifier, the second identifier being used to indicate the number of batches into which the target data is divided and the order of the current batch of data in the target data; or, the second identifier being used to indicate the order of the current batch of data in the target data.

16. The method according to claim 15, characterized in that, The target format data packet also carries a third identifier, which is used to indicate the capacity of the current batch of data.

17. A data synchronization device, characterized in that, The data synchronization device is a first device, which includes a processor and a memory coupled to the processor. The memory is used to store computer program code, which includes computer instructions. The processor reads the computer instructions from the memory, causing the first device to execute the method as described in any one of claims 1-10.

18. A data synchronization device, characterized in that, The data synchronization device is a second device, which includes a processor and a memory coupled to the processor. The memory is used to store computer program code, which includes computer instructions. The processor reads the computer instructions from the memory, causing the second device to execute the method as described in any one of claims 11-16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-10, or cause the computer to perform the method as described in any one of claims 11-16.

20. A computer program product, characterized in that, The computer program product includes: a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-10, or cause the computer to perform the method as described in any one of claims 11-16.

Images