Data transfer method and apparatus, electronic equipment, and storage medium

Abstract

This application relates to a data transfer method and apparatus, electronic equipment, and storage medium, the method comprising: obtaining a first vehicle information list to determine a first vehicle with data processing needs; obtaining a transfer task list to determine the execution status of the transfer task for each first vehicle; screening the vehicle information in the first vehicle information list; removing vehicle information for which a transfer task has been performed; obtaining a second vehicle information list; then executing the transfer task for each second vehicle based on the second vehicle information list; transferring the driving data of each second vehicle to a data processing platform using a multithreaded asynchronous transfer method in accordance with a cyclic queue; thereby allowing the data processing platform to perform subsequent processing on the driving data and updating the execution status of the transfer task in the transfer task list. This method ensures efficient access to the extracted driving data and that the driving data is transferred to the data processing platform without duplication or omission.

Description

Technical Field

[0001] This application relates to the field of vehicle network technologies, and particularly to a data transfer method and apparatus, an electronic device, and a storage medium.

Background Art

[0002] With the rapid development of the automotive industry, vehicle networks are gradually showing a growth trend. The data generated by vehicle networks is increasing, and it has become urgent to synchronize the driving data stored in the vehicle network database to a data processing platform and perform data processing and analysis on these driving data by the data processing platform.

Summary of the Invention

[0003] This application provides a data transfer method and apparatus, an electronic device, and a storage medium, and the technical solution is as follows.

[0004] An embodiment of the first aspect of the present application includes the steps of: obtaining a first vehicle information list, wherein the first vehicle information list includes vehicle information for a plurality of first vehicles, the vehicle information for the first vehicles indicates a vehicle with data processing needs, the data processing needs refer to a data processing platform performing data processing on the driving data of the first vehicles; obtaining a transfer task list, wherein the transfer task list includes the execution status of a transfer task for each first vehicle, the transfer task for the first vehicle refers to transferring the driving data of the first vehicle in a database for storing driving data uploaded by the vehicle via a vehicle network to the data processing platform; and obtaining the first vehicle information list based on the transfer task list. The present invention provides a data transfer method comprising the steps of screening vehicle information in a database and obtaining a second vehicle information list, wherein the second vehicle information list includes vehicle information for at least one second vehicle, and the vehicle information for the second vehicle indicates a vehicle for which a transfer task has not been performed in the plurality of first vehicles; and executing a transfer task for each second vehicle based on the second vehicle information list, and updating the execution status of the transfer task for each second vehicle in the transfer task list, wherein executing a transfer task for a second vehicle includes storing the driving data of the second vehicle in the database in a circular queue and transferring the driving data in the circular queue to the data processing platform using a multithreaded asynchronous transfer method.

[0005] In some embodiments, the execution status of the transfer task includes the execution progress of the transfer task, and the method further includes, if the transfer task list contains a first transfer task which is a transfer task whose execution has been interrupted, continuing the execution of the first transfer task based on the execution progress of the first transfer task in the transfer task list until the execution of the first transfer task is successful.

[0006] In some embodiments, the execution status of the transfer task includes the cause of the execution abnormality of the transfer task, and the method further includes compensating the second transfer task in the transfer task list based on the cause of the execution abnormality of the second transfer task in the transfer task list, if the transfer task list contains a second transfer task which is an abnormally executed transfer task.

[0007] In some embodiments, the method further includes receiving a first timing task instructing to execute a transfer task during a first time period in which the frequency of use of driving data is less than or equal to a first threshold, and receiving a second timing task instructing not to execute a transfer task during a second time period in which the frequency of use of driving data is greater than a second threshold.

[0008] In some embodiments, the execution status of the transfer task includes at least one of the following: vehicle information of the vehicle corresponding to the transfer task, task type of the transfer task, execution status, execution duration, interruption time, execution progress, and cause of execution abnormality.

[0009] Embodiments of a second aspect of the present application include: a first acquisition module for acquiring a first vehicle information list, wherein the first vehicle information list includes vehicle information for a plurality of first vehicles, the vehicle information for the first vehicles indicates a vehicle with data processing needs, the data processing needs refer to a data processing platform performing data processing on the driving data of the first vehicles; a second acquisition module for acquiring a transfer task list, wherein the transfer task list includes the execution status of a transfer task for each first vehicle, the transfer task for the first vehicle refers to transferring the driving data of the first vehicle in a database for storing driving data uploaded by the vehicle via a vehicle network to the data processing platform; and based on the transfer task list, the first vehicle information A screening module that screens vehicle information in a list and obtains a second vehicle information list, wherein the second vehicle information list includes vehicle information for at least one second vehicle, and the vehicle information for the second vehicle indicates a vehicle for which a transfer task has not been performed on any of the plurality of first vehicles; and a task execution module that executes a transfer task for each second vehicle based on the second vehicle information list and updates the execution status of the transfer task for each second vehicle in the transfer task list, wherein executing a transfer task for a second vehicle includes storing the driving data of the second vehicle in the database in a circular queue and transferring the driving data in the circular queue to the data processing platform using a multithreaded asynchronous transfer method.

[0010] In some embodiments, the execution status of the transfer task includes the execution progress of the transfer task, and the task execution module is further used to continue the execution of the first transfer task until the execution of the first transfer task is successful, based on the execution progress of the first transfer task in the transfer task list, if the transfer task list contains a first transfer task which is a transfer task whose execution has been interrupted.

[0011] In some embodiments, the execution status of the transfer task includes the cause of execution abnormality of the transfer task, and the task execution module is further used to compensate the second transfer task based on the cause of execution abnormality of the second transfer task in the transfer task list if the transfer task list contains a second transfer task which is an abnormally executed transfer task.

[0012] In some embodiments, the device further includes a receiving module used for at least one of: receiving a first timing task instructing to execute a transfer task during a first time period in which the frequency of use of driving data is less than or equal to a first threshold; and receiving a second timing task instructing not to execute a transfer task during a second time period in which the frequency of use of driving data is greater than a second threshold.

[0013] In some embodiments, the execution status of the transfer task includes at least one of the following: vehicle information of the vehicle corresponding to the transfer task, task type of the transfer task, execution status, execution duration, interruption time, execution progress, and cause of execution abnormality.

[0014] A third embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data transfer method described in the embodiment.

[0015] A fourth embodiment of the present application provides a computer-readable storage medium on which a computer program is stored and which is executed by a processor in order to realize the data transfer method described in the above embodiment.

[0016] Additional aspects and advantages of the present application are given in part in the following description, some of which become apparent from the following description or are understood through the practice of the present application. [Brief explanation of the drawing]

[0017] The above and / or additional aspects and advantages of the present application will become clearer and easier to understand from the following description of embodiments based on the drawings, where, [Figure 1] This is a flowchart of the data transfer method provided by the embodiment of the present invention. [Figure 2] This is a flowchart of another data transfer method provided by the embodiment of the present invention. [Figure 3] This is a schematic block diagram of a data transfer device provided by an embodiment of the present invention. [Figure 4] This is a schematic diagram of the structure of the electronic device provided by the embodiment of the present application. [Modes for carrying out the invention]

[0018] The embodiments of the present application will be described in detail below, and examples of the above embodiments are shown in the drawings, where the same or similar reference numerals throughout the text represent the same or similar elements, or elements having the same or similar functions. The embodiments described below with reference to the drawings are illustrative and intended for use in interpreting the present application, and should not be understood as limiting the present application.

[0019] It should be explained that information such as vehicle information and transfer task lists relating to the embodiments of this application are all obtained only when the user and each party are fully aware of and authorized to do so, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0020] In related technologies, vehicle driving data is stored using an HBase database, data is extracted and analyzed from the HBase database using Kettle (a data conversion tool), and then transferred to a data processing platform, where the data processing platform performs subsequent processing on the received data.

[0021] The data transfer method and apparatus, electronic equipment, and storage medium of the embodiment of this application will be described below with reference to the drawings. In related technologies, when transferring data from an HBase database to a data processing platform using the Kettle tool, problems exist such as duplicate and missing data extraction, complex flows, low transfer efficiency, and inability to resume from breakpoints. Based on these, the present invention provides a data transfer method in which a first vehicle with data processing needs is determined by obtaining a first vehicle information list, and the execution status of the transfer task for each first vehicle is determined by obtaining a transfer task list. Thereafter, the vehicle information in the first vehicle information list is screened, vehicle information for which a transfer task has been executed is removed, and vehicle information for which a transfer task has not been executed is reserved. A second vehicle information list is then obtained to determine the second vehicles waiting for the execution of a transfer task. Subsequently, the transfer task for each second vehicle is executed based on the second vehicle information list, and the driving data of each second vehicle is transferred to a data processing platform (e.g., a cloud platform) using a multi-threaded asynchronous transfer method in accordance with a circular queue. The data processing platform then performs subsequent processing on the driving data and updates the execution status of the transfer task in the transfer task list. This method allows for efficient access to extracted driving data and ensures that the driving data is transferred to the data processing platform without duplication or loss.

[0022] Specifically, the data transfer method provided by this application is performed by an electronic device, which may be, for example, a terminal or a server, and is not limited thereto. Here, the electronic device is connected to a data processing platform (for example, via a wired or wireless network), and the data processing platform may be a cloud platform for providing cloud computing services, such as a TSP (Telematics Service Provider, Content Service Provider) platform, and is not limited thereto.

[0023] FIG. 1 is a flowchart of a data transfer method provided by an embodiment of the present application. As shown in FIG. 1, the data transfer method is executed by an electronic device, and the data transfer method includes the following steps.

[0024] In step S101, a first vehicle information list is obtained. The first vehicle information list includes vehicle information of a plurality of first vehicles. The vehicle information of the first vehicle indicates a vehicle having data processing needs. The data processing needs refer to performing data processing on the driving data of the first vehicle by a data processing platform.

[0025] Here, the electronic device can access a first relational database (such as mysql, oracle, etc.) for storing a Vehicle vehicle table including vehicle information of multiple vehicles. The first relational database may be arranged on a data processing platform or on the electronic device, and the present application is not limited thereto. Here, the vehicle information includes a vehicle identifier (such as a vin code, a frame number, that is, the unique identifier of each vehicle), a vehicle type, a vehicle production time, etc., and the present application is not limited to the specific content of the vehicle information. In this step, the electronic device screens the vehicle information that meets the screening conditions from the first relational database, and generates a first vehicle information list based on the screened vehicle information. Here, the screening condition is that there are data processing needs in the vehicle. Exemplarily, whether there are data processing needs in the vehicle may be determined based on the online time of the target item. In some embodiments, the electronic device determines whether there are data processing needs in the vehicle based on the production time of the vehicle and the online time of the target item in the vehicle information. Here, if the production time of the vehicle is before the online time of the target item, it indicates that there are data processing needs in the vehicle, and in this case, the vehicle is determined as the first vehicle. For example, the target item is to count the vehicle driving data from YY year to ZZ year in the XX region, etc., and the present application is not limited thereto. Also, the driving data of the vehicle includes, but is not limited to, the driving route of the vehicle, the fuel consumption rate, the driving time length, etc.

[0026] In some embodiments, after screening the vehicle information of multiple first vehicles from the first relational database, the electronic device sorts the vehicle information of the multiple first vehicles, and obtains a first vehicle information list, also called vinList, with the vin code of the vehicle as the index. Exemplarily, the electronic device sorts the vehicle information of the multiple first vehicles based on the production time of the multiple first vehicles to obtain a first vehicle information list. For example, the vehicle information of the multiple first vehicles is sorted in the order from far to near production time, and the present application is not limited thereto.

[0027] Furthermore, embodiments of the present invention may include a timing task, which, when timing task 1 is triggered, screens vehicle information prior to the online time of a target item in the first relational database Vehicle table, sorts the screened vehicle information, and obtains a data list vinList, i.e., a first vehicle information list. Exemplarily, an electronic device receives a first timing task instructing it to perform a transfer task during a first time period, which is a time period during which the frequency of use of driving data is less than or equal to a first threshold. Here, the transfer task refers to the transfer of vehicle driving data from the database to the data processing platform for processing the vehicle driving data. The first time period is a preset time period that can be set according to service requirements, for example, the first time period is from midnight to 6am. The first threshold is a preset threshold that can be set according to service requirements, for example, the first threshold is a data usage frequency of 10, but the present invention is not limited thereto.

[0028] In step S102, a transfer task list is obtained, which includes the execution status of the transfer tasks for each first vehicle, where the transfer task for the first vehicle refers to transferring the first vehicle's driving data, which is stored in a database for storing driving data uploaded by the vehicle via the vehicle network, to the data processing platform.

[0029] Here, the electronic device can access a second relational database (e.g., MySQL, Oracle, etc.) for storing a list of transfer tasks, which may be represented as, for example, "T_COMMERCIAL_TASK". This second relational database may be located on a data processing platform or on the electronic device, and is not limited thereto. For any first vehicle transfer task, the execution status of the transfer task may include, and is not limited thereto, vehicle information of the vehicle corresponding to the transfer task, the task type of the transfer task, the execution status, execution duration, interruption time, execution progress, cause of execution error, etc. In this application, the electronic device can access a database for storing driving data, for example, an HBase database, in which the driving data is uploaded by the vehicle via the vehicle network. It should be understood that the HBase database has good read / write characteristics and can support tables containing large amounts of data, making it suitable for online databases of large amounts of data and simple services, and responsible for aggregating data from each service system.

[0030] In step S103, based on the transfer task list, a screening is performed on the vehicle information in the first vehicle information list to obtain a second vehicle information list, the second vehicle information list includes vehicle information for at least one second vehicle that indicates a vehicle for which a transfer task has not been performed among multiple first vehicles.

[0031] Here, the electronic device screens the vehicle information in the first vehicle information list based on the execution status of each transfer task in the transfer task list, removes vehicle information for which a transfer task has been performed, retains vehicle information for which a transfer task has not been performed, and obtains the second vehicle information list. This process is as follows: the electronic device removes data from the first vehicle information list that meets the removal conditions, such as VIN codes for which a transfer task has been performed, based on the execution status of each transfer task in the transfer task list, and obtains the second vehicle information list.

[0032] In step S104, the transfer task for each second vehicle is executed based on the second vehicle information list, and the execution status of the transfer task for each second vehicle is updated in the transfer task list.

[0033] Here, for multiple second vehicles in a second vehicle information list, the electronic device performing a transfer task for said multiple second vehicles includes storing the driving data of the second vehicles in a database in a circular queue and transferring the driving data in the circular queue to a data processing platform using a multithreaded asynchronous transfer method. Here, the circular queue is, for example, a Disruptor, and said circular queue can simultaneously perform storage and reading. Based on this, the electronic device creates multiple threads, and these multiple threads asynchronously transfer the driving data in the circular queue to the data processing platform. For example, if the number of multiple threads is equal to the number of transfer tasks for multiple second vehicles, one transfer task can be assigned to one thread and executed. Alternatively, for example, if the number of multiple threads is greater than the number of transfer tasks for multiple second vehicles, one transfer task can be assigned to multiple threads and executed (for example, thread A is responsible for transferring a portion of the driving data required for task 1, and thread B is responsible for the remaining driving data), thereby improving data transfer efficiency. The present invention is not limited to a specific implementation method in which the electronic device asynchronously transfers driving data using multithreading.

[0034] In some embodiments, the electronic device determines whether there are any incomplete transfer tasks in the current task list. If there are incomplete tasks in the current task list, it positions the task at the previous execution node and continues executing the incomplete tasks based on the previous execution node until there are no more incomplete tasks in the current task list. That is, if there is a first transfer task in the transfer task list that is an interrupted transfer task, the electronic device continues executing the first transfer task based on the execution progress of the first transfer task in the transfer task list until the execution of the first transfer task is successful. For example, the electronic device determines whether there are any incomplete tasks in the current task list. If there are incomplete tasks in the current task list, it positions the task at the previous execution node (i.e., execution progress, for example, if the driving data waiting for transfer includes ABC and the execution progress indicates that the previous execution node was A) based on the VIN code (i.e., vehicle information) and stop time (i.e., task interruption time), and continues executing the incomplete tasks based on the previous execution node until there are no more incomplete tasks in the current task list, thereby achieving breakpoint restart. If there are no incomplete tasks in the current task list, the system extracts vehicle driving data (i.e., historical message data from the vehicle network) from the HBase database based on the second vehicle information list, stores the extracted driving data in a circular queue, analyzes the driving data using a multi-threaded asynchronous transfer policy, and then transfers it to the data processing platform. The data processing platform then performs subsequent processing and persists the task execution status in the current task list based on the analyzed driving data, i.e., updates the execution status of each second vehicle's transfer task in the transfer task list.

[0035] In some embodiments, the electronic device can first determine whether there are any incomplete transfer tasks in the current task list, and if so, execute those incomplete transfer tasks first. After there are no more incomplete tasks in the current task list, it can then execute the transfer task for the second vehicle indicated by the second vehicle information list. For example, it can execute a pre-configured extraction policy and use a multi-threaded method where one VIN code corresponds to one transfer task, extracting data based on VIN + vehicle type + production time + paging up to the online time of the target item. Specifically, it can extract historical message data of the vehicle network from the HBase database, query data for a predetermined time period using paging with the Java API method, and then place the extracted data into a circular queue Disruptor, performing storage and reading simultaneously. The retrieved data is then transferred to the TSP platform Kafka for data analysis using a multi-threaded asynchronous transfer method, and the task execution status is persisted in the task list.

[0036] In some other embodiments, if the transfer task list contains a second transfer task which is an abnormally executed transfer task, the electronic device compensates the second transfer task based on the cause of the abnormal execution of the second transfer task in the transfer task list. Here, an abnormal execution is, for example, that the driving data waiting to be transferred does not match the driving data that has been transferred. For example, one transfer task can transfer 10G of driving data, but the driving data of a second vehicle is 12G, and when the transfer task for that second vehicle is executed, it can transfer 10G of driving data, but the execution is abnormal, in which case a compensatory transfer can be performed for the remaining 2G of driving data. This compensation process may be manual compensation or automatic compensation, and the present invention is not limited thereto.

[0037] Furthermore, embodiments of the present invention may include a timing task, and when timing task 2 is triggered, the execution of the transfer task is stopped. For example, the electronic device determines whether or not it has received a pre-configured sleep command. If it has received a pre-configured sleep command, it stops extracting the vehicle network history message data from hbase. Exemplarily, the electronic device receives a second timing task indicating that the transfer task should not be executed during a second time period, which is a time period during which the frequency of use of driving data is greater than a second threshold. Here, the second time period is a pre-configured time period that can be set according to service requirements, for example, the second time period refers to the period from 7:00 to 23:00 every day. The second threshold is a pre-configured threshold that can be set according to service requirements, for example, the second threshold is a data usage frequency of 10, but the present invention is not limited thereto. Specifically, embodiments of the present invention can execute transfer tasks during predetermined time periods based on a sleep policy, staggering data usage peak periods, and implement a breakpoint resume function based on the execution status of each transfer task in the transfer task list. Specifically, after receiving a pre-configured sleep command, the sleep policy is executed, the execution of the transfer task is stopped using a thread notification method, the data existing in the queue is processed, and then the thread pool is closed. In some embodiments, after receiving a pre-configured sleep command, compensation for abnormal items in the current task list is further included based on a pre-configured compensation policy. That is, if a second transfer task exists in the transfer task list after receiving a sleep command, the second transfer task is compensated based on the cause of the execution abnormality of the second transfer task in the transfer task list.

[0038] The above data transfer method will be explained below with reference to Figure 2, which is a flowchart of another data transfer method provided by an embodiment of the present application.

[0039] As shown in Figure 2, after timing task 1 is triggered (i.e., the first timing task is received), the system screens and sorts the data in the Vehicle table where the vehicle production time is before the online time of the target item, retrieves the first vehicle information table vinList, filters the vin codes in vinList where the transfer task has been executed, retrieves the second vehicle information table, and determines whether there are any incomplete transfer tasks in the transfer task list T_COMMERCIAL_TASK. If there are, the system positions the task at the last execution node (i.e., execution progress) based on the vin code + stop time, continues task execution, and implements breakpoint restart. If it does not exist, a second vehicle transfer task is executed, where a multithreaded approach is used, with one VIN code corresponding to one task. Based on VIN + vehicle type + production time + paging, mileage data is extracted from the HBase database until the online time of the target item. Based on the data queuing policy, the extracted mileage data is placed into a circular queue Disruptor, where storage and reading are performed simultaneously. Using a multithreaded asynchronous transfer approach, the mileage data analysis in the circular queue is transferred to the TSP platform Kafka, and the task execution status is persisted in the transfer task list.

[0040] After timing task 2 is triggered (i.e., the second timing task is received), the sleep policy is executed, the execution of the transfer task is stopped by a thread notification method (i.e., the extraction of run data from the HBase database is stopped), the thread pool is closed after processing the data already in the queue, and the automatic compensation policy is triggered after the sleep policy is executed to compensate for any transfer tasks in the transfer task list that have failed to execute. Here, the sleep policy means that when sleep is triggered, the data extraction thread stops, does not store data in the queue, completes the technical processing of all data already in the queue, and then closes the thread pool. The compensation policy includes automatic compensation and manual compensation, where automatic compensation means that electronic devices compensate for tasks in the task list that have failed to execute, and manual compensation means that a party (e.g., a big data worker) discovers data anomalies or mismatches and compensates manually.

[0041] The above transfer task list will be explained below with reference to Table 1. Table 1 is a transfer task list provided by this embodiment, in which the electronic device can record the transfer task execution status in T_COMMERCIAL_TASK of the second relational database MySQL by the data processing platform.

[0042] JPEG2026510829000007.jpg91141

[0043] Here, the No.2 field VIN is the frame number, which is the unique identifier for each vehicle, i.e., the vehicle identifier. In the task execution status STATE of a transfer task, "0 Failed" means execution is abnormal, "1 Paused" means execution is interrupted, "2 Running" means the transfer task is running, and "3 Successful" means the transfer task has been completed. This allows for quick filtering of executed transfer tasks and confirmation of the current task status by STATE. In the task type of a transfer task, "0 Normal" indicates that the current task is running normally, and "1 Compensated" indicates that the current task is performing compensation. The STOPTIME field (i.e., stop time) allows positioning to the previous execution node, i.e., execution progress, thereby enabling breakpoint restarts. The ERRORINFO field (i.e., cause of execution abnormality) allows for compensation of the transfer task.

[0044] In summary, the data transfer method provided by the embodiment of the present invention involves obtaining a first vehicle information list to determine the first vehicle with data processing needs, obtaining a transfer task list to determine the execution status of the transfer task for each first vehicle, screening the vehicle information in the first vehicle information list to remove vehicle information for which a transfer task has been performed and to reserve vehicle information for which a transfer task has not been performed, obtaining a second vehicle information list to determine the second vehicle waiting for the execution of a transfer task, then executing the transfer task for each second vehicle based on the second vehicle information list, transferring the vehicle data of each second vehicle to a data processing platform (e.g., a cloud platform) using a multi-threaded asynchronous transfer method in accordance with a cyclic queue, and having the data processing platform perform subsequent processing on the vehicle data and update the execution status of the transfer task in the transfer task list. This method ensures efficient access to extracted driving data and that the driving data is transferred to the data processing platform without duplication or loss.Specifically, the system screens for vehicle information that meets the screening criteria from a pre-configured first relational database, generates a first vehicle information list based on the screened vehicle information, removes data that meets the removal criteria in the first vehicle information list, obtains a second vehicle information list, determines whether there are any incomplete tasks in the current transfer task list, and if there are no incomplete tasks in the current transfer task list, extracts the vehicle network history message data from hbase based on the second vehicle information list and a pre-configured extraction policy, stores the history message data in a pre-configured circular queue, analyzes the history message data using a pre-configured multi-threaded asynchronous transfer policy, and then transfers it to the data processing platform. The data processing platform then persists the task execution status in the current transfer task list based on the analyzed history message data. This solves problems in the data synchronization process such as duplicate and missing data extraction, complex flows, low transfer efficiency, and inability to resume breakpoints, prevents duplicate and missing extracted data, allows efficient access to extracted data, and allows shifting of peak data usage periods by extracting tasks during predetermined time periods.

[0045] Next, with reference to Figure 3, the data transfer device provided by the embodiment of the present application will be described. Figure 3 is a schematic block diagram of the data transfer device provided by the embodiment of the present application.

[0046] As shown in Figure 3, the data transfer device 10 includes a first acquisition module 100, a second acquisition module 200, a screening module 300, and a task execution module 400.

[0047] The first acquisition module 100 is used to acquire a first vehicle information list, the first vehicle information list includes vehicle information for multiple first vehicles, the vehicle information for the first vehicles indicates a vehicle with data processing needs, and the data processing needs refer to performing data processing on the driving data of the first vehicle by a data processing platform.

[0048] The second acquisition module 200 is used to acquire a transfer task list, which includes the execution status of each first vehicle's transfer task, where the first vehicle's transfer task refers to transferring the first vehicle's driving data, which is stored in a database for storing driving data uploaded by the vehicle via the vehicle network, to the data processing platform.

[0049] The screening module 300 is used to screen the vehicle information in the first vehicle information list based on the transfer task list and to obtain a second vehicle information list, the second vehicle information list containing vehicle information for at least one second vehicle, and the vehicle information for the second vehicle indicates a vehicle for which a transfer task has not been performed in the plurality of first vehicles.

[0050] The task execution module 400 is used to execute a transfer task for each second vehicle based on the second vehicle information list and to update the execution status of each second vehicle's transfer task in the transfer task list. Here, executing a transfer task for a second vehicle includes storing the second vehicle's driving data in the database in a circular queue and transferring the driving data in the circular queue to the data processing platform using a multithreaded asynchronous transfer method.

[0051] In some embodiments, the execution status of the transfer task includes the execution progress of the transfer task, and the task execution module is further used to continue the execution of the first transfer task until the execution of the first transfer task is successful, based on the execution progress of the first transfer task in the transfer task list, if the transfer task list contains a first transfer task which is a transfer task whose execution has been interrupted.

[0052] In some embodiments, the execution status of the transfer task includes the cause of execution abnormality of the transfer task, and the task execution module is further used to compensate the second transfer task based on the cause of execution abnormality of the second transfer task in the transfer task list if the transfer task list contains a second transfer task which is an abnormally executed transfer task.

[0053] In some embodiments, the device further includes a receiving module used for at least one of: receiving a first timing task instructing to execute a transfer task during a first time period in which the frequency of use of driving data is less than or equal to a first threshold; and receiving a second timing task instructing not to execute a transfer task during a second time period in which the frequency of use of driving data is greater than a second threshold.

[0054] In some embodiments, the execution status of the transfer task includes at least one of the following: vehicle information of the vehicle corresponding to the transfer task, task type of the transfer task, execution status, execution duration, interruption time, execution progress, and cause of execution abnormality.

[0055] What needs to be explained is that the interpretations and explanations given above regarding the embodiment of the data transfer method also apply to the data transfer device of that embodiment, and therefore, the explanation is omitted here.

[0056] The data transfer device provided by the embodiment of the present invention determines a first vehicle with data processing needs by acquiring a first vehicle information list, and determines the execution status of the transfer task for each first vehicle by acquiring a transfer task list. Thereafter, it screens the vehicle information in the first vehicle information list, removes vehicle information for which a transfer task has been performed, and reserves vehicle information for which a transfer task has not been performed. It then acquires a second vehicle information list, that is, determines the second vehicles waiting for the execution of a transfer task. Subsequently, it executes the transfer task for each second vehicle based on the second vehicle information list, and, in accordance with a cyclic queue, transfers the vehicle data of each second vehicle to a data processing platform (e.g., a cloud platform) using a multi-threaded asynchronous transfer method. The data processing platform then performs subsequent processing on the vehicle data and updates the execution status of the transfer task in the transfer task list. This method ensures efficient access to extracted driving data and that the driving data is transferred to the data processing platform without duplication or loss.

[0057] Figure 4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. The electronic device may include a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402.

[0058] When the processor 402 executes the program, it implements the data transfer method provided by the above embodiment.

[0059] Furthermore, the electronic device further includes a communication interface 403 for communication between the memory 401 and the processor 402.

[0060] Memory 401 is used to store executable computer programs on the processor 402.

[0061] Memory 401 may include high-speed RAM (Random Access Memory) memory and may further include at least one non-volatile memory such as magnetic disk memory.

[0062] If the memory 401, processor 402, and communication interface 403 are implemented independently, the interface 403, memory 401, and processor 402 can be interconnected via a bus to communicate with each other. The bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only one thick line is used in Figure 4, but this does not mean that there is only one bus or only one type of bus.

[0063] In selective and specific implementations, if the memory 401, processor 402, and communication interface 403 are integrated onto a single chip, the memory 401, processor 402, and communication interface 403 can communicate with each other via an internal interface.

[0064] The processor 402 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to perform the embodiments of the present application.

[0065] Embodiments of the present invention further provide a computer-readable storage medium that stores a computer program and implements the above-described data transfer method when the program is executed by a processor.

[0066] In this specification, any reference to terms such as “one embodiment,” “several embodiments,” “example,” “specific example,” or “several examples” means that the specific features, structures, materials, or properties described in combination with such embodiment or example are included in at least one embodiment or example of this application. In this specification, exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or properties described may be combined in an appropriate manner in any one or N embodiments or examples. Moreover, a person skilled in the art can combine different embodiments or examples and features of different embodiments or examples described herein without contradiction.

[0067] Furthermore, the terms "first" and "second" are merely for explanatory purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the quantity of the technical features being referred to. Accordingly, features designated as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specified.

[0068] Any description of a process or method as described in a flowchart or otherwise is understood as a module, fragment, or portion containing code for one or N executable instructions to implement a customized logical function or step of a process, and the scope of preferred embodiments of the present application includes other implementations in which functions can be performed in an order different from the shown or considered order, including essentially simultaneously or in reverse order depending on the functions involved. This should be understood by those skilled in the art.

[0069] It should be understood that each part of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination thereof of technologies known in the art, such as discrete logic circuits having logic gate circuits for implementing logical functions for data signals, dedicated integrated circuits having suitable combinational logic gate circuits, programmable gate arrays, field-programmable gate arrays, etc.

[0070] As those skilled in the art will understand, the implementation of all or some of the steps of the method of the above embodiment can be completed by instructing the relevant hardware by a program, the program can be stored in a computer-readable storage medium, and when the program is executed, it includes one or a combination of the steps of the method embodiment.

[0071] Although embodiments of the present application have been described above, these embodiments are illustrative and should not be understood as limiting the present application. Those skilled in the art will understand that they can modify, alter, substitute, and transform the above embodiments within the scope of the present application.

Claims

1. A data transfer method, A step of obtaining a first vehicle information list, wherein the first vehicle information list includes vehicle information for a plurality of first vehicles, the vehicle information for the first vehicles indicates a vehicle with data processing needs, and the data processing needs refer to performing data processing on the driving data of the first vehicles by a data processing platform. A step of obtaining a transfer task list, wherein the transfer task list includes the execution status of transfer tasks for each first vehicle, and the transfer task for the first vehicle refers to transferring the driving data of the first vehicle in a database for storing driving data uploaded by the vehicle via a vehicle network to the data processing platform. A step of screening the vehicle information in the first vehicle information list based on the transfer task list and obtaining a second vehicle information list, wherein the second vehicle information list includes vehicle information for at least one second vehicle, and the vehicle information for the second vehicle indicates a vehicle for which a transfer task has not been performed among the plurality of first vehicles. A method comprising the steps of executing a transfer task for each second vehicle based on the second vehicle information list and updating the execution status of the transfer task for each second vehicle in the transfer task list, wherein executing the transfer task for the second vehicle includes storing the driving data of the second vehicle in the database in a circular queue and transferring the driving data in the circular queue to the data processing platform using a multithreaded asynchronous transfer method.

2. The execution status of the transfer task includes the execution progress of the transfer task, and the method is The method according to claim 1, further comprising, if the transfer task list contains a first transfer task which is a transfer task whose execution has been interrupted, continuing the execution of the first transfer task based on the execution progress of the first transfer task in the transfer task list until the execution of the first transfer task is successful.

3. The execution status of the transfer task includes the cause of the execution failure of the transfer task, and the method is, The method according to claim 1, further comprising, if the transfer task list contains a second transfer task which is a transfer task that is executed abnormally, compensating the second transfer task based on the cause of the execution abnormality of the second transfer task in the transfer task list.

4. The aforementioned method, Receiving a first timing task instructing the execution of a transfer task during a first time period in which the frequency of use of driving data is below a first threshold, The method according to claim 1, further comprising receiving a second timing task instructing not to perform a transfer task during a second time period in which the frequency of use of driving data is greater than a second threshold.

5. The method according to claim 1, characterized in that the execution status of the transfer task includes at least one of the following: vehicle information of the vehicle corresponding to the transfer task, task type of the transfer task, execution status, execution duration, interruption time, execution progress, and cause of execution abnormality.

6. A data transfer device, A first acquisition module for obtaining a first vehicle information list, wherein the first vehicle information list includes vehicle information for a plurality of first vehicles, the vehicle information for the first vehicles indicates a vehicle with data processing needs, and the data processing needs refer to performing data processing on the driving data of the first vehicles by a data processing platform; A second acquisition module for obtaining a transfer task list, wherein the transfer task list includes the execution status of each first vehicle's transfer task, and the first vehicle's transfer task refers to transferring the first vehicle's driving data in a database for storing driving data uploaded by the vehicle via a vehicle network to the data processing platform. A screening module that screens the vehicle information in the first vehicle information list based on the transfer task list and obtains a second vehicle information list, wherein the second vehicle information list includes vehicle information for at least one second vehicle, and the vehicle information for the second vehicle indicates a vehicle for which a transfer task has not been performed in the plurality of first vehicles. The apparatus includes a task execution module for executing a transfer task for each second vehicle based on the second vehicle information list and updating the execution status of the transfer task for each second vehicle in the transfer task list, wherein executing the transfer task for the second vehicle includes storing the driving data of the second vehicle in the database in a circular queue and transferring the driving data in the circular queue to the data processing platform using a multithreaded asynchronous transfer method.

7. The execution status of the transfer task includes the execution progress of the transfer task, and the task execution module is The apparatus according to claim 6, further characterized in that, if the transfer task list contains a first transfer task which is a transfer task whose execution has been interrupted, the execution of the first transfer task is continued based on the execution progress of the first transfer task in the transfer task list until the execution of the first transfer task is successful.

8. The execution status of the transfer task includes the cause of the execution failure of the transfer task, and the task execution module, The apparatus according to claim 6, further characterized in that, if the transfer task list contains a second transfer task which is a transfer task that is executed abnormally, the second transfer task is compensated based on the cause of the execution abnormality of the second transfer task in the transfer task list.

9. In electronic devices, An electronic device comprising memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program in order to implement the data transfer method described in any one of claims 1 to 5.

10. A computer-readable storage medium in which a computer program is stored, characterized in that the program is executed by a processor in order to realize the data transfer method described in any one of claims 1 to 5.

Images