Data processing method, electronic equipment and storage medium
By dividing the pending data set into multiple data subsets in the Rust programming language and asserting them to different threads for execution as asynchronous tasks, the problem of thread resource waste is solved, and data processing efficiency and thread utilization are improved.
Patent Information
- Application Number
- CN202311086423.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-02
- Filing Date
- 2023-08-25
- Publication Date
- 2024-12-03
Smart Images

Figure CN119065827A_ABST
Abstract
Description
[0001] This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on June 2, 2023, with application number 202310651609.4 and application name “A method and device for supporting automatic splitting of arrays for asynchronous parallel operations of multiple data in Rust”, the entire contents of which are incorporated by reference in this application. Technical Field
[0002] The present application relates to the field of data processing technology, and in particular to a data processing method, an electronic device and a storage medium. Background Art
[0003] When an electronic device processes data, the processor may divide the data to be processed into multiple parts, and process each part of the data as a task, and then execute the multiple tasks in sequence, and obtain the processing result of the data to be processed by combining the execution results of the multiple tasks.
[0004] For example, the Rust programming language supports synchronized parallel iterators, through which data can be traversed and processed sequentially. However, when processing large-scale data sequentially, large-scale data can be processed recursively by binary division, dividing the large-scale data into multiple copies of data equal to the number of threads, and assigning the processing of each copy of data as a task to a thread. When one thread completes the assigned task and another thread has not completed the assigned task, the thread that has completed the assigned task will call the blocking method of the thread that has not completed the assigned task to block the thread that has completed the assigned task. However, since the number of threads is always equal to the number of processor cores of the electronic device, that is, the number of threads is limited, blocking the thread that has completed the assigned task will cause a waste of thread resources. Summary of the invention
[0005] In order to solve the problem that blocking the current working thread will cause a waste of thread resources, an embodiment of the present application provides a data processing method, an electronic device and a storage medium.
[0006] A first aspect of an embodiment of the present application provides a data processing method for an electronic device, the data processing method comprising: based on a first programming language: obtaining a first data set to be processed, dividing the first data set to be processed into M data subsets; generating M tasks corresponding to each data subset in the M data subsets; using N threads to execute N tasks, wherein the M tasks include a first task; determining that the first task is in a waiting state, and using the first thread that executes the first task to execute a second task, wherein the M tasks include the second task; wherein M and N are natural numbers.
[0007] Based on the above solution, when the first task is in a waiting state, the first thread that executes the first task is used to execute the second task instead of blocking the first thread, thereby reducing the waste of thread resources.
[0008] It can be understood that the number of data subsets can be equal to the number of tasks, that is, M can be equal to N. The number of data subsets can also be greater than the number of tasks, that is, M can be greater than N.
[0009] In a possible implementation of the first aspect above, the first programming language is Rust programming language or C++ programming language.
[0010] It can be understood that the first programming language can also be other programming languages that can implement stackless coroutines, and the embodiments of the present application do not make specific limitations.
[0011] The embodiments of the present application can make the parallel iterators in the Rust programming language asynchronous by adopting the above-mentioned data processing method in the Rust programming language, thereby increasing the utilization rate of threads when the number of threads is fixed, reducing the waste of thread resources, and improving the ability to schedule tasks.
[0012] In a possible implementation of the first aspect above, corresponding to the first programming language being the Rust programming language, obtaining a first data set to be processed, and dividing the first data set to be processed into M data subsets, including: obtaining a first data set to be processed, and dividing the first data set to be processed into M data subsets based on a parallel iterator in the Rust programming language.
[0013] It can be understood that the first to-be-processed data set can be divided into multiple data subsets based on the parallel iterator in the Rust programming language, where each data subset can be called a data block or a single block of data. The number of data subsets can be equal to the number of threads in the electronic device, or can be greater than the number of threads in the electronic device.
[0014] In an embodiment of the present application, data processing efficiency can be improved by dividing the first data set to be processed into M data subsets based on a parallel iterator in the Rust programming language.
[0015] In a possible implementation of the first aspect above, the first data set to be processed is divided into M data subsets based on a parallel iterator in the Rust programming language, including: performing binary recursive processing on the first data set to be processed based on a parallel iterator in the Rust programming language to obtain multiple data subsets; corresponding to the number of data subsets in the multiple data subsets being less than the number of threads in the electronic device, performing binary recursive processing on the data subsets again until the number of data subsets is less than the number of threads in the electronic device, and obtaining M data subsets.
[0016] For example, for data 0-N, the parallel iterator can assign data 0-N to thread Thread A for processing. If there are idle threads, the parallel iterator can perform binary recursive processing on data 0-N, divide data 0-N into data 0-N / 2 and data N / 2-N, assign data 0-N / 2 to thread Thread A for processing, and assign data N / 2-N to thread Thread B for processing. If there are idle threads, the parallel iterator can continue to perform binary recursive processing on data 0-N / 2, divide data 0-N / 2 into data 0-N / 4 and data N / 4-N / 2, assign data 0-N / 4 to thread Thread A for processing, and assign data N / 4-N / 2 to thread Thread C for processing. If there are idle threads, the parallel iterator can continue to perform binary recursive processing on the data N / 2~N, dividing the data N / 2~N into data N / 2~N / 2+N / 4 and data N / 2+N / 4~N, and handing the data N / 2~N / 2+N / 4 to thread Thread B for processing, and handing the data N / 2+N / 4~N to thread ThreadD for processing.
[0017] In some optional instances, the first to-be-processed data set can be directly divided into multiple data subsets equal to the number of threads in the electronic device based on the parallel iterator in the Rust programming language.
[0018] In a possible implementation of the first aspect above, the N threads are determined based on the number of cores of a processor of the electronic device.
[0019] In some optional instances, a corresponding number of threads can be generated based on the number of cores of the processor in the electronic device. For example, when the number of cores of the processor of the electronic device is 4, 4 threads can be generated, such as Thread A, Thread B, Thread C and Thread D.
[0020] In the embodiment of the present application, by generating threads equal to the number of cores of the processor in the electronic device, the threads can be fully utilized while reducing the time for data to wait for processing and improving the efficiency of data processing.
[0021] In some optional instances, a corresponding multiple number of threads can be generated based on the number of cores of the processor in the electronic device. For example, when the number of cores of the processor of the electronic device is 4, 8 threads can be generated, such as Thread A, Thread B, Thread C, Thread D, Thread E, Thread F, Thread G, and Thread H.
[0022] In a possible implementation of the first aspect, the N threads include a first thread, and the N threads are used to execute N tasks among the M tasks, including: using the first thread to execute the first task and returning a return value.
[0023] It can be understood that the return value may include a first return value and a second return value, wherein the first return value may indicate that the task is in a waiting state, i.e., the execution is not completed. For example, based on the examples listed above, the processing of data 0 to N / 4 is completed, but the processing of data N / 2+N / 4 to N / 2 is not completed. The second return value may indicate that the task is in a completed state, i.e., the execution is completed. Based on the examples listed above, the processing of data 0 to N / 4 is completed, and the processing of data N / 2+N / 4 to N / 2 is completed.
[0024] In some optional instances, the first return value may be Pending, and the second return value may be Ready.
[0025] In a possible implementation of the first aspect above, determining that the first task is in a waiting state and using the first thread that executes the first task to execute the second task includes: corresponding to the return value being the first return value, determining that the first task is in a waiting state; and using the first thread that executes the first task to execute the second task.
[0026] In a possible implementation of the first aspect above, the M tasks include a third task, and the first task and the third task are in a task execution queue of the first thread; the method also includes: corresponding to the return value being the second return value, determining that the first task is in a completed state; and using the first thread that executes the first task to execute the third task.
[0027] For example, based on the examples listed above, the processing of data 0 to N / 4 is completed, and the processing of data N / 2+N / 4 to N / 2 is completed, and the first thread can steal the task of executing data N / 2 to N / 2+N / 4.
[0028] In a second aspect, the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for executing the data processing method mentioned in the present application.
[0029] In a third aspect, the present application provides a readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the data processing method mentioned in the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 According to some examples of the present application, a schematic diagram of task allocation is shown;
[0031] Figure 2According to some examples of the present application, a framework schematic diagram of task scheduling is shown;
[0032] Figure 3 According to some examples of the present application, another framework schematic diagram of task scheduling is shown;
[0033] Figure 4 According to some examples of the present application, a flow chart of a data processing method is shown;
[0034] Figure 5 According to some examples of the present application, a schematic diagram of a framework for executing asynchronous tasks is shown;
[0035] Figure 6 According to some examples of the present application, a schematic diagram of a process of waiting for asynchronous task execution based on a task handle is shown;
[0036] Figure 7 According to some examples of the present application, a schematic diagram of a processing flow of an asynchronous task is shown;
[0037] Figure 8 According to some examples of the present application, a schematic diagram of the hardware structure of an electronic device is shown. DETAILED DESCRIPTION
[0038] The illustrative embodiments of the present application include, but are not limited to, a data processing method, an electronic device, and a storage medium.
[0039] It can be understood that the technical solution of the present application can be applicable to various task processing scenarios, for example, a scenario in which a processor inside an electronic device performs addition operations on multiple numbers, a scenario in which a processor inside an electronic device responds to multiple request data, a scenario in which a processor inside an electronic device performs speech synthesis on multiple audio data segments, and so on.
[0040] In order to clearly illustrate the scheme mentioned in the embodiments of the present application, the terms involved in the embodiments of the present application are first explained.
[0041] Rust: An emerging programming language released by Mozilla.
[0042] Trait: A feature similar to an interface in the Rust programming language that enables code reusability and extensibility.
[0043] Asynchrony: Asynchrony is the opposite of synchronization. The caller will not be blocked immediately and wait for the other party to return the result. Instead, it will continue to run until the caller notifies that the call result is available or handles the call through a callback function.
[0044] Coroutine: A coroutine is a cooperative multitasking model that uses an asynchronous mechanism, allows multiple entry points, and can suspend and resume execution at specified locations.
[0045] Stackless coroutine: Stackless coroutine does not use stack and context switching to execute asynchronously, but directly uses the system stack. It is essentially a state machine that only retains variables and temporary variables needed after recovery, stores the current state, and relies on the state to decide which part of the code to execute after recovery.
[0046] Future: An interface definition in the Rust programming language that declares the methods and parameters that the stackless coroutine asynchronous type should define. Rust asynchronous functions all implement instances of this interface definition.
[0047] Poll: Polling function, declared in the Rust Future interface definition, is used to promote asynchronous task execution when the Rust asynchronous scheduling framework schedules tasks.
[0048] Wakeup: Contains the wake function, which is responsible for waking up the corresponding asynchronous task.
[0049] Async: A keyword in the Rust programming language. A function with the async keyword is an asynchronous function.
[0050] await: For instances that implement Future, you can use this keyword to call the corresponding poll method at the bottom layer to wait for it asynchronously until the execution is completed and the result is returned.
[0051] Async Runtime, an asynchronous scheduling framework: It is used for the main body of asynchronous scheduling, generally consisting of multiple worker threads and task queues.
[0052] JoinHandle: Rust thread handle, which can be blocked by the Join method to wait for the thread corresponding to the handle to end.
[0053] Iterator: A container iterator type in the Rust programming language, used to traverse each element in a container.
[0054] Context on asyncfunction: A wrapper for the wakeup function. The wakeup function corresponding to the asynchronous task can be obtained through the context of the asynchronous function.
[0055] Working stealing: A mechanism used by the scheduling framework to balance the workload of each thread. The currently idle thread can steal tasks from other threads for execution.
[0056] The following is an introduction to the method of performing data processing based on parallel iterators mentioned in some embodiments using a specific example.
[0057] In some specific implementations, the parallel iterators supported by the Rust programming language can be provided by the community third-party library Rayon, which can generate a number of threads equal to the number of cores of the electronic device's processor (central processing unit, CPU) by default when the electronic device is started, and always maintain this number of threads during the data processing process. Figure 1 As shown, when the number of processor cores of the electronic device is 4, four threads, Thread A, Thread B, Thread C and Thread D, can be generated.
[0058] When processing data, the parallel iterator can perform binary recursive processing on the data, that is, divide the data into two parts, and assign the processing of one part of the data to the current thread as a task, and put the processing of the other part of the data into the task queue as another task, waiting for the current thread to complete the current task before processing the task, or use other threads to process the task. After the parallel iterator divides the data into two parts, it can determine whether it is necessary to perform binary recursive processing on each part of the data again based on whether there are idle threads. If it is necessary to perform binary recursive processing on each part of the data again, continue to perform binary recursive processing on each part of the data until there are no idle threads. The parallel iterator can call the processing method corresponding to the pre-set function to process each part of the data in turn.
[0059] Continue as Figure 1As shown, for data 0-N, the parallel iterator can hand over data 0-N to thread Thread A for processing. If there are idle threads, the parallel iterator can perform binary recursive processing on data 0-N, divide data 0-N into data 0-N / 2 and data N / 2-N, and hand over data 0-N / 2 to thread Thread A for processing, and hand over data N / 2-N to thread Thread B for processing. If there are idle threads, the parallel iterator can continue to perform binary recursive processing on data 0-N / 2, divide data 0-N / 2 into data 0-N / 4 and data N / 4-N / 2, and hand over data 0-N / 4 to thread Thread A for processing, and hand over data N / 4-N / 2 to thread Thread C for processing. If there are idle threads, the parallel iterator can continue to perform binary recursive processing on the data N / 2~N, dividing the data N / 2~N into data N / 2~N / 2+N / 4 and data N / 2+N / 4~N, and handing the data N / 2~N / 2+N / 4 to thread Thread B for processing, and handing the data N / 2+N / 4~N to thread Thread D for processing.
[0060] In some optional examples, the user can define the number of binary recursions, and the number of copies of data obtained by the binary recursion can be greater than the number of threads.
[0061] After the data is processed by binary recursion, the processing of each piece of data can be treated as an asynchronous function, and the execution of the asynchronous function can be handed over to the thread as an asynchronous task. For example, in the Rust programming language, the allocation of multiple asynchronous tasks can be implemented based on parallel iterators, for example, different asynchronous tasks can be assigned to different threads.
[0062] However, the parallel iterator of the existing Rust programming language is a synchronous parallel iterator, that is, the parallel iterator calls the synchronous processing function. When the current thread is executed, it can call the Join function of another thread to block the current thread and wait for the other thread to complete execution, thereby combining the execution results of the two threads. Figure 2 As shown, one copy of data can be processed as function a, and another copy of data can be processed as function b, and the execution of function a can be assigned to the current thread as asynchronous task A, and the execution of function b can be assigned to another thread as asynchronous task C. When asynchronous task A is completed, the synchronous parallel iterator will call the Join method of another thread to block the current thread. At this time, the current thread cannot execute other asynchronous tasks, such as asynchronous task B.
[0063] However, since the number of threads is always equal to the number of processor cores of the electronic device, that is, the number of threads is limited, blocking the current working thread will cause a waste of thread resources.
[0064] In order to solve the above problems, an embodiment of the present application provides a data processing method, in which the processing of each of the multiple copies of data obtained by binary recursive processing can be treated as an asynchronous task and respectively handed over to different threads for execution. When an asynchronous task is completed and another asynchronous task is not completed, that is, the entire data processing is in a waiting state, the thread that has completed the asynchronous task can be switched to execute other asynchronous tasks, so that the waste of thread resources can be reduced.
[0065] It can be understood that the corresponding return value can be returned according to whether the asynchronous task can be executed immediately, for example, a Ready value or a Pending value. If the Ready value is returned, it can indicate that the asynchronous task has been executed and can continue to be executed. If the Pending value is returned, the current state can be recorded, that is, the current execution progress of the asynchronous task can be recorded.
[0066] For example, Figure 3 As shown, the processing of one copy of data can be performed as asynchronous function a, and the processing of another copy of data can be performed as asynchronous function b, and the execution of asynchronous function a can be assigned to the current thread as asynchronous task A, and the execution of asynchronous function b can be put into the task execution queue of the current thread as asynchronous task B. In addition, the processing of part of the data in one copy of data can be assigned to the current thread as asynchronous task A1, and the processing of the other part of the data can be assigned to another thread as asynchronous task A2. When asynchronous task A1 is completed and waits for asynchronous task A2, it can return the Pending value, and the current thread can be switched to execute asynchronous task B without blocking the current thread.
[0067] In some optional instances, when it is found that an asynchronous task that returns a Pending value can continue to execute (such as an event processing module in an electronic device detects that an asynchronous task that returns a Pending value can continue to execute), the wake-up method corresponding to the waker of the asynchronous task can be called to put the asynchronous task back into the task execution queue of the current thread or other threads, so that the current thread or other threads can continue to execute from the last execution progress of the asynchronous task.
[0068] The data processing method mentioned in the embodiments of the present application is introduced in detail below. Figure 4 A schematic flow chart of a data processing method is shown, and the data processing method can be executed by an electronic device, such as Figure 4 As shown, the method may include:
[0069] 401: Obtain a first data set to be processed, and divide the first data set to be processed into multiple data subsets.
[0070] It can be understood that after obtaining the first data set to be processed, the first data set to be processed can be subjected to binary recursive processing, and it can be determined whether there are idle threads. If there are idle threads, the parallel iterator can continue to perform binary recursive processing on the data subsets obtained by the binary recursive processing until the number of data subsets obtained by the binary recursive processing is equal to the number of threads, thereby obtaining multiple data subsets.
[0071] In some optional examples, the parallel iterator may perform binary recursive processing on the first data set to be processed to obtain a first data subset and a second data subset, wherein the first data subset may be referred to as left half data (Left Half), the second data subset may be referred to as right half data (Right Half), and both the first data subset and the second data subset may be referred to as data blocks, or as single block data.
[0072] 402: Generate an asynchronous task based on each data subset in the multiple data subsets to obtain an asynchronous task set.
[0073] It can be understood that the processing of any one of the multiple data subsets can be used as an asynchronous function. For example, the data reading method and data processing method of any data subset can be written in an asynchronous function, and the execution of the asynchronous function can be used as an asynchronous task. In this way, for multiple data subsets, multiple asynchronous tasks can be generated to obtain an asynchronous task set.
[0074] like Figure 5 As shown in the figure, for each data subset in multiple data subsets, a Task structure can be defined, which can wrap a generic Future and require the generic Future to implement the Future trait. When calling the run function of the parallel iterator, all linear operations of the generic Future can be put into the asynchronous function keyword block Async block to convert it into a Future object and encapsulate it into an asynchronous task Task.
[0075] It can be understood that the parallel iterator can use the thread pool started by the underlying scheduling framework of the electronic device, wherein the number of threads in the thread pool can be configured by the user, and each thread (OS Thread) is bound to an execution agent. Among them, the execution agent can be named Worker, each Worker can have a task execution queue, and each Worker can call the poll method corresponding to the asynchronous task to advance the execution progress of the asynchronous task. In addition, after creating each asynchronous task, the Task structure of the asynchronous task can be generated and placed in the Worker's task execution queue. When the asynchronous task is awakened, it will be put back into the Worker's task execution queue. Workers can continuously obtain asynchronous tasks from the task execution queue, and can also steal asynchronous tasks from other Workers' task execution queues.
[0076] In some optional instances, a first asynchronous task may be generated based on a first data subset, and a second asynchronous task may be generated based on a second data subset, that is, the first asynchronous task may be generated based on a left half of the data, and the second asynchronous task may be generated based on a right half of the data.
[0077] 403: Assign the first asynchronous task in the asynchronous task set to the current thread.
[0078] like Figure 5 As shown, after the asynchronous task Task is generated, the asynchronous task Task can be put into the task execution queue of the execution intermediary Worker to wait for execution.
[0079] 404: During the execution of the current thread, if the first asynchronous task is completed, the second asynchronous task in the asynchronous task set is assigned to the current thread.
[0080] It is understandable that the execution intermediary Worker can schedule and execute asynchronous tasks. Users will not directly perceive this, but users need to know how to wait for the asynchronous task to complete. Therefore, after generating an asynchronous task, the task handle of the asynchronous task can be returned. The task handle can be used to asynchronously wait for the corresponding asynchronous task to complete. For example, Figure 6As shown, calling await of the task handle Task Handle can call the poll method of the task handle Task Handle. In the poll method of the task handle Task Handle, it will check whether the asynchronous task has been completed. If the asynchronous task is completed, the Ready value and the execution result of the asynchronous task can be returned. If the asynchronous task is not completed, the Pending value can be returned. Therefore, the .await operation can be performed directly on the task handle Task Handle, so that the current thread can execute other asynchronous tasks first when the currently executed asynchronous task is completed and asynchronously waits for other asynchronous tasks, without blocking and waiting.
[0081] In some optional examples, after generating a first asynchronous task based on the first data subset and generating a second asynchronous task based on the second data subset, task handles of the two asynchronous tasks may be returned.
[0082] In an embodiment of the present application, the above-mentioned data processing method can be used as an implementation mechanism of stackless threads in the Rust programming language, and can also be used as an implementation mechanism of stackless threads in other programming languages such as the C++ programming language.
[0083] In the embodiment of the present application, by making the iterator asynchronous, the thread utilization rate can be increased when the number of threads is fixed, which not only reduces the waste of thread resources but also improves the ability to schedule tasks.
[0084] The following is an introduction to the implementation mechanism of wireless coroutines in the Rust programming language using a specific example.
[0085]
[0086] It can be understood that in the Rust programming language, functions with the keyword async are called asynchronous functions. func, mission1, mission2, and mission3 can all represent asynchronous functions, where the asynchronous function func can call the asynchronous function mission 1, mission 2, or mission 3, and use await to wait for the execution result of the asynchronous function mission 1, mission 2, or mission 3. The compiler can generate a state machine (such as a generator state machine) for each asynchronous function, and await can be regarded as a state of the state machine.
[0087] It can be understood that in the asynchronous function shown above, the asynchronous function func is executed, that is, the poll method of the asynchronous function func is called. When the execution encounters the await point after mission1, the asynchronous function func calls the poll method of mission 1. When the execution encounters the await point after mission2 again, the asynchronous function func calls the poll method of mission 2 and returns the corresponding return value according to whether the poll method of mission 2 can be executed immediately. For example, return the Ready value or return the Pending value. If the Ready value is returned, it can indicate that the poll method of the asynchronous function mission 2 has been executed and continues to execute, that is, the asynchronous function func calls the poll method of mission 3. If the Pending value is returned, the state machine can record the current state, that is, record the current await point. In addition, the execution of the asynchronous function func can be exited. At this time, the asynchronous function func will also return the Pending value. When the poll method of the asynchronous function func is called next time, it will start from the await point that returns the Pending value this time, that is, call the poll method of the asynchronous function mission 2.
[0088] In the embodiment of the present application, the parallel iterator provided by Rayon calls the synchronous processing function, and the parallel iterator in the Rust programming language cannot be executed in the context of an asynchronous function, that is, the existing parallel iterator provided by Rayon cannot perform asynchronous operations on data. The data processing method provided in the embodiment of the present application, by changing the synchronous processing function called by the parallel iterator into an asynchronous processing function, and encapsulating it into a coroutine task, and handing it over to the execution intermediary for execution, can enable the parallel iterator to support asynchronous processing functions, and then enable the parallel iterator to be executed in the context of an asynchronous function, and can integrate IO-intensive tasks and CPU-intensive tasks.
[0089] The data processing method mentioned above is introduced below with a specific example, wherein the data processing method may include a parallel iterator processing flow and a parallel iterator scheduling flow.
[0090] like Figure 7As shown, in the parallel iterator processing flow, after obtaining the data set to be processed, the data set to be processed can be recursively processed by binary division until the data set to be processed is divided into multiple data subsets equal to the number of threads. If there is no idle thread, that is, the smallest block of data has been currently divided, the data set to be processed can be subjected to specified operations, such as addition operations on the data set to be processed. If there is an idle thread, that is, the smallest block of data has not yet been binary divided, the data set to be processed can be recursively processed by binary division to obtain the left half of the data (Left Half) and the right half of the data (Right Half), and an asynchronous task is generated based on the processing of the left half of the data to call an asynchronous processing function, and an asynchronous task is generated based on the processing of the right half of the data to call an asynchronous processing function, and asynchronous waiting is performed through the task handle.
[0091] In the scheduling process of the parallel iterator, different asynchronous tasks can be put into the task execution queue of the execution intermediary Worker, so that the execution intermediary Worker can continuously obtain asynchronous tasks from the task execution queue, and when the execution intermediary cannot perform asynchronous tasks, it can steal asynchronous tasks from the task execution queue of other execution intermediaries, so that all asynchronous tasks are distributed and executed on each execution intermediary Worker.
[0092] The following is an introduction to the hardware structure of electronic equipment. Figure 8 As shown, the electronic device 800 may include a processor 810, an external memory interface 820, an internal memory 821, a universal serial bus (USB) interface 830, a charging management module 840, a power management module 841, a battery 842, an antenna 1, an antenna 2, a mobile communication module 850, a wireless communication module 860, an audio module 870, a speaker 870A, a receiver 870B, a microphone 870C, an earphone interface 870D, a sensor module 880, a button 890, a motor 891, an indicator 892, a camera 893, a display screen 894, and a subscriber identification module (SIM) card interface 895, etc. The sensor module 880 may include a pressure sensor 880A, a gyroscope sensor 880B, an air pressure sensor 880C, a magnetic sensor 880D, an acceleration sensor 880E, a distance sensor 880F, a proximity light sensor 880G, a fingerprint sensor 880H, a temperature sensor 880J, a touch sensor 880K, an ambient light sensor 880L, a bone conduction sensor 880M, and the like.
[0093] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device 800 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0094] A memory may also be provided in the processor 810 for storing instructions and data. In some embodiments, the memory in the processor 810 is a cache memory. The memory may store instructions or data that the processor 810 has just used or circulated. If the processor 810 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 810 is reduced, and the efficiency of the system is improved. For example, the processor 810 may execute the data processing method mentioned in the embodiment of the present application.
[0095] The external memory interface 820 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 800. The external memory card communicates with the processor 810 through the external memory interface 820 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.
[0096] The internal memory 821 can be used to store computer executable program codes, which include instructions. The internal memory 821 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 800 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 821 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 810 executes various functional applications and data processing of the electronic device 800 by running instructions stored in the internal memory 821 and / or instructions stored in a memory provided in the processor.
[0097] In some embodiments, the present application provides a readable medium having instructions stored thereon, which, when executed on an electronic device, enables the electronic device to execute the data processing method in the present application.
[0098] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.
[0099] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.
[0100] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.
[0101] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.
[0102] The above describes the hardware structure that the electronic device may have. It is understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0103] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.
[0104] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit, or a microprocessor.
[0105] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.
[0106] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.
[0107] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.
[0108] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.
[0109] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0110] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.
Claims
1. A data processing method, used in electronic equipment, characterized in that: include: Based on the first programming language: Acquire a first data set to be processed, and divide the first data set to be processed into M data subsets; Generate M tasks corresponding to each data subset in the M data subsets; Using N threads to execute N tasks among the M tasks, wherein the M tasks include the first task; Determining that the first task is in a waiting state, and using a first thread that executes the first task to execute a second task, wherein the M tasks include the second task; Wherein, M and N are natural numbers.
2. The method according to claim 1, characterized in that The first programming language is Rust programming language or C++ programming language.
3. The method according to claim 1, characterized in that: Corresponding to the first programming language being the Rust programming language, The step of obtaining a first set of data to be processed and dividing the first set of data to be processed into M data subsets includes: The first data set to be processed is obtained, and the first data set to be processed is divided into M data subsets based on a parallel iterator in the Rust programming language.
4. The method according to claim 3, characterized in that The dividing the first to-be-processed data set into M data subsets based on the parallel iterator in the Rust programming language includes: Based on the parallel iterator in the Rust programming language, the first to-be-processed data set is subjected to binary recursive processing to obtain multiple data subsets; Corresponding to the number of data subsets in the multiple data subsets being less than the number of threads in the electronic device, the data subsets are recursively binary-partitioned again until the number of data subsets is less than the number of threads in the electronic device, thereby obtaining the M data subsets.
5. The method according to claim 1, characterized in that The N threads are determined based on the number of cores of the processor of the electronic device.
6. The method according to claim 1, characterized in that The N threads include a first thread, The adopting N threads to execute N tasks among the M tasks includes: The first thread is used to execute the first task and return a return value.
7. The method according to claim 6, characterized in that The determining that the first task is in a waiting state and using the first thread that executes the first task to execute the second task includes: Corresponding to the return value being a first return value, determining that the first task is in a waiting state; The second task is executed by using the first thread that executes the first task.
8. The method according to claim 6, characterized in that The M tasks include a third task, and the first task and the third task are in a task execution queue of the first thread; The method further comprises: Corresponding to the return value being a second return value, determining that the first task is in a completed state; The third task is executed by using the first thread that executes the first task.
9. An electronic device, characterized in that: include: A memory, used to store instructions executed by one or more processors of the electronic device, and the processor, which is one of the one or more processors of the electronic device, is used to execute the data processing method described in any one of claims 1-8.
10. A readable storage medium, characterized in that: The readable storage medium stores instructions, and when the instructions are executed on an electronic device, the electronic device executes the data processing method according to any one of claims 1 to 8.
