Data processing method, system, electronic device, medium, product and chip

Abstract

The present disclosure relates to a data processing method, system, electronic device, medium, product and chip. The data processing method comprises: in response to receiving the to-be-processed data sent by a target sensor, generating a standard data unit according to the to-be-processed data, the target sensor being a sensor corresponding to at least one target service in a current running service; constructing an algorithm queue corresponding to the standard data unit according to a target algorithm set subscribed by the at least one target service, an algorithm dependency relationship, a data frame rate and at least one service frame rate, the data frame rate being a frame rate at which the target sensor collects data, and the at least one service frame rate corresponding to the at least one target service in a one-to-one manner; and executing the algorithm queue to generate a processing result for the to-be-processed data. In the present disclosure, computing resources are allocated for the algorithm queue, thereby realizing dynamic allocation of computing resources, matching the allocated computing resources with computing demands, improving the utilization rate of computing resources, and reducing data processing delay.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a data processing method, system, electronic device, medium, product, and chip. Background Technology

[0002] During data processing, the computing resources allocated to tasks are usually constant. However, when business requirements or the characteristics of the data related to those requirements change, a mismatch may occur between the allocated computing resources and the actual needs. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a data processing method, system, electronic device, medium, product, and chip.

[0004] According to a first aspect of the present disclosure, a data processing method is provided, the data processing method comprising: In response to receiving data to be processed sent by a target sensor, a standard data unit is generated based on the data to be processed, wherein the target sensor is a sensor corresponding to at least one target service in the currently running service; Based on the target algorithm set, algorithm dependency, data frame rate, and at least one service frame rate of the target service subscription, construct the algorithm queue corresponding to the standard data unit, wherein the data frame rate is the frame rate of the data collected by the target sensor, and at least one service frame rate corresponds one-to-one with at least one target service; The algorithm queue is executed to generate the processing results for the data to be processed.

[0005] In this embodiment, the data to be processed sent by the target sensor is converted into corresponding standard data units, and a separate algorithm queue is constructed for each standard data unit. This allows computing resources to be allocated to the algorithms in the algorithm queue, realizing dynamic allocation of computing resources. This ensures that the allocated computing resources match the computing needs, improves the utilization rate of computing resources, and reduces data processing latency.

[0006] In some embodiments of this disclosure, generating standard data units based on the data to be processed includes: Generate an identifier corresponding to the data to be processed; The standard data unit is generated based on the identifier and the data to be processed.

[0007] In this embodiment, the data to be processed is uniformly converted into standard data units, which is beneficial for subsequent processing of data to be processed in different frames or multimodal data using the same processing method, thereby reducing the complexity of the data processing process.

[0008] In some embodiments of this disclosure, the identifier includes a data type identifier, a timestamp identifier, a data length identifier, a processing stage identifier, and multiple memory information identifiers.

[0009] In this embodiment, the identifier can identify information such as the data type, timestamp, and data length of the data to be processed included in the standard data unit. It can also identify the current processing stage of the algorithm and the memory information of each stage, so as to facilitate data processing and improve the accuracy of data processing.

[0010] In some embodiments of this disclosure, constructing the algorithm queue corresponding to the standard data unit based on at least one target algorithm set of the target service subscription, algorithm dependencies, data frame rate, and at least one service frame rate includes: The first algorithm set is determined based on the target algorithm set and the priorities of the algorithms in the target algorithm set; A second set of algorithms is determined based on the first set of algorithms, the data frame rate, and at least one of the service frame rates; Based on the second set of algorithms and the algorithm dependencies, a third set of algorithms is determined; The execution order of the algorithms in the third algorithm set is sorted to generate the algorithm queue.

[0011] In this embodiment, in the event of algorithm conflicts, the lowest priority algorithm is not allowed to enter the algorithm queue, and algorithms subscribed to by target services that do not need to be executed in the current frame are not allowed to enter the algorithm queue. Algorithms that need to be relied upon are allowed to enter the algorithm queue, and deduplication is performed. This ensures the necessity and accuracy of the algorithms in the algorithm queue. Furthermore, sorting the execution order of the algorithms can avoid out-of-order execution, ensure the smooth execution of the algorithm queue, and improve the accuracy of the data processing results.

[0012] In some embodiments of this disclosure, determining the first algorithm set based on the target algorithm set and the priorities corresponding to the algorithms in the target algorithm set includes: In response to the fact that all algorithms in the target algorithm set have the lowest priority or none of them have the lowest priority, it is determined that the first algorithm set includes all algorithms in the target algorithm set; In response to the priority of the algorithms in the target algorithm set including the lowest priority and the priority higher than the lowest priority, it is determined that the first algorithm set includes algorithms in the target algorithm set with a priority higher than the lowest priority.

[0013] In this embodiment, if there is an algorithm with a priority higher than the lowest priority, the algorithm with the higher priority will be executed. Only when the corresponding algorithm has the lowest priority will the algorithm with the lowest priority be executed. This can ensure that important algorithms are executed first, guarantee the performance of critical services, and improve the response time of critical services.

[0014] In some embodiments of this disclosure, determining the second algorithm set based on the first algorithm set, the data frame rate, and at least one of the service frame rates includes: In response to the first service frame rate being the same as the data frame rate, it is determined that the second algorithm set includes the algorithm of the target service subscription corresponding to the first service frame rate in the first algorithm set; Wherein, at least one of the service frame rates includes the first service frame rate.

[0015] In this embodiment, for target services with the same service frame rate and data frame rate, the algorithm queue corresponding to each standard data unit includes the algorithm subscribed by the target service, which can optimize the execution timing of the algorithm.

[0016] In some embodiments of this disclosure, determining the second algorithm set based on the first algorithm set, the data frame rate, and at least one of the service frame rates further includes: In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is any one of a plurality of target frame numbers, it is determined that the second algorithm set includes the algorithm of the target service subscription corresponding to the second service frame rate in the first algorithm set; In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is not any of the multiple target frame numbers, it is determined that the second algorithm set does not include the algorithm in the first algorithm set that is only subscribed to by the target service corresponding to the second service frame rate; Wherein, at least one of the service frame rates also includes the second service frame rate, and the difference between any two adjacent target frame numbers is equal to the quotient of the data frame rate divided by the second service frame rate.

[0017] In this embodiment, for target services with different service frame rates and data frame rates, only the algorithm queue corresponding to the standard data unit of the target frame number includes algorithms that are only subscribed to by the target service, which can optimize the execution timing of the algorithms.

[0018] In some embodiments of this disclosure, determining the third algorithm set based on the second algorithm set and the algorithm dependencies includes: In response to determining, based on the algorithm dependency relationship, that an algorithm in the second algorithm set needs to depend on at least one dependent algorithm, the third algorithm set is determined to include all algorithms in the second algorithm set and at least one of the dependent algorithms; In response to determining, based on the algorithm dependencies, that the algorithms in the second algorithm set do not depend on other algorithms, the third algorithm set is determined to include all algorithms in the second algorithm set.

[0019] In this embodiment, the algorithm to be relied upon is added to the algorithm queue to ensure the normal implementation of business functions.

[0020] In some embodiments of this disclosure, sorting the execution order of the algorithms in the third algorithm set to generate the algorithm queue includes: Determine the execution batch of the algorithms in the third algorithm set; The execution order of the algorithms in the third algorithm set is sorted according to their execution batch and priority to generate the algorithm queue.

[0021] In this embodiment, determining the execution batch of the algorithm and sorting it according to the execution batch can ensure that each algorithm can obtain the data required for operation. Algorithms within the same execution batch are sorted in order of priority from high to low, which can ensure that important algorithms are executed first, guarantee the performance of critical business and improve the response time of critical business.

[0022] In some embodiments of this disclosure, determining the execution batch of the algorithms in the third algorithm set includes: In response to the fact that the third algorithm set does not include the dependent algorithm, the execution batch of all algorithms in the third algorithm set is determined as the earliest execution batch. In response to the third algorithm set including the dependent algorithm, the execution batch of algorithms in the third algorithm set that do not need to depend on the dependent algorithm is determined to be the earliest executed execution batch, and the execution batch of algorithms that need to depend on at least one target dependent algorithm is the next executed execution batch of the latest executed execution batch among the execution batches of at least one target dependent algorithm, wherein the dependent algorithm in the third algorithm set includes at least one target dependent algorithm.

[0023] In this embodiment, the execution batch of each algorithm is determined according to the dependencies between algorithms, so as to sort the execution order of the algorithms.

[0024] In some embodiments of this disclosure, sorting the execution order of the algorithms in the third algorithm set according to the execution batch and priority of the algorithms in the third algorithm set to generate the algorithm queue includes: The algorithms in the third algorithm set are sorted in order of execution batch from earliest to latest. The algorithms with the same execution batch are sorted in descending order of priority to generate the algorithm queue.

[0025] In this embodiment, the execution order of the algorithm is sorted according to the execution batch and priority, which ensures the correctness of the execution order and the normal operation of the functions required by the business.

[0026] In some embodiments of this disclosure, the execution of the algorithm queue includes: In response to the requirement that the neural network processor is needed to execute the algorithm queue, the neural network processor is powered on; The data processing method further includes: In response to the completion of the algorithm queue execution, the neural network processor is powered off.

[0027] In this embodiment, the neural network processor can be controlled to power on only when data processing is required, thereby minimizing the power consumption of the neural network processor.

[0028] In some embodiments of this disclosure, the data processing method further includes: In response to the fact that the currently running service does not include the service corresponding to the sensor that is in the powered-on state, the sensor that is in the powered-on state is powered down.

[0029] In this embodiment, sensors that are not in use can be powered off, reducing power consumption.

[0030] In some embodiments of this disclosure, the data processing method further includes: In response to the presence of multiple target sensors, multiple algorithm queues corresponding one-to-one with the multiple target sensors are executed in parallel. In this context, the frame numbers of the multiple standard data units corresponding to the multiple algorithm queues are the same.

[0031] In this embodiment, the algorithm queues corresponding to data with the same frame number output by multiple target sensors can be executed in parallel, realizing parallel processing of multiple services.

[0032] According to a second aspect of the present disclosure, a data processing system is provided, the data processing system comprising: A data processing module is configured to generate standard data units based on data to be processed in response to receiving data to be processed sent by a target sensor, wherein the target sensor is a sensor corresponding to at least one target service in the currently running service. A queue construction module is configured to construct an algorithm queue corresponding to the standard data unit based on a target algorithm set subscribed to by at least one target service, algorithm dependencies, data frame rate, and at least one service frame rate, wherein the data frame rate is the frame rate of the data collected by the target sensor, and at least one service frame rate corresponds one-to-one with at least one target service. An execution module is configured to execute the algorithm queue to generate processing results for the data to be processed.

[0033] In some embodiments of this disclosure, the data processing module is further configured to: Generate an identifier corresponding to the data to be processed; The standard data unit is generated based on the identifier and the data to be processed.

[0034] In some embodiments of this disclosure, the identifier includes a data type identifier, a timestamp identifier, a data length identifier, a processing stage identifier, and multiple memory information identifiers.

[0035] In some embodiments of this disclosure, the queue construction module is further configured to: The first algorithm set is determined based on the target algorithm set and the priorities of the algorithms in the target algorithm set; A second set of algorithms is determined based on the first set of algorithms, the data frame rate, and at least one of the service frame rates; Based on the second set of algorithms and the algorithm dependencies, a third set of algorithms is determined; The execution order of the algorithms in the third algorithm set is sorted to generate the algorithm queue.

[0036] In some embodiments of this disclosure, the queue construction module is further configured to: In response to the fact that all algorithms in the target algorithm set have the lowest priority or none of them have the lowest priority, it is determined that the first algorithm set includes all algorithms in the target algorithm set; In response to the priority of the algorithms in the target algorithm set including the lowest priority and the priority higher than the lowest priority, it is determined that the first algorithm set includes algorithms in the target algorithm set with a priority higher than the lowest priority.

[0037] In some embodiments of this disclosure, the queue construction module is further configured to: In response to the first service frame rate being the same as the data frame rate, it is determined that the second algorithm set includes the algorithm of the target service subscription corresponding to the first service frame rate in the first algorithm set; Wherein, at least one of the service frame rates includes the first service frame rate.

[0038] In some embodiments of this disclosure, the queue construction module is further configured to: In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is any one of a plurality of target frame numbers, it is determined that the second algorithm set includes the algorithm of the target service subscription corresponding to the second service frame rate in the first algorithm set; In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is not any of the multiple target frame numbers, it is determined that the second algorithm set does not include the algorithm in the first algorithm set that is only subscribed to by the target service corresponding to the second service frame rate; Wherein, at least one of the service frame rates also includes the second service frame rate, and the difference between any two adjacent target frame numbers is equal to the quotient of the data frame rate divided by the second service frame rate.

[0039] In some embodiments of this disclosure, the queue construction module is further configured to: In response to determining, based on the algorithm dependency relationship, that an algorithm in the second algorithm set needs to depend on at least one dependent algorithm, the third algorithm set is determined to include all algorithms in the second algorithm set and at least one of the dependent algorithms; In response to determining, based on the algorithm dependencies, that the algorithms in the second algorithm set do not depend on other algorithms, the third algorithm set is determined to include all algorithms in the second algorithm set.

[0040] In some embodiments of this disclosure, the queue construction module is further configured to: Determine the execution batch of the algorithms in the third algorithm set; The execution order of the algorithms in the third algorithm set is sorted according to their execution batch and priority to generate the algorithm queue.

[0041] In some embodiments of this disclosure, the queue construction module is further configured to: In response to the fact that the third algorithm set does not include the dependent algorithm, the execution batch of all algorithms in the third algorithm set is determined as the earliest execution batch. In response to the third algorithm set including the dependent algorithm, the execution batch of algorithms in the third algorithm set that do not need to depend on the dependent algorithm is determined to be the earliest executed execution batch, and the execution batch of algorithms that need to depend on at least one target dependent algorithm is the next executed execution batch of the latest executed execution batch among the execution batches of at least one target dependent algorithm, wherein the dependent algorithm in the third algorithm set includes at least one target dependent algorithm.

[0042] In some embodiments of this disclosure, the queue construction module is further configured to: The algorithms in the third algorithm set are sorted in order of execution batch from earliest to latest. The algorithms with the same execution batch are sorted in descending order of priority to generate the algorithm queue.

[0043] In some embodiments of this disclosure, the execution module is further configured to: In response to the requirement that the neural network processor is needed to execute the algorithm queue, the neural network processor is powered on; In response to the completion of the algorithm queue execution, the neural network processor is powered off.

[0044] In some embodiments of this disclosure, the execution module is further configured to: In response to the fact that the currently running service does not include the service corresponding to the sensor that is in the powered-on state, the sensor that is in the powered-on state is powered down.

[0045] In some embodiments of this disclosure, the execution module is further configured to: In response to the presence of multiple target sensors, multiple algorithm queues corresponding one-to-one with the multiple target sensors are executed in parallel. In this context, the frame numbers of the multiple standard data units corresponding to the multiple algorithm queues are the same.

[0046] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the data processing method described above.

[0047] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the data processing method described above.

[0048] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the data processing method described above.

[0049] According to a sixth aspect of the present disclosure, a chip is provided, the chip including one or more interfaces and one or more processors; The interface is used to receive signals from the memory of the electronic device and send the signals to the processor. The signals include computer instructions stored in the memory. When the processor executes the computer instructions, it causes the electronic device to perform the data processing method described above.

[0050] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In this disclosure, the data to be processed sent by the target sensor is converted into corresponding standard data units, and a separate algorithm queue is constructed for each standard data unit. This allows computing resources to be allocated to the algorithms in the algorithm queue, realizing dynamic allocation of computing resources, matching the allocated computing resources with the computing needs, improving the utilization rate of computing resources, and reducing data processing latency.

[0051] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0053] Figure 1 This is a flowchart illustrating a data processing method according to an exemplary embodiment; Figure 2 This is a flowchart illustrating a data processing method according to another exemplary embodiment; Figure 3 This is a flowchart illustrating a data processing method according to another exemplary embodiment; Figure 4 This is an example diagram illustrating a construction algorithm queue according to an exemplary embodiment; Figure 5 This is a flowchart illustrating a data processing method according to another exemplary embodiment; Figure 6 This is a flowchart illustrating a data processing method according to another exemplary embodiment; Figure 7 This is an example diagram illustrating the parallel execution of multiple algorithm queues according to an exemplary embodiment; Figure 8 This is a flowchart illustrating a data processing method according to another exemplary embodiment; Figure 9 This is a block diagram of a data processing system according to an exemplary embodiment; Figure 10 This is a block diagram of a data processing system according to another exemplary embodiment; Figure 11 This is a block diagram of an electronic device according to an exemplary embodiment.

[0054] In the picture: 1100 - Electronic device; 1102 - Processing component; 1104 - Memory; 1106 - Power supply component; 1108 - Multimedia component; 1110 - Audio component; 1112 - Input / output interface; 1114 - Sensor component; 1116 - Communication component; 1120 - Processor. Detailed Implementation

[0055] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. It should also be understood that the term “and / or” as used in this disclosure refers to any or all possible combinations including one or more of the associated listed items.

[0056] During data processing, the computing resources allocated to tasks are usually constant. However, when business requirements or the characteristics of the corresponding data change, a mismatch may occur between the allocated computing resources and the actual needs, resulting in low utilization of the allocated computing resources or high data processing latency.

[0057] Based on this, this disclosure provides a data processing method that converts the data to be processed sent by the target sensor into corresponding standard data units, and constructs a separate algorithm queue for each standard data unit. This allows computing resources to be allocated to the algorithms in the algorithm queue, realizing dynamic allocation of computing resources, matching the allocated computing resources with the computing needs, improving the utilization rate of computing resources, and reducing data processing latency.

[0058] To facilitate understanding, the application scenarios of the data processing method disclosed herein are first described. The data processing method disclosed herein can be applied to single-modal or multi-modal data processing scenarios. For example, it can be applied to multi-modal data processing scenarios that process image data, audio data, and video stream data in monitoring systems, or to single-modal data processing scenarios that process image data in industrial inspection systems.

[0059] This disclosure provides a data processing method, such as... Figure 1 As shown, the data processing method includes: S100: In response to receiving the data to be processed sent by the target sensor, generate a standard data unit based on the data to be processed.

[0060] The target sensor is the sensor corresponding to at least one target service in the currently running service.

[0061] For example, each service is used to implement a required function, such as facial recognition, gesture recognition, or voice recognition. When a service runs, by executing the algorithm subscribed to by that service, the data collected by the corresponding sensor can be processed to obtain the corresponding data processing result, thereby realizing the function required by that service. Multiple services may correspond to the same sensor. If the currently running service includes at least one target service corresponding to a target sensor, then the data from the target sensor needs to be processed to realize the function required by at least one target service.

[0062] For example, a standard data unit corresponding to the data to be processed may include the data to be processed and an identifier corresponding to the data to be processed. In response to receiving the data to be processed, an identifier corresponding to the data to be processed may be generated, and then the identifier may be added to the data to be processed to generate a standard data unit. The identifiers corresponding to the data to be processed in different modalities may include the same type of identifier.

[0063] S200. Based on the target algorithm set, algorithm dependencies, data frame rate and at least one service frame rate of at least one target service subscription, construct the algorithm queue corresponding to the standard data unit.

[0064] The data frame rate is the frame rate at which the target sensor collects data, and at least one service frame rate corresponds one-to-one with at least one target service.

[0065] For example, the algorithms required for each service subscription. The algorithms for at least one target service subscription can be considered as a target algorithm set. In response to duplicate algorithms being used by multiple target service subscriptions, a first deduplication process is performed, such that the target algorithm set includes one of the duplicate algorithms.

[0066] For example, the first algorithm set can be determined based on the priority of the algorithms in the target algorithm set.

[0067] For example, the data frame rate can be the highest among at least one service frame rate. If a service frame rate is less than the data frame rate, and the frame number corresponding to the standard data unit is not one of the multiple target frame numbers, the algorithms in the first algorithm set that are only subscribed to by the target service corresponding to that service frame rate are deleted to generate a second algorithm set. The difference between any two adjacent target frame numbers is equal to the quotient of the data frame rate divided by the service frame rate. For example, if the data frame rate is 10 fps (Frames Per Second), and there are two target services, with the first target service corresponding to a service frame rate of 10 fps and the second target service corresponding to a service frame rate of 5 fps, then for the second target service, the difference between any two adjacent target frame numbers is equal to 2.

[0068] For example, the algorithm dependency relationship can be an algorithm dependency matrix, which includes the dependencies between all algorithms in the algorithm pool of the data processing system. The dependent algorithms that the algorithms in the second algorithm set depend on can be determined based on the algorithm dependency relationships, and these algorithms, together with the second algorithm set, form a third algorithm set. In response to algorithms in the second algorithm set that have the same dependent algorithm, a second deduplication process is performed, such that the third algorithm set includes one of those dependent algorithms.

[0069] For example, the execution order of the algorithms in the third algorithm set can be sorted to generate an algorithm queue.

[0070] S300: Execute the algorithm queue to generate the processing results for the data to be processed.

[0071] For example, the execution of each algorithm in the algorithm queue may include a preprocessing stage, a computation stage, and a post-processing stage. The preprocessing stage may process the data into a format recognizable by the processor corresponding to the algorithm. The data can be standard data units or the output data of the previous algorithm in the algorithm queue. The computation stage may involve the processor corresponding to the algorithm performing data computations. The post-processing stage may involve processing the computation results of the processor corresponding to the algorithm. The processor corresponding to the algorithm may be, for example, a Neural Processing Unit (NPU). In a business scenario such as face recognition, the computation result of the processor corresponding to the algorithm may include a face, in which case post-processing may involve extracting the face; conversely, the computation result may not include a face, in which case post-processing may involve discarding the data. At each stage, different computing resources can be allocated to the algorithm queue to match the computational needs.

[0072] In this embodiment, the data to be processed sent by the target sensor is converted into corresponding standard data units, and a separate algorithm queue is constructed for each standard data unit. This allows computing resources to be allocated to the algorithms in the algorithm queue, realizing dynamic allocation of computing resources. This ensures that the allocated computing resources match the computing needs, improves the utilization rate of computing resources, and reduces data processing latency.

[0073] In one embodiment, such as Figure 2 As shown, step S100, which generates standard data units based on the data to be processed, includes: S110. Generate the identifier corresponding to the data to be processed; For example, the identifiers corresponding to different data to be processed can include identifiers of the same type. Different data to be processed can be data with different frame numbers sent by the same target sensor, or data of different modalities such as audio data, image data, and video stream data sent by different target sensors. The identifiers corresponding to each data to be processed can include identifiers of data type, timestamp, data length, processing stage, and multiple memory information identifiers.

[0074] S120. Generate standard data units based on the identifier and the data to be processed.

[0075] For example, an identifier corresponding to the data to be processed can be added to the data to be processed to generate a standard data unit.

[0076] In this embodiment, the data to be processed is uniformly converted into standard data units, which is beneficial for subsequent processing of data to be processed in different frames or multimodal data using the same processing method, thereby reducing the complexity of the data processing process.

[0077] In one embodiment, the identifier includes a data type identifier, a timestamp identifier, a data length identifier, a processing stage identifier, and multiple memory information identifiers.

[0078] For example, a data type identifier can be used to store the data type information of the data to be processed. A timestamp identifier can be used to store the data acquisition time information of the data to be processed. A data length identifier can be used to store the data length information of the data to be processed. A processing stage identifier can be used to store the execution stage information of the algorithm being executed in the corresponding algorithm queue, such as the preprocessing stage, computation stage, and postprocessing stage. There can be three memory information identifiers, which are used to store the memory resource information allocated for the preprocessing stage, computation stage, and postprocessing stage of the algorithm being executed in the algorithm queue, respectively.

[0079] In this embodiment, the identifier can identify information such as the data type, timestamp, and data length of the data to be processed included in the standard data unit. It can also identify the current processing stage of the algorithm and the memory information of each stage, so as to facilitate data processing and improve the accuracy of data processing.

[0080] In one embodiment, such as Figure 3 As shown, step S200, which involves constructing an algorithm queue corresponding to a standard data unit based on at least one target service subscription's target algorithm set, algorithm dependencies, data frame rate, and at least one service frame rate, includes: S210. Determine the first algorithm set based on the target algorithm set and the priority of the algorithms in the target algorithm set.

[0081] For example, the priority of an algorithm can be pre-set based on its importance. In response to the condition that all algorithms in the target algorithm set have the lowest priority or none have the lowest priority, the first algorithm set is determined to include all algorithms in the target algorithm set. In response to the condition that the priorities of the algorithms in the target algorithm set include both the lowest priority and priorities higher than the lowest priority, the first algorithm set is determined to include algorithms in the target algorithm set with priorities higher than the lowest priority.

[0082] S220. Determine the second algorithm set based on the first algorithm set, the data frame rate, and at least one service frame rate.

[0083] For example, the data frame rate can be the largest of at least one service frame rate. If a service frame rate is less than the data frame rate, and the frame number corresponding to the standard data unit is not one of the multiple target frame numbers, the algorithms in the first algorithm set that are subscribed to only by the target service corresponding to that service frame rate are deleted to generate a second algorithm set. The difference between any two adjacent target frame numbers is equal to the quotient of the data frame rate divided by the service frame rate.

[0084] S230. Determine the third algorithm set based on the second algorithm set and algorithm dependencies.

[0085] For example, the algorithm dependency relationship can be an algorithm dependency matrix, which includes the dependencies between all algorithms in the algorithm pool of the data processing system. The dependent algorithms that the algorithms in the second algorithm set depend on can be determined based on the algorithm dependency relationships, and these algorithms, together with the second algorithm set, form a third algorithm set. In response to algorithms in the second algorithm set that have the same dependent algorithm, a second deduplication process is performed, such that the third algorithm set includes one of those dependent algorithms.

[0086] S240. Sort the execution order of the algorithms in the third algorithm set to generate an algorithm queue.

[0087] For example, the execution order of algorithms in a third set of algorithms can be sorted according to algorithm dependencies and priorities. During sorting, the execution order of the dependent algorithm can be made earlier than the algorithms that depend on it, and the execution order of the higher priority algorithm can be made earlier than the lower priority algorithm.

[0088] In this embodiment, in the event of algorithm conflicts, the lowest priority algorithm is not allowed to enter the algorithm queue, and algorithms subscribed to by target services that do not need to be executed in the current frame are not allowed to enter the algorithm queue. Algorithms that need to be relied upon are allowed to enter the algorithm queue, and deduplication is performed. This ensures the necessity and accuracy of the algorithms in the algorithm queue. Furthermore, sorting the execution order of the algorithms can avoid out-of-order execution, ensure the smooth execution of the algorithm queue, and improve the accuracy of the data processing results.

[0089] In one embodiment, step S210, determining the first algorithm set based on the target algorithm set and the priorities corresponding to the algorithms in the target algorithm set, includes: determining that the first algorithm set includes all algorithms in the target algorithm set if all algorithms in the target algorithm set have the lowest priority or none have the lowest priority; and determining that the first algorithm set includes algorithms in the target algorithm set with priorities higher than the lowest priority if the priorities corresponding to the algorithms in the target algorithm set include the lowest priority and priorities higher than the lowest priority.

[0090] For example, Figure 4 An example diagram of constructing an algorithm queue is shown. Figure 4 As shown, there are two target services. The first target service subscribes to algorithms GD and ALS, while the second target service subscribes to algorithms DT and GD. The target sensors corresponding to the first and second target services send two frames of data to be processed. The standard data unit corresponding to the first frame of data is the first standard data unit, and the standard data unit corresponding to the second frame of data is the second standard data unit. Because both the first and second target services subscribe to algorithm GD, a first deduplication process is required to ensure that the target algorithm set includes only one algorithm, GD. After the first deduplication process, the target algorithm sets corresponding to the first and second standard data units both include algorithms GD, ALS, and DT. Figure 4The algorithm dependency matrix shows the priority p of each algorithm. The smaller the value of priority p, the lower the priority, with p = 0 being the lowest priority. From the algorithm dependency matrix, we can deduce that the algorithm als in the target algorithm set corresponding to the first standard data unit and the target algorithm set corresponding to the second standard data unit has the lowest priority, while the priorities of algorithms gd and dt are higher than the lowest priority. Therefore, the first algorithm set corresponding to the first standard data unit includes all algorithms in the target algorithm set corresponding to the first standard data unit except for algorithm als, i.e., algorithms gd and dt. Similarly, the first algorithm set corresponding to the second standard data unit includes all algorithms in the target algorithm set corresponding to the second standard data unit except for algorithm als, i.e., algorithms gd and dt. It should be noted that if the priority p values ​​of all algorithms in the target algorithm set are equal to 0 or are not equal to 0, then the corresponding first algorithm set includes all algorithms in that target algorithm set.

[0091] In this embodiment, if there is an algorithm with a priority higher than the lowest priority, the algorithm with the higher priority will be executed. Only when the corresponding algorithm has the lowest priority will the algorithm with the lowest priority be executed. This can ensure that important algorithms are executed first, guarantee the performance of critical services, and improve the response time of critical services.

[0092] In one embodiment, determining the second algorithm set based on the first algorithm set, the data frame rate, and at least one service frame rate in step S220 includes: in response to the first service frame rate being the same as the data frame rate, determining that the second algorithm set includes the algorithm of the target service subscription corresponding to the first service frame rate in the first algorithm set.

[0093] At least one service frame rate includes the first service frame rate.

[0094] For example, such as Figure 4 As shown, the data frame rate is 10fps, and the service frame rate of the first target service is also 10fps, the same as the data frame rate, which is the first service frame rate. The second algorithm set corresponding to the first standard data unit includes the algorithm `gd` subscribed to by the first target service from the first algorithm set corresponding to the first standard data unit. The second algorithm set corresponding to the second standard data unit includes the algorithm `gd` subscribed to by the first target service from the first algorithm set corresponding to the second standard data unit.

[0095] In this embodiment, for target services with the same service frame rate and data frame rate, the algorithm queue corresponding to each standard data unit includes the algorithm subscribed by the target service, which can optimize the execution timing of the algorithm.

[0096] In one embodiment, step S220, determining the second algorithm set based on the first algorithm set, the data frame rate, and at least one service frame rate, further includes: in response to the second service frame rate being different from the data frame rate and the frame number of the standard data unit being any one of multiple target frame numbers, determining that the second algorithm set includes the algorithm in the first algorithm set that is subscribed to by the target service corresponding to the second service frame rate; and in response to the second service frame rate being different from the data frame rate and the frame number of the standard data unit not being any one of multiple target frame numbers, determining that the second algorithm set does not include the algorithm in the first algorithm set that is subscribed to only by the target service corresponding to the second service frame rate.

[0097] At least one service frame rate also includes a second service frame rate, where the difference between any two adjacent target frame numbers is equal to the quotient of the data frame rate divided by the second service frame rate.

[0098] For example, such as Figure 4 As shown, the service frame rate of the second target service is 5fps, which is different from the data frame rate; it is the second service frame rate. The quotient of the data frame rate divided by this second service frame rate is 2. That is, for the second target service, the difference between any two adjacent target frame numbers is equal to 2. Figure 4 For example, if the frame number of the first standard data unit is the target frame number, then the next target frame number is the frame number of the standard data unit corresponding to the third frame of data to be processed. The frame number of the second standard data unit is not the target frame number. The second algorithm set corresponding to the first standard data unit includes the algorithm dt that is only subscribed to by the second target service in the first algorithm set corresponding to the first standard data unit. The second algorithm set corresponding to the second standard data unit does not include the algorithm dt that is only subscribed to by the second target service in the first algorithm set corresponding to the second standard data unit.

[0099] In this embodiment, for target services with different service frame rates and data frame rates, only the algorithm queue corresponding to the standard data unit of the target frame number includes algorithms that are only subscribed to by the target service, which can optimize the execution timing of the algorithms.

[0100] In one embodiment, determining the third algorithm set based on the second algorithm set and algorithm dependencies in step S230 includes: in response to determining that an algorithm in the second algorithm set needs to depend on at least one dependent algorithm based on the algorithm dependencies, determining that the third algorithm set includes all algorithms in the second algorithm set and at least one dependent algorithm; in response to determining that an algorithm in the second algorithm set does not need to depend on other algorithms based on the algorithm dependencies, determining that the third algorithm set includes all algorithms in the second algorithm set.

[0101] For example, such as Figure 4As shown in the diagram, according to the algorithm dependency matrix, in the second algorithm set corresponding to the first standard data unit, algorithm gd depends on algorithm fd, and algorithm dt also depends on algorithm fd. Therefore, algorithm fd is a dependent algorithm and is depended on twice. Based on this, a second deduplication process is needed so that the third algorithm set corresponding to the first standard data unit includes one algorithm fd. After the second deduplication process, the third algorithm set corresponding to the first standard data unit includes algorithms gd and dt from the second algorithm set corresponding to the first standard data unit, and also includes the dependent algorithm fd. It should be noted that if the algorithms in the second algorithm set do not depend on other algorithms, the corresponding third algorithm set only includes all the algorithms in the second algorithm set.

[0102] In this embodiment, the algorithm to be relied upon is added to the algorithm queue to ensure the normal implementation of business functions.

[0103] In one embodiment, such as Figure 5 As shown, step S240, which involves sorting the execution order of algorithms in the third algorithm set to generate an algorithm queue, includes: S241. Determine the execution batch of the algorithms in the third algorithm set.

[0104] For example, in the third set of algorithms, the execution batch of algorithms that do not depend on the dependent algorithm can be determined as the earliest execution batch, and the execution batch of algorithms that depend on at least one target dependent algorithm can be determined as the next execution batch after the latest execution batch among the execution batches of at least one target dependent algorithm.

[0105] S242. Sort the execution order of the algorithms in the third algorithm set according to their execution batches and priorities to generate an algorithm queue.

[0106] For example, when sorting, the algorithms can first be sorted in order of execution batch from earliest to latest, and then the algorithms with the same execution batch can be sorted in order of priority from highest to lowest to generate an algorithm queue.

[0107] In this embodiment, determining the execution batch of the algorithm and sorting it according to the execution batch can ensure that each algorithm can obtain the data required for operation. Algorithms within the same execution batch are sorted in order of priority from high to low, which can ensure that important algorithms are executed first, guarantee the performance of critical business and improve the response time of critical business.

[0108] In one embodiment, determining the execution batch of the algorithms in the third algorithm set in step S241 includes: in response to the third algorithm set not including the dependent algorithm, determining the execution batch of all algorithms in the third algorithm set as the earliest executed execution batch; in response to the third algorithm set including the dependent algorithm, determining the execution batch of algorithms in the third algorithm set that do not need to depend on the dependent algorithm as the earliest executed execution batch, and the execution batch of algorithms that need to depend on at least one target dependent algorithm as the next executed execution batch after the latest executed execution batch among the execution batches of at least one target dependent algorithm.

[0109] The dependent algorithms in the third set of algorithms include at least one target dependent algorithm.

[0110] For example, the initial value of the algorithm's execution batch can be set to 0. A larger value for the algorithm's execution batch indicates a later execution order, and the difference between the values ​​of two adjacent execution batches can be 1. Figure 4 Taking an example, if algorithm fd in the third algorithm set corresponding to the first standard data unit does not depend on other algorithms, then the execution batch value b of algorithm fd is set to 1, indicating that algorithm fd is executed in the first batch. Algorithms gd and dt both depend on algorithm fd, so the execution batch values ​​b of both algorithms gd and dt are set to 2, indicating that algorithms gd and dt are executed in the second batch. Similarly, if algorithm fd in the third algorithm set corresponding to the second standard data unit does not depend on other algorithms, then the execution batch value b of algorithm fd is set to 1, indicating that algorithm fd is executed in the first batch. If algorithm gd depends on algorithm fd, then the execution batch value b of algorithm gd is set to 2, indicating that algorithm gd is executed in the second batch. It should be noted that, in response to all algorithms in the third algorithm set not depending on the algorithms they depend on, the execution batch value of each algorithm is set to 1. In response to one algorithm in the third algorithm set having an execution batch value of 1, another algorithm having an execution batch value of 2, and another algorithm depending on the first two algorithms, then the execution batch value of that algorithm is set to the maximum value of the execution batch values ​​of the algorithms it depends on, plus 1, i.e., set to 3.

[0111] In this embodiment, the execution batch of each algorithm is determined according to the dependencies between algorithms, so as to sort the execution order of the algorithms.

[0112] In one embodiment, such as Figure 6 As shown, step S242, which involves sorting the execution order of algorithms in the third algorithm set according to their execution batches and priorities to generate an algorithm queue, includes: S601. Sort the algorithms in the third algorithm set in order of execution batch from earliest to latest.

[0113] For example, Figure 4 In the algorithm queue, the arrows indicate the execution order of the algorithms from first to last. In the algorithm queue corresponding to the first standard data unit, the execution batch b value of algorithm fd is 1, making it the earliest executed batch. The execution batch b values ​​of algorithms gd and dt are both 2, making them the second executed batches. Therefore, in the execution order, algorithm fd is executed before algorithms gd and dt. In the algorithm queue corresponding to the second standard data unit, the execution batch b value of algorithm fd is 1, making it the earliest executed batch. The execution batch b value of algorithm gd is 2, making it the second executed batch. Therefore, in the execution order, algorithm fd is executed before algorithm gd.

[0114] S602. Sort the algorithms that are executed in the same batch in descending order of priority to generate an algorithm queue.

[0115] For example, Figure 4 In the algorithm queue corresponding to the first standard data unit, algorithms gd and dt are executed in the same batch. Because the priority p of algorithm gd is 1 and the priority p of algorithm dt is 2, meaning that the priority of algorithm dt is higher than that of algorithm gd, algorithm dt is executed before algorithm gd in the algorithm queue.

[0116] In this embodiment, the execution order of the algorithm is sorted according to the execution batch and priority, which ensures the correctness of the execution order and the normal operation of the functions required by the business.

[0117] The processing time for one frame of data can be much shorter than the interval between two frames. If the neural network processor remains powered on continuously, it will result in wasted power. Therefore, in one embodiment, the execution algorithm queue in step S300 includes: powering on the neural network processor in response to the need for the execution algorithm queue to use it. The data processing method further includes: powering off the neural network processor in response to the completion of the algorithm queue execution.

[0118] For example, in response to an algorithm in the algorithm queue corresponding to a standard data unit requiring execution by a neural network processor, the neural network processor can be powered on to allocate computing resources to the corresponding algorithm in the algorithm queue, allowing the neural network processor to process the current standard data unit. In response to the end of algorithm queue execution, the neural network processor can be powered off. The neural network processor can be powered on again the next time it is needed.

[0119] In this embodiment, the neural network processor can be controlled to power on only when data processing is required, thereby minimizing the power consumption of the neural network processor.

[0120] In the application scenarios of the data processing method disclosed herein, multiple sensors may be configured. For example, a monitoring system may include multiple image sensors, sound sensors, etc. When the system in the application scenario starts up, these sensors typically switch from a power-off state to a power-on state. Subsequently, regardless of whether the currently running service uses data collected by a particular sensor, that sensor remains in a power-on state, resulting in wasted power consumption. Based on this, in one embodiment, the data processing method further includes: in response to the fact that the currently running service does not include the service corresponding to the sensor in the power-on state, powering off the sensor in the power-on state.

[0121] For example, a timer can be set in the data processing system. Each time the timer reaches its set time, it is checked whether the currently running service includes a service corresponding to each sensor that is powered on. If there is no service corresponding to a sensor that is powered on, the sensor is powered off.

[0122] In this embodiment, sensors that are not in use can be powered off, reducing power consumption.

[0123] In one embodiment, the data processing method further includes: in response to the number of target sensors being multiple, executing multiple algorithm queues corresponding one-to-one with the multiple target sensors in parallel.

[0124] Multiple algorithm queues correspond to multiple standard data units with the same frame number.

[0125] For example, such as Figure 7 As shown, there are three target sensors. The three algorithm queues corresponding to the three standard data units of the first frame (frame number 1), namely the first, second, and third algorithm queues, are executed in parallel. The three algorithm queues corresponding to the three standard data units of the second frame (frame number 2), namely the fourth, fifth, and sixth algorithm queues, are executed in parallel. The three algorithm queues corresponding to the three standard data units of the third frame (frame number 3), namely the seventh, eighth, and ninth algorithm queues, are executed in parallel.

[0126] In this embodiment, the algorithm queues corresponding to data with the same frame number output by multiple target sensors can be executed in parallel, realizing parallel processing of multiple services.

[0127] This disclosure provides a data processing method, such as... Figure 8 As shown, the data processing methods include: S801. In response to receiving the data to be processed sent by the target sensor, generate an identifier corresponding to the data to be processed.

[0128] The target sensor is the sensor corresponding to at least one target service in the currently running service.

[0129] S802. Generate standard data units based on the identifier and the data to be processed.

[0130] S803. In response to the fact that the priority of each algorithm in the target algorithm set of at least one target service subscription is the lowest priority or none of them are the lowest priority, determine that the first algorithm set includes all algorithms in the target algorithm set.

[0131] S804. In response to the priority of the algorithms in the target algorithm set, including the lowest priority and the priority higher than the lowest priority, determine that the first algorithm set includes the algorithms in the target algorithm set with priorities higher than the lowest priority.

[0132] S805. In response to the first service frame rate being the same as the data frame rate, determine that the second algorithm set includes the algorithm of the target service subscription corresponding to the first service frame rate in the first algorithm set.

[0133] S806. In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is any one of multiple target frame numbers, determine that the second algorithm set includes the algorithm of the target service subscription corresponding to the second service frame rate in the first algorithm set.

[0134] S807. In response to the fact that the second service frame rate is different from the data frame rate and the frame number of the standard data unit is not any of the multiple target frame numbers, determine the second algorithm set that does not include the first algorithm set and is only subscribed to by the target service corresponding to the second service frame rate.

[0135] S808. In response to determining that an algorithm in the second algorithm set depends on at least one dependent algorithm based on algorithm dependencies, the third algorithm set is determined to include all algorithms in the second algorithm set and at least one dependent algorithm.

[0136] S809. In response to determining that the algorithms in the second algorithm set do not depend on other algorithms based on algorithm dependencies, the third algorithm set is determined to include all algorithms in the second algorithm set.

[0137] S810. In response to the fact that the third algorithm set does not include the dependent algorithm, the execution batch of all algorithms in the third algorithm set is determined as the earliest execution batch.

[0138] S811. In response to the third algorithm set including the dependent algorithm, determine the execution batch of the algorithm in the third algorithm set that does not need to depend on the dependent algorithm as the earliest execution batch, and the execution batch of the algorithm that needs to depend on at least one target dependent algorithm as the next execution batch of the latest execution batch among the execution batches of at least one target dependent algorithm.

[0139] S812. Sort the algorithms in the third algorithm set in order of execution batch from earliest to latest.

[0140] S813. Sort the algorithms that are executed in the same batch according to their priority from high to low to generate an algorithm queue.

[0141] This disclosure provides a data processing system, such as... Figure 9 As shown, the data processing system includes a data processing module 91, a queue construction module 92, and an execution module 93. The data processing module 91 is configured to generate standard data units based on the data to be processed received from a target sensor. The target sensor is a sensor corresponding to at least one target service in the currently running service. The queue construction module 92 is configured to construct an algorithm queue corresponding to the standard data units based on the target algorithm set subscribed to by at least one target service, algorithm dependencies, data frame rate, and at least one service frame rate. The data frame rate is the frame rate at which the target sensor collects data, and at least one service frame rate corresponds one-to-one with at least one target service. The execution module 93 is configured to execute the algorithm queue to generate the processing result of the data to be processed.

[0142] In one embodiment, the data processing module 91 is further configured to: Generate identifiers corresponding to the data to be processed; Standard data units are generated based on the identifier and the data to be processed.

[0143] In one embodiment, the identifier includes a data type identifier, a timestamp identifier, a data length identifier, a processing stage identifier, and multiple memory information identifiers.

[0144] In one embodiment, the queue construction module 92 is further configured to: The first algorithm set is determined based on the target algorithm set and the priority of the algorithms in the target algorithm set; The second algorithm set is determined based on the first algorithm set, the data frame rate, and at least one service frame rate; The third set of algorithms is determined based on the second set of algorithms and their dependencies. The execution order of the algorithms in the third set of algorithms is sorted to generate an algorithm queue.

[0145] In one embodiment, the queue construction module 92 is further configured to: In response to the fact that all algorithms in the target algorithm set have the lowest priority or none of them have the lowest priority, it is determined that the first algorithm set includes all algorithms in the target algorithm set; In response to the priority of the algorithms in the target algorithm set, including the lowest priority and the priority higher than the lowest priority, the first algorithm set is determined to include the algorithms in the target algorithm set with priorities higher than the lowest priority.

[0146] In one embodiment, the queue construction module 92 is further configured to: In response to the first service frame rate being the same as the data frame rate, the second algorithm set is determined to include the algorithm of the target service subscription corresponding to the first service frame rate in the first algorithm set; Among them, at least one service frame rate includes the first service frame rate.

[0147] In one embodiment, the queue construction module 92 is further configured to: In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is any one of multiple target frame numbers, the second algorithm set is determined to include the algorithm of the target service subscription corresponding to the second service frame rate in the first algorithm set; In response to the fact that the second service frame rate is different from the data frame rate and the frame number of the standard data unit is not any of the multiple target frame numbers, the algorithm set is determined to be an algorithm that is not included in the first algorithm set and is only subscribed to by the target service corresponding to the second service frame rate. Among them, at least one service frame rate also includes a second service frame rate, and the difference between any two adjacent target frame numbers is equal to the quotient of the data frame rate divided by the second service frame rate.

[0148] In one embodiment, the queue construction module 92 is further configured to: In response to determining that an algorithm in the second algorithm set depends on at least one dependent algorithm based on algorithm dependencies, the third algorithm set is determined to include all algorithms in the second algorithm set and at least one dependent algorithm. In response to the determination that the algorithms in the second algorithm set do not depend on other algorithms based on algorithm dependencies, the third algorithm set is determined to include all algorithms in the second algorithm set.

[0149] In one embodiment, the queue construction module 92 is further configured to: Determine the execution batch of the algorithms in the third algorithm set; Based on the execution batch and priority of the algorithms in the third algorithm set, the execution order of the algorithms in the third algorithm set is sorted to generate an algorithm queue.

[0150] In one embodiment, the queue construction module 92 is further configured to: Since the third algorithm set does not include the dependent algorithm, the execution batch of all algorithms in the third algorithm set is determined as the earliest execution batch. In response to the fact that the third set of algorithms includes dependent algorithms, the execution batch of algorithms in the third set that do not need to depend on dependent algorithms is determined to be the earliest executed execution batch, and the execution batch of algorithms that need to depend on at least one target dependent algorithm is determined to be the next executed execution batch of the latest executed execution batch among the execution batches of at least one target dependent algorithm, wherein the dependent algorithms in the third set include at least one target dependent algorithm.

[0151] In one embodiment, the queue construction module 92 is further configured to: Sort the algorithms in the third algorithm set in order of execution batch from earliest to latest; Algorithms that execute in the same batch are sorted in descending order of priority to generate an algorithm queue.

[0152] In one embodiment, the execution module 93 is further configured to: In response to the requirement of using a neural network processor in the algorithm execution queue, power on the neural network processor; In response to the completion of the algorithm queue execution, the neural network processor is powered off.

[0153] In one embodiment, the execution module 93 is further configured to: In response to the fact that the currently running services do not include the services corresponding to sensors that are powered on, the sensors that are powered on will be powered off.

[0154] In one embodiment, the execution module 93 is further configured to: In response to the fact that there are multiple target sensors, multiple algorithm queues corresponding to the multiple target sensors are executed in parallel. Among them, multiple standard data units corresponding to multiple algorithm queues have the same frame number.

[0155] In one exemplary embodiment, such as Figure 10 As shown, a data processing system is illustrated. Figure 10 The data processing system in the system comprises an application layer, a service layer, an engine layer, and a hardware layer. The application layer supports the parallel execution of multiple business processes, such as... Figure 10As shown, services one, two, three, and four run in parallel. The service layer includes a data processing module, a scheduling service module, and an algorithm management module. The data service module can be the data processing module 91 in the above embodiment, used to convert multimodal data to be processed into standardized data units with a unified format. The scheduling service module can be used for queue management, algorithm execution batch management, algorithm priority management, and frame rate alignment between algorithms and services. The algorithm management module can be used to register algorithms, set algorithm priorities and dependencies between algorithms, count algorithms subscribed by services, and report the execution results of algorithms to the corresponding services. The engine layer includes a queue construction module, a scheduling decision module, and a neural network processor execution module. The queue construction module can be the queue construction module 92 in the above embodiment. The scheduling decision module can be used to monitor computing resources, formulate scheduling strategies for computing resources, and formulate execution plans for algorithms. The neural network processor execution module can be used to allocate computing tasks to neural network processors, collect data processing results from neural network processors, and control the state synchronization of each layer. The hardware layer can include multiple sensors, neural network processors, general-purpose processors, and memory. Multiple sensors may include image data sensors for generating image data, sound data sensors for generating sound data, and motion data sensors for generating motion data. A neural network processor is used for artificial intelligence-related data processing. A general-purpose processor is responsible for controlling and coordinating the data processing system. Memory is used to store data and enable cross-core data transfer. The scheduling decision module, the neural network processor execution module, and the neural network processor and general-purpose processor can work together to implement the functions of execution module 93 in the above embodiments.

[0156] This disclosure provides an electronic device, such as a multi-sensor embedded device, a mobile phone, a laptop computer, a tablet computer, and a wearable device.

[0157] refer to Figure 11 As shown, the electronic device 1100 may include one or more of the following components: processing component 1102, memory 1104, power supply component 1106, multimedia component 1108, audio component 1110, input / output (I / O) interface 1112, sensor component 1114, and communication component 1116.

[0158] Processing component 1102 typically controls the overall operation of electronic device 1100, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1102 may include one or more processors 1120 to execute instructions to complete all or part of the steps of the data processing method described above. Furthermore, processing component 1102 may include one or more modules to facilitate interaction between processing component 1102 and other components. For example, processing component 1102 may include a multimedia module to facilitate interaction between multimedia component 1108 and processing component 1102.

[0159] Memory 1104 is configured to store various types of data to support the operation of electronic device 1100. Examples of such data include instructions for any application or method operating on electronic device 1100, contact data, phonebook data, messages, pictures, videos, etc. Memory 1104 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0160] Power supply component 1106 provides power to various components of electronic device 1100. Power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1100.

[0161] Multimedia component 1108 includes a screen that provides an output interface between electronic device 1100 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1108 includes a front-facing camera module and / or a rear-facing camera module. When electronic device 1100 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera module and / or the rear-facing camera module may receive external multimedia data. Each front-facing camera module and rear-facing camera module may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0162] Audio component 1110 is configured to output and / or input audio signals. For example, audio component 1110 includes a microphone (MIC) configured to receive external audio signals when electronic device 1100 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1104 or transmitted via communication component 1116. In some embodiments, audio component 1110 also includes a speaker for outputting audio signals.

[0163] I / O interface 1112 provides an interface between processing component 1102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0164] Sensor assembly 1114 includes one or more sensors for providing state assessments of various aspects of electronic device 1100. For example, sensor assembly 1114 may detect the on / off state of electronic device 1100, the relative positioning of components such as the display and keypad of electronic device 1100, changes in position of electronic device 1100 or a component of electronic device 1100, the presence or absence of user contact with electronic device 1100, the orientation or acceleration / deceleration of electronic device 1100, and temperature changes of electronic device 1100. Sensor assembly 1114 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1114 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1114 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0165] Communication component 1116 is configured to facilitate wired or wireless communication between electronic device 1100 and other terminals. Electronic device 1100 can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 1116 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1116 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0166] In an exemplary embodiment, the electronic device 1100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing terminals (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the data processing methods shown in the above embodiments or combinations thereof.

[0167] This disclosure provides a non-transitory computer-readable storage medium including instructions, such as a memory 1104 including instructions, which can be executed by a processor 1120 of an electronic device 1100 to perform the data processing methods shown in the above embodiments or combinations thereof. For example, the non-transitory computer-readable storage medium may be a ROM, CD-ROM, magnetic tape, floppy disk, or optical data storage terminal, etc. When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the data processing methods shown in the above embodiments or combinations thereof.

[0168] This disclosure provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the data processing method described above.

[0169] This disclosure provides a chip. The chip includes one or more interfaces and one or more processors. The interfaces are used to receive signals from the memory of an electronic device and send signals to the processors. The signals include computer instructions stored in the memory. When the processor executes the computer instructions, it causes the electronic device to perform the data processing method described above.

[0170] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0171] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0172] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0173] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0174] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0175] The examples in this document may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and rules. In the examples, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0176] In this manual and the sample solutions, any processing of personal information will be conducted only under legal grounds (such as obtaining the consent of the data subject or being necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

Claims

1. A data processing method, characterized in that, The data processing method includes: In response to receiving data to be processed sent by a target sensor, a standard data unit is generated based on the data to be processed, wherein the target sensor is a sensor corresponding to at least one target service in the currently running service; Based on the target algorithm set, algorithm dependency, data frame rate, and at least one service frame rate of the target service subscription, construct the algorithm queue corresponding to the standard data unit, wherein the data frame rate is the frame rate of the data collected by the target sensor, and at least one service frame rate corresponds one-to-one with at least one target service; The algorithm queue is executed to generate the processing results for the data to be processed.

2. The data processing method according to claim 1, characterized in that, The step of generating standard data units based on the data to be processed includes: Generate an identifier corresponding to the data to be processed; The standard data unit is generated based on the identifier and the data to be processed.

3. The data processing method according to claim 2, characterized in that, The identifiers include data type identifiers, timestamp identifiers, data length identifiers, processing stage identifiers, and multiple memory information identifiers.

4. The data processing method according to claim 1, characterized in that, The step of constructing the algorithm queue corresponding to the standard data unit based on at least one target algorithm set subscribed to by the target service, algorithm dependencies, data frame rate, and at least one service frame rate includes: The first algorithm set is determined based on the target algorithm set and the priorities of the algorithms in the target algorithm set; A second set of algorithms is determined based on the first set of algorithms, the data frame rate, and at least one of the service frame rates; Based on the second set of algorithms and the algorithm dependencies, a third set of algorithms is determined; The execution order of the algorithms in the third algorithm set is sorted to generate the algorithm queue.

5. The data processing method according to claim 4, characterized in that, The step of determining the first algorithm set based on the target algorithm set and the priorities corresponding to the algorithms in the target algorithm set includes: In response to the fact that all algorithms in the target algorithm set have the lowest priority or none of them have the lowest priority, it is determined that the first algorithm set includes all algorithms in the target algorithm set; In response to the fact that the priority of the algorithms in the target algorithm set includes the lowest priority and the priority higher than the lowest priority, it is determined that the first algorithm set includes the algorithms in the target algorithm set whose priority is higher than the lowest priority.

6. The data processing method according to claim 4, characterized in that, Determining the second algorithm set based on the first algorithm set, the data frame rate, and at least one of the service frame rates includes: In response to the first service frame rate being the same as the data frame rate, it is determined that the second algorithm set includes the algorithm of the target service subscription corresponding to the first service frame rate in the first algorithm set; Wherein, at least one of the service frame rates includes the first service frame rate.

7. The data processing method according to claim 6, characterized in that, The step of determining the second algorithm set based on the first algorithm set, the data frame rate, and at least one of the service frame rates further includes: In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is any one of a plurality of target frame numbers, it is determined that the second algorithm set includes the algorithm of the target service subscription corresponding to the second service frame rate in the first algorithm set; In response to the fact that the second service frame rate is different from the data frame rate, and the frame number of the standard data unit is not any of the multiple target frame numbers, it is determined that the second algorithm set does not include the algorithm in the first algorithm set that is only subscribed to by the target service corresponding to the second service frame rate; Wherein, at least one of the service frame rates also includes the second service frame rate, and the difference between any two adjacent target frame numbers is equal to the quotient of the data frame rate divided by the second service frame rate.

8. The data processing method according to claim 4, characterized in that, The step of determining the third algorithm set based on the second algorithm set and the algorithm dependencies includes: In response to determining, based on the algorithm dependency relationship, that an algorithm in the second algorithm set needs to depend on at least one dependent algorithm, the third algorithm set is determined to include all algorithms in the second algorithm set and at least one of the dependent algorithms; In response to determining, based on the algorithm dependencies, that the algorithms in the second algorithm set do not depend on other algorithms, the third algorithm set is determined to include all algorithms in the second algorithm set.

9. The data processing method according to claim 8, characterized in that, The step of sorting the execution order of the algorithms in the third algorithm set to generate the algorithm queue includes: Determine the execution batch of the algorithms in the third algorithm set; The execution order of the algorithms in the third algorithm set is sorted according to their execution batch and priority to generate the algorithm queue.

10. The data processing method according to claim 9, characterized in that, Determining the execution batch of the algorithms in the third algorithm set includes: In response to the fact that the third algorithm set does not include the dependent algorithm, the execution batch of all algorithms in the third algorithm set is determined as the earliest execution batch. In response to the third algorithm set including the dependent algorithm, the execution batch of algorithms in the third algorithm set that do not need to depend on the dependent algorithm is determined to be the earliest executed execution batch, and the execution batch of algorithms that need to depend on at least one target dependent algorithm is the next executed execution batch of the latest executed execution batch among the execution batches of at least one target dependent algorithm, wherein the dependent algorithm in the third algorithm set includes at least one target dependent algorithm.

11. The data processing method according to claim 10, characterized in that, The step of sorting the execution order of the algorithms in the third algorithm set according to their execution batches and priorities to generate the algorithm queue includes: The algorithms in the third algorithm set are sorted in order of execution batch from earliest to latest. The algorithms with the same execution batch are sorted in descending order of priority to generate the algorithm queue.

12. The data processing method according to claim 1, characterized in that, The queue for executing the algorithm includes: In response to the requirement that the neural network processor is needed to execute the algorithm queue, the neural network processor is powered on; The data processing method further includes: In response to the completion of the algorithm queue execution, the neural network processor is powered off.

13. The data processing method according to claim 1, characterized in that, The data processing method further includes: In response to the fact that the currently running service does not include the service corresponding to the sensor that is in the powered-on state, the sensor that is in the powered-on state is powered down.

14. The data processing method according to claim 1, characterized in that, The data processing method further includes: In response to the presence of multiple target sensors, multiple algorithm queues corresponding one-to-one with the multiple target sensors are executed in parallel. In this context, the frame numbers of the multiple standard data units corresponding to the multiple algorithm queues are the same.

15. A data processing system, characterized in that, The data processing system includes: A data processing module is configured to generate standard data units based on data to be processed in response to receiving data to be processed sent by a target sensor, wherein the target sensor is a sensor corresponding to at least one target service in the currently running service. A queue construction module is configured to construct an algorithm queue corresponding to the standard data unit based on a target algorithm set subscribed to by at least one target service, algorithm dependencies, data frame rate, and at least one service frame rate, wherein the data frame rate is the frame rate of the data collected by the target sensor, and at least one service frame rate corresponds one-to-one with at least one target service. An execution module is configured to execute the algorithm queue to generate processing results for the data to be processed.

16. The data processing system according to claim 15, characterized in that, The data processing module is also configured to: Generate an identifier corresponding to the data to be processed; The standard data unit is generated based on the identifier and the data to be processed.

17. The data processing system according to claim 15, characterized in that, The queue construction module is also configured to: The first algorithm set is determined based on the target algorithm set and the priorities of the algorithms in the target algorithm set; A second set of algorithms is determined based on the first set of algorithms, the data frame rate, and at least one of the service frame rates; Based on the second set of algorithms and the algorithm dependencies, a third set of algorithms is determined; The execution order of the algorithms in the third algorithm set is sorted to generate the algorithm queue.

18. The data processing system according to claim 15, characterized in that, The execution module is also configured to: In response to the requirement that the neural network processor is needed to execute the algorithm queue, the neural network processor is powered on; In response to the completion of the algorithm queue execution, the neural network processor is powered off.

19. The data processing system according to claim 15, characterized in that, The execution module is also configured to: In response to the fact that the currently running service does not include the service corresponding to the sensor that is in the powered-on state, the sensor that is in the powered-on state is powered down.

20. The data processing system according to claim 15, characterized in that, The execution module is also configured to: In response to the presence of multiple target sensors, multiple algorithm queues corresponding one-to-one with the multiple target sensors are executed in parallel. In this context, the frame numbers of the multiple standard data units corresponding to the multiple algorithm queues are the same.

21. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to perform the data processing method as described in any one of claims 1 to 14.

22. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the data processing method as described in any one of claims 1 to 14.

23. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the data processing method as described in any one of claims 1 to 14.

24. A chip, characterized in that, The chip includes one or more interfaces and one or more processors; The interface is used to receive signals from the memory of the electronic device and send the signals to the processor, the signals including computer instructions stored in the memory, which, when executed by the processor, cause the electronic device to perform the data processing method as described in any one of claims 1 to 14.

Images