Heterogeneous sensing resource runtime selection and efficiency updating method based on data multiplexing
The method optimizes heterogeneous perception resource management by enabling data reuse and dynamic threshold/time period adjustments, addressing inefficiencies in existing methods to enhance accuracy and reduce energy consumption.
Patent Information
- Application Number
- CN202411138349.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-12-27
AI Technical Summary
Existing methods for selecting and managing heterogeneous perception resources are inefficient due to energy consumption proportional to the number of activated sensors, lack of data reuse, and inadequate characterization of runtime efficiency, leading to unreliable task execution.
A method for runtime selection and performance update of heterogeneous perception resources based on data reuse, involving registration, parameter initialization, performance calculation, dynamic threshold and time period adjustment, and fault-aware resource management to optimize energy consumption and accuracy.
Improves utilization efficiency of perception resources and reduces energy consumption while ensuring accurate and stable task execution by dynamically adjusting thresholds and time periods based on historical data evaluation.
Smart Images
Figure CN119204209A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of perception technology, and specifically relates to a method for runtime selection and performance update of heterogeneous perception resources based on data reuse. Background Art
[0002] Perception resource devices are one of the important ways for us to obtain data. Real-world applications such as autonomous driving, target tracking, AR reconstruction, and emotion recognition all directly or indirectly rely on perception resources to collect data for subsequent analysis and research. With the unprecedented rise of close-fitting devices such as smartphones, wearable devices, and IoT devices, research predicts that by 2025, each person will have more than 9 sensor devices with different functions. How to efficiently and accurately perceive in a heterogeneous perception resource environment is a key research area in the field of mobile sensing.
[0003] Rich sensor devices can provide more accurate and effective perception experience. Data fusion of multiple homogeneous or heterogeneous devices can achieve higher accuracy and precision in human activity recognition and emotion recognition. With the development of deep learning, deep learning-based fusion technology has emerged to effectively connect multiple sensor streams in the early or later stages to achieve higher accuracy. However, fusing heterogeneous perception data streams from multiple different devices will lead to many system problems, such as time synchronization, data loss, different sampling rates, etc., which is neither practical nor scalable in highly mobile and dynamic environments. Selecting an optimal sensor to perform a perception task is one of the effective ways to solve the above problems. On the one hand, the system energy consumption is often proportional to the number of activated sensor devices. Selecting an optimal sensor for a specific task can effectively reduce energy consumption and system cost; more importantly, through effective orchestration, sensor selection can achieve almost the same accuracy as sensor fusion, but with much less energy consumption. Most of the existing technical methods assume that a perception resource can only respond to a certain type of specific task, ignoring the potential opportunities brought by data reuse; on the other hand, the existing technical methods use the marginal sampling value of the inference model at a certain moment to quantify the efficiency value of the perception resource, without considering the possible impact of the runtime characteristics of the perception resource efficiency value, resulting in the calculated efficiency value being unable to effectively characterize the runtime characteristics, ultimately affecting the reliability of perception resource selection and task execution effect. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the present invention provides a method for selecting and updating the performance of heterogeneous perception resources at runtime based on data reuse, including the following steps: registering the available perception resource devices and the corresponding reasoning models and evaluation models to different task partition directories of the main control computing device; initializing parameters; calculating and updating the performance values of the perception resources and reasoning models; selecting the reasoning model and perception resource with the highest performance value under the corresponding partition to perform the task according to the task request category; dynamically adjusting the time period and performance threshold size of the selected task according to whether the performance of the perception resource task execution is lower than the selection performance threshold; dynamically adjusting the time period and threshold size of the performance update according to whether the performance of the perception resource task execution is lower than the update performance threshold; quickly removing faulty perception resources and accessing new perception resources. The method of the present invention uses data reuse, and the data from the same perception resource supports different types of tasks, thereby improving the utilization efficiency of perception resources and data and reducing the energy consumption caused by perception resource selection and performance update.
[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:
[0006] Step 1: Registration of perception resources and reasoning models;
[0007] Register the available perception resource devices and the corresponding reasoning models and evaluation models to different task partition directories of the master computing device; the same perception resource can respond to requests of different task types to achieve data reuse, so multiple models can be registered to different task partition directories;
[0008] Step 2: Initialize parameters;
[0009] Each successfully registered inference model needs to initialize 4 parameters, namely the initial selection time period Initial update time period Initial selection threshold Initial Update Threshold Where i represents the i-th sensor device, and t0 represents the time t0;
[0010] Step 3: Calculate the effectiveness of updating the perception resources and reasoning model;
[0011] The calculation and update of the perception resource efficiency value is performed according to the time cycle. When the time cycle of the running perception resource arrives, all the perception resources under the task partition will perform data collection and task reasoning, and input the historical efficiency value into the evaluation network for calculation to evaluate the efficiency value of the latest state.
[0012] After the performance values of all sensing resources in the partition are updated, the sensing resources are sorted according to the updated performance values. If there is a higher performance value, the sensing resource is selected and switched to the running state to execute the task, and the sensing resource in the running state at the last moment is closed.
[0013] Step 4: Selection of perceptual resources:
[0014] When a task request arrives, the inference model and perception resource with the highest efficiency value are selected and activated according to the partition corresponding to the task request type, and data collection, task inference and request response are performed. At the same time, there is only one perception resource responding to the same type of task request and it is in the running state. There may be multiple task requests of different types at the same time.
[0015] Step 5: Dynamic update of the perception resource selection time period and threshold size:
[0016] Dynamically adjust the time period and threshold value. When the perceived resource efficiency value F in the running state i <θs i When , the selection task is triggered to execute, the sensor with the highest efficiency value is selected from the candidate sensing resource list and activated, and the current sensing resource selection time period is reduced and the selection efficiency threshold is increased; on the contrary, when two consecutive time periods Ts i No F appears i <θs i , then increase the current perception resource selection time period and reduce the efficiency threshold;
[0017] Step 6: Dynamic update of the perceived resource efficiency value update time period and threshold value:
[0018] Dynamically adjust the time period and threshold value. When the perceived resource efficiency value F in the running state i <θu i , the efficiency value of the sensing resource is updated; each sensing resource in the candidate list collects data and inputs the inference model to output the efficiency value at the current moment, and then splices the historical efficiency value of the sensing resource and inputs it into the evaluation network to obtain the updated efficiency value F′ i , while reducing the current perception resource selection time period Tu i , increase the threshold θu i , on the contrary, when two consecutive time periods Tu i F does not appear in i <θu i , then increase the current perception resource selection time period and reduce the efficiency threshold;
[0019] Step 7: Remove fault-aware resources and add new awareness resources:
[0020] Provide hot deployment to ensure the removal and new access of sensing resources; registered sensing resources need to send heartbeat information to the main control computing device regularly to ensure that they are in an available state.
[0021] Preferably, in step 2, the average performance value in the threshold size training process is initialized, and the cycle size is initialized to 100 times the inverse of the perception resource sampling cycle, i.e., the sampling frequency; at the same time, at each moment, each type of task has one and only one perception resource selected for activation, and the perception resource will perform data collection - model inference - result output.
[0022] The beneficial effects of the present invention are as follows:
[0023] The method of the present invention uses data reuse to support different types of tasks from the data of the same perception resource, thereby improving the utilization efficiency of perception resources and data and reducing the energy consumption caused by perception resource selection and performance update; at the same time, the runtime performance value of the perception resource is effectively characterized by evaluating the network model combined with historical data, and a strategy combining time period and threshold is adopted to improve the accuracy and stability of perception resource selection and performance update; it lays a solid foundation for further developing heterogeneous perception resource applications to improve task execution effects and reduce energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a schematic diagram of heterogeneous sensing resource data multiplexing in the method of the present invention;
[0025] Figure 2 It is a schematic diagram of the overall process of the method of the present invention;
[0026] Figure 3 It is a flowchart of the heterogeneous sensing resource selection method of the present invention;
[0027] Figure 4 It is a flowchart of the heterogeneous perception resource performance update method of the present invention. DETAILED DESCRIPTION
[0028] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0029] The present invention proposes a method for runtime selection and performance update of heterogeneous perception resources based on data reuse, which realizes data reuse by adopting a method of one perception resource supporting multiple task requests, and at the same time evaluates the performance value of the perception resource by inputting historical data with the help of a neural network; further, a method combining performance thresholds and time periods is adopted to dynamically select and update the performance of perception resources, so as to effectively reduce the energy consumption during the execution of perception tasks while ensuring the accuracy of tasks.
[0030] The technical solution of the present invention is: a method for selecting and updating the performance of heterogeneous sensing resources at runtime based on data reuse, and the method of the present invention comprises the following steps:
[0031] S1: Registration of perception resources and reasoning models:
[0032] Register the available sensing resource devices and the corresponding reasoning models and evaluation models in different task partition directories of the master computing device; in particular, the method of the present invention proposes a one-to-many relationship between sensing resources and task categories, that is, the same sensing resource can respond to requests of different task types to achieve data reuse, so multiple models can be registered in different task partition directories;
[0033] S2: Initialization parameters:
[0034] Before the method of the present invention is executed, each successfully registered inference model needs to initialize 4 parameters, namely, the initial selection time period Initial update time period Initial selection threshold Initial Update Threshold (i represents the i-th sensor device, t0 represents the time t0), where the average performance value of the threshold size training process is initialized, and the cycle size is initialized to 100 times the sampling cycle of the perception resource (the inverse of the sampling frequency). At the same time, at each moment, each type of task has only one perception resource selected for activation, and the perception resource will perform data collection-model inference-result output.
[0035] S3: Calculate and update the performance value of the perception resources and reasoning model:
[0036] According to experimental data, the performance of sensing resources in executing tasks fluctuates with time and environmental changes. Therefore, the method of the present invention calculates and updates the performance value of sensing resources according to the time period. Specifically, when the time period of the operating state sensing resource arrives, all sensing resources under the task partition will perform data collection and task reasoning once, and combine the historical performance value to input into the evaluation network for calculation, and evaluate the performance value of the latest state.
[0037] Furthermore, after the performance values of all perception resources under the partition are updated, the method of the present invention will sort the perception resources according to the updated performance values. If there is a higher performance value, the perception resource will be selected to switch to the running state and execute the task, while the perception resource in the running state at the previous moment will be closed.
[0038] S4: Selection of perception resources:
[0039] When a task request arrives, the method of the present invention selects the inference model and perception resource with the highest efficiency value under the partition corresponding to the task request type and activates them, performs data collection, task inference and request response, and at the same time, there is only one perception resource response task and is in the running state for the same type of task request, and there may be multiple task requests of different types at the same time;
[0040] S5: Dynamic update of the perception resource selection time period and threshold size:
[0041] Considering that the time period and threshold value will affect the selection result and task execution effect, the fixed time period and threshold value cannot adapt to the runtime characteristics of the perceived resource efficiency value during the method operation process. The time period Ts of the perceived resource selection i Too large and threshold θs i Too low will lead to a decrease in selection frequency and a decrease in the accuracy of the selection results; on the contrary, the time period Ts i and threshold θs i Too large a value will lead to an increase in selection frequency, increased energy consumption caused by performance value updates and perception resource selection.
[0042] Furthermore, the method of the present invention dynamically adjusts the time period and the threshold value. When the perceived resource efficiency value F in the running state i <θs i When , the selection task is triggered to execute, the sensor with the highest efficiency value is selected from the candidate sensing resource list and activated, and the current sensing resource selection time period is reduced and the selection efficiency threshold is increased; on the contrary, when two consecutive time periods Ts i No F appears i <θs i , then increase the current perception resource selection time period and reduce the efficiency threshold. While ensuring the accuracy and stability of the selection results, effectively reducing energy consumption is the most important optimization goal of the method of the present invention.
[0043] S6: Dynamic update of the perceived resource efficiency value update time period and threshold value:
[0044] Considering the runtime characteristics of perceived resource efficiency value, the time period Tu i Too large and threshold θu i If it is too low, the updated performance value will not be able to effectively describe the runtime characteristics. On the contrary, the time period Tu i and threshold θu i Too large a value will lead to increased energy consumption caused by performance value update and perception resource selection.
[0045] Furthermore, the method of the present invention dynamically adjusts the time period and the threshold value. When the perceived resource efficiency value F in the running state i <θu i, triggers the update of the efficiency value of the sensing resource. Each sensing resource in the candidate list collects data and inputs the inference model to output the efficiency value at the current moment, then splices the historical efficiency value of the sensing resource and inputs it into the evaluation network to obtain the updated efficiency value F′i, while reducing the current sensing resource selection time period Tu i , increase the threshold θu i , on the contrary, when two consecutive time periods Tu i F does not appear in i <θu i , then increase the current perception resource selection time period and reduce the efficiency threshold.
[0046] S7: Quickly remove fault-aware resources and access new awareness resources:
[0047] In the scenario where heterogeneous sensing resources perform tasks, they are affected by dynamic changes in the environment and the energy of the sensing resources themselves, and there may be sensing resource failures or new sensing resources connected. For this, the method of the present invention provides a hot deployment method to ensure the rapid removal and new access of sensing resources. Registered sensing resources need to send heartbeat information to the master computing device regularly to ensure that they are in an available state.
[0048] Example:
[0049] Step 1: Register the available sensing resource devices and the corresponding reasoning models to different task partition directories of the master computing device; in particular, the method of the present invention proposes a one-to-many relationship between sensing resources and task categories, that is, the same sensing resource can respond to requests of different task types to achieve data reuse. Figure 1 As shown, S i represents the i-th sensing resource, Cj represents the j-th task request, It represents the reasoning model of the i-th sensing resource performing the j-th task. As can be seen from the figure, one sensing resource can train multiple reasoning models to perform different tasks and execute tasks at the same time. The same task can be responded to by different sensing resources and corresponding models. A possible task scheduling execution situation is as follows: Figure 1 shown.
[0050] Step 2: Parameter initialization and task reasoning
[0051] When registering, the sensing resources need to initialize the thresholds and time periods corresponding to their respective selection and performance updates. When a task request arrives, it starts responding to the request and executing the reasoning task.
[0052] like Figure 2 As shown:
[0053] ①Accept task requests;
[0054] ②When a task request arrives, redundancy judgment is performed. If a task of the current type has already been executed, the execution result of the current task is directly synchronized; otherwise, the perception resource selection module is executed;
[0055] ③Execute perception resource selection and output the selected perception resources;
[0056] ④Notify and activate the selected sensing resources to start responding to the task;
[0057] ⑤ The sensing resources collect data and send it to the central control computer;
[0058] ⑥ The central control computer processes data and inputs the model for reasoning;
[0059] ⑦The model outputs the results and returns them to the task requester.
[0060] During the task execution, it is necessary to continuously run "data collection-model inference-result output" until the task request is completed.
[0061] Step 3: Calculate the performance value of updating perception resources and reasoning models
[0062] When the time period of the running state perception resource arrives, all perception resources under the task partition will perform data collection and task reasoning, and input the historical efficiency value into the evaluation network for calculation to evaluate the efficiency value of the latest state, eff i =Q(S i-T , S i-T+1 , ..., S i ; ω; Θ), where Q is the evaluation network, which inputs the performance values of the past T moments and outputs the evaluation result of the performance value at the current moment, and ω and Θ are the evaluation network parameters.
[0063] Step 4: Choice of Perception Resources
[0064] The process of perceiving resource selection is as follows Figure 3 As shown in:
[0065] 1. First, enter the task request, and each task request will perform the selection process separately;
[0066] 2. Query the list of sensing resources that can respond to the task;
[0067] 3. Traverse the sensor list and read the efficiency value eff in turn i ;
[0068] 4. During the traversal process, determine whether the sensing resource is in an activated state. If so, the corresponding efficiency value eTf i + = μ;
[0069] 5. Sort by performance value and return the perception resource with the highest performance value.
[0070] Step 5: Dynamically adjust the time period and threshold size of the perception resource selection
[0071] Since the effect of perceptual resources in executing tasks has obvious runtime characteristics, it is necessary to dynamically adjust the time period and threshold size of perceptual resource selection during task execution and dynamically select the perceptual resources that respond to the task, such as Figure 2 shown.
[0072] ⑧ The perception resource selection result synchronously updates the status of the perception resource and task execution. The selected perception resource is updated to the running state, and the task request is updated to the execution state.
[0073] ⑨ When the model reasoning output result is returned to the task requester, a threshold judgment needs to be made. According to the judgment result, it is decided whether to trigger the perception resource selection module; if the reasoning result is not lower than the threshold, the current execution state is continued;
[0074] ⑩If the model inference result is lower than the threshold, that is, F i <θs i , then the perception resource selection is triggered, the process is as follows Figure 3 As shown;
[0075] Adjust the time period of the sensing resource according to the model reasoning result. If the reasoning result is lower than the threshold, reduce the time period size Ts i =Ts i -Ts i *0.1; increase the selection efficiency threshold θs i =θs i +θs i *0.01; on the contrary, when two consecutive time periods Ts i No F appears i <θs i , then increase the current sensing resource selection time period, Ts i =Ts i +0.05*Ts i , reduce the efficiency threshold θs i =θs i -θs i *0.01.
[0076] When the time period arrives, the perception resource selection module is triggered. The process is as follows Figure 3 As shown;
[0077] If the monitoring sense resource fails or is abnormal, the sense resource selection module (such as Figure 3As shown), select new perception resources to perform the task.
[0078] Step 6: Dynamically update the time period and threshold value of the perceived resource performance update
[0079] The sensor resource selection module selects the device with the highest efficiency after sorting. The selected sensor resources perform the task and perform the selection irregularly according to the trigger conditions during the task execution. However, the efficiency of sensor resources has obvious runtime characteristics, so the efficiency of sensor resources needs to be updated in real time. The specific process is as follows Figure 4 As shown:
[0080] 1. First enter the ID of the perception resource that needs to be updated;
[0081] 2. Activate the sensing resource corresponding to the ID, establish a communication connection, and respond to the task request;
[0082] 3. The sensing resource performs data collection, model inference, and performance evaluation within a time period, and obtains the marginal sampling S of the model inference output at that moment. i = p1-p2 (where p1 and p2 represent the maximum and second-largest values of the probability distribution output by the inference model, respectively, and the difference between the two reflects the confidence of the inference result);
[0083] 4. Combine the historical performance data of the past T moments into the evaluation network to obtain the latest evaluation performance value of the sensing resource, eff_i = Q(S i-T , S i-T+1 , ..., S i ; ω; Θ), where ω and Θ are evaluation network parameters.
[0084] The update of the effectiveness of the perceived resources is also an event loop in the task execution process, which depends on external conditions to trigger execution, such as Figure 2 As shown:
[0085] Determine whether the model inference result is lower than the efficiency update threshold of the corresponding perception resource. If not, keep the current perception resource efficiency value unchanged. If it does not fall below the efficiency value within two time periods, increase the update period of the perception resource.
[0086] If the inference result is lower than the efficiency threshold, that is, F i <θu i , then increase the time period Tu i =Tu i +Tu i *0.1; increase the selection efficiency threshold θu i =θu i +θu i*0.01; on the contrary, when two consecutive time periods Tu i No F appears i <θs i , then increase the current perception resource selection time period, Tu i =Tu i +0.05*Tu i , reduce the efficiency threshold θu i =θu i -θu i *0.01. At the same time, the performance update module is executed. The specific process is as follows Figure 4 As shown; when the update cycle of the perception resource arrives, the performance update of the perception resource is triggered. The specific process Figure 4 shown.
[0087] After each performance update, the corresponding perception resource status will be modified synchronously.
[0088] Step 7:
[0089] In the scenario where heterogeneous sensing resources perform tasks, they may fail or be newly connected to sensing resources due to the influence of dynamic changes in the environment and the energy of the sensing resources themselves. In this regard, the method of the present invention provides a hot deployment method to ensure the rapid removal and new access of sensing resources. The method of the present invention monitors the status of sensing resources ; The registered sensing resources need to send heartbeat information to the master computing device regularly , make sure it is available.
[0090] Furthermore, if the heartbeat information from the sensing resource is not received within a timeout period, the master computing device will actively initiate two requests to the sensing resource. If no reply is received, the sensing resource is considered to be faulty and removed from the list of registered sensing resources. The corresponding reasoning model is removed with a delay (10 time units) to avoid pseudo-failures caused by communication delays and other reasons, which leads to energy consumption caused by re-registering the sensing resource in a short period of time. When a new sensing resource is accessed, it is only necessary to provide the sensing resource hardware information and the corresponding reasoning model, as well as the supported task type, send a message to the main control computing device and register, and the two parties can establish communication. The entire removal and new access of sensing resources does not affect the running status of the method.
Claims
1. A method for selecting and updating the performance of heterogeneous sensing resources at runtime based on data reuse, characterized in that: The steps include: Step 1: Registration of perception resources and reasoning models; Register the available perception resource devices and the corresponding reasoning models and evaluation models in different task partition directories of the master computing device; The same perception resource can respond to requests of different task types to achieve data reuse, so multiple models can be registered in different task partition directories; Step 2: Initialize parameters; Each successfully registered inference model needs to initialize 4 parameters, namely the initial selection time period Initial update time period Initial selection threshold Initial Update Threshold Where i represents the i-th sensor device, and t0 represents the time t0; Step 3: Calculate the effectiveness of updating the perception resources and reasoning model; The calculation and update of the perception resource efficiency value is performed according to the time cycle. When the time cycle of the running perception resource arrives, all the perception resources under the task partition will perform data collection and task reasoning, and input the historical efficiency value into the evaluation network for calculation to evaluate the efficiency value of the latest state. After the performance values of all sensing resources in the partition are updated, the sensing resources are sorted according to the updated performance values. If there is a higher performance value, the sensing resource is selected and switched to the running state to execute the task, and the sensing resource in the running state at the last moment is closed. Step 4: Selection of perceptual resources: When a task request arrives, the inference model and perception resource with the highest efficiency value are selected and activated according to the partition corresponding to the task request type, and data collection, task inference and request response are performed. At the same time, there is only one perception resource responding to the same type of task request and it is in the running state. There may be multiple task requests of different types at the same time. Step 5: Dynamic update of the perception resource selection time period and threshold size: Dynamically adjust the time period and threshold value. When the perceived resource efficiency value F in the running state i <θs i When , the selection task is triggered to execute, the sensor with the highest efficiency value is selected from the candidate sensing resource list and activated, and the current sensing resource selection time period is reduced and the selection efficiency threshold is increased; on the contrary, when two consecutive time periods Ts i No F appears i <θs i , then increase the current perception resource selection time period and reduce the efficiency threshold; Step 6: Dynamic update of the perceived resource efficiency value update time period and threshold value: Dynamically adjust the time period and threshold value. When the perceived resource efficiency value F in the running state i <θu i , the efficiency value of the sensing resource is updated; each sensing resource in the candidate list collects data and inputs the inference model to output the efficiency value at the current moment, and then splices the historical efficiency value of the sensing resource and inputs it into the evaluation network to obtain the updated efficiency value F′ i , while reducing the current perception resource selection time period Tu i , increase the threshold θu i , on the contrary, when two consecutive time periods Tu i F does not appear in i <θu i , then increase the current perception resource selection time period and reduce the efficiency threshold; Step 7: Remove fault-aware resources and add new awareness resources: Provide hot deployment to ensure the removal and new access of sensing resources; registered sensing resources need to send heartbeat information to the main control computing device regularly to ensure that they are in an available state.
2. According to the method of claim 1, the method is characterized in that: In step 2, the average performance value in the threshold size training process is initialized, and the cycle size is initialized to 100 times the reciprocal of the perception resource sampling cycle, i.e., the sampling frequency; at the same time, at each moment, each type of task has one and only one perception resource selected for activation, and the perception resource will perform data collection - model inference - result output.