Intelligent internet of things multi-modal data sensing method

Abstract

The present application relates to a kind of intelligent internet of things multi-modal data sensing method, belong to internet of things field.First, the weight of space-time attention mechanism is constructed when sensing platform to capture the influence of space-time relationship on worker mobility prediction, the mobility of sensing worker is predicted using GRU prediction model with space-time attention mechanism.Then based on the prediction result, using the multi-modal sensing task utility maximization task allocation method based on mobility prediction to the sensing worker in different time period appears in multi-modal sensing POI task area is allocated multi-modal sensing task.Effectively improve the prediction accuracy of sensing worker in different time period located in different multi-modal sensing task area, assign different sensing worker suitable multi-modal sensing task, improve the utility of sensing platform and the completion rate of multi-modal sensing task.

Description

Technical Field

[0001] This invention belongs to the field of Internet of Things (IoT) and relates to a method for intelligent IoT multimodal data sensing. Background Technology

[0002] In recent years, the widespread adoption of Internet of Things (IoT) terminal devices has led to an explosive growth in multimodal data and connections. This massive data flow necessitates IoT network architectures with higher computing power to enable timely analysis and processing of multimodal data. With the approaching era of ubiquitous connectivity, IoT and AI are seen as key to reshaping future business models and even transforming human lifestyles. Upgrades in communication technology have only addressed the networking aspect of IoT. The bottleneck lies in how to flexibly and accurately sense and intelligently integrate multimodal data—which is diverse, massive, and has low value density—to provide timely, intelligent, and personalized services to IoT users. The development of big data and AI technologies allows for the analysis and mining of the vast amounts of multimodal sensing data collected by sensing platforms, generating more applications and value. The development of multimodal sensing stems from technological advancements in mobile internet, social networks, IoT, big data, and AI, providing strong support for intelligent, automated, and efficient data collection and analysis.

[0003] In smart cities, sensing data naturally exists in a "multimodal" form. This multimodality manifests in the diverse data information generated by various IoT terminals' multimodal perception of the environment. The realization of scenarios such as autonomous driving, smart healthcare, and smart offices in smart cities relies heavily on the perception of multimodal data. Sensing platforms collect multimodal data by recruiting mobile workers. Since most workers' movements are repetitive, they systematically cover specific multimodal sensing task areas at different times. Different multimodal sensing task areas exhibit spatiotemporal correlation. Analyzing the movement trajectories of workers performing tasks reveals their movement patterns, which is beneficial for the sensing platform to allocate multimodal sensing data tasks, thereby improving task completion rates and the quality of multimodal sensing data. Firstly, sensing costs affect multimodal sensing task allocation. After task allocation, the sensing platform rewards workers who complete multimodal sensing tasks based on task costs. However, each worker's sensing ability and credibility differ. Giving all workers the same reward can discourage workers providing high-quality multimodal sensing data from accepting tasks, thus affecting the completion rate of multimodal sensing tasks. Secondly, worker mobility is influenced by spatiotemporal characteristics. For example, some workers may visit specific multimodal sensing data task locations during certain time periods, while others may frequently visit specific multimodal sensing data locations. The above research neglects the impact of spatiotemporal characteristics on mobility prediction, leading to low accuracy in worker mobility prediction, affecting multimodal sensing task allocation, reducing multimodal sensing task completion rates, and ultimately impacting the interests of the sensing platform.

[0004] Currently, there are some research works on multimodal data sensing methods. Wang E, Yang Y, and Jie W, in “An Efficient Prediction-Based User Recruitment for Mobile Crowdsensing” [in IEEE Transactions on Mobile Computing, PP(1):1-1, 2018], used different price plans of sensing workers to divide workers into two types: pay-as-you-go PAYG and pay-per-month PAYM. They used Markov models to predict the time probability distribution of workers arriving at the POI area of ​​the multimodal sensing task, and proposed a worker recruitment algorithm based on mobility prediction to minimize the cost of uploading multimodal sensing data. Yang Y, Liu W, and Wang E, in “A prediction-based user selection framework for heterogeneous mobile crowdsensing” [in IEEE Transactions on Mobile Computing, 18(11):2460-2473, 2019], considering the different spatiotemporal requirements of multimodal sensing tasks, firstly, a Markov model-based sensing worker mobility prediction model is used to obtain the probability of workers completing multimodal sensing tasks. Then, a greedy offline algorithm is proposed for sensing worker recruitment. For online mode, an online multimodal sensing task allocation algorithm is proposed, which effectively improves the completion rate of multimodal sensing tasks. Zhu X, Luo Y, and Liu A, in “A Deep Learning-Based Mobile Crowdsensing Scheme by Predicting VehicleMobility” [in IEEE Transactions on Intelligent Transportation Systems, PP(99):1-12, 2020],

[0005] To recruit mobile vehicles to collect multimodal sensing data in cities, a deep learning-based LSTM model is proposed to predict vehicle mobility. Then, based on the prediction results, an online greedy algorithm is used to assign multimodal sensing tasks to mobile vehicles, increasing the amount of multimodal sensing data collected. Zhang J and Zhang X, in "Multi-Task Allocation in Mobile Crowd Sensing with Mobility Prediction" [in IEEE Transactions on Mobile Computing, PP(99):1-1, 2021], studied the multimodal sensing task allocation problem for mobile workers. They used fuzzy logic to predict the mobility of sensing workers and then proposed a global heuristic algorithm, GGPSO, based on the prediction results, effectively improving the accuracy of sensing worker mobility prediction and the completion rate of multimodal sensing tasks. However, the above methods neglect the influence of spatiotemporal relationships on sensing worker mobility prediction, leading to inaccurate prediction, reduced quality of multimodal sensing data collection, and impaired effectiveness of the sensing platform. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a multimodal data sensing method for intelligent Internet of Things (IoT). Firstly, since workers systematically visit specific multimodal sensing task POI areas at different time periods, a temporal attention mechanism adaptively assigns greater weight to locations with higher relevance for target location prediction, thereby improving prediction accuracy. Therefore, temporal attention weights can be extracted by quantitatively analyzing the travel patterns of potential time in different sensing worker location sequences by comparing the values ​​of these weights. Since workers tend to complete tasks closer to themselves, tasks in adjacent task areas are more relevant. To capture the influence of spatial factors on sensing worker mobility prediction, spatial attention weights are constructed based on a Gaussian kernel function, thereby uncovering the impact of spatiotemporal relationships on sensing worker mobility prediction. A GRU worker mobility prediction model with a spatiotemporal attention mechanism is used to predict the mobility of sensing workers. After obtaining the movement patterns of different sensing workers, a task allocation method based on maximizing the utility of multimodal sensing tasks based on mobility prediction is provided to improve the utility and completion rate of multimodal sensing tasks.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for multimodal data sensing in intelligent Internet of Things (IoT) includes the following steps:

[0009] S1: Construct a basic Gated Recurrent Unit (GRU) prediction model; GRU consists of hidden states and unit memories, which store information about past input sequences and control the information flow between input and output through a gating mechanism; the following recursive equation represents the working principle of GRU;

[0010] r t =σ(W r ·[h t-1 ,x t ])

[0011] z t =σ(W z ·[h t-1 ,x t ])

[0012]

[0013]

[0014] Among them, memory increment Formed by the accumulation of current input information and past memories, the reset gate r t It controls the importance of past memories in the increment of current memories, and updates the gate z. t Controlling the ratio of past memory increments to current memory increments, current memory h t Subject to update gate z t and memory increment The influence of x; t It is the input vector, W r W z , This is the linear transformation matrix of the learning parameters;

[0015] S2: Constructing a GRU prediction model based on encoding and decoding: The GRU prediction framework based on encoding and decoding consists of an encoder network and a decoder network, which read and generate variable-length sequences respectively; the encoder network recursively inputs the representative worker w. i A trajectory length sequence WT = {WT1, WT2, ..., WT} of length T T}, and through formula h t =GRU(x t ,h t-1 Update the hidden state vector h at each time step t. t , where h t Depends on the current input x t And the previous hidden state h t-1 Finally, after time steps T, the encoder will process the entire input sequence WT = {WT1, WT2, ..., WT}. T The summation is into the final vector h. T ;

[0016] S3: Constructing a temporal attention mechanism: The decoder uses h passed from the encoder. T As its initial unit storage state vector, h 0′ =h T ;pass Hide its state h T′ The hidden state h of the GRU unit in the decoding phase at the previous moment t-1′ and the predicted value Y at time t-1 t-1′ Nonlinear combination, then through Normalize the temporal attention weights, and then normalize the weights. and hidden state The weights H′ of the hidden states are obtained by performing a weighted summation.

[0017] S4: Constructing a spatial attention mechanism: using a Gaussian-based kernel function To assign weights between different task POI regions, where The distance between POI regions for different tasks, where κ is the distance scaling parameter; using Calculate the spatial attention score function, and then calculate... The weights of spatial attention are ultimately used. Obtain the weights of the encoder's t-th hidden state; where W i ′, W i U i b is the parameter matrix for the model training process. i It is the bias vector;

[0018] S4: Construct a GRU prediction model with a spatiotemporal attention mechanism: predict worker mobility, and after decoding, output the predicted probability function pre of the worker's location at time t using a softmax function after passing through a fully connected layer. ij ;

[0019]

[0020] S5: The perception platform pre-acquires the task requirements of multimodal perception tasks;

[0021] S6: Construct a multimodal sensing data quality score model for sensing workers;

[0022] S7: Construct a credibility model for perceived workers;

[0023] S8: Construct a model for maximizing the utility of a perception platform based on data perception quality requirements and perception cost constraints;

[0024] S9: Construct a multimodal perception task utility maximization task allocation method based on mobility prediction to allocate multimodal perception tasks;

[0025] S10: Sensing workers are assigned multimodal sensing tasks, complete these tasks, and then upload the collected multimodal sensing data to the sensing platform. The sensing platform processes and aggregates the multimodal sensing data and then feeds back the results to the task requester and distributes rewards to the workers.

[0026] S3: Constructing a group interest model: Based on the individual interest prediction model, analyze the group of users = {u1, u2, ..., u...} i ,...,u M By performing joint reasoning on the entity interest list of}, we can obtain the entity interest list of group users. in The entity e is represented i The popularity, where M represents the total number of users.

[0027] Optionally, S6 and S7 are specifically as follows:

[0028] Firstly, the accuracy of multimodal sensing data is often affected by the data submitted by workers. TQ and perceived worker credibility W RP Impact, workers w j Complete the multimodal perception task t j The mean of the multimodal sensing data is worker w j Complete task t j The standard deviation of the perceived data is Then, the standard deviation of the multimodal sensing data is normalized to obtain the accuracy W of the multimodal sensing data submitted by the workers. TQ When the accuracy of the user's multimodal perception data W TQ The closer the value is to 1, the more reliable the multimodal perception data submitted by the user; conversely, the less reliable the multimodal perception data submitted by the user is.

[0029]

[0030] The calculation of the perceived credibility of workers, where s ij On behalf of the task requester to worker w i Execute multimodal tasks t j The degree of satisfaction, s ij ∈(0,1); λ h -e is the decay function, because as the number of times the multimodal sensing task is completed increases, the weight of the completed multimodal sensing task on the worker's credit assessment becomes lower and lower.

[0031]

[0032] Optionally, S8 specifically includes:

[0033] First, using the data quality constraints and task requirement constraints of the multimodal perception task, the following optimization objective is established, where the optimization objective system utility function U(T) is the task utility. Platform utility The sum of, where pre ij For workers w i Appeared in task t j The probability; constraint C1 is a cost constraint, namely the total reward ultimately paid to the worker by the sensing platform. It will not exceed the total cost B of the multimodal perception task, where x ij It is a 0-1 vector, when worker w i Accepting multimodal perception tasks t j The value is 1 if the constraint is active, and 0 otherwise; constraint C2 is worker w. i Dwell time at POI location in multimodal perception task w t During the perception time of the multimodal perception task j Within; C3 requires that the perception quality score of the multimodal perception data submitted by workers must not be lower than the threshold T. θ ;

[0034]

[0035] Optionally, S9 specifically includes:

[0036] First, the utility maximization model of the perception platform is solved using a multimodal perception task utility maximization task allocation method based on mobility prediction, which includes three steps: initial particle generation process, particle mutation optimization, and multimodal perception task allocation process.

[0037] Initial particle optimization process: The traditional particle swarm optimization (PSO) algorithm is an evolutionary algorithm that simulates the foraging behavior of birds. A flock of birds finds food through cooperation and information sharing among individuals within the group; using formula V... t+1 =wv t +c1r1(pbest t -x t )+c2r2(gbest t -x t ) and x t+1 =x t +v t+1 The bird continuously changes its position and speed to search for the area closest to food; where t is the current time step, w is the inertia weight, c1 and c2 are acceleration coefficients, r1 and r2 are random numbers in [0,1], and v... t and xt These are the velocity and position of particle i at time t, respectively. t For particle i's individual optimal position, gbest t It is the optimal position globally; the particle swarm is initialized randomly; however, it suffers from poor balance and low particle diversity.

[0038] Particle mutation optimization process: Chaotic sequences have randomness and ergodicity. Chaotic sequences are used during initialization to improve the distribution and diversity of particles.

[0039]

[0040]

[0041] Among them l i,d Let d be the coordinates of the particle. max and d min The boundary representing the particle, y i (·) represents a cubic mapping mathematical model; to improve the diversity of mapping distribution, when position information is embedded in particle generation, it needs to satisfy cos(x) in formula (17). i ,x j () is less than the threshold ε;

[0042] To overcome premature convergence of the algorithm, chaotic mutation is introduced to help the particle escape local optima; the particle's current optimal position p is... best Mapped into the Logistic domain; then through the Logistic equation For y k After iteration, a chaotic sequence is obtained. Then, the reliable solution vector is obtained by calculating the adaptive value of each feasible solution vector and comparing it with the original optimal solution.

[0043]

[0044]

[0045] Multimodal sensing task allocation process: For the multimodal sensing task allocation problem based on data sensing quality requirements and sensing cost constraints, a penalty function is introduced to calculate the adaptivity function F. fitness (l);

[0046]

[0047]

[0048] Where ρ is a random number between 0 and 1, x ij r is a 0-1 vector ijTo perceive the reward for workers after completing a multimodal perception task, B ij T represents the budgeted cost of the multimodal perception task. sq T represents the quality score for the multimodal perception task. θ U(T) is the task quality threshold for the multimodal perception tasks released by the perception platform, and U(T) is the utility function of the perception platform.

[0049] The average optimal position (mbest) of all quantum particles is calculated using the wave function. t Using p i (t)=rp best (t)+(1-r)g best (t) and The particle position update method is obtained; where M is the number of particles in the swarm, p best (t) and g best (t) represents the individual optimal position and the global optimal position of the particle at time t, respectively, and r is a control parameter with a 50% probability of being positive or negative 1. u A random number uniformly distributed between (0,1);

[0050] Finally, workers accept multimodal perception tasks based on their daily movement routes, submit multimodal perception data after completing the tasks, and the perception platform processes and aggregates the multimodal perception data provided by the workers. Then, it issues rewards to the workers who complete the multimodal perception tasks and returns the multimodal perception data to the task requesters.

[0051] The beneficial effects of this invention are as follows: The proposed intelligent IoT multimodal data sensing method first utilizes the fact that sensing workers systematically access POI areas for multimodal sensing data tasks at different time periods. A spatiotemporal attention mechanism is constructed using temporal and spatial attention weights. This mechanism adaptively allocates greater weight to locations with higher relevance to improve the accuracy of sensing worker location prediction for multimodal data tasks. Then, based on the mobility prediction results of sensing workers, a task allocation method that maximizes the utility of multimodal sensing tasks based on mobility prediction is proposed to assign multimodal sensing tasks to sensing workers. This effectively improves the utility of the sensing platform and the completion rate of multimodal sensing tasks while ensuring the quality of multimodal sensing data.

[0052] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0054] Figure 1 A model diagram of the multimodal sensing task allocation system provided by this invention;

[0055] Figure 2 A diagram of the GRU-based worker mobility prediction model with a spatiotemporal attention mechanism provided by this invention.

[0056] Figure 3 The system flowchart for multimodal sensing task allocation designed for this invention is shown. Detailed Implementation

[0057] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0058] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0059] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0060] Please see Figure 1 This is a model diagram of a multimodal perception task allocation system, which specifically includes the following components:

[0061] Task requester: Provides a certain cost for the multimodal perception task, sends a task request to the perception platform, and obtains the perception data of the multimodal perception task from the perception platform.

[0062] Perception Platform: For multimodal perception tasks issued by task requesters, the platform first utilizes the historical movement records of sensing workers.

[0063] A GRU prediction model with a spatiotemporal attention mechanism is used to predict the movement trajectories of sensing workers. Then, based on the predicted workers appearing in different POI regions of multimodal sensing tasks at different times, multimodal sensing tasks are assigned using a task allocation method that maximizes the utility of multimodal sensing tasks based on mobility prediction. Finally, after processing the multimodal sensing data uploaded by the workers, the collected multimodal sensing data is returned to the task requester, and rewards are distributed to the workers who complete the tasks.

[0064] Sensing worker: Responsible for performing multimodal sensing tasks on daily routes, collecting multimodal sensing data, and then transmitting it to the sensing platform for data processing.

[0065] Please see Figure 2 This is a GRU-based model for predicting worker mobility with a spatiotemporal attention mechanism. The method specifically includes the following steps:

[0066] 1) GRU prediction model construction: First, a GRU neural network model is built, and then a prediction model with an encoding and decoding structure is constructed.

[0067] 2) Temporal attention weight construction: Since sensing workers will systematically visit other specific multimodal sensing task POI regions at different time periods, the temporal attention mechanism can adaptively allocate greater weight to locations with higher relevance for multimodal sensing task location prediction, thereby improving the accuracy of sensing worker mobility prediction.

[0068] 3) Spatial Attention Mechanism Construction: Since perception workers tend to complete multimodal perception tasks that are closer to them, tasks in adjacent multimodal perception task regions are more relevant. A Gaussian kernel function is used to assign weights between different POI regions for multimodal perception tasks.

[0069] 3) GRU prediction model with spatiotemporal attention mechanism: The constructed GRU prediction model with spatiotemporal attention mechanism is used to predict the mobility of perceived workers and the prediction results are recorded.

[0070] Please see Figure 3 , Figure 3 The diagram shown is a system flowchart for multimodal sensing task allocation designed in this invention. The method specifically includes the following steps:

[0071] The entire system process mainly consists of four steps: First, the task requester publishes a multimodal perception task request to the perception platform. Upon receiving the multimodal perception task, the platform first captures the impact of spatiotemporal relationships on worker mobility prediction based on a spatiotemporal attention mechanism weighting, and then uses a GRU prediction model with a spatiotemporal attention mechanism to predict worker mobility. Next, based on the prediction results, a multimodal perception task allocation method that maximizes the utility of multimodal perception tasks based on mobility prediction is used to allocate multimodal perception tasks to workers appearing in the multimodal perception POI task area at different time periods. Finally, workers accept multimodal perception tasks based on their daily travel routes, complete the tasks, and submit multimodal perception data. The perception platform processes and aggregates the multimodal perception data provided by the workers, then issues rewards to workers who complete the tasks and returns the multimodal perception data to the task requester.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for multimodal data sensing in intelligent Internet of Things, characterized in that: The method includes the following steps: S1: Construct a basic Gated Recurrent Unit (GRU) prediction model; GRU consists of hidden states and unit memories, which store information about past input sequences and control the information flow between input and output through a gating mechanism; the following recursive equation represents the working principle of GRU; Among them, memory increment Formed by the accumulation of current input information and past memories, the reset gate r t It controls the importance of past memories in the increment of current memories, and updates the gate z. t Controlling the ratio of past memory increments to current memory increments, current memory h t Subject to update gate z t and memory increment The influence; where x t It is the input vector, W r W z , This is the linear transformation matrix of the learning parameters; S2: Constructing a GRU prediction model based on encoding and decoding: The GRU prediction framework based on encoding and decoding consists of an encoder network and a decoder network, which read and generate variable-length sequences respectively; the encoder network recursively inputs the representative worker w. i A trajectory length sequence WT = {WT1, WT2, ..., WT} of length T T }, and through formula h t =GRU(x t ,h t-1 Update the hidden state vector h at each time step t. t , where h t Depends on the current input x t And the previous hidden state h t-1 Finally, after time steps T, the encoder will process the entire input sequence WT = {WT1, WT2, ..., WT}. T The summation is into the final vector h. T ; S3: Constructing a temporal attention mechanism: The decoder uses h passed from the encoder. T As its initial unit storage state vector, h 0′ =h T ;pass Hide its state h T′ The hidden state h of the GRU unit in the decoding phase at the previous moment t-1′ and the predicted value Y at time t-1 t-1′ Nonlinear combination, then through Normalize the temporal attention weights, and then normalize the weights. and hidden state h T′ The weights H′ of the hidden states are obtained by performing a weighted summation. S4: Constructing a spatial attention mechanism: using a Gaussian-based kernel function To assign weights between different task POI regions, where The distance between POI regions for different tasks, where κ is the distance scaling parameter; using Calculate the spatial attention score function, and then calculate... The weights of spatial attention are ultimately used. Obtain the weights of the encoder's t-th hidden state; where W i ′, W i U i b is the parameter matrix for the model training process. i It is the bias vector; S5: Construct a GRU prediction model with a spatiotemporal attention mechanism: predict worker mobility, and after decoding, output the predicted probability function pre of the worker's location at time t using a softmax function after passing through a fully connected layer. ij ; S6: The perception platform pre-acquires the task requirements of the multimodal perception task; S7: Construct a multimodal sensing data quality score model for sensing workers; S8: Construct a credibility model for perceived workers; S9: Construct a model for maximizing the utility of a perception platform based on data perception quality requirements and perception cost constraints; S10: Based on the GRU prediction model with spatiotemporal attention mechanism, worker mobility is predicted. Based on the prediction results, the multimodal perception task allocation method based on the multimodal perception task utility maximization method is used to allocate multimodal perception tasks to the sensing workers who appear in the multimodal perception POI task area at different time periods. S11: The perception worker is assigned a multimodal perception task to complete the multimodal perception task, and then the collected multimodal perception data is uploaded to the perception platform. The perception platform processes and aggregates the multimodal perception data and then feeds back the multimodal perception data results to the task requester and distributes rewards to the worker. S12: Constructing a group interest model: Based on the individual interest prediction model, analyze the group of users u={u1,u2,...,u i ,...,u M By performing joint reasoning on the entity interest list of}, we can obtain the entity interest list of group users. in The entity e is represented i The popularity, where M represents the total number of users.

2. The intelligent Internet of Things multimodal data sensing method according to claim 1, characterized in that: Specifically, S7 and S8 are: Firstly, the accuracy of multimodal sensing data is often affected by the accuracy of the multimodal sensing data submitted by workers. TQ and perceived worker credibility W RP Impact, workers w j Complete the multimodal perception task t j The mean of the multimodal sensing data is worker w j Complete task t j The standard deviation of the perceived data is Then, the standard deviation of the multimodal sensing data is normalized to obtain the accuracy W of the multimodal sensing data submitted by the workers. TQ When the accuracy of the user's multimodal perception data W TQ The closer the value is to 1, the more reliable the multimodal perception data submitted by the user; conversely, the less reliable the multimodal perception data submitted by the user is. The calculation of the perceived credibility of workers, where s ij On behalf of the task requester to worker w i Execute multimodal tasks t j The degree of satisfaction, s ij ∈(0,1); λ h-e As the decay function increases, the weight of the completed multimodal sensing tasks on the worker's creditworthiness assessment decreases.

3. The intelligent Internet of Things multimodal data sensing method according to claim 1, characterized in that: Specifically, S9 is: First, using the data quality constraints and task requirement constraints of the multimodal perception task, the following optimization objective is established, where the optimization objective system utility function U(T) is the task utility. Platform utility The sum of, where pre ij For workers w i Appeared in task t j The probability; constraint C1 is a cost constraint, namely the total reward ultimately paid to the worker by the sensing platform. It will not exceed the total cost B of the multimodal perception task, where x ij It is a 0-1 vector, when worker w i Accepting multimodal perception tasks t j The value is 1 if the constraint is active, and 0 otherwise; constraint C2 is worker w. i Dwell time at POI location in multimodal perception task w t During the perception time of the multimodal perception task j Within; C3 requires that the perception quality score of the multimodal perception data submitted by workers must not be lower than the threshold T. θ ; 4. The intelligent Internet of Things multimodal data sensing method according to claim 1, characterized in that: Specifically, S10 is: First, the utility maximization model of the perception platform is solved using a multimodal perception task utility maximization task allocation method based on mobility prediction, which includes three steps: initial particle generation process, particle mutation optimization, and multimodal perception task allocation process. Initial particle optimization process: The traditional particle swarm optimization (PSO) algorithm is an evolutionary algorithm that simulates the foraging behavior of birds. A flock of birds finds food through cooperation and information sharing among individuals within the group; using formula V... t+1 =wv t +c1r1(pbest t -x t )+c2r2(gbest t -x t ) and x t+1 =x t +v t+1 The bird continuously changes its position and speed to search for the area closest to food; where t is the current time step, w is the inertia weight, c1 and c2 are acceleration coefficients, r1 and r2 are random numbers in [0,1], and v... t and x t These are the velocity and position of particle i at time t, respectively. t For particle i's individual optimal position, gbest t It is the optimal position globally; the particle swarm is initialized randomly; however, it suffers from poor balance and low particle diversity. Particle mutation optimization process: Chaotic sequences have randomness and ergodicity. Chaotic sequences are used during initialization to improve the distribution and diversity of particles. Among them l i,d Let d be the coordinates of the particle. max and d min The boundary representing the particle, y i (·) represents a cubic mapping mathematical model; to improve the diversity of mapping distribution, when position information is embedded in particle generation, it needs to satisfy cos(x) in formula (17). i ,x j () is less than the threshold ε; To overcome premature convergence of the algorithm, chaotic mutation is introduced to help the particle escape local optima; the particle's current optimal position p is... best Mapped into the Logistic domain; then through the Logistic equation For y k After iteration, a chaotic sequence is obtained. Then, the reliable solution vector is obtained by calculating the adaptive value of each feasible solution vector and comparing it with the original optimal solution. Multimodal sensing task allocation process: For the multimodal sensing task allocation problem based on data sensing quality requirements and sensing cost constraints, a penalty function is introduced to calculate the adaptivity function F. fitness (l); Where ρ is a random number between 0 and 1, x ij r is a 0-1 vector ij To perceive the reward for workers after completing a multimodal perception task, B ij T represents the budgeted cost of the multimodal perception task. sq T represents the quality score for the multimodal perception task. θ U(T) is the task quality threshold for the multimodal perception tasks released by the perception platform, and U(T) is the utility function of the perception platform. The average optimal position (mbest) of all quantum particles is calculated using the wave function. t Using p i (t)=rp best (t)+(1-r)g best (t) and The particle position update method is obtained; where M is the number of particles in the swarm, p best (t) and g best (t) represents the individual best position and the global best position of the particle at time t, respectively; r is a control parameter with a probability of 50% positive and 50% negative; and u is a random number uniformly distributed between (0,1). Finally, workers accept multimodal perception tasks based on their daily movement routes, submit multimodal perception data after completing the tasks, and the perception platform processes and aggregates the multimodal perception data provided by the workers. Then, it issues rewards to the workers who complete the multimodal perception tasks and returns the multimodal perception data to the task requesters.

Images