A trust evaluation method, device and equipment

Abstract

The application provides a trust evaluation method, device and equipment. In the method, a first device requests a trust evaluation node to perform trust evaluation on a plurality of candidate devices. The trust evaluation node obtains an initial trust value of each candidate device and divides the plurality of candidate devices into at least two trust types. Taking a target device as an example, the trust evaluation node determines a weight value corresponding to a direct trust value according to each type of candidate device, and determines a plurality of recommended devices and a weight value corresponding to each recommended device. The trust evaluation node determines a trust value of the target device according to the weight value corresponding to the direct trust value, the plurality of recommended devices and the weight value corresponding to each recommended device. After the trust evaluation node determines the trust value of each candidate device, the trust evaluation node determines a trusted device and feeds back to the first device. Through the scheme, the trust evaluation node can determine the parameter value required when calculating the trust value based on the candidate devices of different trust types, and improve the accuracy of trust evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and in particular to a trust evaluation method, device and equipment. BACKGROUND

[0002] With the gradual commercialization of the 5th generation mobile communication technology (5G) network, more and more researches begin to focus on the future network vision. At present, the industry generally believes that the integration of heterogeneous networks is an inevitable trend of future network development. Future networks will move from independent and closed networks to heterogeneous interconnection, such as the integration of cellular networks, Internet Protocol (IP) networks, and satellite networks. The integration of heterogeneous networks has the following advantages: network integration can expand the coverage of the network, making the network more scalable; network integration can make full use of existing network resources, reduce operating costs, and enhance competitiveness; network integration can provide various services to different users and better meet the needs of future network users; network integration can improve network reliability, attack resistance, and other capabilities. However, different heterogeneous networks have different hardware network devices, and how to solve the mutual trust problem between different network devices is one of the great challenges faced by network integration.

[0003] Trust modeling based on data theory is currently a method for determining whether a device is a trusted device. In this method, a model of dynamic trust relationship is established by various mathematical methods and tools, the direct trust and indirect trust values corresponding to the device are combined together to form a final trust value. However, this method analyzes the trust relationship in a specific scenario, and then uses mathematical methods to combine the trust relationship to form a complete trust model. Therefore, this method has the problem of serious scene customization and poor portability. In addition, the trust model in this method involves many weight values and parameters. These weight values or parameter values are empirical values of technical personnel, and setting different weight values or parameters has a great impact on the performance of the trust model. Therefore, the robustness of the trust modeling method based on data theory is not high. SUMMARY

[0004] The present application provides a trust evaluation method, device and equipment to provide an accurate trust evaluation method suitable for various scenarios.

[0005] In a first aspect, the present application provides a trust evaluation method, which can be applied to a trust evaluation node. The method comprises:

[0006] receive a trust evaluation request sent by a first device, the trust evaluation request comprising identifications of a plurality of candidate devices to be evaluated; obtain an initial trust value of each candidate device in the plurality of candidate devices, and divide the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices; perform the following operations on each candidate device in the plurality of candidate devices to obtain a trust value of the plurality of candidate devices: determine a weight value corresponding to a direct trust value of a target device according to candidate devices corresponding to each trust type; and determine a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device according to the candidate devices corresponding to each trust type; determine the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the plurality of recommended devices corresponding to the target device, and the weight value corresponding to each recommended device; wherein the target device is any device in the plurality of candidate devices; the direct trust value of the target device is used to indicate a trust degree of the target device relative to the first device; and determine a trusted device in the plurality of candidate devices according to the trust values of the plurality of candidate devices, and send an identification of the trusted device to the first device.

[0007] In the above method, after receiving the trust evaluation request sent by the first device, the trust evaluation node can obtain the initial trust values of the plurality of candidate devices requested by the first device for trust evaluation, and divide the plurality of candidate devices into at least two trust types. The trust evaluation node determines the weight value corresponding to the direct trust value of each candidate device according to the candidate devices corresponding to each trust type, and determines the plurality of recommended devices corresponding to each candidate device and the weight value corresponding to each recommended device, so as to obtain the parameter value required when calculating the trust value of the candidate device, so as to obtain an accurate trust value and improve the accuracy of trust evaluation. Through the scheme, when the trust of the candidate device is evaluated, the trust of the device in a specific scene is not analyzed, but after the candidate devices are classified according to the initial trust values of the devices, the parameters required in the trust evaluation process are determined according to the classified candidate devices of each type. Therefore, the trust evaluation method provided by the present application can be applied to various trust evaluation scenes, has wide applicability, and can solve the device trust evaluation problem in various communication networks.

[0008] In one possible design, the determination of the weight value corresponding to the direct trust value of the target device according to the candidate devices corresponding to each trust type comprises:

[0009] performing a plurality of rounds of weight calculation processes on the target device based on the first reinforcement learning model to obtain a plurality of candidate weight values; and determining the weight value corresponding to the direct trust value of the target device according to the plurality of candidate weight values;

[0010] wherein any round of weight calculation process comprises:

[0011] determine the interaction device selection strategy of the current weight calculation process as a first strategy or a second strategy, determine at least one interaction device according to the interaction device selection strategy of the current weight calculation process; wherein the first strategy is to determine a target trust type from the at least two trust types with a preset probability, and select a candidate device corresponding to the target trust type as an interaction device; the second strategy is to select a node with the highest trust value in the neighbor nodes of the target device as an interaction device, wherein the trust evaluation node stores the trust values of the neighbor nodes of the target device; obtain interaction data of the at least one interaction device and the target device; determine a first reward value and a first state value of the current weight calculation process based on the first reinforcement learning model according to the interaction data, and store the first reward value of the current weight calculation process; calculate a candidate weight value corresponding to the current weight calculation process according to the first reward value and the first state value of the current weight calculation process; wherein the first reward value is used to indicate the performance of the first reinforcement learning model in the current weight calculation process; and the first state value is used to indicate the communication state of the target device and the at least one interaction device.

[0012] Through the design, the trust evaluation node can learn the calculation method of the weight value corresponding to the direct trust value based on the first reinforcement learning model, so as to determine the weight value corresponding to the direct trust value matched with the target device, and improve the accuracy of trust evaluation.

[0013] In one possible design, the determination of the interaction device selection strategy of the current weight calculation as the first strategy or the second strategy includes: determining a probability value of the interaction device selection strategy as the first strategy according to a preset decision function, and determining the interaction device selection strategy according to a random function corresponding to the probability value.

[0014] Through the design, when the first reinforcement learning model determines the weight value corresponding to the direct trust value of the target device, the interaction device selection strategy of each round of weight calculation process is determined with different probability values as the multiple rounds of weight calculation are performed, so as to accelerate the optimization speed of the reinforcement learning model, and solve the problems of slow convergence speed and unstable reward value of the existing reinforcement learning model.

[0015] In one possible design, the determination of the weight value corresponding to the direct trust value of the target device according to the multiple candidate weight values includes: selecting a candidate weight value calculated in a round of weight calculation with the highest first reward value from the multiple candidate weight values; and taking the selected candidate weight value as the weight value corresponding to the direct trust value of the target device.

[0016] Through the design, the trust evaluation node can take the result of the weight calculation of the round with the highest reward value as the weight value corresponding to the direct trust value of the target device, so that the result of the weight calculation of the round with the best model performance in the multiple rounds of weight calculation of the first reinforcement learning model is taken as the final result, and the accuracy of the weight calculation is improved.

[0017] In a possible design, the method further includes: after the M rounds of weight calculation for the target device, and / or, if the difference between the first reward values of any two adjacent rounds of weight calculation in the last N rounds of weight calculation for the target device is less than a preset threshold, stopping the multiple rounds of weight calculation; where M and N are positive integers, and M is greater than N.

[0018] Through the design, if the difference between the reward values of any two adjacent rounds of weight calculation in the last N rounds of weight calculation for the target device is less than the preset threshold, it can be considered that the first reward value converges, indicating that the performance of the first reinforcement learning model tends to be stable, and at this time, the multiple rounds of weight calculation can be stopped; or the number of rounds of weight calculation of the first reinforcement learning model can be preset, and after the current number of rounds reaches the preset number of rounds, the multiple rounds of weight calculation are stopped, to prevent the first reinforcement learning model from entering an infinite loop and reduce efficiency.

[0019] In a possible design, the interaction data includes the number of successful communications and the number of failed communications between the target device and the at least one interaction device.

[0020] Through the design, the interaction data obtained by the first reinforcement learning model can be the number of successful communications and the number of failed communications between the target device and the at least one interaction device, so that the state value of the environment corresponding to the first reinforcement learning model can be determined according to the interaction data, and then the strategy of the first reinforcement learning model can be adjusted to obtain a first reinforcement learning model with better performance.

[0021] In a possible design, the determining, according to the candidate devices corresponding to each trust type, of the multiple recommended devices corresponding to the target device and the weight value corresponding to each recommended device includes: obtaining multiple groups of candidate results by performing, based on the second reinforcement learning model, multiple rounds of recommended device selection processes for the target device, each group of candidate results including multiple candidate recommended devices and the weight value corresponding to each candidate recommended device; and determining a target result from the multiple groups of candidate results, the target result including the multiple recommended devices corresponding to the target device and the weight value corresponding to each recommended device.

[0022] The any-round recommended device selection process comprises: determining that a recommended device selection strategy of the current recommended device selection process is a third strategy or a fourth strategy, determining a plurality of first candidate recommended devices according to the recommended device selection strategy; the third strategy is to select a candidate device that has communicated with the target device from candidate devices of each trust type in a preset proportion as a recommended device; the fourth strategy is to select a preset number of nodes that have communicated with the target device from neighbor nodes of the target device as recommended devices; determining a second state value of the current recommended device selection process based on the second reinforcement learning model according to the plurality of first candidate recommended devices; the second state value is used to indicate a communication state of the target device and the plurality of first candidate recommended devices; performing a plurality of rounds of weight calculation processes based on the second reinforcement learning model to determine a weight value corresponding to each first candidate recommended device; determining a second reward value of the current recommended device selection process according to the plurality of first candidate recommended devices and the weight value corresponding to each first candidate recommended device, and storing the second reward value; the second reward value is used to indicate the performance of the second reinforcement learning model in the current recommended device selection process.

[0023] Through the design, the trust evaluation node can learn the selection manner of the recommended device and the manner of calculating the weight value corresponding to the recommended device based on the second reinforcement learning model, so as to determine the recommended device matched with the target device and the weight value corresponding to each recommended device, and improve the accuracy of trust evaluation.

[0024] In a possible design, the determining that the recommended device selection strategy of the current recommended device selection is the third strategy or the fourth strategy comprises: determining a probability value of the recommended device selection strategy being the third strategy according to a preset decision function, and determining the recommended device selection strategy according to a random function corresponding to the probability value.

[0025] In a possible design, the determining the target result from the plurality of groups of candidate results comprises: selecting a candidate result determined by a round of recommended device selection with the highest second reward value from the plurality of groups of candidate results; and taking the selected candidate result as the target result.

[0026] In a possible design, the method further comprises: stopping the plurality of rounds of recommended device selection when Q rounds of recommended device selection are performed for the target device, and / or the difference between reward values of any two adjacent rounds of recommended device selection in the last P rounds of recommended device selection performed for the target device is less than a preset threshold; wherein Q and P are positive integers, and Q is greater than P.

[0027] In a possible design, the obtaining of the initial trust value of each candidate device in the plurality of candidate devices comprises: receiving trust-related data of the plurality of candidate devices sent by the first device, and determining the initial trust value of each candidate device according to the trust-related data of each candidate device; wherein the trust-related data comprises at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location; or receiving the initial trust value of the plurality of candidate devices sent by the first device; the initial trust value of the plurality of candidate devices is obtained by the first device from a blockchain or an interplanetary file system.

[0028] Through this design, the trust evaluation node can receive the trust-related data of the plurality of candidate devices sent by the first device, and determine the initial trust value of each candidate device according to the trust-related data of each candidate device; or the first device can obtain the initial trust value of each candidate device from the blockchain or the interplanetary file system, and then send the initial trust value of each candidate device to the trust evaluation node, so as to flexibly obtain the initial trust value to start the trust evaluation process of the candidate device.

[0029] In a possible design, after sending the identification of the trusted device to the first device, the method further comprises: receiving interaction data between the first device and the trusted device sent by the first device, and updating at least one of the following according to the interaction data between the first device and the trusted device: the model parameter of the first reinforcement learning model, the model parameter of the second reinforcement learning model, and the trust value of the trusted device.

[0030] Through this design, the trust evaluation node can optimize the model parameters of the first reinforcement learning model and the second reinforcement learning model according to the real interaction data between the first device and the trusted device, and further improve the performance of the two reinforcement learning models. Moreover, the first reinforcement learning model can determine the weight value corresponding to the direct trust value of the trusted device based on the interaction data between the first device and the trusted device, the second reinforcement learning model can determine the plurality of recommended devices corresponding to the trusted device and the weight value corresponding to each recommended device based on the interaction data between the first device and the trusted device, and then the trust evaluation node can redetermine the trust value of the trusted device, so as to update the trust value of the trusted device according to the real interaction data, and further improve the accuracy of the trust evaluation.

[0031] In a possible design, the determining the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the plurality of recommended devices corresponding to the target device, and the weight value corresponding to each recommended device includes: determining an indirect trust value of the target device according to the plurality of recommended devices corresponding to the target device and the weight value corresponding to each recommended device, where the indirect trust value is used to indicate a trust degree of the target device relative to the plurality of recommended devices corresponding to the target device; determining a direct trust value of the target device; and determining the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the direct trust value, and the indirect trust value.

[0032] By this design, the trust evaluation node can determine the indirect trust value based on the plurality of recommended devices corresponding to the target device and the weight value of each recommended device determined by the second reinforcement learning model, and determine the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the direct trust value, and the indirect trust value determined by the first reinforcement learning model, thereby improving the accuracy of trust evaluation and ensuring communication security.

[0033] In a second aspect, an embodiment of the present application provides a trust evaluation apparatus, which includes a plurality of function modules; the plurality of function modules interact with each other to implement the method in the first aspect and each of the implementation manners thereof. The plurality of function modules can be implemented based on software, hardware, or a combination of software and hardware, and the plurality of function modules can be combined or divided based on actual implementation.

[0034] In a third aspect, an embodiment of the present application provides an electronic device including a processor and a memory, where the memory stores computer program instructions, and when the electronic device is running, the processor executes the method provided in the first aspect.

[0035] In a fourth aspect, an embodiment of the present application further provides a computer program, which, when running on a computer, enables the computer to execute the method provided in any of the aspects.

[0036] In a fifth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a computer, the computer program enables the computer to execute the method provided in any of the aspects.

[0037] In a sixth aspect, an embodiment of the present application further provides a chip, which is used to read a computer program stored in a memory and execute the method provided in any of the aspects.

[0038] Seventhly, embodiments of this application also provide a chip system including a processor for supporting a computer device in implementing the methods provided in any of the above aspects. In one possible design, the chip system further includes a memory for storing programs and data necessary for the computer device. The chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0039] Figure 1 A schematic diagram of a trust mathematical model provided in an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of two machine learning-based classification models;

[0041] Figure 3 A schematic diagram illustrating a scenario to which the trust assessment method provided in the embodiments of this application is applicable;

[0042] Figure 4 This is a schematic diagram of the structure of a trust evaluation node provided in an embodiment of this application;

[0043] Figure 5 This application provides a functional diagram of each module within a trust assessment node, as illustrated in an embodiment of the present application.

[0044] Figure 6 A flowchart illustrating a trust assessment method provided in this application embodiment;

[0045] Figure 7 A flowchart illustrating a trust assessment method provided in this application embodiment;

[0046] Figure 8 A flowchart illustrating a trust assessment method provided in this application embodiment;

[0047] Figure 9 A flowchart illustrating a trust assessment method provided in this application embodiment;

[0048] Figure 10 This is a schematic diagram of the structure of a trust assessment device provided in an embodiment of this application;

[0049] Figure 11 This is a schematic diagram of the structure of a trust assessment device provided in an embodiment of this application. Detailed Implementation

[0050] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the following will further describe the embodiments of the present application with reference to the drawings. In the description of the embodiments of the present application, the terms "first", "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can be explicitly or implicitly included one or more of the features.

[0051] It should be understood that "at least one" in the embodiments of the present application means one or more, and "multiple" means two or more. The association relationship of the associated objects is described, which means that there can be three relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b or c can mean a, b, c, a and b, a and c, b and c, or a, b and c, where a, b and c can be single or multiple.

[0052] The terms related to the embodiments of the present application are explained as follows:

[0053] (1) Reinforcement learning (RL) is an important branch of machine learning. Reinforcement learning learns through an evaluation feedback mechanism. The difference between reinforcement learning and supervised learning is that the target of reinforcement learning does not have label information. Reinforcement learning is usually described by Markov Decision Process (MDP), which includes two objects: Agent and Environment. In the process of reinforcement learning, the environment is in a specific state, and the agent can change the state of the environment by performing a specific action. The environment returns an observation to the agent after the state of the environment changes, which can be a state value indicating the current state of the environment. At the same time, the environment also returns a reward value to the agent, which can be positive or negative. When the reward is negative, it can be regarded as a punishment. The agent takes a new action according to the state value and the reward value returned by the environment. The way the agent selects the action can be called a policy. Repeat the above process, and the task of reinforcement learning is to find a policy to maximize the reward value.

[0054] Actor-Critic (AC) algorithm is a reinforcement learning method combining policy gradient and temporal difference learning. The actor network is a policy function, which is generally implemented by a neural network. The input of the actor network is the current state, and the output is an action. The critic network is a value function. The critic network can evaluate the performance of the current actor network. The actor network can update the policy according to the evaluation returned by the critic network. The purpose of training the actor network is to maximize the expected cumulative return.

[0055] (2) Deep belief network (DBN) is a network based on Boltzmann pre-training structure, which belongs to a deep neural network. DBN can be used for unsupervised learning and supervised learning. In the embodiment of the application, the DBN model can classify the input data by training the weights between neurons.

[0056] (3) Blockchain is a chain composed of one block after another. Each block saves a certain amount of information, and multiple blocks are connected into a chain according to the time sequence of their generation. This chain is saved in all servers. As long as one server in the entire system can work, the entire blockchain is safe. These servers are called nodes in the blockchain system, which provide storage space and computing power support for the entire blockchain system. If you want to modify the information in the blockchain, you must obtain the consent of more than half of the nodes and modify the information in all nodes, and these nodes are usually in the hands of different subjects, so it is extremely difficult to tamper with the information in the blockchain. Compared with traditional networks, blockchain has two core features: data is difficult to tamper with and is decentralized. Based on these two features, the information recorded by the blockchain is more real and reliable, which can help solve the problem of mutual distrust. In the embodiment of the application, the blockchain can refer to a storage system based on the blockchain data structure to store data, such as one or more servers based on the blockchain data structure to store data.

[0057] The embodiments of the application are described below:

[0058] With the gradual commercialization of the 5th generation mobile communication technology (5G) network, more and more researches begin to focus on the future network vision. At present, the industry generally believes that the integration of heterogeneous networks is the inevitable trend of future network development. Future networks will move from independent and closed networks to heterogeneous interconnection, such as the integration of cellular networks, Internet Protocol (IP) networks, and satellite networks. The integration of heterogeneous networks has the following advantages: network integration can expand the coverage of the network, making the network more scalable; network integration can make full use of existing network resources, reduce operating costs, and enhance competitiveness; network integration can provide various services to different users and better meet the needs of future network users; network integration can improve network reliability, attack resistance, and other capabilities. However, different heterogeneous networks have different hardware network devices. To ensure communication efficiency and security, how to solve the mutual trust problem between different network devices is one of the great challenges faced by network integration.

[0059] In an optional implementation, trust modeling can be performed through data theory to establish a trust mathematical model that can dynamically evaluate the trust value of a device. For example, Figure 1 A schematic diagram of a trust mathematical model provided by an embodiment of the present application. Referring to Figure 1 The trust mathematical model is used to determine the trust value of a trusted device relative to a trust device. For ease of description, the trust device is referred to as device i and the trusted device is referred to as device j. When calculating the trust value of device j relative to device i, the direct trust value of device j and the indirect trust value of device j can be calculated respectively. The direct trust value of device j can be determined based on the interaction data between device j and device i, which can be the number of successful communications and the number of failed communications between device i and device j. The indirect trust value of device j depends on whether the recommended device corresponding to device j is trustworthy and the degree of trust of the recommended device for device j. We call the recommended device of device j as device k. Referring to Figure 1 The trust value of device j in the trust mathematical model can satisfy the following formula 1:

[0060] T i,j (t)=α i,j D i,j (t)+(1-α i,j )I i,j (t) Formula 1

[0061] Where T i,j (t) is the trust value of device j relative to device i, D i,j (t) is the direct trust value of device j relative to device i, and αi,j is a weight value corresponding to the direct trust value of device i, I i,j (t) is an indirect trust value of device j.

[0062] Optionally, the direct trust value of device j relative to device i can satisfy the following formula 2 or formula 3:

[0063]

[0064]

[0065] wherein, is the number of normal data packets in the interaction data between device i and device j, is the number of malicious data packets in the interaction data between device i and device j, p is a decay index, and h is a constant. It should be noted that the normal data packet can be a data packet of successful communication, and the malicious data packet can be a data packet of failed communication.

[0066] Optionally, the indirect trust value of device B can satisfy the following formula 4:

[0067]

[0068] wherein, S i,j (t) is a set of devices that have communicated with device j, C i,k (t) is a trust value of device k relative to device i, R k,j (t) is a trust value of device k relative to device j.

[0069] It should be noted that C i,k (t) in formula 4 is one setting mode of the weight value corresponding to device k, and the weight value corresponding to device k can also be set as an experienced value of a technician.

[0070] Optionally, the weight value corresponding to the direct trust value can satisfy the following formula 5 or formula 6:

[0071]

[0072]

[0073] wherein, is the total number of data packets in the interaction data between device i and device j, c is a constant, and b is a decay coefficient.

[0074] As can be seen from formula 5 and formula 6, as the number of communication interactions between device i and device j increases, the weight value corresponding to the direct trust value increases, and at this time, the direct trust value between device i and device j has a greater influence on the trust value of device j.

[0075] It can be seen from the above formula that the trust mathematical model includes many weight factors and parameters, for example, the weight value a corresponding to the direct trust value in formula 1 i,j , the parameter c in formula 5, and the like. These weight factors and parameters are generally empirical values obtained by technical personnel based on simulation data. Setting different weight factors and parameters has a great influence on the performance of the trust mathematical model. Therefore, the robustness of the trust modeling method based on data theory is not high. In addition, in different application scenarios, the trust relationship will also change. The trust mathematical model can only analyze the trust relationship in a specific scenario, resulting in poor scene customization verification and migration of the trust mathematical model, and it is difficult to apply the trust mathematical model to the trust evaluation of an actual network system.

[0076] In another optional implementation, a model for performing device trust evaluation can be trained based on machine learning. Since the trust modeling of machine learning has weak dependence on the trust relationship, mathematical modeling of the trust relationship for a specific scene is generally not required. Therefore, compared with the above trust mathematical model, the trust modeling of machine learning can be applied to more scenarios. In the trust modeling based on machine learning, a classification algorithm of machine learning can be used to classify devices according to device data to determine whether the devices are trustworthy. In the trust modeling based on machine learning, sample data with labels are required to train the classification model. However, it is difficult to obtain trust data with labels in the trust scenario. For example, the trust type of a device may be different in different scenarios. Therefore, it is difficult to determine whether the trust label of the data of the device is trustworthy or untrustworthy. Therefore, the trust modeling based on machine learning is difficult to obtain sample data for model training, and thus is difficult to be practically applied to device trust evaluation. In addition, when performing trust evaluation on a device, some devices may not have interaction data, that is, these devices have not communicated with other devices. However, the classification model based on machine learning cannot determine whether the device without interaction data is trustworthy. It can be seen that how to cold start is also a problem that the classification model based on machine learning cannot solve.

[0077] For example, Figure 2 FIGS. 1 and 2 are schematic diagrams of two classification models based on machine learning. Referring to FIG. 1, Figure 2 FIG. 1(a), Figure 2 FIG. 1(a) shows the distribution of sample data with different trust labels. When classifying the device data of the device to be evaluated, a circular region is divided with the device data position of the device to be evaluated as the center and a classification radius K. In the region, if the number of sample data with a trust label of trustworthy is large, the SVM model determines that the device to be evaluated is trustworthy. If the number of sample data with a trust label of untrustworthy is large, the SVM model determines that the device to be evaluated is untrustworthy. Based on Figure 2As can be seen in (a), this classification model requires a large amount of sample data with trust labels, and different values ​​of the classification radius K have a significant impact on the performance of the SVM model. For example, as... Figure 2 As shown in (a), when K is 3, the SVM determines the device to be evaluated to be untrustworthy because the number of samples labeled as untrustworthy is greater than the number of samples labeled as trustworthy. When K is 7, the SVM determines the device to be evaluated to be trustworthy because the number of samples labeled as trustworthy is greater than the number of samples labeled as untrustworthy. Since the value of K is generally an empirical value used by technical personnel, the performance of the SVM classification model in trust assessment of devices is not stable.

[0078] Figure 2 Figure (b) shows the distribution of sample data with different trust labels. This classification model needs to determine a hyperplane based on the distribution of the sample data using a regression algorithm. This hyperplane can classify the trust types of the sample data to achieve trust assessment of the device. It is evident that this classification model also requires a large amount of sample data with trust labels to obtain a relatively accurate hyperplane.

[0079] In summary, current solutions for trust assessment of devices suffer from problems such as severe scenario customization, difficulty in obtaining large amounts of sample data with trust labels, low robustness, and difficulty in cold start, making them unsuitable for trust assessment of devices in actual network systems.

[0080] To address the aforementioned issues, this application provides a trust assessment method that is accurate and applicable to various scenarios. Figure 3 This is a schematic diagram illustrating a scenario to which the trust assessment method provided in this application is applicable. (Reference) Figure 3 This scenario includes a primary device, multiple candidate devices, a trust evaluation node, and a storage system. Furthermore, the storage system can be a blockchain or the Interplanetary File System (IPFS). The primary device initiates the trust evaluation, and the multiple candidate devices are those for which the primary device requests trust evaluation. For example, before communicating with the multiple candidate devices, the primary device requests the trust evaluation node to perform trust evaluations on the candidate devices. The trust evaluation node can be used to perform trust evaluations on the multiple candidate devices to determine whether each candidate device is trustworthy. The blockchain can be used to store and transmit data from each device. IPFS can also be used to store data from each device to alleviate the storage pressure on the blockchain.

[0081] In some embodiments of this application, reference is made to Figure 3In the scenario shown, a first device can send a trust assessment request to a trust assessment node. This request may include the identifiers of multiple candidate devices to be assessed. Upon receiving the trust assessment request from the first device, the trust assessment node can obtain the initial trust value of each candidate device and, based on these initial trust values, categorize the candidate devices into at least two trust types. The trust assessment node can then perform a trust assessment on each candidate device to determine its trust value. Taking the trust assessment node's trust assessment of a target device as an example, the target device is any one of the multiple candidate devices. The trust assessment node can determine the weight value corresponding to the direct trust value of the target device based on a first reinforcement learning model, and, based on the candidate devices corresponding to each trust type, determine the recommended devices and their corresponding weight values ​​based on a second reinforcement learning model. The trust assessment node can then determine the target device's trust value based on the weight value corresponding to its direct trust value, the recommended devices, and their corresponding weight values. In this way, the trust assessment node can perform a trust assessment on each candidate device, thereby determining its trust value. Then, the trust evaluation node can determine the trustworthy device from the multiple candidate devices based on their trust values, and send the identifier of the trustworthy device to the first device. After receiving the identifier of the trustworthy device sent by the trust evaluation node, the first device can communicate and interact with the trustworthy device to ensure communication efficiency and security.

[0082] The functions performed by the trust evaluation node in the trust evaluation method provided in this application embodiment are further described below:

[0083] Figure 4 This is a schematic diagram of the structure of a trust evaluation node provided in an embodiment of this application. (Reference) Figure 4 The trust assessment node may include a data collection module, a classification module, a reinforcement learning module, a basic mathematical model (BMM) module, and a trust decision module. It is understood that these multiple modules can be integrated into the same device or deployed on different devices; this application embodiment does not limit this. The following... Figure 4 The functions of each module in the trust assessment node shown are introduced below:

[0084] Figure 5 This is a functional diagram of each module within a trust assessment node provided in an embodiment of this application.

[0085] refer to Figure 5The data collection module is configured to collect trust-related data of the device. Optionally, the trust-related data of the device can include at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location. The trust-related data of the device can be used to determine an initial trust value of the device.

[0086] The classification module, which can be a deep belief network (DBN)-based classification module, is configured to classify the device according to the initial trust value of the device. For example, the DBN-based classification module can determine a trust type of each candidate device according to the initial trust values of the candidate devices in the scenario shown in FIG. 2. Figure 3 The classification module, which can be a deep belief network (DBN)-based classification module, is configured to classify the device according to the initial trust value of the device. For example, the DBN-based classification module can determine a trust type of each candidate device according to the initial trust values of the candidate devices in the scenario shown in FIG. 2.

[0087] The reinforcement learning module, which can be a double actor-critic (DAC)-based reinforcement learning module, is configured to learn a weight value corresponding to the direct trust value and learn how to select the recommended device and the weight value corresponding to each recommended device. Optionally, the DAC-based reinforcement learning module can include two reinforcement learning models, such as a first reinforcement learning model and a second reinforcement learning model. The first reinforcement learning model can be configured to learn the weight value corresponding to the direct trust value or the weight value corresponding to the indirect trust value, and the second reinforcement learning model can be configured to learn how to select the recommended device and the weight value corresponding to each recommended device. It should be noted that the sum of the weight value corresponding to the direct trust value and the weight value corresponding to the indirect trust value is 1. The first reinforcement learning model can learn either the weight value corresponding to the direct trust value or the weight value corresponding to the indirect trust value, and thus determine the other.

[0088] The BMM module is configured to calculate the direct trust value and the indirect trust value of the device, and perform weighted calculation on the direct trust value and the indirect trust value to obtain the trust value of the device.

[0089] The trust decision module is configured to determine a trust evaluation result of the device according to the trust value of the device. For example, the trust decision module can determine a trusted device from the plurality of devices according to the trust values of the plurality of devices.

[0090] Based on the above, Figure 4 and Figure 5The various modules in the trust evaluation node are shown, and the steps of the trust evaluation method provided by the embodiments of the application are introduced below. Figure 6 A flowchart of a trust evaluation method provided by the embodiments of the application is shown in FIG. 1. Figure 6 The method comprises the following steps:

[0091] S601: The data collection module acquires trust-related data of each candidate device in the plurality of candidate devices.

[0092] The trust-related data of each candidate device can comprise at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location.

[0093] S602: The data collection module sends the trust-related data of each candidate device to the classification module.

[0094] S603: The classification module determines an initial trust value of each candidate device according to the trust-related data of each candidate device, and classifies the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices.

[0095] Optionally, candidate devices of the same trust type can be taken as a set. For example, assuming that the classification module classifies the candidate devices based on a DBN model of binary classification, the plurality of candidate devices can be classified into a set of trusted devices and a set of untrusted devices according to the trust types determined by the classification module.

[0096] S604: The classification module sends the trust type of each candidate device in the plurality of candidate devices to a first reinforcement learning model and a second reinforcement learning model in the reinforcement learning module.

[0097] The first reinforcement learning model is used to determine a weight value corresponding to a direct trust value of each candidate device, and the second reinforcement learning model is used to determine a plurality of recommended devices corresponding to each candidate device and a weight value corresponding to each recommended device in the plurality of recommended devices.

[0098] S605: The first reinforcement learning model determines whether the interaction device selection strategy of the current weight calculation process for the target device is the first strategy? If yes, go to S606; otherwise, determine that the interaction device selection strategy of the current weight calculation process is the second strategy, and go to S607.

[0099] The target device is any device in the plurality of candidate devices.

[0100] In the embodiments of the present application, the first reinforcement learning model can perform multiple rounds of weight calculation on the target device to determine the weight value corresponding to the direct trust value of the target device. In each round of weight calculation, at least one interaction device can be selected to interact with the target device to obtain interaction data. The interaction device selection strategy in any round of weight calculation can be the first strategy or the second strategy. The first strategy can also be referred to as an exploration strategy, and the second strategy can also be referred to as an exploitation strategy. Specifically, the first strategy is to determine a target trust type from at least two trust types with a preset probability, and select a candidate device corresponding to the target trust type as the interaction device. The second strategy is to select a node with the highest trust value in the neighbor nodes of the target device as the interaction device.

[0101] In an optional implementation, the first reinforcement learning model can determine the interaction device selection strategy in the current round of weight calculation according to a preset decision function. For example, the preset decision function can be Formula 7 as follows:

[0102]

[0103] wherein a, β, and σ are constants, Pr(exp) is the probability value of the interaction device selection strategy in the current round of weight calculation being the exploration strategy, and t is the time length from the start of weight calculation on the target device to the current round of weight calculation, or t can be the round number of the current round of weight calculation in the weight calculation performed on the target device. For example, if the current round of weight calculation is the third round of weight calculation performed on the target device, t can take the value of 3.

[0104] It should be noted that the result obtained by the prediction function shown in Formula 7 is the probability value of the interaction device selection strategy in the current round of weight calculation being the exploration strategy. After determining the probability value of the interaction device selection strategy in the current round of weight calculation being the exploration strategy according to Formula 7, the first reinforcement learning model can determine whether the interaction device selection strategy in the current round of weight calculation is the exploration strategy or the exploitation strategy according to the random function corresponding to the determined probability value. For example, the first reinforcement learning model can determine the interaction device selection strategy in the current round of weight calculation by using the random function Random(X), wherein X is the probability value. For example, when X = 0.7, Random(0.7) indicates that the probability of the interaction device selection strategy in the current round of weight calculation being the exploration strategy is 0.7, and the first reinforcement learning model randomly selects the interaction device selection strategy in the current round of weight calculation with this probability.

[0105] In some embodiments of the present application, in the multi-round weight calculation process for the target device, the first reinforcement learning model can learn how to determine the weight value corresponding to the direct trust value. The strategy of the first reinforcement learning model for determining the weight value corresponding to the direct trust value is adjusted through multi-round weight calculation, and the result output by the first reinforcement learning model after multi-round optimization is taken as the weight value corresponding to the direct trust value of the target device. The interaction device selection strategy of each round of weight calculation process can be an exploration strategy or an exploitation strategy. When the interaction device selection strategy of a round of weight calculation process is an exploration strategy, it means that the current target device has less interaction data with other devices, and candidate devices and the target device can be randomly selected for interaction to obtain interaction data in this round of weight calculation process. When the interaction device selection strategy of a round of weight calculation process is an exploitation strategy, it means that the current target device is a device that has communicated with other devices for many times, and neighbor nodes that have communicated with the target device can be selected for interaction with the target device to obtain interaction data in this round of weight calculation process. Therefore, referring to formula 7, according to the decision function provided by the embodiments of the present application, the greater the value of t corresponding to a round of weight calculation, the more rounds of weight calculation have been performed before this round of weight calculation, and the target device has communicated with multiple candidate devices or neighbor nodes. At this time, the probability value of the exploration strategy of the interaction device selection strategy determined by the first reinforcement learning module in this round of weight calculation process is smaller, and the interaction device selection strategy determined by the random function based on the probability value in this round of weight calculation process is more likely to be the exploitation strategy.

[0106] For example, formula 7 can be set as:

[0107]

[0108] Suppose that t in the above formula is the number of rounds of weight calculation in the weight calculation process for the target device, when t = 1, the probability value of the exploration strategy of the interaction device selection strategy in this round of weight calculation process calculated according to the formula is about 0.956, and the interaction device selection strategy in this round of weight calculation process determined by the random function based on the probability value of 0.956 is more likely to be the exploration strategy. When t = 30, the probability value of the exploration strategy of the interaction device selection strategy in this round of weight calculation process calculated according to the formula is about 0.301, and the interaction device selection strategy in this round of weight calculation process determined by the random function based on the probability value of 0.301 is more likely to be the exploitation strategy.

[0109] Through the above design, in the trust evaluation method provided by the embodiments of the present application, when the weight value corresponding to the direct trust value of the target device is determined based on the reinforcement learning model, the interaction device selection strategy of each round of weight calculation process is determined with different probability values as the multiple rounds of weight calculation are performed, so as to accelerate the optimization speed of the reinforcement learning model, and solve the problems of slow convergence speed and unstable reward value in the existing reinforcement learning model.

[0110] S606: The first reinforcement learning model determines a target trust type from the at least two trust types with a preset probability, and selects a candidate device corresponding to the target trust type as the interaction device.

[0111] In an optional implementation, when the interaction device selection strategy of the current round of weight calculation process is the exploration strategy, the first reinforcement learning model can determine a target trust type from the at least two trust types classified by the classification model with a preset probability, and select a candidate device corresponding to the target trust type as the interaction device. For example, it is assumed that the classification module classifies the plurality of candidate devices into two trust types: trusted and untrusted in S603, and the plurality of candidate devices can be classified into a trusted device set and an untrusted device set according to the trust types determined by the classification module. The first reinforcement learning model can determine the target trust type according to a random function of the probability α, for example, if it is determined that the target trust type is trusted, a candidate device is randomly selected from the trusted device set as the interaction device; if it is determined that the target trust type is untrusted, a candidate device is randomly selected from the untrusted device set as the interaction device.

[0112] S607: The first reinforcement learning model selects a node with the highest trust value in the neighbor nodes of the target device as the interaction device.

[0113] In an optional implementation, when the interaction device selection strategy of the current round of weight calculation process is the utilization strategy, the first reinforcement learning model can select a node in the neighbor nodes of the target device as the interaction device, for example, a node with the highest trust value in the neighbor nodes as the interaction device.

[0114] S608: The first reinforcement learning model obtains interaction data of at least one interaction device communicating with the target device, determines a first reward value and a first state value of the current round of weight calculation process according to the obtained interaction data, and determines a candidate weight value corresponding to the current round of weight calculation process according to the first reward value and the first state value of the current round of weight calculation process.

[0115] Optionally, the interaction data can be the number of communication successes and the number of communication failures when the target device communicates with the at least one interaction device.

[0116] In the embodiments of the present application, the first reinforcement learning model can determine the first reward value and the first state value of the current weight calculation process according to the obtained interaction data, wherein the first reward value of the current weight calculation process can be used to indicate the performance of the first reinforcement learning model in the current weight calculation process, and the first state value can be used to indicate the communication state of the target device and the at least one interaction device. The first reinforcement learning model can determine the candidate weight value calculated in the current weight calculation process according to the first reward value and the first state value of the current weight calculation process. The determination of the candidate weight value by the first reinforcement learning model can be regarded as a new action taken by the first reinforcement learning model according to the first reward value and the first state value. Through multiple rounds of weight calculation, the first reinforcement learning model can optimize the strategy for determining the candidate weight value, and then determine the weight value corresponding to the direct trust value of the target device.

[0117] S609: The first reinforcement learning model determines whether the multiple rounds of weight calculation can be ended, and if yes, proceeds to S610; otherwise, returns to S605.

[0118] In an optional implementation, the first reinforcement learning model can stop the multiple rounds of weight calculation when the reward value converges and / or the current round number reaches a preset round number. For example, after M rounds of weight calculation for the target device, and / or the difference between the reward values of any two adjacent rounds of weight calculation in the last N rounds of weight calculation for the target device is less than a preset threshold, the multiple rounds of weight calculation are stopped; wherein M and N are positive integers, and M is greater than N.

[0119] It can be understood that the convergence of the reward value can indicate that the performance of the first reinforcement learning model tends to be stable, and at this time the multiple rounds of weight calculation can be stopped; or a preset round number of weight calculation for the first reinforcement learning model can be set, and the multiple rounds of weight calculation are stopped when the current round number reaches the preset round number, to prevent the first reinforcement learning model from entering an infinite loop and reducing efficiency.

[0120] S610: The first reinforcement learning model determines the weight value corresponding to the direct trust value of the target device.

[0121] Optionally, the first reinforcement learning model can take the candidate weight value calculated in the round of weight calculation with the highest reward value in the multiple rounds of weight calculation as the weight value corresponding to the direct trust value of the target device.

[0122] S611: The second reinforcement learning model determines whether the recommendation device selection strategy of the current recommendation device selection process for the target device is the third strategy, and if yes, proceeds to S612; otherwise, determines that the recommendation device selection strategy of the current recommendation device selection process is the fourth strategy, and proceeds to S613.

[0123] In the embodiments of the present application, the second reinforcement learning model can perform multiple rounds of recommended device selection for the target device to determine the multiple recommended devices corresponding to the target device and the weight values corresponding to each recommended device. In each round of recommended device selection, multiple candidate recommended devices can be selected and the weight values corresponding to each candidate recommended device can be determined. The recommended device selection strategy of any round of recommended device selection can be the third strategy or the fourth strategy. The third strategy can also be referred to as an exploration strategy, and the fourth strategy can also be referred to as an exploitation strategy. Specifically, the third strategy is to select, from candidate devices of each trust type, a candidate device that has communicated with the target device as a recommended device at a preset ratio. The fourth strategy is to select, from the neighbor nodes of the target device, a preset number of nodes that have communicated with the target device as recommended devices.

[0124] Optionally, when determining the recommended device selection strategy of the current round of recommended device selection, the second reinforcement learning model can determine, according to the decision function (such as formula 7) provided in the embodiments of the present application, a probability value of the recommended device selection strategy of the current round of recommended device selection being the exploration strategy, and then determine, according to the random function corresponding to the determined probability value, the recommended device selection strategy of the current round of recommended device selection being the exploration strategy or the exploitation strategy. For specific implementation, reference can be made to S605, and repeated parts will not be described herein.

[0125] S612: The second reinforcement learning model selects, from candidate devices of each trust type, a candidate device that has communicated with the target device as a first candidate recommended device at a preset ratio.

[0126] In some optional embodiments, when the recommended device selection strategy of the current round of recommended device selection is the exploration strategy, the second reinforcement learning model can select, from candidate devices of each trust type, a candidate device that has communicated with the target device as a first candidate recommended device, and the number of candidate devices selected as the first candidate recommended device from candidate devices of each trust type conforms to a preset ratio. For example, assuming that the classification module divides the multiple candidate devices into two trust types: trusted and untrusted in S603, the multiple candidate devices can be divided into a trusted device set and an untrusted device set according to the trust types determined by the classification module. The second reinforcement learning model can select recommended devices from the trusted device set and the untrusted device set at a preset ratio of X:Y. Specifically, the second reinforcement learning model can select X candidate devices from the trusted device set as first candidate recommended devices, and Y devices from the untrusted device set as first candidate recommended devices, where X and Y are positive integers, for example, X can be 1 and Y can be 2.

[0127] S613: The second reinforcement learning model selects, from the neighbor nodes of the target device, a preset number of nodes that have communicated with the target device as first candidate recommended devices.

[0128] In some optional embodiments, when the recommendation device selection strategy of the current round of recommendation device selection process is the utilization strategy, the second reinforcement learning model can select a preset number of nodes from the neighbor nodes of the target device as the first candidate recommendation devices, such as selecting Q nodes that have communicated with the target device from the neighbor nodes of the target device as the first candidate recommendation devices, Q being a positive integer.

[0129] S614: The second reinforcement learning module determines the second reward value and the second state value of the current round of recommendation device selection process according to the selected multiple first candidate recommendation devices.

[0130] The second reward value of the current round of recommendation device selection can be used to indicate the performance of the second reinforcement learning model in the current round of recommendation device selection, and the second state value can be used to indicate the communication state of the target device and the multiple first candidate recommendation devices obtained through the current round of recommendation device selection.

[0131] In an optional embodiment, the determination of the multiple first candidate recommendation devices by the second reinforcement learning model in the current round of recommendation device selection can be regarded as a new action taken by the second reinforcement learning model, and the second reinforcement learning model can determine the second state value of the current round of recommendation device selection process according to the determined multiple recommendation devices. For the determined multiple first candidate recommendation devices, the second reinforcement learning model can determine the weight value corresponding to each first candidate recommendation device through multiple rounds of weight calculation. In a specific implementation, the second reinforcement learning model can determine the type of each round of weight calculation according to the decision function (such as formula 7) provided in the embodiments of the present application, and then determine the weight value corresponding to each first candidate recommendation device through multiple rounds of weight calculation. The second reinforcement learning model can determine the second reward value of the current round of recommendation device selection process according to the multiple first candidate recommendation devices determined in each round of recommendation device selection process and the weight value corresponding to each first candidate recommendation device. By repeating the above process, the second reinforcement learning model performs multiple rounds of recommendation device selection to obtain multiple sets of candidate results, each set of candidate results including multiple candidate recommendation devices and the weight value corresponding to each candidate recommendation device. Through multiple rounds of recommendation device selection, the strategy of the second reinforcement learning model for selecting recommendation devices and the strategy of the second reinforcement learning model for determining the weight value corresponding to each recommendation device can be optimized, and then the multiple recommendation devices corresponding to the target device and the weight value corresponding to each recommendation device can be determined according to the optimized second reinforcement learning model.

[0132] S615: The second reinforcement learning model determines whether the multiple rounds of recommendation device selection can be ended; if yes, proceed to S616; otherwise, return to S611.

[0133] In an optional implementation, the second reinforcement learning model can stop the multi-round recommended device selection when the reward value converges and / or the current round number reaches a preset round number. For example, the multi-round recommended device selection is stopped after Q rounds of recommended device selection for the target device, and / or the difference between the reward values of any two adjacent rounds of recommended device selection in the last P rounds of recommended device selection for the target device is less than a preset threshold. Q and P are positive integers, and Q is greater than P.

[0134] S616: The second reinforcement learning model determines the multiple recommended devices corresponding to the target device and the weight value corresponding to each recommended device.

[0135] In an optional implementation, after the second reinforcement learning model performs the multi-round recommended device selection for the target device, multiple groups of candidate results can be obtained, each group of candidate results including multiple candidate recommended devices and the weight value corresponding to each candidate recommended device. After the multi-round recommended device selection is stopped, the second reinforcement learning model determines a target result from the multiple groups of candidate results, where the target result includes the multiple recommended devices corresponding to the target device and the weight value corresponding to each recommended device.

[0136] Optionally, the first reinforcement learning model can select, from the multiple groups of candidate results, a candidate result determined by a round of recommended device selection with the highest second reward value, and take the selected candidate result as the target result.

[0137] S617: The reinforcement learning module sends the weight value corresponding to the direct trust value of the target device, the multiple recommended devices corresponding to the target device, and the weight value of each recommended device to the BMM module.

[0138] S618: The BMM module determines the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the multiple recommended devices corresponding to the target device, and the weight value of each recommended device.

[0139] In some embodiments of the present application, the BMM module can determine the indirect trust value of the target device according to the multiple recommended devices corresponding to the target device and the weight value corresponding to each recommended device. The indirect trust value of the target device can be used to indicate the trust degree of the target device relative to the multiple recommended devices of the target device. For example, the BMM module can determine the indirect trust value of the target device according to the following formula 4: i,k(t) replacing the weight value corresponding to each recommended device determined by the second reinforcement model in the embodiment of the present application, thereby calculating the indirect trust value of the target device. The BMM module can also determine the direct trust value of the target device. The direct trust value of the target device can be used to indicate the degree of trust of the target device relative to the first device. For example, the BMM module can calculate the direct trust value of the target device according to formula 2 or formula 3. After determining the direct trust value and the indirect trust value of the target device, the BMM module can determine the trust value of the target device according to the direct trust value, the indirect trust value of the target device, and the weight value corresponding to the direct trust value. For example, the BMM module can calculate the trust value of the target device according to formula 1.

[0140] Repeat S605-S618 until the trust value of each candidate device in the plurality of candidate devices is determined.

[0141] S619: The BMM module sends the trust value of each candidate device in the plurality of candidate devices to the trust decision module.

[0142] S620: The trust decision module determines the trust evaluation result according to the trust value of each candidate device in the plurality of candidate devices.

[0143] For example, the trust decision module determines the device identifier of the trustable device in the plurality of candidate devices according to the trust value of each candidate device in the plurality of candidate devices.

[0144] Through the above method, the trust evaluation node can perform trust evaluation on the plurality of candidate devices that the first device requests to perform trust evaluation, to determine whether the plurality of candidate devices are trustable devices. The trust evaluation method provided in the embodiments of the present application can be applied to various trust evaluation scenarios, and can solve the cold start trust evaluation problem of the to-be-evaluated device that does not exist interaction data, thereby improving the accuracy of trust evaluation and ensuring communication security.

[0145] The trust evaluation method provided in the embodiments of the present application is further introduced below through two examples:

[0146] Example one

[0147] Figure 7 A flowchart of a trust evaluation method provided in the embodiments of the present application. The method can be performed by the first device, the trust evaluation node and the storage system in the scenario shown in FIG. 7, and in this example, the storage system can be a blockchain. Figure 3

[0148] S701: The trust evaluation node obtains trust-related data of a plurality of devices.

[0149] The trust-related data of each device can include at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location.​

[0150] S702: The trust evaluation node determines an initial trust value of each device according to the trust-related data of each device.

[0151] S703: The trust evaluation node sends the initial trust value of each device to the blockchain.

[0152] In some embodiments of the present application, the blockchain can store the trust value of each device. Optionally, the trust value of each device can be the initial trust value determined by the trust evaluation node according to the trust-related data of the device, or can be the trust value obtained after the trust evaluation node performs trust evaluation on the device based on the trust evaluation method provided in the embodiments of the present application.

[0153] S704: The first device sends a request message to the blockchain, and the request message is used to request the initial trust values of the plurality of candidate devices to be evaluated from the blockchain.

[0154] S705: The blockchain sends the initial trust value of each candidate device in the plurality of candidate devices to the first device.

[0155] S706: The first device sends a trust evaluation request to the trust evaluation node.

[0156] Optionally, the trust evaluation request can include the identification of the plurality of candidate devices to be evaluated and the initial trust value of each candidate device.

[0157] S707: The trust evaluation node divides the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices.

[0158] S708: The trust evaluation node determines a weight value corresponding to the direct trust value of each candidate device based on the first reinforcement learning model according to the candidate devices corresponding to each trust type, and determines a recommended device corresponding to each candidate device and a weight value corresponding to each recommended device based on the second reinforcement learning model according to the candidate devices corresponding to each trust type; determines the trust value of each candidate device according to the weight value corresponding to the direct trust value of each candidate device, the recommended device corresponding to each candidate device, and the weight value corresponding to each recommended device.

[0159] It should be noted that the method for determining the trust value of each candidate device by the trust evaluation node can refer to the embodiments shown in Figure 6 and will not be described here again.

[0160] S709: The trust evaluation node determines the trustable device in the plurality of candidate devices according to the trust values of the plurality of candidate devices.

[0161] S710: The trust evaluation node sends the identification of the trustable device to the first device.

[0162] S711: The first device communicates with the trusted devices, and records interaction data of the first device communicating with each trusted device.

[0163] S712: The first device sends the interaction data of the first device communicating with each trusted device to the trust evaluation node.

[0164] S713: The trust evaluation node updates the model parameters of the first reinforcement learning model, the model parameters of the second reinforcement learning model, and the trust value of the trusted device according to the interaction data between the first device and the trusted device.

[0165] In an optional implementation, the first reinforcement learning model can determine a state value according to the interaction data between the first device and the trusted device, and determine a weight value corresponding to the direct trust value of the trusted device according to the state value. Through this process, the model parameters of the first reinforcement learning model can be optimized based on real interaction data, and the performance of the first reinforcement learning model can be further improved.

[0166] Similarly, the second reinforcement learning model can determine a state value according to the interaction data between the first device and the trusted device, and determine a weight value corresponding to each recommended device according to the state value. Through this process, the model parameters of the second reinforcement learning model can be optimized based on real interaction data, and the performance of the second reinforcement learning model can be further improved.

[0167] In some embodiments, the first reinforcement learning model can determine a weight value corresponding to the direct trust value of the trusted device based on the interaction data between the first device and the trusted device, and the second reinforcement learning model can determine a weight value corresponding to each recommended device based on the interaction data between the first device and the trusted device. Then, the trust evaluation node can re-determine the trust value of the trusted device to update the trust value of the trusted device according to real interaction data, and further improve the accuracy of trust evaluation.

[0168] S714: The trust evaluation node sends the updated trust value of the trusted device to the blockchain.

[0169] Example Two

[0170] Figure 8 A flowchart of a trust evaluation method provided by an embodiment of the present application. The method can be executed by Figure 3 the first device, the trust evaluation node, and the storage system in the scenario shown in the figure. In this example, the storage system can be a blockchain and an IPFS.

[0171] S801: The trust evaluation node obtains trust-related data of a plurality of devices.

[0172] The trust-related data of each device can include at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location.

[0173] S802: The trust evaluation node determines an initial trust value of each device according to the trust-related data of each device.

[0174] S803: The trust evaluation node sends the initial trust value of each device and the trust-related data of each device to the IPFS.

[0175] S804: The trust evaluation node sends a hash value of the initial trust value of each device to the blockchain.

[0176] In an optional implementation, the trust evaluation node can send the trust value of the device and the trust-related data of the device to the IPFS, and the IPFS can store the trust value of the device and the trust-related data of the device, so as to reduce the data storage pressure of the blockchain. The trust evaluation node can also send the hash value of the trust value of the device to the blockchain, and the blockchain stores the hash value of the trust value of the device, so as to realize data integrity protection, prevent the trust value of the device stored in the IPFS from being tampered with, and ensure data security. The trust value of the device stored in the IPFS can be the initial trust value determined by the trust evaluation node according to the trust-related data of the device, or the trust value obtained by the trust evaluation node after performing trust evaluation on the device based on the trust evaluation method provided in the embodiments of the present application.

[0177] S805: The first device sends a request message to the IPFS, and the request message is used to request the initial trust values of the plurality of candidate devices to be evaluated from the IPFS.

[0178] S806: The IPFS sends the initial trust value of each candidate device in the plurality of candidate devices to the first device.

[0179] In an optional implementation, the first device can request the initial trust values of the plurality of candidate devices to be evaluated from the IPFS, and the IPFS sends the initial trust values of the plurality of candidate devices to be evaluated to the first device.

[0180] In another optional implementation, the first device can also request the hash values of the initial trust values of the plurality of candidate devices to be evaluated from the blockchain, and after the blockchain sends the hash values of the initial trust values of the plurality of candidate devices to the first device, the first device can send the hash values of the initial trust values of the plurality of candidate devices to the IPFS to obtain the initial trust values of the plurality of candidate devices stored in the IPFS. Through this design, it can be ensured that the initial trust values of the plurality of candidate devices obtained by the first device are unaltered data, and data security is ensured.

[0181] S807: The first device sends a trust evaluation request to the trust evaluation node.

[0182] Optionally, the trust evaluation request can include the identification of the plurality of candidate devices to be evaluated and the initial trust value of each candidate device.

[0183] S808: The trust evaluation node divides the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices.

[0184] S809: The trust evaluation node determines the weight value corresponding to the direct trust value of each candidate device based on the first reinforcement learning model according to the candidate devices corresponding to each trust type, and determines the plurality of recommended devices corresponding to each candidate device and the weight value corresponding to each recommended device based on the second reinforcement learning model according to the candidate devices corresponding to each trust type; determines the trust value of each candidate device according to the weight value corresponding to the direct trust value of each candidate device, the plurality of recommended devices corresponding to each candidate device, and the weight value corresponding to each recommended device.

[0185] It should be noted that the method for determining the trust value of each candidate device by the trust evaluation node can refer to the embodiment shown in Figure 6 , which will not be described here.

[0186] S810: The trust evaluation node determines the trustable device in the plurality of candidate devices according to the trust values of the plurality of candidate devices.

[0187] S811: The trust evaluation node sends the identification of the trustable device to the first device.

[0188] S812: The first device communicates with the trustable device, and records the interaction data of the first device and each trustable device.

[0189] S813: The first device sends the interaction data of the first device and each trustable device to the trust evaluation node.

[0190] S814: The trust evaluation node updates the model parameters of the first reinforcement learning model, the model parameters of the second reinforcement learning model, and the trust value of the trustable device according to the interaction data between the first device and the trustable device.

[0191] S815: The trust evaluation node sends the updated trust value of the trustable device to the IPFS.

[0192] S816: The trust evaluation node sends the hash value of the updated trust value of the trustable device to the blockchain.

[0193] Based on the same technical concept, the present application also provides a trust evaluation method, which can be executed by Figure 3 a trust evaluation node and a first device in the scenario shown in Figure 9 FIG. 1. Figure 9 A flowchart of a trust evaluation method provided by an embodiment of the present application is shown in FIG. 2. Referring to

[0194] , the method comprises the following steps:

[0195] S901: The first device sends a trust evaluation request to the trust evaluation node.

[0196] The trust evaluation request comprises the identities of a plurality of candidate devices to be evaluated.

[0197] S902: The trust evaluation node obtains an initial trust value of each candidate device in the plurality of candidate devices, and divides the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices.

[0198] S903: The trust evaluation node performs the following operations on each candidate device in the plurality of candidate devices to obtain the trust values of the plurality of candidate devices: determines a weight value corresponding to a direct trust value of a target device according to the candidate devices corresponding to each trust type; and determines a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device according to the candidate devices corresponding to each trust type. The trust evaluation node determines the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the plurality of recommended devices corresponding to the target device, and the weight value corresponding to each recommended device.

[0199] The target device is any device in the plurality of candidate devices; and the direct trust value of the target device is used to indicate the degree of trust of the target device relative to the first device.

[0200] S904: The trust evaluation node determines a trusted device in the plurality of candidate devices according to the trust values of the plurality of candidate devices.

[0201] S905: The trust evaluation node sends the identity of the trusted device to the first device. Figure 9 It should be noted that the trust evaluation method shown in may refer to the above-mentioned embodiments of the present application in specific implementation, and the repeated parts will not be described herein.

[0202] Based on the same technical concept, the present application also provides a trust evaluation device 1000, which can be applied to Figure 3 a trust evaluation node in the scenario shown in Figure 6- Figure 9 to implement the functions performed by the trust evaluation node in the trust evaluation method shown in Figure 10A structural diagram of a trust evaluation apparatus 1000 is provided for an embodiment of the present application. The trust evaluation apparatus 1000 includes a receiving unit 1001, a processing unit 1002, and a sending unit 1003. The processing unit 1002 can also be divided into a plurality of modules as shown in the figure, and different modules perform different functions as shown in the figure, which will not be described here again. Figure 4 Figure 5

[0203] The functions of each unit in the trust evaluation apparatus 1000 will be introduced below.

[0204] The receiving unit 1001 is configured to receive a trust evaluation request sent by a first device, the trust evaluation request including identifiers of a plurality of candidate devices to be evaluated.

[0205] The processing unit 1002 is configured to obtain an initial trust value of each candidate device in the plurality of candidate devices, divide the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices, and perform the following operations on each candidate device in the plurality of candidate devices to obtain trust values of the plurality of candidate devices: determine a weight value corresponding to a direct trust value of a target device according to candidate devices corresponding to each trust type; and determine a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device according to the candidate devices corresponding to each trust type; determine the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the plurality of recommended devices corresponding to the target device, and the weight value corresponding to each recommended device; wherein the target device is any device in the plurality of candidate devices; the direct trust value of the target device is used to indicate a trust degree of the target device relative to the first device; and determine a trusted device in the plurality of candidate devices according to the trust values of the plurality of candidate devices.

[0206] The sending unit 1003 is configured to send an identifier of the trusted device to the first device.

[0207] In an embodiment, the processing unit 1002 is specifically configured to perform a plurality of rounds of weight calculation processes on the target device based on a first reinforcement learning model to obtain a plurality of alternative weight values, and determine the weight value corresponding to the direct trust value of the target device according to the plurality of alternative weight values.

[0208] ​​The any-round weight calculation process comprises: determining that the interaction device selection strategy of the current round of weight calculation process is a first strategy or a second strategy, and determining at least one interaction device according to the interaction device selection strategy of the current round of weight calculation process; the first strategy is to determine a target trust type from the at least two trust types with a preset probability, and select a candidate device corresponding to the target trust type as an interaction device; the second strategy is to select a node with the highest trust value in the neighbor nodes of the target device as an interaction device, wherein the trust evaluation node stores the trust values of the neighbor nodes of the target device; obtaining interaction data of the at least one interaction device and the target device; determining a first reward value and a first state value of the current round of weight calculation process based on the first reinforcement learning model according to the interaction data, and storing the first reward value of the current round of weight calculation process; calculating a candidate weight value corresponding to the current round of weight calculation process according to the first reward value and the first state value of the current round of weight calculation process; the first reward value is used to indicate the performance of the first reinforcement learning model in the current round of weight calculation process; and the first state value is used to indicate the communication state of the target device and the at least one interaction device.

[0209] In an embodiment, the processing unit 1002 is specifically configured to: determine a probability value of the interaction device selection strategy being the first strategy according to a preset decision function, and determine the interaction device selection strategy according to a random function corresponding to the probability value.

[0210] In an embodiment, the processing unit 1002 is specifically configured to: select, from the plurality of candidate weight values, a candidate weight value calculated by a round of weight calculation process with the highest first reward value; and take the selected candidate weight value as the weight value corresponding to the direct trust value of the target device.

[0211] In an embodiment, the processing unit 1002 is further configured to: stop the multiple rounds of weight calculation when the M rounds of weight calculation for the target device are completed, and / or the difference between the reward values of any two adjacent rounds of weight calculation in the last N rounds of weight calculation for the target device is less than a preset threshold; wherein M and N are positive integers, and M is greater than N.

[0212] In an embodiment, the processing unit 1002 is specifically configured to: the interaction data comprises the number of communication successes and the number of communication failures of the target device and the at least one interaction device in the communication interaction.

[0213] In an embodiment, the processing unit 1002 is specifically configured to: perform a plurality of rounds of recommendation device selection processes for the target device based on the second reinforcement learning model, to obtain a plurality of groups of candidate results, each group of candidate results including a plurality of candidate recommended devices and a weight value corresponding to each candidate recommended device; and determine a target result from the plurality of groups of candidate results, the target result including a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device.

[0214] In any round of the recommendation device selection process, the processing unit 1002 is specifically configured to: determine that the recommendation device selection strategy of the current round of the recommendation device selection process is a third strategy or a fourth strategy, determine a plurality of first candidate recommended devices according to the recommendation device selection strategy; the third strategy is to select a candidate device that has communicated with the target device from candidate devices of each trust type as a recommended device at a preset proportion; the fourth strategy is to select a preset number of nodes that have communicated with the target device from neighbor nodes of the target device as recommended devices; determine a second state value of the current round of the recommendation device selection process based on the second reinforcement learning model according to the plurality of first candidate recommended devices; the second state value is used to indicate a communication state between the target device and the plurality of first candidate recommended devices; perform a plurality of rounds of weight calculation processes based on the second reinforcement learning model, to determine a weight value corresponding to each first candidate recommended device; determine a second reward value of the current round of the recommendation device selection process according to the plurality of first candidate recommended devices and the weight value corresponding to each first candidate recommended device, and store the second reward value; the second reward value is used to indicate a performance of the second reinforcement learning model in the current round of the recommendation device selection process.

[0215] In an embodiment, the processing unit 1002 is specifically configured to: determine a probability value of the recommendation device selection strategy being the third strategy according to a preset decision function, and determine the recommendation device selection strategy according to a random function corresponding to the probability value.

[0216] In an embodiment, the processing unit 1002 is specifically configured to: select, from the plurality of groups of candidate results, a candidate result determined by a round of the recommendation device selection process with the highest second reward value; and take the selected candidate result as the target result.

[0217] In an embodiment, the processing unit 1002 is further configured to: stop the plurality of rounds of the recommendation device selection process when a Q rounds of the recommendation device selection process are performed for the target device, and / or a difference between reward values of any two adjacent rounds of the recommendation device selection process in the last P rounds of the recommendation device selection process performed for the target device is less than a preset threshold; wherein Q and P are positive integers, and Q is greater than P.

[0218] In an embodiment, the processing unit 1002 is specifically configured to: receive, by the receiving unit 1001, the trust-related data of the plurality of candidate devices sent by the first device, and determine an initial trust value of each candidate device according to the trust-related data of each candidate device; wherein the trust-related data comprises at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location; or receive, by the receiving unit 1001, the initial trust values of the plurality of candidate devices sent by the first device; the initial trust values of the plurality of candidate devices are obtained by the first device from a blockchain or an interplanetary file system.

[0219] In an embodiment, the processing unit 1002 is further configured to: receive, by the receiving unit 1001, the interaction data between the first device and the trusted device sent by the first device, and update at least one of the following according to the interaction data between the first device and the trusted device: a model parameter of the first reinforcement learning model, a model parameter of the second reinforcement learning model, and a trust value of the trusted device.

[0220] The sending unit 1003 is further configured to: send the updated trust value of the trusted device to a blockchain or an interplanetary file system.

[0221] In an embodiment, the processing unit 1002 is specifically configured to: determine an indirect trust value of the target device according to a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device, wherein the indirect trust value is used to indicate a trust degree of the target device relative to the plurality of recommended devices corresponding to the target device; determine a direct trust value of the target device; and determine a trust value of the target device according to the direct trust value of the target device, the indirect trust value, and a weight value corresponding to the direct trust value.

[0222] Based on the same technical concept, the present application also provides a trust evaluation device 1100, Figure 11 A structural diagram of a trust evaluation device 1100 provided by an embodiment of the present application, which can be used to implement Figure 6- Figure 9 The functions performed by the trust evaluation node in the trust evaluation method shown. Referring to Figure 11 As shown, the trust evaluation device 1100 comprises a communication interface 1101, a processor 1102, and a memory 1103.

[0223] Optionally, the communication interface 1101, the processor 1102 and the memory 1103 are connected with each other through a bus 1104. The bus 1104 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 Only one thick line is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0224] The communication interface 1101 is configured to receive and send data, and realize communication interaction with other devices. For example, the communication interface 1101 can be a transceiver, a circuit, a bus, a module or other types of communication interfaces.

[0225] In an embodiment of the present application, the trust evaluation device 1100 can implement the trust evaluation method of the present application Figure 6- Figure 9 The functions of the trust evaluation node in the trust evaluation method shown and the functions of the processor 1102 are described as follows:

[0226] The processor 1102 is configured to receive a trust evaluation request sent by a first device through the communication interface 1101, wherein the trust evaluation request comprises the identities of a plurality of candidate devices to be evaluated; obtain an initial trust value of each candidate device in the plurality of candidate devices; divide the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices; perform the following operations on each candidate device in the plurality of candidate devices to obtain the trust values of the plurality of candidate devices: determine a weight value corresponding to a direct trust value of a target device according to the candidate devices corresponding to each trust type; and determine a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device according to the candidate devices corresponding to each trust type; determine the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the plurality of recommended devices corresponding to the target device and the weight value corresponding to each recommended device; wherein the target device is any device in the plurality of candidate devices; the direct trust value of the target device is used to indicate the trust degree of the target device relative to the first device; determine a trusted device in the plurality of candidate devices according to the trust values of the plurality of candidate devices; and send the identity of the trusted device to the first device through the communication interface 1101.

[0227] In an implementation, the processor 1102 is specifically configured to: perform a plurality of rounds of weight calculation processes on the target device based on the first reinforcement learning model to obtain a plurality of candidate weight values; and determine the weight value corresponding to the direct trust value of the target device according to the plurality of candidate weight values.

[0228] In any round of weight calculation process, the processor 1102 is specifically configured to: determine the interaction device selection strategy of the current round of weight calculation process as a first strategy or a second strategy, and determine at least one interaction device according to the interaction device selection strategy of the current round of weight calculation process; the first strategy is to determine a target trust type from the at least two trust types with a preset probability, and select one candidate device corresponding to the target trust type as an interaction device; the second strategy is to select a node with the highest trust value in the neighbor nodes of the target device as an interaction device, wherein the trust evaluation node stores the trust values of the neighbor nodes of the target device; obtain interaction data of the at least one interaction device and the target device; determine a first reward value and a first state value of the current round of weight calculation process based on the first reinforcement learning model according to the interaction data; store the first reward value of the current round of weight calculation process; and calculate a candidate weight value corresponding to the current round of weight calculation process according to the first reward value and the first state value of the current round of weight calculation process; the first reward value is used to indicate the performance of the first reinforcement learning model in the current round of weight calculation process; and the first state value is used to indicate the communication state of the target device and the at least one interaction device.

[0229] In an implementation, the processor 1102 is specifically configured to: determine a probability value of the interaction device selection strategy being the first strategy according to a preset decision function, and determine the interaction device selection strategy according to a random function corresponding to the probability value.

[0230] In an implementation, the processor 1102 is specifically configured to: select, from the plurality of candidate weight values, a candidate weight value calculated by a round of weight calculation process with the highest first reward value; and take the selected candidate weight value as the weight value corresponding to the direct trust value of the target device.

[0231] In an implementation, the processor 1102 is further configured to: stop the plurality of rounds of weight calculation when M rounds of weight calculation are performed on the target device, and / or the difference between the reward values of any two adjacent rounds of weight calculation in the last N rounds of weight calculation performed on the target device is less than a preset threshold; wherein M and N are positive integers, and M is greater than N.

[0232] In an embodiment, the processor 1102 is specifically configured to: the interaction data comprises a number of successful communication interactions and a number of failed communication interactions between the target device and the at least one interaction device.

[0233] In an embodiment, the processor 1102 is specifically configured to: perform a plurality of rounds of recommendation device selection processes on the target device based on the second reinforcement learning model to obtain a plurality of groups of candidate results, each group of candidate results comprising a plurality of candidate recommended devices and a weight value corresponding to each candidate recommended device; and determine a target result from the plurality of groups of candidate results, the target result comprising a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device.

[0234] In an embodiment, any round of recommendation device selection process comprises: determining that a recommendation device selection strategy of the round of recommendation device selection process is a third strategy or a fourth strategy, and determining a plurality of first candidate recommended devices according to the recommendation device selection strategy; the third strategy is to select, from candidate devices of each trust type, a candidate device that has had a communication interaction with the target device as a recommended device at a preset proportion; the fourth strategy is to select a preset number of nodes that have had a communication interaction with the target device from neighbor nodes of the target device as recommended devices; determining a second state value of the round of recommendation device selection process based on the second reinforcement learning model according to the plurality of first candidate recommended devices; the second state value is used to indicate a communication state of the target device and the plurality of first candidate recommended devices; performing a plurality of rounds of weight calculation processes based on the second reinforcement learning model to determine a weight value corresponding to each first candidate recommended device; determining a second reward value of the round of recommendation device selection process according to the plurality of first candidate recommended devices and the weight value corresponding to each first candidate recommended device, and storing the second reward value; the second reward value is used to indicate a performance of the second reinforcement learning model in the round of recommendation device selection process.

[0235] In an embodiment, the processor 1102 is specifically configured to: determine a probability value of the recommendation device selection strategy being the third strategy according to a preset decision function, and determine the recommendation device selection strategy according to a random function corresponding to the probability value.

[0236] In an embodiment, the processor 1102 is specifically configured to: select, from the plurality of groups of candidate results, a candidate result determined by a round of recommendation device selection process with the highest second reward value; and determine the selected candidate result as the target result.

[0237] In an implementation, the processor 1102 is further configured to: stop the multiple rounds of recommended device selection after Q rounds of recommended device selection for the target device, and / or a difference between reward values of any two adjacent rounds of recommended device selection in the last P rounds of recommended device selection for the target device is less than a preset threshold; wherein Q and P are positive integers, and Q is greater than P.

[0238] In an implementation, the processor 1102 is specifically configured to: receive, by the communication interface 1101, the trust-related data of the plurality of candidate devices sent by the first device, and determine an initial trust value of each candidate device according to the trust-related data of each candidate device; wherein the trust-related data comprises at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location; or receive, by the communication interface 1101, the initial trust values of the plurality of candidate devices sent by the first device; the initial trust values of the plurality of candidate devices are obtained by the first device from a blockchain or an interstellar file storage system.

[0239] In an implementation, the processor 1102 is further configured to: receive, by the communication interface 1101, the interaction data between the first device and the trusted device sent by the first device, and update at least one of the following according to the interaction data between the first device and the trusted device: a model parameter of the first reinforcement learning model, a model parameter of the second reinforcement learning model, and a trust value of the trusted device.

[0240] The communication interface 1101 is further configured to: send the updated trust value of the trusted device to a blockchain or an interstellar file storage system.

[0241] In an implementation, the processor 1102 is specifically configured to: determine an indirect trust value of the target device according to the plurality of recommended devices corresponding to the target device and the weight values corresponding to each recommended device, wherein the indirect trust value is used to indicate a trust degree of the target device relative to the plurality of recommended devices corresponding to the target device; determine a direct trust value of the target device; and determine a trust value of the target device according to the direct trust value of the target device, the indirect trust value, and the weight value corresponding to the direct trust value.

[0242] It can be understood that the present application Figure 11The memory 1103 in the foregoing embodiments can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0243] Based on the foregoing embodiments, the present application also provides a computer program, which, when executed on a computer, causes the computer to perform the methods described in the embodiments of the present application.

[0244] Based on the foregoing embodiments, the present application also provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a computer, causes the computer to perform the methods described in the embodiments of the present application.

[0245] Based on the foregoing embodiments, the present application also provides a chip for reading a computer program stored in a memory, which implements the methods described in the embodiments of the present application.

[0246] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the methods described in the embodiments of this application. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. This chip system may be composed of chips or may include chips and other discrete devices.

[0247] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0248] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure One One or more processes and / or boxes Figure One A device that provides the functions specified in one or more boxes.

[0249] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure One One or more processes and / or boxes Figure One The function specified in one or more boxes.

[0250] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure One One or more processes and / or boxes Figure One The steps of the function specified in one or more boxes.

[0251] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A trust assessment method characterized by, The method is applied to a trust evaluation node, and the method comprises the following steps: receiving a trust evaluation request sent by a first device, wherein the trust evaluation request comprises the identities of a plurality of candidate devices to be evaluated; obtaining an initial trust value of each candidate device in the plurality of candidate devices, and dividing the plurality of candidate devices into at least two trust types according to the initial trust values of the plurality of candidate devices; for each candidate device in the plurality of candidate devices, the following operations are performed to obtain the trust values of the plurality of candidate devices: determining a weight value corresponding to a direct trust value of a target device according to the candidate devices corresponding to each trust type; determining a plurality of recommended devices corresponding to the target device and a weight value corresponding to each recommended device according to the candidate devices corresponding to each trust type; and determining the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the plurality of recommended devices corresponding to the target device and the weight value corresponding to each recommended device; wherein the target device is any device in the plurality of candidate devices, and the direct trust value of the target device is used to indicate the trust degree of the target device relative to the first device; determining a trusted device in the plurality of candidate devices according to the trust values of the plurality of candidate devices, and sending the identity of the trusted device to the first device.

2. The method of claim 1, wherein, The method comprises the following steps: performing a plurality of rounds of weight calculation processes on the target device based on a first reinforcement learning model to obtain a plurality of candidate weight values; and determining the weight value corresponding to the direct trust value of the target device according to the plurality of candidate weight values. Any round of weight calculation process comprises the following steps: determining that the interaction device selection strategy of the current round of weight calculation process is a first strategy or a second strategy, and determining at least one interaction device according to the interaction device selection strategy of the current round of weight calculation process; wherein the first strategy is to determine a target trust type from the at least two trust types with a preset probability, and to select one candidate device as an interaction device from the candidate devices corresponding to the target trust type; and the second strategy is to select a node with the highest trust value in the neighbor nodes of the target device as an interaction device, wherein the trust values of the neighbor nodes of the target device are stored in the trust evaluation node; obtaining interaction data of communication interaction between the at least one interaction device and the target device; determining a first reward value and a first state value of the current round of weight calculation process based on the first reinforcement learning model according to the interaction data, and storing the first reward value of the current round of weight calculation process; and calculating a candidate weight value corresponding to the current round of weight calculation process according to the first reward value and the first state value of the current round of weight calculation process; wherein the first reward value is used to indicate the performance of the first reinforcement learning model in the current round of weight calculation process; and the first state value is used to indicate the communication state of the target device and the at least one interaction device.

3. The method of claim 2, wherein, The method comprises the following steps: The probability value of the interaction device selection strategy being the first strategy is determined according to a preset decision function, and the interaction device selection strategy is determined according to a random function corresponding to the probability value.

4. The method of claim 2 or 3, wherein, The weight value corresponding to the direct trust value of the target device is determined according to the plurality of alternative weight values, including: Among the plurality of alternative weight values, an alternative weight value calculated by a weight calculation process of a round with the highest first reward value is selected; and the selected alternative weight value is taken as the weight value corresponding to the direct trust value of the target device.

5. The method according to any one of claims 2 to 4, wherein, The method further includes: After M rounds of weight calculation for the target device, and / or, the difference between the first reward values of any two adjacent rounds of weight calculation in the last N rounds of weight calculation for the target device is less than a preset threshold, the plurality of rounds of weight calculation is stopped; wherein M and N are positive integers, and M is greater than N.

6. The method according to any one of claims 2 to 5, wherein, The interaction data includes the number of successful communication interactions and the number of failed communication interactions between the target device and the at least one interaction device.

7. The method according to any one of claims 1 to 6, wherein The plurality of recommended devices corresponding to the target device and the weight value corresponding to each recommended device are determined according to the candidate devices corresponding to each trust type, including: A plurality of sets of alternative results are obtained by performing a plurality of rounds of recommended device selection processes for the target device based on a second reinforcement learning model, each set of alternative results including a plurality of alternative recommended devices and a weight value corresponding to each alternative recommended device; a target result is determined from the plurality of sets of alternative results, the target result including the plurality of recommended devices corresponding to the target device and the weight value corresponding to each recommended device; Any round of recommended device selection process includes: The recommended device selection strategy of the current round of recommended device selection process is determined to be a third strategy or a fourth strategy, and a plurality of first alternative recommended devices are determined according to the recommended device selection strategy; wherein the third strategy is to select candidate devices that have communicated with the target device from candidate devices of each trust type at a preset proportion; and the fourth strategy is to select a preset number of nodes that have communicated with the target device from neighbor nodes of the target device as recommended devices; A second state value of the current round of recommended device selection process is determined based on the second reinforcement learning model according to the plurality of first alternative recommended devices; the second state value is used to indicate the communication state of the target device and the plurality of first alternative recommended devices; a plurality of rounds of weight calculation processes are performed based on the second reinforcement learning model to determine the weight value corresponding to each first alternative recommended device; a second reward value of the current round of recommended device selection process is determined according to the plurality of first alternative recommended devices and the weight value corresponding to each first alternative recommended device, and the second reward value is stored; the second reward value is used to indicate the performance of the second reinforcement learning model in the current round of recommended device selection process.

8. The method of claim 7, wherein, The recommended device selection strategy of the current round of recommended device selection is determined to be the third strategy or the fourth strategy, including: The probability value of the recommended device selection strategy being the third strategy is determined according to a preset decision function, and the recommended device selection strategy is determined according to a random function corresponding to the probability value.

9. The method of claim 7 or 8, wherein, The determining the target result from the multiple groups of candidate results comprises: selecting a candidate result determined by a round of recommended device selection with the highest second reward value from the multiple groups of candidate results; and taking the selected candidate result as the target result.

10. The method according to any one of claims 7 to 9, characterized in that, The method further comprises: stopping the multiple rounds of recommended device selection after Q rounds of recommended device selection for the target device, and / or the difference between reward values of any two adjacent rounds of recommended device selection in the last P rounds of recommended device selection for the target device being less than a preset threshold; wherein Q and P are positive integers, and Q is greater than P.

11. The method of any one of claims 1-10, wherein, The obtaining the initial trust value of each candidate device from the multiple candidate devices comprises: receiving trust-related data of the multiple candidate devices sent by the first device, and determining the initial trust value of each candidate device according to the trust-related data of each candidate device; wherein the trust-related data comprises at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location; or receiving the initial trust values of the multiple candidate devices sent by the first device; the initial trust values of the multiple candidate devices being obtained by the first device from a blockchain or an interplanetary file system.

12. The method of any one of claims 2-11, wherein, After sending the identifier of the trusted device to the first device, the method further comprises: receiving interaction data between the first device and the trusted device sent by the first device, and updating at least one of the following according to the interaction data between the first device and the trusted device: a model parameter of the first reinforcement learning model, a model parameter of the second reinforcement learning model, and a trust value of the trusted device.

13. The method of any one of claims 1-12, wherein, The determining the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the multiple recommended devices corresponding to the target device, and the weight value corresponding to each recommended device comprises: determining an indirect trust value of the target device according to the multiple recommended devices corresponding to the target device and the weight value corresponding to each recommended device, wherein the indirect trust value is used to indicate a trust degree of the target device relative to the multiple recommended devices corresponding to the target device; determining a direct trust value of the target device; determining the trust value of the target device according to the direct trust value of the target device, the indirect trust value, and the weight value corresponding to the direct trust value.

14. A trust assessment apparatus, characterized by comprising: The trust evaluation apparatus comprises a receiving unit, a processing unit, and a sending unit, the receiving unit is configured to receive a trust evaluation request sent by a first device, the trust evaluation request comprising identifiers of multiple candidate devices to be evaluated; the processing unit is configured to obtain an initial trust value of each candidate device from the multiple candidate devices, and divide the multiple candidate devices into at least two trust types according to the initial trust values of the multiple candidate devices; The processing unit is specifically configured to: determine a weight value corresponding to the direct trust value of the target device according to the candidate device corresponding to each trust type; and determine the trust value of the target device according to the weight value corresponding to the direct trust value of the target device, the plurality of recommended devices corresponding to the target device, and the weight value corresponding to each recommended device; wherein the target device is any device in the plurality of candidate devices; the direct trust value of the target device is used to indicate the trust degree of the target device relative to the first device; and the trusted device is determined from the plurality of candidate devices according to the trust values of the plurality of candidate devices.

15. The apparatus of claim 14, wherein, The sending unit is configured to send the identifier of the trusted device to the first device. The processing unit is specifically configured to: perform a plurality of rounds of weight calculation processes on the target device based on a first reinforcement learning model to obtain a plurality of alternative weight values; and determine the weight value corresponding to the direct trust value of the target device according to the plurality of alternative weight values. Any round of weight calculation process includes: determining that the interaction device selection strategy of the current round of weight calculation process is a first strategy or a second strategy, and determining at least one interaction device according to the interaction device selection strategy of the current round of weight calculation process; wherein the first strategy is to determine a target trust type from the at least two trust types with a preset probability, and to select one candidate device corresponding to the target trust type as an interaction device; and the second strategy is to select a node with the highest trust value in the neighbor nodes of the target device as an interaction device, wherein the trust evaluation device stores the trust values of the neighbor nodes of the target device; 16. The apparatus of claim 15, wherein, obtain interaction data of communication interaction between the at least one interaction device and the target device; determine a first reward value and a first state value of the current round of weight calculation process based on the first reinforcement learning model according to the interaction data, store the first reward value of the current round of weight calculation process, and calculate the alternative weight value corresponding to the current round of weight calculation process according to the first reward value and the first state value of the current round of weight calculation process; wherein the first reward value is used to indicate the performance of the first reinforcement learning model in the current round of weight calculation process; and the first state value is used to indicate the communication state of the target device and the at least one interaction device. The processing unit is specifically configured to:

17. The apparatus of claim 15 or 16, wherein, determine the probability value of the interaction device selection strategy being the first strategy according to a preset decision function, and determine the interaction device selection strategy according to a random function corresponding to the probability value. The processing unit is specifically configured to:

18. The apparatus of any one of claims 15-17, wherein, select the alternative weight value calculated by the round of weight calculation process with the highest first reward value from the plurality of alternative weight values; and take the selected alternative weight value as the weight value corresponding to the direct trust value of the target device. The processing unit is further configured to: After M rounds of weight calculation are performed for the target device, and / or, a difference between reward values of any two adjacent rounds of weight calculation in the last N rounds of weight calculation performed for the target device is less than a preset threshold, the multiple rounds of weight calculation are stopped; wherein M and N are positive integers, and M is greater than N.

19. The apparatus of any one of claims 15-18, wherein, The interaction data includes a number of successful communication interactions and a number of failed communication interactions between the target device and the at least one interaction device.

20. The apparatus of any one of claims 14-19, wherein, The processing unit is specifically configured to: perform a multiple-round recommendation device selection process for the target device based on the second reinforcement learning model, to obtain multiple groups of candidate results, each group of candidate results including multiple candidate recommendation devices and a weight value corresponding to each candidate recommendation device; determine a target result from the multiple groups of candidate results, the target result including multiple recommendation devices corresponding to the target device and a weight value corresponding to each recommendation device; wherein any round of recommendation device selection process includes: determining that a recommendation device selection strategy of the current round of recommendation device selection process is a third strategy or a fourth strategy, and determining multiple first candidate recommendation devices according to the recommendation device selection strategy; wherein the third strategy is to select a candidate device that has communicated with the target device from candidate devices of each trust type as a recommendation device at a preset proportion; and the fourth strategy is to select a preset number of nodes that have communicated with the target device from neighbor nodes of the target device as recommendation devices; determining a second state value of the current round of recommendation device selection process based on the second reinforcement learning model according to the multiple first candidate recommendation devices; the second state value is used to indicate a communication state between the target device and the multiple first candidate recommendation devices; performing a multiple-round weight calculation process based on the second reinforcement learning model to determine a weight value corresponding to each first candidate recommendation device; determining a second reward value of the current round of recommendation device selection process according to the multiple first candidate recommendation devices and the weight value corresponding to each first candidate recommendation device, and storing the second reward value; the second reward value is used to indicate a performance of the second reinforcement learning model in the current round of recommendation device selection process.

21. The apparatus of claim 20, wherein, The processing unit is specifically configured to: determine a probability value of the recommendation device selection strategy being the third strategy according to a preset decision function, and determine the recommendation device selection strategy according to a random function corresponding to the probability value.

22. The apparatus of claim 20 or 21, wherein, The processing unit is specifically configured to: select a candidate result determined by a round of recommendation device selection process with the highest second reward value from the multiple groups of candidate results; and take the selected candidate result as the target result.

23. The apparatus of any one of claims 20-22, wherein, The processing unit is further configured to: after Q rounds of recommendation device selection are performed for the target device, and / or, a difference between reward values of any two adjacent rounds of recommendation device selection in the last P rounds of recommendation device selection performed for the target device is less than a preset threshold, stop the multiple rounds of recommendation device selection; wherein Q and P are positive integers, and Q is greater than P.

24. The apparatus of any one of claims 14-23, wherein, The processing unit is specifically configured to: receive, by the receiving unit, the trust-related data of the plurality of candidate devices sent by the first device, and determine an initial trust value of each candidate device according to the trust-related data of each candidate device; wherein the trust-related data comprises at least one of a device manufacturer, a device usage range, a device importance, and a device deployment location; or receive, by the receiving unit, the initial trust values of the plurality of candidate devices sent by the first device; the initial trust values of the plurality of candidate devices are obtained by the first device from a blockchain or an interstellar file storage system.

25. The apparatus of any one of claims 15-24, wherein, The processing unit is further configured to: receive, by the receiving unit, the interaction data between the first device and the trusted device sent by the first device, and update at least one of the following according to the interaction data between the first device and the trusted device: model parameters of the first reinforcement learning model, model parameters of the second reinforcement learning model, and the trust value of the trusted device.

26. The apparatus of any one of claims 14-25, wherein, The processing unit is specifically configured to: determine an indirect trust value of the target device according to the plurality of recommended devices corresponding to the target device and the weight values corresponding to each recommended device, wherein the indirect trust value is used to indicate a trust degree of the target device relative to the plurality of recommended devices corresponding to the target device; determine a direct trust value of the target device; determine a trust value of the target device according to the direct trust value of the target device, the indirect trust value, and the weight value corresponding to the direct trust value.

27. A trust assessment device, characterized by comprising a processor and a memory; the memory stores computer program instructions; when the trust evaluation device is running, the processor executes the computer program instructions stored in the memory to realize the operation steps of the method of any one of claims 1 to 13.

28. A computer-readable storage medium, characterized in that, comprising computer instructions, when the computer instructions are executed by a processor, causing a trust evaluation device to perform the method of any one of claims 1 to 13.

29. A computer program product, characterised in that, when the computer program product is executed on a processor, causing a trust evaluation device to perform the method of any one of claims 1 to 13.

Images