Knowledge-based intelligent space system automatic application conflict detection method and device

Abstract

The invention relates to a knowledge-based intelligent space system automatic application conflict detection method and device. The method comprises the following steps: mapping key elements of an automation application to an ECA control network, wherein the key elements comprise events, conditions and behaviors; semantic information of an ECA control network is supplemented through a knowledge graph, the semantic information comprises dependency and mutual exclusion relations between services of equipment and influences of the services of the equipment on a physical environment, and an automatic application network is formed; and performing search matching on the automatic application network based on the defined conflict mode, and detecting conflicts existing between the automatic applications in combination with semantic information. According to the method, the defects of effectiveness and practicability of an automatic application conflict detection method of an existing intelligent space system are overcome, an efficient and practical automatic application conflict detection scheme based on graph pattern matching is achieved, and the conflict problem between automatic applications can be detected more accurately and efficiently.

Description

Technical Field

[0001] The present invention belongs to the field of software technology, and particularly relates to a method and device for automatically detecting application conflicts in a knowledge-based intelligent space system. Background Art

[0002] The Internet of Things is becoming more and more popular in the field of intelligent space. An intelligent space system connects various Internet of Things devices and provides a programming framework for users to establish an automated control system. Currently, Trigger Action Programming (TAP) is the most popular programming paradigm and is supported by platforms such as IFTTT, Samsung SmartThings, Home Assistant, OpenHab, and Zapier. Through TAP, users of the intelligent space system can create various forms of event-driven automated application programs. These application programs satisfy the form of "IF a trigger occurs, then perform an action". However, the expressiveness of TAP is limited, making it difficult for users to build large and complex automated systems. To meet the requirements of scenarios, users need to design multiple application programs and coordinate them to complete complex tasks. However, Internet of Things devices have some inherent characteristics, such as implicit dependencies between device services and implicit interactions through physical channels. These characteristics may lead to unexpected interaction conflicts between automated application programs, resulting in anomalies and even security risks. Therefore, designing a detection method for automated application conflicts is of great significance for ensuring the reliability of intelligent space systems.

[0003] At present, most of the automated application conflict detection works for smart space systems are based on model checking methods. These works systematically explore the system state to check whether the system meets the given specifications. For example, Soteria (Z. Berkay Celik, Patrick McDaniel, and Gang Tan. "SOTERIA: Automated IoT Safety and Security Analysis" USENIX ATC 2018) models automated applications and converts them into intermediate representations based on program control relationships. Then, conflicts between automated applications are detected based on predefined attributes, where the predefined attributes are similar to specifications, such as "the door must be locked when the user is not at home" and "the air conditioner and heater cannot be turned on at the same time." This work detects conflicts based on some attributes defined by the author, which lacks generality. iRuler (Qi Wang, Pubali Datta, Wei Yang, Si Liu, Adam Bates, and Carl A. Gunter. "Charting the Attack Surface of Trigger-Action IoT Platforms." CCS2019) models the environment, location, and time of the IoT, and classifies conflicts into categories such as conditional prohibition, behavior conflict, loop, and redundancy. They parse automated applications into triggers and behaviors, and finally check for conflicts based on rewriting the SMT model. These model-based checks have poor efficiency, and it takes up to 30 minutes to detect conflicts between two applications. At the same time, the scalability is poor, and when new devices are connected, they often need to be remodeled. In addition, the current method has a large modeling granularity for device functions, which is prone to underreporting.

[0004] Compared with the model checking-based method, the knowledge-based detection method can quickly and effectively identify the conflict problems of automated applications in the intelligent space system. The knowledge-based method requires modeling the device capabilities and the impact of device capabilities on physical environmental factors. Then, based on this knowledge and combined with the graph detection algorithm, the conflicts are detected. Huang et al. (Bing Huang, Hai Dong, and Athman Bouguettaya. “Conflict Detection in IoT-based Smart Homes.” ICWS 2021) characterized the capabilities of IoT devices by obtaining knowledge from the common sense graph based on NLP technology, and then detected the automated applications with opposite operations on the same physical environment based on this knowledge. However, the device capability modeling obtained from the common sense graph in this work is not comprehensive and correct, and it is difficult to correctly detect the conflict problems in automated applications.

[0005] In summary, the intelligent space system is widely used, and it is very important to ensure its reliability. The conflict problems of automated applications in the intelligent space system will affect the correctness of the system and then threaten the system reliability. However, the existing technologies cannot effectively and quickly detect the conflict problems of automated applications. Therefore, designing and developing an extensible and fast conflict detection method has very important value and significance for improving the reliability of the intelligent space system. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies in the effectiveness and practicality of the existing conflict detection methods for automated applications in the intelligent space system, and propose an efficient and practical conflict detection method for automated applications based on graph pattern matching.

[0007] The technical solution of the present invention is as follows:

[0008] A knowledge-based method for detecting conflicts in automated applications in an intelligent space system, comprising the following steps:

[0009] Map the key elements of the automated application to the ECA control network, where the key elements include events, conditions, and actions;

[0010] Supplement the semantic information of the ECA control network through the knowledge graph, including the dependency and mutual exclusion relationships between the services of the devices and the impact of the services of the devices on the physical environment, to form an automated application network;

[0011] Search and match the automated application network based on the defined conflict patterns, and detect the conflicts existing between automated applications in combination with the semantic information.

[0012] Further, the knowledge graph is a knowledge graph depicting the relationship between devices and the environment in the Internet of Things space. The construction steps of the knowledge graph include:

[0013] Establish an ontology model of the knowledge graph to depict the relationships between devices, capabilities, instructions, services, and the environment;

[0014] Perform data collection and extraction, and fill the content of the knowledge graph through entity recognition and matching methods;

[0015] Further, the relationships between the devices, capabilities, instructions, services, and the environment include: a device has multiple capabilities, and each capability is realized according to the corresponding instruction; the device provides services externally, and a service is composed of the specific capabilities and specific instructions of the device; there are two relationships between the services provided by the device, namely dependence and mutual exclusion. The dependence relationship means that the invocation of one device service depends on the invocation of another device service, and the mutual exclusion relationship indicates that two device services cannot occur simultaneously; the relationship between the service and the environment is the increase and decrease effects generated by the device's service on the environment.

[0016] Further, the performing of data collection and extraction includes three steps of preprocessing, namely stop word removal, lemmatization, and entity matching.

[0017] Further, the automated application network is represented as AG = (Vs, Vp, Vc, Ec, Ee, Ei, Ex), where:

[0018] Vs: a set of Internet of Things service vertices, and the Internet of Things service vertices are composed of Internet of Things services Si, where Si = <devType, prodId, deviceId, serviceId>, representing device type, product identifier, device identifier, and service identifier respectively;

[0019] Vp: a set of physical channel vertices, and the physical channel vertices represent the types of physical contexts existing in the smart home platform;

[0020] Vc: a set of condition vertices, and the condition vertices are nodes representing the physical factor constraints of the application program. The condition vertices are defined as <env, operator, args>, representing physical environment identifier, operator, and parameter respectively;

[0021] Ec: a set of control edges, and the control edges are directed edges flowing in the vertex set V = Vs ∪ Vp ∪ Vc, representing control relationships;

[0022] Ee: a set of influence edges, and each influence edge is a directed edge flowing from the Internet of Things service vertex to the physical channel vertex, representing the influence of the Internet of Things service on the corresponding physical channel, including positive or incremental effects and negative or decreasing effects;

[0023] Ei: A set of service dependency edges, where each dependency edge is a directed edge connecting two Internet of Things services;

[0024] Ex: A set of service mutual exclusion edges, where each service mutual exclusion edge is a directed or bidirectional edge connecting two Internet of Things services, indicating that the two Internet of Things services cannot be executed simultaneously.

[0025] Furthermore, the conflict pattern includes 4 conflict categories and 10 sub - graph patterns; the conflict categories include conflict, cycle, redundancy, and blockage.

[0026] Furthermore, a corresponding detection report is formed for the detected conflicts, and the content of the detection report includes the corresponding conflict type, the automated applications involved, and the information about the edges and nodes involved.

[0027] A knowledge - based intelligent space system automated application conflict detection device, which includes:

[0028] An application parsing module, used to map the key elements of the automated application to the ECA control network, and the key elements include events, conditions, and actions;

[0029] An automated application network construction module, used to supplement the semantic information of the ECA control network through the knowledge graph, including the dependency and mutual exclusion relationships between the services of the devices and the impact of the services of the devices on the physical environment, so as to form an automated application network;

[0030] A pattern matching module, used to search and match the automated application network based on the defined conflict pattern, and detect the conflicts existing between automated applications in combination with the semantic information.

[0031] Compared with the prior art, the present invention has the following technical advantages:

[0032] 1. The present invention conducts fine - grained modeling on the devices and environment in the Internet of Things space, making up for the problem that the existing modeling scheme has a large granularity and cannot accurately reflect the relationship between the services of Internet of Things devices, resulting in insufficient accuracy of conflict detection.

[0033] 2. The present invention proposes a knowledge - based automated application network construction method, which has higher scalability than the model - checking - based method. In addition, the efficiency of conflict detection on the automated application network is also higher than that of related methods.

[0034] 3. The present invention fully considers the semantic information of nodes and edges when detecting conflicts, and can more accurately detect the conflict problems between automated applications. Description of the Drawings

[0035] Figure 1 It is an overall process example diagram of the method of the present invention.

[0036] Figure 2 It is a flow chart for constructing a knowledge graph.

[0037] Figure 3 It is a schematic diagram of the ontology structure of the knowledge graph.

[0038] Figure 4 It is a diagram of an example of an automated application.

[0039] Figure 5 It is a schematic diagram of a conflict mode.

[0040] Figure 6 It is an automated application network for an example application. Specific embodiments

[0041] Next, in conjunction with the accompanying drawings, the technical solutions in the embodiments of the present invention will be clearly and completely described. The technical solution of the present invention includes a method for detecting conflicts in automated applications of a knowledge-based intelligent space system. In this detection method, a fine-grained device knowledge graph is used to enrich the semantics of the automated application network, different conflict subgraph patterns are defined, and finally, pattern matching is used to detect conflicts in automated applications.

[0042] The overall process of the method of the present invention is as Figure 1 shown. It maps the key elements of the automated application, namely Event (event), Condition (condition), and Action (behavior), to the ECA control network, and enriches the semantic information of the ECA control network through the knowledge graph, supplements the dependency and mutual exclusion relationships between the services of the device and the impact of the device service on the physical environment, and forms an automated application network. Finally, the automated application network is searched and matched with the subgraph of the conflict mode, and the semantic information is combined to detect the conflicts existing between the automated applications.

[0043] In a first aspect, a knowledge graph is provided that depicts the relationship between devices and the environment in the Internet of Things space, and the construction steps are as Figure 2 shown. The steps include:

[0044] Step 1: Ontology modeling of the knowledge graph. Characterize the relationships between devices, capabilities, instructions, services, and the environment. The ontology model of the present invention is as Figure 3As shown in the figure. A device itself has multiple capabilities, and each capability can be realized according to the corresponding instruction. The device provides services externally, and a service is composed of the specific capabilities and specific instructions of the device. There are two relationships between the services provided by the device, namely dependence and mutual exclusion. The dependence relationship means that the invocation of one device service depends on the invocation of another device service. For example, the heating service of an air conditioner depends on the turning-on service of the air conditioner. The mutual exclusion relationship indicates that two device services cannot occur simultaneously. In addition, the services of the device may have an increasing or decreasing impact on environmental factors. For example, the heating service of an air conditioner will cause an increase in the environmental factor of temperature.

[0045] Step 2: Data collection and extraction. The method uses the general knowledge graph ConceptNet as the data source, which covers a wide range of fields and concepts. By inputting the corresponding device name, three types of relationship data can be obtained, such as Figure 2 As shown in the figure, specifically including isUsedFor (used for), isCapableOf (able to do something), and derivedFrom (derived from). isUsedFor and isCapableOf describe the capabilities of the device. Through entity recognition and matching methods, the content of the knowledge graph is filled in this invention. Specifically, three steps of preprocessing are carried out, namely stop word removal, lemmatization, and entity matching. The first step is to filter out common stop words that are less important in the meaning of the sentence. For example, articles (such as "the" and "a"), prepositions (such as "of" and "to"), and pronouns (such as "that" and "this") are all common stop words. The next step is lemmatization. Texts sometimes use different forms of words with similar meanings (such as organize, organizes, and organizing). The goal of lemmatization is to reduce the inflected forms of a word to a common basic form. For example, after lemmatization, the words "car, cars, car's, cars'" can be transformed into the basic word "car". Finally, entity linking is performed on these texts by matching the processed texts with predefined model entities (capabilities, instructions, services, and environmental factors). The impact on the environment is mainly obtained by measuring the described physical channels and keywords for positive and negative feedback effects.

[0046] In a second aspect, a method for detecting conflicts in automated applications of a knowledge-based intelligent space system is provided, including the following steps:

[0047] Step 1: Parse a set of automated applications, and extract the information of the corresponding conditions Condition, events Event, and actions Action from them. Based on the control relationship, form the ECA control network of the application. First, parse the ECA elements of the automated application. Such as Figure 4As shown, the present invention transforms automated applications into a structured JSON format. For the parsing of the Condition node, it is necessary to extract the type of condition, namely the physical field. The present invention focuses on 7 physical environments (physical), temperature, humidity, movement, gas leakage, location, smoke, and light. Then it is necessary to parse the params field in the condition, and then construct the Condition node, where the node Id is the concatenation of the physical field, params.operator (the operator of the parameter field), and params.range (the numerical range of the parameter field). The parsing content of the Event and Action nodes is similar. It is necessary to extract the device type devType, product identifier prodId, device identifier deviceId, capability identifier capabilityId, instruction command, and parameter value. Then, the service identifier serviceId is obtained from the knowledge graph through devType, prodId, capabilityId, command, and value. Finally, the corresponding nodes are formed. Figure 4 In Figure 4 , eventId represents the event identifier, actionId represents the action identifier, delay represents the delay, and delaySync represents the specific parameters of the delay.

[0048] Step 2: Supplement semantic information to the ECA control network through the knowledge graph, supplement the dependencies and mutual exclusions between the services of the devices and the impact of the services of the devices on the physical environment, and form an automated application network.

[0049] The automated application network is represented as AG = (Vs, Vp, Vc, Ec, Ee, Ei, Ex).

[0050] Vs: A set of Internet of Things service vertices. The Internet of Things service vertices are composed of Internet of Things services Si, where Si = <devType, prodId, deviceId, serviceId>, representing the device type, product identifier, device identifier, and service identifier respectively. For example, an Internet of Things service node <00A, 001T, testDeviceId#001, speaker - AISpeaker_audioPlayState_pause_1> indicates that the AI speaker is in the paused playback state. It should be noted that Vs can semantically represent both the event Event and the action Action.

[0051] Vp: A set of physical channel vertices. Physical channel vertices represent the types of physical context present in the smart home platform. We considered the seven physical channels discussed previously. The vertex Id is set to its channel name. These vertices can be used to capture the impact of IoT services on physical channels.

[0052] Vc: A set of condition vertices. Condition vertices are nodes that represent the physical factor constraints (seven physical channels and time) of an application. A condition vertex is defined as <env, operator, args>, representing the physical environment flag, operator, and arguments respectively. For example, the condition "when the temperature rises to 25°C" can be represented as <temperature, >, 25>.

[0053] Ec: A set of control edges. Control edges are directed edges that flow in the vertex set V = Vs ∪ Vp ∪ Vc, representing control relationships. For a simple application without conditions, the source of the control edge is the Event, and the target is the Action. Assuming both the event and the action are IoT services, the application can be represented as Vs event ->Vs action , where Vs event represents the event service node, and Vs action represents the behavior service node. An application with a condition Condition can be represented as: Vc->Vs event ->Vs action , where Vc represents the condition node.

[0054] Ee: A set of impact edges. Each impact edge is a directed edge that flows from an IoT service vertex to a physical channel vertex, and can be represented as Vs-eff->Vp, where eff ∈ {hasIncreased, hasDecreased}. Impact edges represent the impact of IoT services on the corresponding physical channels. The value "hasIncreased" indicates that the IoT service has a positive or incremental effect on a specific physical channel, while the other value "hasDecreased" indicates a negative or decreasing effect.

[0055] Ei: A set of service dependency edges. Each dependency edge is a directed edge that connects two IoT services. The dependency edge Vs i -ind->Vs j . represents that the execution of service Vs i depends on service Vs jExecution. For example, <00A,001T,testDeviceId#001,speaker-AISpeaker_audioPlayState_pause_1>-ind-><00A,001T,testDeviceId#001,speaker-AISpeaker_switchOnOff_on_1>.

[0056] Ex: A set of service mutually exclusive edges. Each service mutually exclusive edge is a directed or bidirectional edge connecting two IoT services. Such an edge indicates that the two IoT services cannot be executed simultaneously. For example, we find that the camera and the flash of a mobile phone cannot be turned on simultaneously.

[0057] Step 3: Based on the defined conflict subgraph patterns, perform pattern matching in the automated application network, detect corresponding types of automation conflicts, and generate a detection report. Further, the conflict patterns include 4 conflict categories and 10 subgraph patterns. The conflict categories include conflict, cycle, redundancy, and blockage. The specific subgraph patterns are as Figure 5 shown.

[0058] P1. Conflicting actions. Conflicting actions occur when different applications specify opposite actions for the same device, or when the actions of the device have opposite effects on the same physical channel. Specifically, it includes P1.1. Device conflict and P1.2. Environment conflict. Device conflict means that different automated applications send conflicting instructions to the device, such as turning on and turning off. Environment conflict means that different automated applications define device services that have opposite effects on the same physical environment, such as cooling and heating. For the detection of P1.1 device conflict, maintain a list of devices during the graph construction phase, which records all the service nodes of the device. To detect device conflict, it is necessary to traverse the device list, analyze the conflicting device services, and then define the corresponding automated application. For the detection of P1.2 environment conflict, locate all the physical environment nodes, and then analyze all their incoming edges. If the edge impact information of two incoming edges is opposite, i.e., hasIncreased and hasDecreased, then it is considered that there is a corresponding environment conflict.

[0059] P2. Looping. Operations from multiple applications can trigger each other infinitely in a loop. We call this "looping" and divide it into two sub-categories, P2.1 Device Looping and P2.2 Environment Looping. Device Looping has a loop formed by service nodes, while Environment Looping is caused by the influence of environmental factor changes. For example, "When the temperature is greater than 25 degrees Celsius, turn on the air conditioning cooling service." and "When the temperature is lower than 20 degrees Celsius, turn on the air conditioning heating service", these two automated applications will form a loop due to the change of temperature environmental factors. For the detection of P2.1 Device Looping, it only needs to judge whether there is a loop on the graph. By finding strongly connected components and using the Targan algorithm to find the loop, and then locating the set of automated applications that produce the loop. For the detection of P2.2 Environment Looping, it needs to be further detected based on P1.2 Environment Conflict. After detecting the opposite operation device service for P1.2, it is necessary to traverse its incoming edges and judge whether its condition node Vc is related to the change of the corresponding environmental factor. Taking the above example, air conditioning cooling will cause the temperature to drop, resulting in a situation where the temperature is lower than 20 degrees Celsius. Therefore, it is necessary to judge whether there is a corresponding relationship between the influence hasDecreased of the behavior of Application 1 and the condition judgment of Application 2 that the temperature drops. And it is necessary to judge whether there is a corresponding relationship between the influence hasIncreased of the behavior of Application 2 and the judgment of Application 1 that the temperature rises. If both exist, it is considered that there is an environment loop.

[0060] P3. Redundancy. Application programs may be redundant because they are the same or there are dependencies between their events or behaviors. Redundant programs may trigger the execution of certain device behaviors multiple times. For the detection of P3 redundant application programs, it is mainly to analyze the semantic information on the application edges. If there are multiple control edges of different applications between two nodes, it is considered that redundancy occurs. In addition, if there are dependencies between the corresponding event and behavior nodes, it is also considered that there is redundancy between these applications. Figure 5 In P3 redundancy, T 1 、T 2 represents the trigger node Trigger, which specifically includes the event node Event and the condition node Condition. A 1 、A 2 represents the action node Action, imp. represents the dependency relationship implied, abbreviated as imp.

[0061] P4. Blocking behavior. There may be a mutual exclusion relationship between nodes, resulting in the inability of the application to execute. This type of interference is called blocking behavior and can occur even within a single application. For example, when the camera on a mobile phone is turned on, the flashlight cannot be turned on. In addition, blocking problems may also occur between multiple applications. For example, an application that shuts off the water supply may prevent another application that depends on the water supply from executing. For the blocking behavior caused by a single application, it is necessary to determine whether there is a mutual exclusion relationship in addition to the control relationship on the edge. For the latter case, it is necessary to determine whether there is a mutual exclusion relationship between the behaviors of the two applications. If so, it is considered that blocking behavior will occur.

[0062] Finally, for the detected conflicts, corresponding detection reports will be formed. The report content includes the corresponding conflict types, the automated applications involved, and the information of the edges and nodes involved.

[0063] Specific embodiments are provided below. In this embodiment, a set of automated applications is given, including 4 automated applications.

[0064] Application 1: "When the temperature is higher than 26 degrees Celsius, turn on the air conditioner's cooling mode."

[0065] Application 2: "When the temperature is lower than 20 degrees Celsius, turn on the air conditioner's heating mode."

[0066] Application 3: "When the air conditioner is turned on, turn on the humidifier."

[0067] Application 4: "When the mobile phone camera is turned on, turn on the flashlight."

[0068] The graph data involved in the above 4 applications are shown in Tables 1 and 2 below. Table 1 is the device modeling data, which is used to describe the device identification, device service, and the impact of the device service on the environment. Table 2 is the data on the relationship between device services, which is used to describe the relationships existing between different device services, including mutual exclusion and dependency relationships. In the first row of Table 1, ID is the data ID, DEVICE is the device, NAME is the device name, DEVTYPE is the device type, PRODID is the product logo, CAPABILITY is the capability logo, COMMAND is the instruction logo, VALUETYPE is the parameter type, ENUM_VALUE is the enumerated type parameter range, VALUE is the specific parameter value, MIN is the lower bound of the range type parameter, MAX is the upper bound of the range type parameter, SERVICEID is the device service logo, CHANNEL is the physical environment logo affected, and TYPE is the impact on the physical environment. In the first row of Table 2, Id is the data Id, sourceServiceId is the source device service, targetServiceId is the target device service, and rel is the service relationship.

[0069] Table 1

[0070]

[0071]

[0072] Table 2

[0073]

[0074] (1) For this group of four applications, using the method of the present invention, first extract their condition, event, and action, and then construct an ECA control network based on the control relationship. For the extraction of Condition node information, mainly extract the constraints of the physical environment it involves, including the type of physical environment and the specific constraint range. The information of the Condition node consists of physicalType|physicalEnvironment|parms. For the Event and Action nodes, extract the device information involved. From the applications, the directly extractable methods are devType, prodId, capability, command, value, and deviceId. Then, based on the values of devType, prodId, capability, command, and value extracted, the method obtains the device, name, and serviceId of the device from the graph. The information of the Event and Action nodes consists of devType|prodId|deviceId|device-name_serviceId. The node information extracted from these four applications is as follows:

[0075] Application 1: "When the temperature is higher than 26 degrees Celsius, turn on the cooling mode of the air conditioner."

[0076] Condition: / / Condition node, the content is that the temperature is higher than 26 degrees Celsius

[0077] cond.temperature|temperature|{"operator":">","value":{"range":"26"}}

[0078] Action: / / Action node, the content is the cooling service provided by the air conditioner (DevType: 012, ProdId: 100z)

[0079] 012|100z|tempACDeviceId|AC-MiAC_mode_setMode_2

[0080] Application 2: "When the temperature is below 20 degrees Celsius, turn on the heating mode of the air conditioner."

[0081] Condition: / / Condition node, the content is that the temperature is below 20 degrees Celsius

[0082] cond.temperature|temperature|{"operator":"<","value":{"range":"20"}}

[0083] Action: / / Action node, the content is the heating service provided by the air conditioner (DevType: 012, ProdId: 100z)

[0084] 012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0085] Application 3: "When the air conditioner is turned on, turn on the humidifier."

[0086] Event: / / Event node, the content is the turn-on service provided by the air conditioner (DevType: 012, ProdId: 100z)

[0087] 012|100z|tempACDeviceId|AC-MiAC_switchOnOff_on_1

[0088] Action: / / Action node, the content is the turn-on service provided by the humidifier (DevType: 01D, ProdId: 101X)

[0089] 01D|101X|tempHumidityDeviceId|humidity_POVOS_switchOnOff_on_1

[0090] Application 4: "When the mobile phone camera is turned on, turn on the flashlight."

[0091] Event: / / Event node, the content is the camera turn-on service provided by the mobile phone (DevType: X03, ProdId: mobileX03)

[0092] mobile|X03|mobileX03|phone-iPhoneX_camera_switch_on

[0093] Action: / / Action node, the content is the flashlight turn-on service provided by the mobile phone (DevType: X03, ProdId: mobileX03)

[0094] mobile|X03|mobileX03|phone-iPhoneX_flashlight_switch_on

[0095] Based on the above-extracted ECA information, the method constructs an ECA control network based on the control relationship, where the Condition information is represented as the conditional node Vc, and the Event and Action information are represented as the device service node Vs. The edges between the nodes are represented as the control relationship Ec. The following shows the node and edge information of the ECA network formed by this group of applications.

[0096] [Source]cond.temperature|temperature|{"operator":">","value":{"range":"26"}}

[0097] [Relation]Control

[0098] [Target]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_2

[0099] [Source]cond.temperature|temperature|{"operator":"<","value":{"range":"20"}}

[0100] [Relation]Control

[0101] [Target]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0102] [Source]012|100z|tempACDeviceId|AC-MiAC_switchOnOff_on_1

[0103] [Relation]Control

[0104] [Target]01D|101X|tempHumidityDeviceId|humidity_POVOS_switchOnOff_on_1

[0105] [Source]mobile|X03|mobileX03|phone-iPhoneX_camera_switch_on

[0106] [Relation]Control

[0107] [Target]mobile|X03|mobileX03|phone-iPhoneX_flashlight_switch_on

[0108] (2) After forming the ECA control network, it is necessary to supplement the semantic information of the ECA control network through the knowledge graph, supplement the dependency and mutual exclusion relationships between device services and the impact of device services on the physical environment, and form an automated application network.

[0109] While obtaining the device service serviceId through devType, prodId, capability, command, and value, this method also needs to obtain the impact of the device service on the physical environment. For example, in the graph data of ID3, the heating mode of the air conditioner will cause the temperature of the physical environment to rise. Therefore, after constructing the corresponding device service node, the method will also construct the corresponding physical environment node and add the impact edge. The following is the edge and node information of the impact of device services on the physical environment that need to be added for this group of applications.

[0110] [Source]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0111] [Relation]hasIncreased

[0112] [Target]temperature

[0113] [Source]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_2

[0114] [Relation]hasDecreased

[0115] [Target]temperature

[0116] [Source]01D|101X|tempHumidityDeviceId|humidity_POVOS_switchOnOff_on_1

[0117] [Relation]hasDecreased

[0118] [Target]humidity

[0119] Next, the method adds the relationships between device services, including mutually exclusive and dependency relationships. The method queries the relationships REL existing between different device services from the graph and adds the corresponding edge information to the graph. The following is the newly added edge information.

[0120] [Source]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0121] [Relation]Implied

[0122] [Target]012|100z|tempACDeviceId|AC-MiAC_switchOnOff_on_1

[0123] [Source]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_2

[0124] [Relation]Implied

[0125] [Target]012|100z|tempACDeviceId|AC-MiAC_switchOnOff_on_1

[0126] [Source]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_2

[0127] [Relation]Exclusive

[0128] [Target]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0129] [Source]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0130] [Relation]Exclusive

[0131] [Target]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_2

[0132] [Source]mobile|X03|mobileX03|phone-iPhoneX_camera_switch_on

[0133] [Relation]Exclusive

[0134] [Target]mobile|X03|mobileX03|phone - iPhoneX_flashlight_switch_on

[0135] [Source]mobile|X03|mobileX03|phone - iPhoneX_flashlight_switch_on

[0136] [Relation]Exclusive

[0137] [Target]mobile|X03|mobileX03|phone - iPhoneX_camera_switch_on

[0138] (3) Figure 6 The automated application network of this example is shown. Then, based on the defined conflict sub - graph patterns, the method performs pattern matching in the automated application network, detects automated conflicts of the corresponding type, and generates a detection report. The conflict categories include conflicts, loops, redundancies, and blocks. The specific sub - graph patterns are as Figure 5 shown. This set of applications has two types of conflicts, namely environmental loops and blocks. The specific edge and node information involved is as follows.

[0139] Environmental loop:

[0140] [Source]cond.temperature|temperature|{"operator":">","value":{"range":"26"}}

[0141] [Relation]Control

[0142] [Target]012|100z|tempACDeviceId|AC - MiAC_mode_setMode_2

[0143] [Source]012|100z|tempACDeviceId|AC - MiAC_mode_setMode_2

[0144] [Relation]hasDecreased

[0145] [Target]temperature

[0146] [Source]cond.temperature|temperature|{"operator":"<","value":{"range":"20"}}

[0147] [Relation]Control

[0148] [Target]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0149] [Source]012|100z|tempACDeviceId|AC-MiAC_mode_setMode_1

[0150] [Relation]hasIncreased

[0151] [Target]temperature

[0152] Blocking:

[0153] [Source]mobile|X03|mobileX03|phone-iPhoneX_camera_switch_on

[0154] [Relation]Control

[0155] [Target]mobile|X03|mobileX03|phone-iPhoneX_flashlight_switch_on

[0156] [Source]mobile|X03|mobileX03|phone-iPhoneX_camera_switch_on

[0157] [Relation]Exclusive

[0158] [Target]mobile|X03|mobileX03|phone-iPhoneX_flashlight_switch_on

[0159] Finally, based on the above detection results, the method generates a detection report, which includes the types of conflicts of this group of applications, the applications involved, and the information of the edges and nodes involved.

[0160] Another embodiment of the present invention provides a knowledge-based intelligent space system automated application conflict detection device, which includes:

[0161] An application analysis module, configured to map key elements of an automation application to an ECA control network, where the key elements include events, conditions, and actions;

[0162] An automation application network construction module, configured to supplement semantic information of the ECA control network through a knowledge graph, including dependencies and mutual exclusions between services of devices and the impact of services of devices on the physical environment, to form an automation application network;

[0163] A pattern matching module, configured to search and match the automation application network based on defined conflict patterns, and detect conflicts existing between automation applications in combination with semantic information.

[0164] For the specific implementation processes of each module, refer to the description of the method of the present invention above.

[0165] Based on the same inventive concept, another embodiment of the present invention provides a computer device (such as a computer, a server, a smart phone, etc.), which includes a memory and a processor. The memory stores a computer program, and the computer program is configured to be executed by the processor. The computer program includes instructions for executing each step in the method of the present invention.

[0166] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, a disk, an optical disc). The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, each step of the method of the present invention is implemented.

[0167] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. Those of ordinary skill in the art can understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification, and the protection scope of the present invention is subject to the scope defined by the claims.

Claims

1. A knowledge-based intelligent space system automatic application conflict detection method, characterized in that: The following steps are involved: Mapping key elements of automation applications to ECA control networks, including events, conditions, and behaviors; The knowledge graph is used to supplement the semantic information of the ECA control network, including the dependency and mutual exclusion relationship between device services and the impact of device services on the physical environment, to form an automated application network. The automation application network is searched and matched based on the defined conflict patterns, and the conflicts between automation applications are detected in combination with semantic information.

2. The method according to claim 1, characterized in that The knowledge graph is a knowledge graph that describes the relationship between devices and the environment in the IoT space. The steps of constructing the knowledge graph include: Establish an ontology model of the knowledge graph to describe the relationship between devices, capabilities, instructions, services, and environment; Collect and extract data, and fill the content of the knowledge graph through entity recognition and matching methods; 3. The method according to claim 2, characterized in that The relationship between the device, capabilities, instructions, services and environment includes: a device has multiple capabilities, each capability is implemented according to the corresponding instructions; the device provides services to the outside, and a service is composed of the specific capabilities and specific instructions of the device; there are two types of relationships between the services provided by the device, namely dependency and mutual exclusion. The dependency relationship means that the call of one device service depends on the call of another device service, and the mutual exclusion relationship means that two device services cannot occur at the same time; the relationship between the service and the environment is the increase and decrease of the impact of the device service on the environment.

4. The method according to claim 2, characterized in that: The data collection and extraction includes three steps of preprocessing, namely, stop word removal, word form restoration and entity matching.

5. The method according to claim 1, characterized in that The automation application network is represented by AG=(Vs, Vp, Vc, Ec, Ee, Ei, Ex), where: Vs: a group of IoT service vertices, which are composed of IoT services Si, where Si =<devType,prodId,deviceId,serviceId> , representing equipment type, product mark, equipment mark, and service mark respectively; Vp: a set of physical channel vertices, which represent the types of physical contexts in the smart home platform; Vc: a set of conditional vertices, which are nodes representing the physical factor constraints of the application. The conditional vertices are defined as<env,operator,args> , representing physical environment flags, operators, and parameters respectively; Ec: a set of control edges, which are directed edges flowing in the vertex set V = Vs ∪ Vp ∪ Vc, representing control relationships; Ee: a set of influence edges, each of which is a directed edge flowing from an IoT service vertex to a physical channel vertex, representing the impact of the IoT service on the corresponding physical channel, including positive or incremental effects and negative or degrading effects; Ei: a set of service dependency edges, each dependency edge is a directed edge connecting two IoT services; Ex: A set of service mutually exclusive edges, each service mutually exclusive edge is a directed or bidirectional edge connecting two IoT services, indicating that the two IoT services cannot be executed at the same time.

6. The method according to claim 1, characterized in that The conflict mode includes 4 conflict categories and 10 subgraph modes; the conflict categories include conflict, loop, redundancy and blocking.

7. The method according to claim 6, characterized in that A corresponding detection report is generated for the detected conflict, and the content of the detection report includes the corresponding conflict type, the automation application involved, and the information of the edges and nodes involved.

8. A knowledge-based intelligent space system automatic application conflict detection device, characterized in that: include: An application parsing module, used to map key elements of automation applications to the ECA control network, wherein the key elements include events, conditions, and behaviors; The automated application network building module is used to supplement the semantic information of the ECA control network through the knowledge graph, including the dependency and mutual exclusion relationship between the services of the equipment and the impact of the equipment's services on the physical environment, to form an automated application network; The pattern matching module is used to search and match the automation application network based on the defined conflict pattern, and detect the conflicts between automation applications in combination with semantic information.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images