An intelligent collaborative control method and system based on multi-source heterogeneous device access

Abstract

The application provides a kind of intelligent collaborative control method and system based on multi-source heterogeneous device access, belongs to industrial internet of things and intelligent control technical field.The method includes: obtaining the access information and running data of heterogeneous device, obtaining device capability representation information by protocol analysis and semantic mapping;Determine control response association according to device capability representation information and running data, and construct device collaborative relationship graph;According to the target state of collaborative control task and the current state of device, search candidate control path, determine target control sequence through executability verification and action conflict analysis, and issue control instruction;According to the actual state evolution process, verify the control result, and modify the device collaborative relationship graph and re-plan the unexecuted control action when the verification fails.The application aims to improve the problem that it is difficult to realize cross-device collaborative control and dynamic adjustment according to the actual state after heterogeneous device access.

Description

Technical Field

[0001] This application belongs to the field of industrial Internet of Things and intelligent control technology, and in particular relates to an intelligent collaborative control method and system based on the access of multi-source heterogeneous devices. Background Technology

[0002] With the development of the Industrial Internet of Things (IIoT), smart cities, and intelligent control technologies, an increasing number of sensing devices, controllers, and actuators are being connected to a unified control platform. These devices then work together to handle anomalies or adjust processes. Because different devices use different communication protocols, data formats, and control interfaces, existing technologies can utilize IoT gateways or protocol adaptation modules for protocol parsing and data conversion. Then, control commands are sent to the corresponding devices according to preset event triggering rules or fixed control procedures, thereby achieving unified access and basic linkage of multi-source heterogeneous devices.

[0003] However, when equipment operating status, available capacity, or inter-equipment response relationships change, fixed control flows struggle to determine executable alternative control paths and handle state dependencies and mutual influences between different control actions. Therefore, existing technologies still suffer from insufficient adaptability to cross-equipment collaborative control and inadequate closed-loop control capabilities. Summary of the Invention

[0004] This application provides an intelligent collaborative control method and system based on the access of multiple heterogeneous devices, which is used to improve the above-mentioned problems.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application provides an intelligent collaborative control method based on the access of multiple heterogeneous devices, the method comprising: The system acquires access information and operational data from multiple heterogeneous devices, performs protocol parsing on the access information, and maps the parsed device operation semantics and device status semantics to a preset device semantic space to obtain capability representation information for each heterogeneous device. Based on the capability characterization information and operational data of each heterogeneous device, the control response correlation between different device control actions and device state changes is determined, and a device collaboration relationship diagram is constructed based on the control response correlation. Obtain the collaborative control task, and search for candidate control paths that can achieve the target state in the device collaboration relationship graph based on the target state corresponding to the collaborative control task and the current state of each heterogeneous device. Perform executability verification and action conflict analysis on candidate control paths, determine the target control sequence, convert the control actions in the target control sequence into control instructions supported by the corresponding heterogeneous devices, and issue control instructions according to the target control sequence. Obtain the execution result data corresponding to the control command, verify the actual state evolution process represented by the execution result data based on the control response association, and when the actual state evolution process does not meet the expected state evolution process corresponding to the target control sequence, correct the equipment coordination relationship diagram, and re-plan the unexecuted control actions based on the corrected equipment coordination relationship diagram.

[0006] In one possible approach of the first aspect, the access information is parsed according to a protocol, and the parsed device operation semantics and device state semantics are mapped to a preset device semantic space, including: The protocol features of the interactive data sent by heterogeneous devices are identified, and the protocol parsing rules that match the identification results are invoked. Identify device operations and their corresponding status feedback from the interaction data according to the protocol parsing rules; Establish mapping relationships between device operation and status feedback and standard control actions and standard device states in the preset device semantic space.

[0007] In one possible approach of the first aspect, determining the control response correlation between different equipment control actions and equipment state changes includes: The operating data of multiple heterogeneous devices are time-aligned according to the execution time of the control action to obtain the device state evolution sequence before and after the execution of the control action; Identify the state change process that has a stable response relationship with the control action from the equipment state evolution sequence; Based on the evolution direction and response sequence of the state change process relative to the control action, a control response correlation is established between the control action and the state change process.

[0008] In one possible approach of the first aspect, a device collaboration graph is constructed based on control response association, including: Write the control actions that each heterogeneous device can perform and the state transitions caused by the control actions into the initial relationship diagram; When a state transition caused by a control action can change the execution conditions of another control action, a cooperative connection is established between the corresponding control actions. The reliability of collaborative connections is determined based on the consistency of responses during historical operation, and a device collaboration relationship graph is obtained.

[0009] In one possible approach of the first aspect, searching the device coordination graph for candidate control paths that can achieve the target state includes: Based on the current state of each heterogeneous device, determine the initial state position in the device collaboration relationship diagram; Perform a forward search from the initial state position along the state evolution direction, and a reverse search from the target state along the state evolution direction; Based on the intersection of positive and negative search results, candidate control paths for the evolution from the current state to the target state are determined.

[0010] In one possible approach of the first aspect, the executability verification of the candidate control path includes: Based on the execution conditions of the control actions in each candidate control path, determine whether the corresponding control action is executable in the current state of the heterogeneous device; Remove paths containing non-executable control actions from the candidate control paths; Candidate control sequences are generated according to the state transition order corresponding to each control action in the remaining candidate control paths.

[0011] In one possible approach of the first aspect, action conflict analysis is performed on candidate control paths to determine the target control sequence, including: Compare the state effects corresponding to different control actions in the candidate control sequences; When the state effect of one control action causes the execution conditions of another control action to fail, or causes the device state to deviate from the target state, it is determined that there is a conflict between the corresponding control actions. The execution order of conflicting control actions is adjusted based on the state dependencies between different control actions, and control actions without conflict are coordinated and arranged to obtain the target control sequence.

[0012] In one possible approach of the first aspect, control commands are issued according to the target control sequence, including: Based on the current state of the heterogeneous devices, determine the current control action in the target control sequence that meets the execution conditions; Convert the current control action into control commands supported by the target heterogeneous device and issue them; After the state change corresponding to the current control action satisfies the control response correlation, the subsequent control actions in the target control sequence are determined. If the state change corresponding to the current control action does not satisfy the control response association, the issuance of subsequent control actions will be paused.

[0013] In one possible approach of the first aspect, verifying the actual state evolution process represented by the execution result data based on the control response correlation includes: Based on the timing of the control command issuance, the execution result data obtained after executing the control command is time-aligned to generate the actual state evolution sequence; Generate the expected state evolution sequence corresponding to the control command based on the control response association; By comparing the consistency of state transitions and response timing between the actual state evolution sequence and the expected state evolution sequence, the execution verification results of the control commands are obtained.

[0014] In one possible approach of the first aspect, the device coordination diagram is revised, and unexecuted control actions are replanned based on the revised device coordination diagram, including: Determine the target state transition where the actual state evolution sequence first deviates from the expected state evolution sequence; Based on the position of the target state transition in the device collaboration relationship diagram, determine the abnormal device capabilities or abnormal collaboration connections, and correct the corresponding availability state or association credibility. Using the device states already reached in the actual state evolution sequence as the new initial states, a new control path to the target state is searched in the corrected device cooperation relationship graph. Generate alternative control sequences based on the control paths obtained from the re-search.

[0015] In one possible approach of the first aspect, the method is executed collaboratively by a collaborative control platform and an edge control node, and the method further includes: Synchronize the device collaboration relationship diagram verified by the collaborative control platform to the edge control node; When communication between the edge control node and the collaborative control platform is interrupted, the edge control node updates the current status of each heterogeneous device based on locally collected operational data. When the control path corresponding to the collaborative control task is within the control range of the edge control node, the edge control node generates and executes the target control sequence based on the local device collaborative relationship diagram. After communication is restored, the state evolution process generated during the communication interruption will be sent to the collaborative control platform.

[0016] In one possible approach of the first aspect, the method also includes: Based on the execution verification results of multiple control commands, determine the response deviation between the actual state evolution process and the expected state evolution process corresponding to the same control action; When the response deviation meets the relationship update condition, the corresponding control response association is corrected according to the actual state evolution process, and an updated equipment coordination relationship diagram is generated. Use historical collaborative control tasks to replay the control path in the updated device collaboration relationship diagram; When the replay of the state evolution process satisfies the corresponding target state, the updated device collaboration relationship diagram will be used for subsequent collaborative control tasks.

[0017] The intelligent collaborative control method based on multi-source heterogeneous device access provided in this application converts device operations and states under different protocols into unified capability representation information by parsing and semantically mapping the access information of heterogeneous devices. This enables the collaborative control process to perform cross-device processing based on the actual control capabilities of the devices. Furthermore, it determines the control response correlation between device control actions and device state changes based on device capability representation information and operational data, and constructs a device collaboration graph. This allows for the search of executable control paths by combining the target state of the collaborative control task with the current state of the devices, reducing the mismatch between fixed control flows and the actual operating state of the devices. On this basis, by performing executability verification and action conflict analysis on candidate control paths, the possibility of control processes failing due to device unavailability or mutual interference of control actions can be reduced. Verifying the control results through the actual state evolution process, and correcting the device collaboration graph and replanning unexecuted control actions when verification fails, ensures that subsequent control processes are aligned with the actual states already reached by the devices, thereby improving the adaptability, reliability, and closed-loop control capability of multi-source heterogeneous device collaborative control.

[0018] Secondly, this application provides an intelligent collaborative control system based on the access of multiple heterogeneous devices, comprising: The device access module is used to obtain access information and operation data of multiple heterogeneous devices, perform protocol parsing on the access information, and map the parsed device operation semantics and device status semantics to a preset device semantic space to obtain the capability representation information of each heterogeneous device. The relationship building module is used to determine the control response correlation between different device control actions and device state changes based on the capability characterization information and operation data of each heterogeneous device, and to build a device collaboration relationship diagram based on the control response correlation. The path planning module is used to obtain the collaborative control task and search for candidate control paths that can reach the target state in the device collaboration relationship graph based on the target state corresponding to the collaborative control task and the current state of each heterogeneous device. The collaborative control module is used to perform executability verification and action conflict analysis on candidate control paths, determine the target control sequence, convert the control actions in the target control sequence into control instructions supported by the corresponding heterogeneous devices, and issue control instructions according to the target control sequence. The feedback adjustment module is used to obtain the execution result data corresponding to the control command, verify the actual state evolution process represented by the execution result data according to the control response association, and correct the equipment coordination relationship diagram when the actual state evolution process does not meet the expected state evolution process corresponding to the target control sequence, and re-plan the unexecuted control actions based on the corrected equipment coordination relationship diagram.

[0019] Thirdly, this application provides a computer-readable storage medium including computer instructions; when the computer instructions are executed on an electronic device, they cause the electronic device to perform the method as described in any of the possible modes of the first aspect above.

[0020] Fourthly, this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform a method as described in any of the possible ways of the first aspect.

[0021] The technical effects of the second to fourth aspects refer to the technical effects of the first aspect and any of its methods, and will not be repeated here. Attached Figure Description

[0022] Figure 1 A schematic flowchart of the intelligent collaborative control method based on multi-source heterogeneous device access provided for an embodiment of this application; Figure 2 A second schematic flowchart of the intelligent collaborative control method based on multi-source heterogeneous device access provided for the embodiments of this application; Figure 3 The third schematic diagram of the steps of the intelligent collaborative control method based on multi-source heterogeneous device access provided for the embodiments of this application; Figure 4 A block diagram of a type of intelligent collaborative control system based on multi-source heterogeneous device access provided for embodiments of this application; Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0024] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0025] In the following description, the terms "first," "second," etc., are used for descriptive convenience only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more. For example, a plurality of processing circuits refers to two or more processing circuits.

[0026] In this application, the term "module" typically refers to a logically divided functional structure. A "module" can be implemented purely in hardware, or a combination of hardware and software. In this application, "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, or both A and B existing simultaneously.

[0027] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0028] With the development of the Industrial Internet of Things (IIoT), smart cities, and industrial safety monitoring technologies, an increasing number of sensing, control, and execution devices are being connected to unified equipment management platforms. Different devices can be used to collect environmental parameters, monitor equipment status, or execute control actions. When equipment malfunctions or environmental risks occur, multiple devices typically need to work collaboratively in a specific sequence. For example, upon detecting an abnormal gas concentration in a target area, alarms, ventilation, and valve control can be triggered in a coordinated manner to limit the further spread of the abnormal condition.

[0029] Since these devices may be provided by different manufacturers, their communication protocols, data encoding methods, and control interfaces typically differ. Related technologies utilize IoT gateways or protocol adaptation modules to parse device communication messages, converting different device data formats into a unified data format recognizable by the platform, and converting platform-generated control commands into protocol instructions supported by the corresponding devices. This method enables data access and command forwarding for multiple types of devices, but primarily addresses communication compatibility issues between devices and the platform. For different connected devices, the specific control actions they can perform, the necessary operating conditions for these actions, and the resulting device state changes are usually described separately in the interface documentation of each device, making it difficult to directly support cross-device collaborative control.

[0030] To achieve linkage between multiple devices, related technologies may pre-configure event trigger rules or control flows. When a certain device generates an alarm signal, the platform sequentially issues control instructions to associated devices according to the preset rules. This approach relies on pre-determined device relationships and a fixed control sequence. When the device type, access location or operating state changes, the original control flow may no longer be executable. For example, some devices in the control flow may be in an offline, faulty or inoperable state; different control actions may also act on the same controlled object, or change the device state on which other control actions depend. In this case, the fixed control flow is difficult to re-determine an executable cooperative control process according to the actual available state of the devices, and is also difficult to handle the mutual influence between different control actions.

[0031] In addition, there is usually not only an instruction execution sequence relationship between the control actions of different devices, but also a correlation in state response. The state change generated after one device executes a control action may constitute a trigger condition for another device to execute a subsequent control action, or may affect the operating state of other devices. Due to differences in the response time, state change process and data reporting period of different devices, controlling devices solely based on the instruction issuing order is prone to problems such as subsequent control actions being executed in advance, multiple control actions interfering with each other, or entering the next control stage before the device state is stabilized.

[0032] Related technologies usually judge whether a control instruction is successfully executed according to the instruction reception receipt returned by the device or a single status field. However, the instruction reception receipt can only indicate that the device has received the control instruction, and cannot directly indicate that the device has completed the corresponding action; the local state change of a single device is also difficult to indicate that the control action has produced the expected effect on the associated device or the target object. When the result returned by the device is inconsistent with the expectation, the existing control method is still difficult to determine whether the abnormality occurs in the instruction transmission, device execution or the cooperative response process between devices, and thus cannot adjust the unexecuted control actions accordingly.

[0033] Some device cooperative control processes also rely on a remote platform for unified decision-making. When the communication between the edge control node and the remote platform is interrupted, the remote platform cannot obtain the latest status of the on-site devices in time. Although the edge control node can perform some local operations according to the pre-stored control rules, the preset control rules are usually difficult to adapt to changes in device status and device availability during the communication interruption, which may lead to interruption of the control flow or mismatch between the control result and the actual on-site state.

[0034] Therefore, while related technologies can achieve unified access to multi-source heterogeneous devices and device linkage under preset rules, they still struggle to determine executable collaborative control processes based on the actual capabilities of the devices, cross-device state response relationships, and real-time operating status. Furthermore, they are difficult to verify the collaborative control results and adjust subsequent control processes based on actual state changes caused by control actions. Improving the adaptability and closed-loop control capabilities of cross-device collaborative control based on the access of multi-source heterogeneous devices remains a technical problem that needs to be solved.

[0035] To address the aforementioned issues, this application provides an intelligent collaborative control method and system based on the access of multiple heterogeneous devices. By parsing and semantically mapping the access information of heterogeneous devices, device capability representation information that can be used across protocols is obtained. Furthermore, a control response correlation between device control actions and device state changes is established by combining device operating data. This allows for the planning of collaborative control processes based on the state interaction relationships between different devices. Further, the control path is verified and adjusted according to the current device state, and the control results are validated using the actual state evolution process formed after the execution of control commands. This allows for the replanning of subsequent control actions when devices are unavailable, have abnormal state responses, or the original control path cannot continue, thereby improving the adaptability and closed-loop control capability of multi-source heterogeneous device collaborative control.

[0036] The intelligent collaborative control method based on multi-source heterogeneous device access provided in this application can be applied to equipment control scenarios requiring the collaborative work of multiple sensing devices, control devices, and execution devices, such as industrial production, park management, municipal facilities, security monitoring, fire protection linkage, or energy management. In specific applications, the accessed heterogeneous devices can adopt different communication protocols and respectively undertake status acquisition, anomaly detection, or control execution functions. When equipment control tasks or risk handling tasks are generated, the collaborative control process that can achieve the target state can be determined by combining the current state of each device and the control response correlation between devices.

[0037] The intelligent collaborative control method based on multi-source heterogeneous device access provided in this application can be executed by an electronic device with device access, data processing, and command control capabilities, such as a server, industrial computer, edge controller, or IoT gateway. Alternatively, it can be a collaborative control system composed of a collaborative control platform and one or more edge control nodes. The following description uses a collaborative control platform as the executing entity to illustrate the method provided in this application.

[0038] Figure 1 This is a flowchart illustrating an intelligent collaborative control method based on multi-source heterogeneous device access, provided as an embodiment of this application. See also... Figure 1 As shown, the method may include S101 to S105.

[0039] S101: Obtain access information and operation data of multiple heterogeneous devices, perform protocol parsing on the access information, and map the parsed device operation semantics and device status semantics to a preset device semantic space to obtain capability representation information of each heterogeneous device.

[0040] Heterogeneous devices refer to devices that differ in origin, communication protocol, or control method. Heterogeneous devices can be used to collect operational data from a target object or to execute control actions on that object. Multiple heterogeneous devices can be connected to a collaborative control platform, which will then manage them uniformly.

[0041] Access information is used to characterize the information required for data interaction between the collaborative control platform and heterogeneous devices. Operational data is used to characterize the actual operating status of heterogeneous devices or the objects they act upon. The collaborative control platform can obtain access information and operational data from heterogeneous devices, device access gateways, or edge control nodes.

[0042] Because different heterogeneous devices use different communication protocols, the operation or status of the same device may be expressed through different data fields, status codes, or command formats. Therefore, the collaborative control platform can perform protocol parsing on the access information according to the communication protocols used by the corresponding heterogeneous devices, and identify the device operation semantics and device status semantics from them.

[0043] Device operation semantics characterizes the operations that heterogeneous devices can perform, while device status semantics characterizes the states that heterogeneous devices can present or respond to during operation. For example, different devices can use different instructions to represent startup operations, or different status codes to indicate that the device has started. The collaborative control platform can map these different expressions to the same semantic in a preset device semantic space, enabling device operations and device status under different protocols to be processed in a unified manner.

[0044] After semantic mapping is completed, capability representation information for the heterogeneous device can be generated based on the device operation semantics and device state semantics corresponding to the heterogeneous device. The capability representation information is used to characterize the control actions that the heterogeneous device can perform and the device states associated with the corresponding control actions, thereby converting the native protocol capabilities of the heterogeneous device into unified capability information that can be used in subsequent collaborative control processes.

[0045] S102: Based on the capability characterization information and operating data of each heterogeneous device, determine the control response correlation between different device control actions and device state changes, and construct a device collaboration relationship diagram based on the control response correlation.

[0046] Specifically, the collaborative control platform can determine the device control actions that each heterogeneous device can perform based on the capability characterization information of the heterogeneous devices. For the device control actions that have already been executed, the changes in the device state before and after the execution of the device control action can be determined from the operational data.

[0047] Equipment state changes can be generated by the equipment itself or detected by other heterogeneous equipment. For example, after a heterogeneous equipment executes a control action, it can change its own operating state, which in turn causes other heterogeneous equipment acting on the same target object to detect the corresponding state change. Therefore, the control response correlation between different equipment control actions and equipment state changes can be determined based on the correspondence between equipment control actions and equipment state changes during the execution process.

[0048] Control response correlation is used to characterize the state changes that can be caused by executing a device control action. By using control response correlation, it is possible to determine the responses that the executing device itself or other heterogeneous devices may produce after a heterogeneous device executes a control action, thereby reflecting the cooperative relationship between different heterogeneous devices based on control actions and state changes.

[0049] The collaborative control platform can combine the control response associations of multiple heterogeneous devices to construct a device collaboration diagram. This diagram represents device control actions, device state changes, and the response relationships between them. A device state change caused by a single control action can serve as the state basis for subsequent control actions. Therefore, through the device collaboration diagram, a control path can be formed where one or more device control actions progressively drive changes in device state.

[0050] S103: Obtain the collaborative control task, and search for candidate control paths that can achieve the target state in the device collaboration relationship graph based on the target state corresponding to the collaborative control task and the current state of each heterogeneous device.

[0051] The collaborative control task is used to characterize the control objective that needs to be achieved through the collaboration of multiple heterogeneous devices. The collaborative control task can be generated based on abnormal operational data uploaded by the heterogeneous devices, or it can be sent by other control platforms or operating terminals.

[0052] The target state characterizes the device state or target object state that needs to be achieved after completing the collaborative control task. The current state of heterogeneous devices can be determined based on the operational data obtained in S101. By comparing the current state and the target state, the state transitions that the collaborative control task needs to complete can be determined.

[0053] Specifically, the collaborative control platform can map the current state of each heterogeneous device and the target state corresponding to the collaborative control task to a device collaboration relationship diagram. Subsequently, based on the control response associations in the device collaboration relationship diagram, it can search for control paths that enable heterogeneous devices to gradually transition from their current state to the target state.

[0054] A control path represents the equipment control actions and equipment state changes required to reach a target state. The same collaborative control task may have multiple implementation methods. For example, different heterogeneous devices may have the same or similar control capabilities, or they may achieve the same target state through different combinations of equipment control actions. Therefore, one or more control paths may be found in the equipment collaboration graph.

[0055] When multiple control paths are obtained, each path can be designated as a candidate control path. A candidate control path only indicates that the target state can be achieved according to the control response association in the device coordination diagram. Whether it can be executed under the current device operating conditions still needs to be further judged in conjunction with the current state of the heterogeneous devices.

[0056] S104: Perform executability verification and action conflict analysis on the candidate control path, determine the target control sequence, convert the control actions in the target control sequence into control instructions supported by the corresponding heterogeneous devices, and issue control instructions according to the target control sequence.

[0057] Specifically, the collaborative control platform can perform executability checks on the device control actions in candidate control paths based on the current state of each heterogeneous device. The executability check determines whether the corresponding heterogeneous device can execute the device control actions in the candidate control path under its current state. If the device control actions in the candidate control path cannot be executed by the corresponding heterogeneous device, then the candidate control path cannot reach the target state under the current operating conditions.

[0058] For candidate control paths that pass the executability verification, the collaborative control platform can further perform action conflict analysis. Action conflict analysis is used to determine whether different device control actions in the candidate control path will produce incompatible state changes during execution. If there are mutual influences between different device control actions, the execution relationship of the device control actions can be adjusted based on the control response correlation between the corresponding control actions and device state changes, or an executable control path can be determined from other candidate control paths.

[0059] The collaborative control platform can determine the target control sequence corresponding to this collaborative control task based on the executability verification results and action conflict analysis results. The target control sequence is used to characterize the control actions to be executed and the execution relationships between each control action.

[0060] The control actions in the target control sequence employ a unified semantic expression within a predefined device semantic space. However, the control commands actually received by different heterogeneous devices need to conform to the communication protocols used by those devices. Therefore, based on the heterogeneous devices and their access information corresponding to each control action, the control actions in the target control sequence can be converted into control commands supported by the respective heterogeneous devices.

[0061] After completing the control command conversion, the collaborative control platform can send control commands to the corresponding heterogeneous devices according to the execution relationship represented by the target control sequence, so that multiple heterogeneous devices can jointly complete the collaborative control task according to the target control sequence.

[0062] S105: Obtain the execution result data corresponding to the control command, verify the actual state evolution process represented by the execution result data according to the control response association, and when the actual state evolution process does not meet the expected state evolution process corresponding to the target control sequence, correct the equipment coordination relationship diagram, and re-plan the unexecuted control actions based on the corrected equipment coordination relationship diagram.

[0063] Among them, execution result data refers to data obtained after the control command is issued, which can characterize the execution result of heterogeneous devices or changes in device status. Execution result data can be returned by the heterogeneous device that performs the control action, or it can originate from other heterogeneous devices that have a control response association with the corresponding control action.

[0064] Specifically, after issuing control commands, the collaborative control platform can continue to acquire operational data from relevant heterogeneous devices and determine the actual state evolution process based on the device state changes resulting from the execution of control commands. The actual state evolution process is used to characterize the actual state transitions that occur after heterogeneous devices execute the target control sequence.

[0065] Based on the control actions in the target control sequence and the corresponding control responses, the expected state evolution process of the target control sequence can be determined. The expected state evolution process is used to characterize the state transitions that each heterogeneous device or target object should form when the target control sequence is executed normally.

[0066] The collaborative control platform can compare the actual state evolution process represented by the execution result data with the expected state evolution process corresponding to the target control sequence. If the actual state evolution process meets the expected state evolution process, it can be determined that the corresponding control action has formed the expected state change, and subsequent control actions can continue to be executed according to the target control sequence.

[0067] If the actual state evolution process does not meet the expected state evolution process, it indicates that the actual available state of the heterogeneous equipment or the control response relationship between different equipment may have changed. In this case, the equipment coordination relationship diagram can be revised based on the actual state changes reflected in the execution result data, so that the revised equipment coordination relationship diagram is adapted to the actual operating conditions of the heterogeneous equipment.

[0068] After revising the device coordination diagram, the completed control actions in the target control sequence can be retained. The current actual state of the heterogeneous devices is then used as the starting state for replanning. A new control path to the target state is searched in the revised device coordination diagram. Based on the re-searched control path, the unexecuted control actions in the target control sequence can be replanned, thus avoiding the duplication of completed control actions and ensuring that subsequent control processes can seamlessly connect to the current actual state.

[0069] The intelligent collaborative control method based on multi-source heterogeneous device access provided in this application can not only realize the unified access of devices with different protocols, but also plan cross-device control paths based on the control response correlation between device control actions and device state changes, and verify and adjust the collaborative control process according to the actual state evolution process after the execution of control commands, thereby improving the adaptability and closed-loop control capability of multi-source heterogeneous device collaborative control.

[0070] To achieve unified capability expression among devices with different protocols, in one feasible implementation, protocol parsing and semantic mapping of access information may include: identifying protocol features of interactive data sent by heterogeneous devices and determining protocol parsing rules that match the heterogeneous devices; identifying device operations and status feedback expressed by the interactive data according to the protocol parsing rules; and mapping the identified device operations and status feedback to standard control actions and standard device states in a preset device semantic space.

[0071] The protocol characteristics can be determined based on the registration and interaction process, message organization method, or interface description information of heterogeneous devices. For pre-supported communication protocols, the collaborative control platform can call the corresponding protocol adaptation component for parsing; for heterogeneous devices using custom communication protocols, protocol parsing rules can be configured based on the interface description information provided by the corresponding device, and the configured protocol parsing rules can be associated with the device identifier.

[0072] Protocol parsing rules are used to identify device operations and status feedback with actual control meaning from device interaction data, rather than simply converting the interaction data format. For example, different heterogeneous devices may use different control fields to represent startup operations, or different status codes to indicate that the device has entered the running state. The collaborative control platform can map different protocol expressions to the same semantics in a preset device semantic space according to the actual meaning of the corresponding device operations or status feedback.

[0073] After semantic mapping is completed, standard control actions and standard device states can be mapped to corresponding heterogeneous devices, forming capability representation information for those heterogeneous devices. This allows subsequent collaborative control processes to determine the control actions that heterogeneous devices can execute and the device states they can report based on this capability representation information, without needing to reinterpret the native protocols used by the heterogeneous devices each time control path planning is performed.

[0074] To identify the state-to-state relationships between different devices from actual operational data, in one feasible implementation, determining control response associations may include: time-aligning the operational data of multiple heterogeneous devices based on the execution time of the device control action; determining the device state before the execution of the device control action, and forming a device state evolution sequence based on the operational data obtained after the execution of the control action; and determining the state change process with a stable response relationship to the control action based on the device state evolution sequence formed by executing the same control action multiple times.

[0075] Specifically, the collaborative control platform can use the execution time of the control action as the benchmark for state analysis, determine the initial state of the equipment from the operating data before the corresponding execution time, and determine the changes in the equipment state over time from the operating data after the execution time. Thus, a state evolution sequence from the initial state of the equipment to subsequent states can be obtained.

[0076] When the state of a device changes in a similar direction after multiple executions of the same control action, and there is a stable temporal relationship between the corresponding state changes and the control actions, it can be considered that the state change is caused by the corresponding control action. The collaborative control platform can establish a control response association between the control action and the state change process.

[0077] Control response correlations can characterize state changes generated by the executing device itself, as well as state changes generated by other heterogeneous devices. For example, after the first heterogeneous device executes a control action, its own operating state changes; a second heterogeneous device acting on the same target object as the first heterogeneous device detects that the operating state of the target object has changed accordingly. In this case, control response correlations can be established separately between the control action of the first heterogeneous device and its own state change, and between the control action of the first heterogeneous device and the state change detected by the second heterogeneous device.

[0078] In some embodiments, for a state change that occurs only once, a candidate response relationship between the control action and the state change can be temporarily retained; during subsequent operation, when the candidate response relationship reappears and meets the preset response conditions, it is then determined as a control response association. This reduces the impact of occasional data fluctuations on the identification results of device collaboration relationships.

[0079] The following explains the specific process of constructing the device collaboration diagram. (See attached document.) Figure 2 It may include S201 to S204.

[0080] S201: Based on the capability characterization information of each heterogeneous device, determine the device control actions and their corresponding device states.

[0081] Specifically, the standard control actions supported by each heterogeneous device can be read from the capability characterization information of each heterogeneous device, and the device state involved in executing the corresponding standard control action can be determined. The device state can include the state before executing the control action and the expected state after executing the control action.

[0082] For example, when a heterogeneous device can only perform a startup action while in standby mode, the standby mode can be considered the pre-action state corresponding to the startup action, and the operating state formed after the startup action is executed can be considered the post-action state. By determining the pre-action and post-action states corresponding to a control action, the device control action can be converted into a transition process between device states.

[0083] S202: Based on the control response association, write the device control actions and the resulting state transitions into the initial relationship diagram.

[0084] Specifically, for each device control action, the state change that the control action can cause can be determined based on the corresponding control response association, and the device control action, the state before the action, and the state after the action are written into the initial relationship diagram.

[0085] When the same device control action can cause changes in the state of multiple devices, the corresponding state changes can be written into the initial relationship diagram, so that the initial relationship diagram can simultaneously represent the impact of the control action on the state of the executing device itself and the state of other heterogeneous devices.

[0086] At this point, the initial relationship diagram can represent the state transition corresponding to a single control action, but a cooperative relationship that can be executed continuously has not yet been formed between different control actions.

[0087] S203: Establish a collaborative connection between device control actions based on the state transitions corresponding to different device control actions.

[0088] Specifically, the post-action state corresponding to the first control action can be compared with the pre-action state corresponding to the second control action. When the post-action state formed by the first control action can satisfy the execution conditions of the second control action, it can be determined that there is a sequential cooperative relationship between the first and second control actions, and a cooperative connection from the first control action to the second control action can be established.

[0089] When the state change caused by the first control action alters the execution conditions of the second control action but does not directly satisfy those conditions, a cooperative connection can be established between the first and second control actions to characterize the state influence relationship between them. During subsequent control path planning, this state influence relationship can be used to determine whether the second control action is still executable.

[0090] In this way, control actions belonging to different heterogeneous devices can be connected, so that the control actions of multiple heterogeneous devices can form a continuous state evolution process.

[0091] S204: Determine the association credibility based on the historical response data corresponding to the collaborative connection, and obtain the device collaborative relationship diagram.

[0092] Specifically, it is possible to query the control actions and state changes corresponding to the collaborative connection during its historical operation, and determine whether the state changes on which the second control action depends occur in accordance with the control response after the first control action is executed.

[0093] If the corresponding state change can be formed in multiple control processes and can support the continued execution of the second control action, the reliability of the collaborative connection can be improved. If the expected state change is not formed after the first control action is executed, or if the formed state change cannot support the execution of the second control action, the reliability of the collaborative connection can be reduced.

[0094] Association reliability reflects the degree to which a state change resulting from a control action can stably support subsequent control actions. After determining the association reliability of each cooperative connection, a device cooperation relationship graph can be obtained. During subsequent control path search, the association reliability of cooperative connections can be combined to select the control path with higher execution stability from multiple control paths that can reach the target state.

[0095] To determine the control path based on the collaborative control task, in one feasible implementation, searching for candidate control paths in the device collaboration graph may include: determining the initial state position based on the current state of each heterogeneous device, determining the target state position based on the target state corresponding to the collaborative control task; performing a forward search from the initial state position along the evolution direction of the device state, and a reverse search from the target state position along the evolution direction of the device state; and determining the candidate control path based on the intersection relationship between the forward search results and the reverse search results.

[0096] The forward search is used to determine the control actions that can be performed starting from the current state of the heterogeneous device, and the device state that can be reached after performing the corresponding control actions. The backward search is used to determine the device states that need to be passed through to reach the target state, and the control actions that can form the corresponding device state.

[0097] When the device states in the forward search results and the device states in the reverse search results are interconnected, the continuous path formed by the corresponding control actions and device states can be determined as a candidate control path. By performing forward and reverse searches simultaneously, the traversal of device control actions unrelated to the current or target state can be reduced, thus improving the efficiency of candidate control path determination.

[0098] When multiple heterogeneous devices can produce the same or similar state changes, different candidate control paths can be formed for each. For example, if the first heterogeneous device is currently unavailable, another candidate control path can be formed by a second heterogeneous device with corresponding control capabilities, thus providing an alternative for the subsequent control process.

[0099] To exclude candidate control paths that cannot be executed under the current operating conditions, in one feasible implementation, the executability of the candidate control paths can be verified, which may include: determining the execution conditions of each control action in sequence according to the state evolution order corresponding to the candidate control path; and judging whether the execution conditions of each control action can be met based on the current state of the heterogeneous device or the state formed by the preceding control action.

[0100] If the execution conditions of a control action can be directly satisfied by the current state of the heterogeneous device, then the control action can be determined to be currently executable. If the execution conditions of a control action need to be formed by a preceding control action, then the corresponding execution conditions can be determined based on the control response associations of the preceding control actions to determine whether they can be formed before the execution of the control action.

[0101] If any control action in a candidate control path cannot obtain the device state required for its execution, the candidate control path can be determined to be unexecutable under the current operating conditions, and the candidate control path can be deleted. For candidate control paths that pass the executability check, a candidate control sequence can be generated according to the state transition order corresponding to each control action.

[0102] To reduce the mutual influence between different control actions, in one feasible implementation, action conflict analysis can be performed on different control actions in the candidate control sequence. Specifically, the state effects corresponding to different control actions can be compared. When the state change caused by the first control action invalidates the execution conditions of the second control action, or causes the device state to deviate from the target state corresponding to the cooperative control task, an action conflict can be determined between the first and second control actions.

[0103] For control actions that conflict, the execution order can be adjusted based on the state dependency between them. When the action conflict cannot be eliminated by adjusting the execution order, other control paths that do not contain the corresponding action conflict can be selected from the candidate control paths.

[0104] For multiple control actions that do not conflict and have no state dependencies on each other, they can be programmed into cooperative control actions. If a subsequent control action depends on the result of a preceding control action, the preceding control action can be executed first. Based on the adjusted control actions and their execution relationships, the target control sequence can be obtained.

[0105] In order to keep the control command issuance process consistent with the actual changes in the device state, in one feasible implementation, the current control action that meets the execution conditions can be determined from the target control sequence based on the current state of the heterogeneous device, and the current control action can be converted into a control command supported by the corresponding heterogeneous device according to the communication protocol used by the heterogeneous device executing the control action.

[0106] After issuing control commands, the collaborative control platform can continue to acquire operational data from relevant heterogeneous devices and determine whether the current control action has resulted in the expected state change based on the correlation of control responses. When the expected state change has occurred, it can be determined that the current control action has been completed, and subsequent control actions in the target control sequence can be determined.

[0107] When the expected state change has not yet occurred, we can continue to wait for state feedback from the corresponding heterogeneous devices. If the expected state change still has not occurred after a response time that matches the control response, we can pause issuing subsequent control actions. This avoids using fixed waiting times to control multiple heterogeneous devices and reduces the possibility of executing subsequent control actions before the preceding devices have reached their corresponding states.

[0108] To improve the accuracy of control result verification, in one feasible implementation, verifying the actual state evolution process based on control response correlation may include: time-aligning the execution result data obtained after the control command is issued to generate an actual state evolution sequence based on the timing of the control command issuance; generating an expected state evolution sequence based on the control response correlations corresponding to each control action in the target control sequence; and comparing the consistency of state transitions and response timing between the actual state evolution sequence and the expected state evolution sequence to obtain the execution verification result.

[0109] State transition consistency characterizes whether the actual equipment state change matches the expected equipment state change. Response timing consistency characterizes whether the corresponding equipment state change occurs within the response time corresponding to the control action.

[0110] If the device returns a message indicating successful execution of the control command, but no corresponding state change occurs in the actual state evolution sequence, it can be determined that the actual state evolution process does not meet the expected state evolution process. Therefore, the control result is no longer judged solely based on whether the control command is received by the device, but can be further verified by whether the device's control actions result in the expected state effect.

[0111] The following explains the process of revising the equipment coordination diagram and replanning control actions. (See reference...) Figure 3 It may include S301 to S305.

[0112] S301: Determine the target state transition where the actual state evolution sequence deviates from the expected state evolution sequence for the first time.

[0113] Specifically, the actual state evolution sequence and the expected state evolution sequence can be compared segment by segment according to the execution order of each control action in the target control sequence. When the actual state evolution sequence does not form the expected state transition corresponding to the corresponding control action, or when the time for forming the corresponding state transition does not meet the control response correlation, this state transition can be identified as the first deviation from the target state transition.

[0114] By identifying the target state transition where the first deviation occurs, the scope of anomaly analysis can be limited to the corresponding control action and its subsequent state response process, thus avoiding identifying control actions that have already been completed normally as anomalies.

[0115] S302: Determine the abnormal device capability or abnormal cooperative connection based on the position of the target state transition in the device coordination relationship diagram.

[0116] Specifically, the control actions and collaborative connections that determine the target state transition can be queried in the device collaboration relationship diagram. When a heterogeneous device executes a control action but fails to generate the expected state change corresponding to that control action, the device capability of that heterogeneous device can be identified as an abnormal device capability.

[0117] When the executing device itself has formed the expected state change, but the state change does not further cause other heterogeneous devices to generate state changes that are consistent with the control response, the cooperative connection that connects the corresponding control action or device state can be identified as an abnormal cooperative connection.

[0118] By mapping the deviations in the actual state evolution sequence to the equipment coordination relationship diagram, coordination control anomalies can be located at the equipment capabilities or the coordination relationships between equipment.

[0119] S303: Correct the device collaboration relationship diagram based on the abnormal location results.

[0120] Specifically, when an abnormal device capability is identified, the availability of that capability can be reduced, causing subsequent control path search processes to either no longer select or reduce the priority of selecting that device capability. When an abnormal cooperative connection is identified, the association reliability of that cooperative connection can be reduced to reflect that the cooperative connection is currently unable to stably form the expected state change.

[0121] For state transitions that have been completed in the actual state evolution sequence and are consistent with the expected state evolution sequence, the current state of the corresponding equipment capabilities and cooperative connections can be retained, so that the correction range of the equipment cooperative relationship diagram corresponds to the actual anomaly location.

[0122] S304: Determine the current actual state of the heterogeneous device based on the actual state evolution sequence.

[0123] Specifically, the most recently verified device state of each heterogeneous device can be extracted from the actual state evolution sequence, and this device state can be determined as the current state to be used when replanning control actions.

[0124] When some control actions in the target control sequence have been completed, the device state formed by the corresponding control actions can serve as the state basis for replanning subsequent control actions, without having to restore all heterogeneous devices to their initial state before the target control sequence was executed.

[0125] S305: Using the current actual equipment state as the new initial state, re-plan the unexecuted control actions.

[0126] Specifically, the current actual device state of each heterogeneous device can be mapped to the corrected device collaboration relationship diagram, and the corresponding device state can be used as the new initial state position to re-search for control paths that can achieve the target state corresponding to the collaborative control task.

[0127] When a new control path is found, an alternative control sequence can be generated based on the new control path. The alternative control sequence is used to replace the control actions that have not yet been executed in the original target control sequence. Control actions that have been completed and verified in the original target control sequence do not need to be executed again.

[0128] When a substitute capability exists for the malfunctioning device, the new control path can select another heterogeneous device to perform the corresponding control action. When no heterogeneous device can directly replace the malfunctioning device's capability, a control path can be searched to reach the target state through other state evolution processes. This allows the redesigned control process to connect with the actual states already reached by the heterogeneous devices and reduces the possibility of a single heterogeneous device malfunction causing the entire collaborative control process to terminate.

[0129] To maintain field control capabilities in the event of communication failures on the collaborative control platform, one feasible implementation involves synchronizing a device collaboration relationship diagram verified by the collaborative control platform to the edge control node. The edge control node can then store device collaboration relationships related to field devices based on locally connected heterogeneous devices and their control ranges.

[0130] When communication between the edge control node and the collaborative control platform is interrupted, the edge control node can continue to acquire the operating data of local heterogeneous devices and use the operating data to update the current state of the heterogeneous devices. If all heterogeneous devices involved in the collaborative control task can be accessed by the edge control node, and the control path from the current state to the target state can be determined based on the local device collaboration relationship graph, then the collaborative control task can be considered to be within the control range of the edge control node.

[0131] Edge control nodes can generate and execute target control sequences based on local device collaboration diagrams, and record control commands and corresponding state evolution processes according to the execution order of control actions. After communication is restored, the records generated during the communication interruption can be sent to the collaborative control platform, enabling the platform to resume continuous management of field device status and control processes based on the corresponding records.

[0132] In order to adapt the control response to the performance changes that occur during the long-term operation of heterogeneous devices, in one feasible implementation, the response deviation between the actual state evolution process and the expected state evolution process corresponding to the same control action can be determined based on the execution verification results of multiple control commands.

[0133] When a response deviation occurs only in a single control process, it can be recorded as an abnormal execution result. When the same or similar response deviations occur continuously in multiple control processes, it can be determined that the corresponding control response association can no longer accurately represent the current state interaction relationship of heterogeneous devices, and the response deviation can be judged to meet the relationship update condition.

[0134] The collaborative control platform can correct the corresponding control response associations based on the actual state evolution process, and generate an updated equipment collaboration diagram based on the corrected control response associations. Before using the updated equipment collaboration diagram for actual collaborative control, control paths can be replayed using historical collaborative control tasks to determine whether the control paths generated based on the updated equipment collaboration diagram can still enable the equipment to reach the corresponding target state.

[0135] When the replay of the state evolution process satisfies the corresponding target state, the updated device coordination diagram can be used for subsequent coordinated control tasks. When the replay result does not satisfy the corresponding target state, the original device coordination diagram can be retained, and the operational data of heterogeneous devices can continue to be collected. This allows for adaptation to changes in device operating characteristics while reducing the possibility of occasional abnormal data causing erroneous corrections to the device coordination diagram.

[0136] Figure 4 This is a block diagram of an intelligent collaborative control system based on the access of multiple heterogeneous devices, provided as an embodiment of this application. Figure 4 As shown, the intelligent collaborative control system 400 based on multi-source heterogeneous device access includes: The device access module 401 is used to obtain access information and operation data of multiple heterogeneous devices, perform protocol parsing on the access information, and map the parsed device operation semantics and device status semantics to a preset device semantic space to obtain the capability representation information of each heterogeneous device. The relationship building module 402 is used to determine the control response association between different device control actions and device state changes based on the capability characterization information and operation data of each heterogeneous device, and to build a device collaboration relationship diagram based on the control response association; The path planning module 403 is used to obtain the collaborative control task and search for candidate control paths that can reach the target state in the device collaboration relationship graph based on the target state corresponding to the collaborative control task and the current state of each heterogeneous device. The collaborative control module 404 is used to perform executability verification and action conflict analysis on candidate control paths, determine the target control sequence, convert the control actions in the target control sequence into control instructions supported by the corresponding heterogeneous devices, and issue control instructions according to the target control sequence. The feedback adjustment module 405 is used to obtain the execution result data corresponding to the control command, verify the actual state evolution process represented by the execution result data according to the control response association, and correct the equipment coordination relationship diagram when the actual state evolution process does not meet the expected state evolution process corresponding to the target control sequence, and re-plan the unexecuted control actions based on the corrected equipment coordination relationship diagram.

[0137] The intelligent collaborative control system based on multi-source heterogeneous device access provided in this application embodiment can execute the methods shown in the above method embodiments. Its implementation principle and beneficial effects can be found in the relevant descriptions in the method embodiments, and will not be repeated here. Furthermore, each module in the above-mentioned intelligent collaborative control system based on multi-source heterogeneous device access can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0138] In other embodiments, an electronic device is provided, which can be an intelligent collaborative control device for multi-source heterogeneous device access in the above method embodiments, used to execute the method steps performed by the terminal in the above method flow. The internal structure diagram of this electronic device can be as follows: Figure 5 As shown, the device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an intelligent collaborative control method for multi-source heterogeneous device access. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the electronic device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the electronic device, or external keyboards, touchpads, or mice, etc.

[0139] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0140] This embodiment also provides a computer-readable storage medium storing computer instructions that, when executed on an electronic device, cause the electronic device to perform the various functions or steps described in the above method embodiments.

[0141] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps in the above method embodiments.

[0142] In this embodiment, the electronic device, computer-readable storage medium, and computer program product are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A smart collaborative control method based on the access of multi-source heterogeneous devices, characterized in that, include: The system acquires access information and operation data of multiple heterogeneous devices, performs protocol parsing on the access information, and maps the parsed device operation semantics and device state semantics to a preset device semantic space to obtain capability representation information of each heterogeneous device. Based on the capability characterization information and operating data of each heterogeneous device, the control response correlation between different device control actions and device state changes is determined, and a device collaboration relationship diagram is constructed based on the control response correlation. Obtain the collaborative control task, and search for candidate control paths that can reach the target state in the device collaboration relationship graph based on the target state corresponding to the collaborative control task and the current state of each heterogeneous device; The candidate control path is subjected to executability verification and action conflict analysis to determine the target control sequence. The control actions in the target control sequence are converted into control instructions supported by the corresponding heterogeneous devices, and the control instructions are issued according to the target control sequence. Obtain the execution result data corresponding to the control command, verify the actual state evolution process represented by the execution result data according to the control response association, and when the actual state evolution process does not meet the expected state evolution process corresponding to the target control sequence, correct the device coordination relationship diagram, and re-plan the unexecuted control actions based on the corrected device coordination relationship diagram.

2. The method according to claim 1, characterized in that, The step of parsing the access information according to the protocol and mapping the parsed device operation semantics and device state semantics to a preset device semantic space includes: The protocol features of the interactive data sent by the heterogeneous devices are identified, and the protocol parsing rules that match the identification results are invoked. Based on the protocol parsing rules, identify device operations and their corresponding status feedback from the interaction data; The device operation and the status feedback are respectively mapped to the standard control actions and standard device states in the preset device semantic space.

3. The method according to claim 1, characterized in that, The determination of the control response correlation between different equipment control actions and equipment state changes includes: The operating data of multiple heterogeneous devices are time-aligned according to the execution time of the control action to obtain the device state evolution sequence before and after the execution of the control action; Determine the state change process that has a stable response relationship with the control action from the device state evolution sequence; Based on the evolution direction and response timing of the state change process relative to the control action, a control response association is established between the control action and the state change process.

4. The method according to claim 3, characterized in that, The construction of the device collaboration relationship diagram based on the control response association includes: Write the control actions that each of the heterogeneous devices can perform and the state transitions caused by the control actions into the initial relationship diagram; When a state transition caused by a control action can change the execution conditions of another control action, a cooperative connection is established between the corresponding control actions. The reliability of the collaborative connection is determined based on the consistency of the responses during the historical operation of the collaborative connection, and the device collaborative relationship graph is obtained.

5. The method according to claim 1, characterized in that, The step of searching for candidate control paths that can achieve the target state in the device collaboration graph includes: Based on the current state of each heterogeneous device, determine the initial state position in the device collaboration relationship diagram; A forward search is performed from the initial state position along the state evolution direction, and a reverse search is performed from the target state along the state evolution direction. Based on the intersection of forward and reverse search results, candidate control paths for the evolution from the current state to the target state are determined.

6. The method according to claim 5, characterized in that, The executability verification of the candidate control path includes: Based on the execution conditions of the control actions in each candidate control path, determine whether the corresponding control action is executable in the current state of the heterogeneous device; Remove paths containing non-executable control actions from the candidate control paths; Candidate control sequences are generated according to the state transition order corresponding to each control action in the remaining candidate control paths.

7. The method according to claim 6, characterized in that, The step of performing action conflict analysis on the candidate control paths to determine the target control sequence includes: Compare the state action results corresponding to different control actions in the candidate control sequences; When the state effect of one control action causes the execution conditions of another control action to fail, or causes the device state to deviate from the target state, it is determined that there is a conflict between the corresponding control actions. The execution order of conflicting control actions is adjusted according to the state dependencies between different control actions, and control actions without conflict are coordinated and arranged to obtain the target control sequence.

8. The method according to claim 7, characterized in that, The step of issuing the control command according to the target control sequence includes: Based on the current state of the heterogeneous device, determine the current control action in the target control sequence that meets the execution conditions; The current control action is converted into a control command supported by the target heterogeneous device and then issued. After the state change corresponding to the current control action satisfies the control response association, the subsequent control actions in the target control sequence are determined; If the state change corresponding to the current control action does not satisfy the control response association, the issuance of subsequent control actions shall be suspended.

9. The method according to claim 1, characterized in that, The verification of the actual state evolution process represented by the execution result data based on the control response association includes: Based on the issuance time of the control command, the execution result data obtained after executing the control command is time-aligned to generate an actual state evolution sequence; Based on the control response, a predicted state evolution sequence corresponding to the control command is generated; By comparing the consistency of state transitions and response timing between the actual state evolution sequence and the expected state evolution sequence, the execution verification results of the control command are obtained.

10. An intelligent collaborative control system based on the access of multi-source heterogeneous devices, characterized in that, include: The device access module is used to obtain access information and operation data of multiple heterogeneous devices, perform protocol parsing on the access information, and map the parsed device operation semantics and device status semantics to a preset device semantic space to obtain capability representation information of each heterogeneous device. The relationship building module is used to determine the control response association between different device control actions and device state changes based on the capability characterization information and operating data of each heterogeneous device, and to build a device collaboration relationship diagram based on the control response association; The path planning module is used to obtain the collaborative control task and search for candidate control paths that can reach the target state in the device collaboration relationship graph based on the target state corresponding to the collaborative control task and the current state of each heterogeneous device. The collaborative control module is used to perform executability verification and action conflict analysis on the candidate control path, determine the target control sequence, convert the control actions in the target control sequence into control instructions supported by the corresponding heterogeneous devices, and issue the control instructions according to the target control sequence. The feedback adjustment module is used to obtain the execution result data corresponding to the control command, verify the actual state evolution process represented by the execution result data according to the control response association, and when the actual state evolution process does not meet the expected state evolution process corresponding to the target control sequence, correct the device coordination relationship diagram, and re-plan the unexecuted control actions based on the corrected device coordination relationship diagram.

Images