Communication method and system between devices and electronic device

Abstract

The invention relates to the technical field of Internet of Things communication, and discloses an inter-device communication method and system and an electronic device.The method comprises the steps that a first Internet of Things device establishes communication connection with a plurality of second Internet of Things devices, the first Internet of Things device has the natural language generation capacity, and the second Internet of Things device has the natural language generation capacity; the second Internet of Things equipment has a natural language understanding capability; based on the current control intention, generating a first natural language instruction for at least one second Internet of Things device; and sending the corresponding first natural language instruction to the at least one second Internet of Things device, so that the at least one second Internet of Things device understands and executes the corresponding first natural language instruction. Through the above mode, accurate and effective communication between devices can be realized.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things communication technology, and in particular to a method, system and electronic device for inter-device communication. Background Art

[0002] With the rapid development of Internet of Things technology, smart devices have been widely used in home, transportation, urban management and other fields.

[0003] At present, IoT devices mainly establish communication connections through communication protocols such as WIFI, Bluetooth, Zigbee, and 5G. However, even if communication connections are established between IoT devices, the control instructions generated by the IoT devices at the control end cannot accurately reflect the control requirements, and the IoT devices at the controlled end cannot accurately understand the control requirements, resulting in abnormal communication between the IoT devices at the control end and the IoT devices at the controlled end, that is, the IoT devices at the control end cannot accurately and effectively control the IoT devices at the controlled end. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a method, system and electronic device for communication between devices, which can realize accurate and effective communication between devices.

[0005] To solve the above technical problems, the first aspect of the present application provides a method for communication between devices, which method includes: a first Internet of Things device establishes communication connections with several second Internet of Things devices respectively, wherein the first Internet of Things device has natural language generation capability and the second Internet of Things device has natural language understanding capability; based on the current control intention, a first natural language instruction is generated to at least one second Internet of Things device; and corresponding first natural language instructions are sent to at least one second Internet of Things device respectively, so that at least one second Internet of Things device understands and executes the corresponding first natural language instruction.

[0006] Among them, before generating a first natural language instruction to at least one second Internet of Things device based on the current control intention, the device communication method also includes: obtaining a device capability description of the second Internet of Things device, and determining the functions that can be implemented by the second Internet of Things device based on the device capability description; based on the current control intention, generating a first natural language instruction to at least one second Internet of Things device, including: based on the current control intention, generating a first natural language instruction that matches the function of at least one second Internet of Things device.

[0007] Among them, the first Internet of Things device establishes communication connections with several second Internet of Things devices respectively, including: scanning the broadcast message of the second Internet of Things device; sending a service query to the second Internet of Things device; receiving the query response fed back by the second Internet of Things device, wherein the query response includes the connection information of the second Internet of Things device; and establishing a communication connection with the second Internet of Things device based on the connection information.

[0008] Among them, the broadcast message includes a device capability description, and the device capability description is used to determine the functions that the second Internet of Things device can implement; and / or, the broadcast message includes a service type; before sending a service query to the second Internet of Things device, the device communication method also includes: based on the service type, determining whether the second Internet of Things device has natural language understanding capabilities; in response to the second Internet of Things device having natural language understanding capabilities, executing sending a service query to the second Internet of Things device.

[0009] The device capability description includes the functions supported by the device, and also includes at least one of the device type, function parameter range and current status; and / or the device capability description is described in natural language, and the first IoT device has natural language understanding capability and can perform natural language understanding on the device capability description to determine the functions that the second IoT device can implement.

[0010] Among them, before generating a first natural language instruction to at least one second Internet of Things device based on the current control intention, the device communication method also includes: receiving control requirement information, wherein the control requirement information is input by the user or sent by a third Internet of Things device, and / or the control requirement information is text, voice or image; analyzing the control requirement information to determine the current control intention.

[0011] Among them, after establishing communication connections with several second Internet of Things devices respectively, the device communication method also includes: receiving the current status of the second Internet of Things device; executing at least one of the following steps: saving the current status; displaying the current status; and feeding back the current status to the user or the fourth Internet of Things device.

[0012] Receiving the current state of the second IoT device includes at least one of the following steps: sending a query request to the second IoT device, receiving first state information sent by the second IoT device, parsing the first state information to obtain the current state, wherein at least one of the query request and the first state information is described in natural language; receiving second state information sent by the second IoT device and described in natural language, parsing the second state information to obtain the current state, wherein the second state information is sent by the second IoT device when it detects that its own state has changed.

[0013] Among them, sending a query request to the second Internet of Things device includes: receiving query demand information, wherein the query demand information is input by a user or sent by a fourth Internet of Things device, and / or the query demand information is text, voice or image; analyzing the query demand information to determine the current query intent; and generating a query request for the second Internet of Things device based on the current query intent.

[0014] Among them, the first Internet of Things device has natural language understanding capability, and the second Internet of Things device has natural language generation capability; after sending the corresponding first natural language instructions to at least one second Internet of Things device respectively, the device communication method also includes: receiving a natural language response sent by at least one second Internet of Things device, wherein the natural language response represents the response result of the second Internet of Things device to the first natural language instruction; performing natural language understanding on each natural language response respectively to obtain the response result of each first natural language instruction; executing at least one of the following steps: based on the response result, determining whether to re-send the first natural language instruction to the corresponding second Internet of Things device; based on the response result, generating natural language information, and natural language feedback information fed back to the control demander, wherein the current control intention is determined based on the control demand information provided by the control demander, and the control demander is a user or a third Internet of Things device; based on the response result, generating a second natural language instruction to at least one second Internet of Things device, and sending the corresponding second natural language instruction to at least one second Internet of Things device.

[0015] Among them, the first Internet of Things device has natural language understanding capability, and at least one second Internet of Things device has natural language generation capability; the inter-device communication method also includes: receiving a third natural language instruction sent by the second Internet of Things device with natural language generation capability; performing natural language understanding on the third natural language instruction to obtain an understanding result; and executing an operation corresponding to the understanding result.

[0016] Among them, before sending the corresponding first natural language instructions to at least one second Internet of Things device respectively, the device communication method also includes: respectively using the application layer protocol or transport layer protocol supported by at least one second Internet of Things device to encapsulate the corresponding first natural language instructions; and / or, respectively sending the corresponding first natural language instructions to at least one second Internet of Things device, including: respectively sending the corresponding first natural language instructions to at least one second Internet of Things device through the physical layer protocol supported by at least one second Internet of Things device.

[0017] Among them, the natural language processing task of the first Internet of Things device or the second Internet of Things device is executed by a natural language model, and different natural language processing tasks are selected to be implemented by a locally deployed natural language model or an online natural language model, and the online natural language model includes at least one of a natural language model deployed on a cloud device and a natural language model deployed on an edge device; and / or, the natural language processing tasks of the first Internet of Things device and the second Internet of Things device are executed relying on the device domain knowledge graph; and / or, the natural language used between the first Internet of Things device and the second Internet of Things device is a language that can be understood by the receiving end or a language that matches the location; and / or, the inter-device communication method also includes: during the process of establishing a communication connection or before sending a first natural language instruction, identity authentication is performed with the second Internet of Things device, wherein establishing a communication connection or sending a first natural language instruction is performed when the identity authentication is passed, and at least one interaction data between the first Internet of Things device and the second Internet of Things device in the identity authentication process is described in natural language.

[0018] The first IoT device and the second IoT device are cross-domain devices, or are different devices in a smart home system, or are different devices in an industrial system.

[0019] To solve the above-mentioned technical problems, the second aspect of the present application provides a method for communication between devices, which method includes: establishing a communication connection between a second Internet of Things device and a first Internet of Things device, wherein the second Internet of Things device has natural language understanding capabilities and the first Internet of Things device has natural language generation capabilities; receiving a first natural language instruction sent by the first Internet of Things device, and understanding and executing the first natural language instruction.

[0020] Among them, the second Internet of Things device establishes a communication connection with the first Internet of Things device, including: the second Internet of Things device broadcasts a message; receives a service query sent by the first Internet of Things device; and sends a query response including connection information to the first Internet of Things device, so that the first Internet of Things device establishes a communication connection with the second Internet of Things device based on the connection information.

[0021] The broadcast message includes a device capability description, which is used to characterize the functions that can be implemented by the second IoT device; and / or the broadcast message includes a service type, so that the first IoT device determines whether the second IoT device has natural language understanding capabilities based on the service type before sending a service query to the second IoT device, and sends a service query to the second IoT device in response to the second IoT device having natural language understanding capabilities.

[0022] Among them, after the second Internet of Things device establishes a communication connection with the first Internet of Things device, the inter-device communication method also includes at least one of the following steps: receiving a query request sent by the first Internet of Things device, sending first status information to the first Internet of Things device, so that the first Internet of Things device parses the first status information to obtain the current status of the second Internet of Things device; sending second status information to the first Internet of Things device, so that the first Internet of Things device parses the second status information to obtain the current status of the second Internet of Things device, wherein the second status information is sent by the second Internet of Things device when it detects that its own status has changed.

[0023] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, which includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the inter-device communication method of the above first aspect.

[0024] In order to solve the above technical problems, the fourth aspect of the present application provides an electronic device, which includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the inter-device communication method of the second aspect.

[0025] To solve the above technical problems, the fifth aspect of the present application provides an inter-device communication system, which includes a first Internet of Things device and a second Internet of Things device, the first Internet of Things device is the device of the third aspect above, and the second Internet of Things device is the device of the fourth aspect above.

[0026] In the above technical solution, since the first IoT device has the ability to generate natural language, the first IoT device can accurately generate the first natural language instruction reflecting the current control intention based on the current control intention. Since the second IoT device has the ability to understand natural language, the second IoT device can accurately understand and execute the first natural language instruction generated by the first IoT device. Therefore, the first IoT device can communicate with the second IoT device accurately and effectively, that is, the first IoT device can control the second IoT device accurately and effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flowchart of an embodiment of a method for communication between devices provided by the present application; Figure 2 yes Figure 1 The flowchart of step S11 is shown as an embodiment; Figure 3 It is a flowchart of another embodiment of the inter-device communication method provided by the present application; Figure 4 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application; Figure 5 is a structural schematic diagram of another embodiment of the electronic device provided by the present application; Figure 6 It is a structural diagram of an embodiment of an inter-device communication system provided in the present application. DETAILED DESCRIPTION

[0028] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0029] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0030] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0031] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of the inter-device communication method provided by the present application. It should be noted that if there are substantially the same results, this embodiment does not necessarily refer to the embodiment of the present invention. Figure 1 The process sequence shown is limited. Figure 1 As shown, the execution subject of this embodiment is a first IoT device, and this embodiment includes: Step S11: The first IoT device establishes communication connections with a plurality of second IoT devices respectively.

[0032] In this implementation, a first IoT device establishes communication connections with a plurality of second IoT devices respectively, wherein the first IoT device has a natural language generation capability and the second IoT device has a natural language understanding capability.

[0033] By establishing communication connections between the first IoT device and several second IoT devices, the first IoT device can communicate with each second IoT device. For example, by establishing communication connections between the first IoT device and several second IoT devices, the first IoT device can control each second IoT device, thereby realizing the control of the second IoT device by the first IoT device. For another example, by establishing communication connections between the first IoT device and several second IoT devices, the first IoT device can query the relevant device status of each second IoT device, thereby realizing the query of the second IoT device by the first IoT device.

[0034] In addition, since the first IoT device has the ability to generate natural language, the first IoT device can accurately generate natural language instructions that reflect communication needs (such as control needs, query needs, etc.). Since the second IoT device has the ability to understand natural language, the second IoT device can accurately understand and execute the natural language instructions generated by the first IoT device. Therefore, by configuring the first IoT device to have the ability to generate natural language and the second IoT device to have the ability to understand natural language, accurate and effective communication between the first IoT device and the second IoT device can be achieved.

[0035] Therefore, by establishing communication connections between a first IoT device with natural language generation capability and a plurality of second IoT devices with natural language understanding capability, the first IoT device and each second IoT device can communicate accurately and effectively.

[0036] In one implementation, a first IoT device may establish a communication connection with a second IoT device, and subsequently the first IoT device may communicate accurately and effectively with the second IoT device.

[0037] Of course, in other implementations, the first IoT device may also establish a communication connection with at least two second IoT devices, and subsequently the first IoT device may select second IoT devices that match the communication requirements to communicate accurately and effectively based on the communication requirements.

[0038] In one embodiment, the first IoT device and the second IoT device are different devices in a smart home system. For example, the first IoT device is a smart central control, and the second IoT device is a smart TV, smart curtains, smart air conditioner, smart lighting, etc.

[0039] In one embodiment, the first IoT device and the second IoT device are different devices in an industrial system. For example, the first IoT device is a central control system, and the second IoT device is a production line robot, a material delivery system, a quality inspection system, an AGV, etc.

[0040] In one embodiment, the first IoT device and the second IoT device are cross-domain devices. For example, the first IoT device is a traffic light system and a smart street light system; the second IoT device is a signal light and lighting.

[0041] In one embodiment, after establishing communication connections with several second IoT devices respectively, the first IoT device also receives the current status of the second IoT devices; saves the current status, and / or displays the current status, and / or feeds back the current status to the user or the fourth IoT device.

[0042] That is to say, after the first IoT device establishes communication connections with several second IoT devices respectively, it will receive the current status of the second IoT devices, so that the status of each second IoT device known by the first IoT device is synchronized with the current status of the second IoT devices.

[0043] Among them, the first IoT device can save the current status of each second IoT device to record the current status of each second IoT device in time. The first IoT device can also display the current status of each second IoT device to display the current status of each second IoT device in real time. The first IoT device can also feedback the current status of each second IoT device to the user or the fourth IoT device so that the user or the fourth IoT device can know the current status of each second IoT device in time.

[0044] It should be noted that the fourth Internet of Things device may be other Internet of Things devices that need to know the current status of each second Internet of Things device.

[0045] In a specific implementation, receiving the current state of the second IoT device includes: sending a query request to the second IoT device, receiving first state information sent by the second IoT device, parsing the first state information, and obtaining the current state, wherein at least one of the query request and the first state information is described in natural language.

[0046] That is to say, after the first IoT device sends a query request to the second IoT device, the second IoT device will send the first status information to the first IoT device, so that the first IoT device can parse the first status information to obtain the current status of the second IoT device. That is, when the first IoT device queries the current status of the second IoT device, it can know the current status of the second IoT device, so that the status of the second IoT device known by the first IoT device is synchronized with the current status of the second IoT device.

[0047] In a specific implementation, sending a query request to the second IoT device specifically includes: receiving query demand information; analyzing the query demand information to determine the current query intent; and generating a query request for the second IoT device based on the current query intent. That is, after the first IoT device receives the query demand information, it will analyze the query demand information to determine the current query intent. By analyzing the query demand information, the current query intent can be accurately analyzed, so that a query request reflecting the query demand can be accurately generated based on the current query intent. The second IoT device can then understand and execute the corresponding query, thereby enabling the first IoT device to accurately query the second IoT device based on the query demand.

[0048] In a specific implementation, the query demand information is input by a user or sent by a fourth IoT device. It should be noted that the fourth IoT device may be any second IoT device, or may be another IoT device other than the second IoT device.

[0049] For example, take the query demand information as user input: the query demand information input by the user is "What is the wind speed and temperature of the air conditioner"; the central control analyzes the query demand information input by the user and determines that the current query intention is "Determine the wind speed and temperature of the air conditioner."

[0050] Furthermore, based on the current query intent, the central control generates a query request for the air conditioner, "What is the current temperature and wind speed of the air conditioner?" and sends it to the air conditioner. After the air conditioner receives the query request sent by the central control, it will understand and return the query result "The current temperature is 26 degrees, and the wind speed is medium."

[0051] For another example, take the query demand information as user input: the query demand information input by the user is "What is the current status of the devices at home?"; the central control analyzes the query demand information input by the user, and determines that the current query intention is "determine the current status of each device at home."

[0052] Furthermore, based on the current query intent, the central control generates a query request for the curtains, lighting, and TV, "Please feedback the current status" and sends it to the curtains, lighting, and TV. After the curtains receive the query request sent by the central control, they will understand and return the query result "The curtains are currently closed", after the lighting receives the query request sent by the central control, they will understand and return the query result "The current brightness of the light is 20, the color temperature is 2700K", and after the TV receives the query request sent by the central control, it will understand and return the query result "The TV is turned on, HDMI1, and the volume is 40".

[0053] Furthermore, the central control system provides users with feedback on the current status of each device in the home: "The living room curtains are closed, the lights are in theater mode, the TV is playing on HDMI1, and the volume is 40."

[0054] It should be noted that since the status of each second IoT device known by the first IoT device is synchronized with the current status of the second IoT device, when there is a fourth IoT device or a user queries the current status of any second IoT device in the future, the status of the second IoT device known by itself can be directly fed back to the user or the fourth IoT device without the need for query, thereby improving query efficiency.

[0055] In a specific implementation, the query demand information is in the form of text, voice or image. That is, the information form of the query demand information can be flexibly set.

[0056] In a specific implementation, the first Internet of Things device may actively query the current status of each second Internet of Things device.

[0057] In a specific implementation, receiving the current state of the second IoT device includes: receiving second state information described in natural language and sent by the second IoT device, parsing the second state information, and obtaining the current state, wherein the second state information is sent by the second IoT device when detecting a change in its own state.

[0058] That is, when the second IoT device detects a change in its own state, it actively sends the second state information to the first IoT device, so that the first IoT device can parse the second state information to obtain the current state of the second IoT device. That is, when the second IoT device detects a change in its own state, it will promptly inform the first IoT device, so that the state of the second IoT device known by the first IoT device is synchronized with the current state of the second IoT device.

[0059] Step S12: Based on the current control intention, generate a first natural language instruction to at least one second IoT device.

[0060] In this embodiment, based on the current control intention, a first natural language instruction corresponding to at least one second IoT device is generated. Since the first IoT device has a natural language generation capability, the first IoT device can accurately generate the first natural language instruction reflecting the current control intention.

[0061] It should be noted that, since the current control intention does not necessarily involve controlling each second IoT device, only the first natural language instruction of the second IoT device corresponding to the current control intention will be generated based on the current control intention.

[0062] In one embodiment, the first natural language instruction generated based on the current control intention may be a first natural language text, that is, the generated first natural language instruction may be in text form. Of course, in other embodiments, the first natural language instruction generated based on the current control intention may also be in voice form, etc., which is not limited here.

[0063] In one embodiment, before generating a first natural language instruction to at least one second IoT device based on the current control intent, the first IoT device also obtains the device capability description of the second IoT device, and determines the functions that the second IoT device can implement based on the device capability description; at this time, generating a first natural language instruction to at least one second IoT device based on the current control intent is specifically: generating a first natural language instruction that matches the function of at least one second IoT device based on the current control intent. Since the functions of each second IoT device are different and limited, the first natural language instruction that matches its function is generated based on the functions that it can implement, so that the subsequent second IoT device can accurately understand and execute the first natural language instruction generated by the first IoT device, and realize the accurate and effective control of the second IoT device by the first IoT device.

[0064] In one embodiment, before generating a first natural language instruction to at least one second IoT device based on the current control intention, the first IoT device will also receive control demand information; analyze the control demand information to determine the current control intention. That is, after the first IoT device receives the control demand information, it will analyze the control demand information to determine the current control intention. By analyzing the control demand information, the current control intention can be accurately analyzed, so that the first natural language instruction reflecting the control demand can be accurately generated based on the current control intention, and the second IoT device can understand and execute the corresponding first natural language instruction, so that the first IoT device can accurately control the second IoT device based on the control demand.

[0065] In a specific implementation, the control requirement information is input by a user or sent by a third IoT device. It should be noted that the third IoT device may be any second IoT device, or may be another IoT device other than the second IoT device.

[0066] For example, take the control demand information as user input: the control demand information input by the user is "adjust the air conditioner to 26 degrees, and set the wind speed to medium wind speed"; the central control analyzes the control demand information input by the user, and determines that the current control intention is "adjust the temperature of the air conditioner to 26 degrees, and adjust the wind speed of the air conditioner to medium wind speed".

[0067] Furthermore, based on the current control intention, the central control generates a first natural language instruction to the air conditioner, "Please adjust the air conditioner temperature to 26 degrees and set the air speed to medium" and sends it to the air conditioner. After the air conditioner receives the first natural language instruction sent by the central control, it will understand and execute the first natural language instruction to adjust the air conditioner temperature to 26 degrees and the air conditioner wind speed to medium wind speed.

[0068] For another example, take the control demand information as user input: the control demand information input by the user is "I want to watch a movie"; the central control analyzes the control demand information input by the user, and determines that the current control intention is "need to close the curtains, need to adjust the lighting to theater mode, need to turn on the TV."

[0069] Furthermore, based on the current control intention, a first natural language instruction for the curtains is generated, "Please close the curtains", and a first natural language instruction for the lighting is generated, "Please adjust to cinema mode, adjust the brightness to 20, and adjust the color temperature to 2700K", and a first natural language instruction for the TV is generated, "Please turn on, switch to HDMI1, and adjust the color temperature to 40". After the curtains receive the first natural language instruction sent by the central control, they will understand and execute the first natural language instruction to close the curtains; and, after the lighting receives the first natural language instruction sent by the central control, it will understand and execute the first natural language instruction to adjust the lighting to cinema mode, adjust the lighting brightness to 20, and adjust the lighting color temperature to 2700K; and, after the TV receives the first natural language instruction sent by the central control, it will understand and execute the first natural language instruction to turn on the TV, switch the TV to HDMI1, and adjust the TV color temperature to 40.

[0070] For another example, take the case where the control demand information is sent to other IoT devices except the second IoT device: the first IoT device is the traffic light system and the street light system, and the second IoT device is the traffic light and the street light; the control demand information "Emergency, request for priority passage" sent by the ambulance is received. The traffic light system receives the control demand information sent by the ambulance, analyzes the control demand information, and determines that the current control intention is "switch the traffic light at the intersection ahead to green"; further, based on the current control intention, the traffic light system generates a first natural language instruction to the traffic light at the intersection ahead, "Please switch the traffic light to green"; subsequently, after the traffic light at the intersection ahead receives the first natural language instruction sent by the traffic light system, it will understand and execute the first natural language instruction to switch the traffic light to green. At the same time, the street light system receives the control demand information sent by the ambulance, analyzes the control demand information, and determines that the current control intention is "increase the brightness of the street light on the road section ahead"; further, the street light system generates a first natural language instruction to the street light on the road section ahead based on the current control intention, "Please increase the brightness of the street light"; subsequently, after the street light on the road section ahead receives the first natural language instruction sent by the street light system, it will understand and execute the first natural language instruction to increase the brightness of the street light.

[0071] In a specific implementation, the control requirement information is in the form of text, voice or image. In other words, the interactive form of the control requirement information can be flexibly selected.

[0072] In one embodiment, before sending the corresponding first natural language instruction to at least one second Internet of Things device respectively, the first Internet of Things device will also respectively use the application layer protocol or transport layer protocol supported by at least one second Internet of Things device to encapsulate the corresponding first natural language instruction; at this time, sending the corresponding first natural language instruction to at least one second Internet of Things device respectively is specifically: sending the corresponding first natural language instruction to at least one second Internet of Things device respectively through the physical layer protocol supported by at least one second Internet of Things device.

[0073] That is to say, before the first IoT device sends the corresponding first natural language instruction to the second IoT device, the first natural language instruction will be encapsulated to convert the first natural language instruction into a format that can be understood by the second IoT device; on the one hand, if other IoT devices do not support this application layer protocol or transport layer protocol, they cannot understand the encapsulated first natural language instruction, thereby ensuring data security; on the other hand, in resource-constrained IoT devices, encapsulation helps optimize message size and transmission efficiency.

[0074] The application layer protocols supported by different second IoT devices may be different or the same. For example, the application layer protocol may be MQTT, CoAP, HTTP, WebSocket, etc., which are not limited here. In addition, the transport layer protocol may be TCP, UDP, etc., which are not limited here. In addition, the physical layer protocol is not limited, for example, the physical layer protocol is WIFI, 5G, etc.

[0075] In one embodiment, the natural language processing task of the first IoT device is performed by a natural language model, and different natural language processing tasks are selected to be implemented by a locally deployed natural language model or an online natural language model. The online natural language model includes at least one of a natural language model deployed on a cloud device and a natural language model deployed on an edge device.

[0076] That is to say, the natural language processing task of the first IoT device is performed by the deployed natural language model, such as the first IoT device calls the natural language model to generate a first natural language instruction to at least one second IoT device based on the current control intention.

[0077] It should be noted that the deployment location of the natural language model corresponding to the first IoT device can be flexible, such as local, cloud device, edge device, etc. Of course, the natural language model corresponding to the first IoT device can also be deployed at multiple locations at the same time, and the natural language models at different deployment locations can be used to perform different natural language processing tasks of the first IoT device, and can also be used to perform the same natural language processing task of the first IoT device.

[0078] In one embodiment, during the process of establishing a communication connection or before sending a first natural language instruction, identity authentication is performed with the second IoT device, wherein the establishment of the communication connection or the sending of the first natural language instruction is performed when the identity authentication is passed, and at least one interactive data between the first IoT device and the second IoT device during the identity authentication process is described in natural language. By performing identity authentication with the second IoT device during the process of establishing a communication connection or before sending the first natural language instruction, it is possible to ensure that the communication between the first IoT device and the second IoT device is credible, prevent unauthorized devices from accessing, and ensure the reliability and stability of the communication.

[0079] Step S13: Send the corresponding first natural language instruction to at least one second IoT device respectively, so that the at least one second IoT device understands and executes the corresponding first natural language instruction.

[0080] In this embodiment, the corresponding first natural language instruction is sent to at least one second IoT device respectively, so that at least one second IoT device understands and executes the corresponding first natural language instruction. Since the second IoT device has the ability to understand natural language, after the second IoT device receives the corresponding first natural language instruction sent by the first IoT device, it can accurately understand and execute the natural language instruction generated by the first IoT device, thereby realizing accurate and effective control of the second IoT device by the first IoT device, that is, realizing accurate and effective communication between the first IoT device and the second IoT device.

[0081] Specifically, after receiving the first natural language instruction sent by the first IoT device, the second IoT device will understand the first natural language instruction to determine the control operation to be performed; then, the second IoT device performs the corresponding control operation, thereby realizing the control of the second IoT device by the first IoT device.

[0082] In one implementation method, the first natural language instruction sent by the first IoT device and received by the second IoT device is encapsulated by the first IoT device based on the application layer protocol supported by the second IoT device, that is, the first natural language instruction received by the second IoT device is encapsulated based on the application layer protocol supported by itself. Therefore, after the second IoT device receives the encapsulated first natural language instruction sent by the first IoT device, it first unpacks the encapsulated first natural language instruction to obtain the first natural language instruction; then, it understands the first natural language instruction to determine the control operation to be performed; then, the second IoT device performs the corresponding control operation, thereby realizing the control of the second IoT device by the first IoT device.

[0083] In one implementation, after the second IoT device understands and executes the corresponding first natural language instruction, it will feed back the execution result to the first IoT device.

[0084] In one embodiment, the natural language processing task of the second IoT device is performed by a natural language model, and different natural language processing tasks are selected to be implemented by a locally deployed natural language model or an online natural language model. The online natural language model includes at least one of a natural language model deployed on a cloud device and a natural language model deployed on an edge device.

[0085] That is to say, the natural language processing task of the second IoT device is performed by the deployed natural language model, such as the second IoT device calls the natural language model to understand the first natural language instruction sent by the first IoT device.

[0086] It should be noted that the deployment location of the natural language model corresponding to the second IoT device can be flexible, such as local, cloud device, edge device, etc. Of course, the natural language model corresponding to the second IoT device can also be deployed at multiple locations at the same time, and the natural language models at different deployment locations can be used to perform different natural language processing tasks of the second IoT device, and can also be used to perform the same natural language processing task of the second IoT device.

[0087] In one embodiment, after receiving the query request sent by the first IoT device, the second IoT device will understand the query request to determine the content to be queried; then, the second IoT device performs the query and generates the query result, which is then fed back to the first IoT device, thereby enabling the first IoT device to query the second IoT device.

[0088] In one embodiment, the natural language processing tasks of the first IoT device and the second IoT device are performed based on the device domain knowledge graph. The device domain knowledge graph contains a large amount of entity, relationship and attribute information related to the device, which provides rich background knowledge for the natural language processing task and helps the natural language processing task to more accurately understand the language input and output of the device. By combining the natural language processing task with the device domain knowledge graph, the natural language processing task can map the entities and relationships in the text to the corresponding entities and relationships in the knowledge graph; in this way, the natural language processing task can use the knowledge in the knowledge graph to solve ambiguous problems, infer the relationship between entities, and even answer complex queries from users. In the natural language processing tasks of IoT devices, the device domain knowledge graph provides the necessary semantic information and contextual support, which enables more accurate execution of tasks and improves the intelligence level of the device.

[0089] In one implementation, the natural language used between the first IoT device and the second IoT device is a language that the receiving end can understand, wherein the language that the receiving end can understand is not limited.

[0090] It should be noted that when the user input or the demand information sent by other IoT devices is not in a language that the receiving end can understand, the first IoT device can translate it, and generate a first natural language instruction based on the translated content, and then send it to the corresponding second IoT device. In other words, the communication between the first IoT device and the second IoT device supports multiple languages.

[0091] In one embodiment, the natural language used between the first IoT device and the second IoT device is a language that matches the location of the receiving end. In other words, the communication language between the first IoT device and the second IoT device can be changed based on the geographical location to the default language of the geographical location. In one embodiment, after the second IoT device receives the first natural language instruction sent by the first IoT device, it can verify the legitimacy of the first natural language instruction, thereby preventing security issues caused by malicious attacks or misoperation.

[0092] In one implementation, before the first IoT device sends the first natural language instruction to the corresponding second IoT device, the first natural language instruction may also be encrypted to ensure that the transmitted data is not eavesdropped or tampered with.

[0093] In one embodiment, the first Internet of Things device has natural language understanding capability, and the second Internet of Things device has natural language generation capability; after sending corresponding first natural language instructions to at least one second Internet of Things device respectively, the first Internet of Things device receives a natural language response sent by at least one second Internet of Things device, wherein the natural language response represents a response result of the second Internet of Things device to the first natural language instruction; natural language understanding is performed on each natural language response respectively to obtain a response result of each first natural language instruction.

[0094] Do at least one of the following: Based on the response result, determine whether to resend the first natural language instruction to the corresponding second Internet of Things device.

[0095] Based on the response result, natural language feedback information is generated and fed back to the control demander, wherein the current control intention is determined based on the control demand information provided by the control demander, and the control demander is a user or a third IoT device.

[0096] Based on the response result, a second natural language instruction for at least one second Internet of Things device is generated, and the corresponding second natural language instruction is sent to the at least one second Internet of Things device.

[0097] It should be noted that the third IoT device may be a second IoT device, or may be another IoT device other than the second IoT device.

[0098] For example, the first IoT device is a traffic control system with natural language understanding capabilities; the second IoT device is a bus with natural language generation capabilities; the second IoT device also includes a roadside unit, a charging pile, and traffic lights along the route. The bus is a control demander, and the current control intention is "vehicle number B12345, current location [116.3, 39.9], power 20%, request the nearest charging station location and route"; the traffic control system sends the corresponding first natural language instruction to the roadside unit "Please check the road conditions along [116.3, 39.9] to [116.4, 39.95]", and sends the corresponding first natural language instruction to the charging pile "Query the idle status of the charging position at Qianliang Hutong Station". The first IoT device receives the natural language response sent by the roadside unit "The current traffic volume on this section is large, it is recommended to delay for 10 minutes", and receives the natural language response sent by the charging pile "There are currently 2 fast charging positions available, and the estimated waiting time is 5 minutes". The first IoT device performs natural language understanding on the natural language responses sent by the roadside unit and the charging pile, obtains the response results of each first natural language instruction, and generates natural language feedback information based on the response results, "It is recommended to go to Qianliang Hutong Charging Station in 10 minutes. The route has been planned and priority will be given to vehicles passing through traffic lights", and feeds back natural language feedback information to the bus, "It is recommended to go to Qianliang Hutong Charging Station in 10 minutes. The route has been planned and priority will be given to vehicles passing through traffic lights". In addition, based on the response results, the first IoT device generates a second natural language instruction to the traffic lights along the route, "Vehicle B12345 will be given priority in 10 minutes", and sends the corresponding second natural language instruction "Vehicle B12345 will be given priority in 10 minutes" to the traffic lights along the route.

[0099] In one embodiment, a first IoT device has natural language understanding capability, and at least one second IoT device has natural language generation capability; the first IoT device receives a third natural language instruction sent by a second IoT device with natural language generation capability; performs natural language understanding on the third natural language instruction to obtain an understanding result; and executes an operation corresponding to the understanding result.

[0100] For example, the first IoT device is a traffic control system with natural language understanding capability; the second IoT device is a bus with natural language generation capability. The bus generates a third natural language instruction "vehicle number B12345, current location [116.3, 39.9], power 20%, request the nearest charging station location and route", and sends it to the traffic control system; the traffic control system performs natural language understanding on the third natural language instruction "vehicle number B12345, current location [116.3, 39.9], power 20%, request the nearest charging station location and route", obtains the understanding result, and performs an operation corresponding to the understanding result. The operation corresponding to the understanding result may be to ask the second IoT device-roadside unit to check the road conditions along [116.3, 39.9] to [116.4,39.95], ask the charging pile to check the idle status of the charging position at the Qianliang Hutong station, and determine the location of the nearest charging pile and the corresponding route, and feedback to the bus.

[0101] See also Figure 2 , Figure 2 yes Figure 1 It should be noted that if there is substantially the same result, this embodiment does not use Figure 2 The process sequence shown is limited. Figure 2 As shown, this embodiment includes: Step S21: Scanning a broadcast message of a second IoT device.

[0102] In this embodiment, the broadcast message of the second IoT device is scanned. Specifically, the second IoT device broadcasts a message; the first IoT device scans, thereby scanning the broadcast message of the second IoT device.

[0103] In one implementation, the second IoT device needs to start the mDNS service before broadcasting the message. The second IoT device starts the mDNS service so that the second IoT device can broadcast the message without a DNS server.

[0104] In one embodiment, the broadcast message includes a device capability description, and the device capability description is used to determine the functions that the second IoT device can implement. The second IoT device broadcasts the device capability description so that after the first IoT device scans the broadcast message of the second IoT device, it can understand the functions that the second IoT device can implement. Subsequently, the first IoT device can determine whether it can successfully communicate with the second IoT device based on the functions that the second IoT device can implement. In other words, the first IoT device can subsequently generate natural language instructions that match the functions that the second IoT device can implement based on the functions that the second IoT device can implement, thereby successfully communicating with the second IoT device.

[0105] In a specific implementation, the device capability description includes functions supported by the device and at least one of a device type, a function parameter range, and a current state.

[0106] For example, the first IoT device is a smart central control, and the second IoT device is a smart TV, smart curtain, smart air conditioner, and smart lighting. The device capability description includes the functions supported by the device, the device type, the function parameter range, and the current status. The capability description of the smart curtain broadcast device is "This device is a smart curtain; supports curtain switching, curtain opening and closing adjustment; curtain opening and closing adjustment range: 0-100%; current status: off". The capability description of the smart lighting broadcast device is "This device is a smart lighting; supports lighting switching, lighting brightness adjustment, and lighting color temperature adjustment; lighting brightness adjustment range: 1-100, lighting color temperature adjustment range: 2700K-6500K; current status: off". The capability description of the smart TV broadcast device is "This device is a smart TV; supports TV power on and off, TV volume adjustment, and TV source switching; TV volume adjustment range: 0-100, TV source switching range: HDMI1 / HDMI2 / network; current status: standby". The capability description of the smart air conditioner broadcasting device is "This device is a smart air conditioner; it supports air conditioner on / off, air conditioner temperature adjustment, and air conditioner wind speed adjustment; air conditioner temperature adjustment range: 16-30℃, air conditioner wind speed adjustment range: low / medium / high; current status: off".

[0107] In a specific implementation, the device capability description is described in natural language, and the first IoT device has the ability to understand natural language, and can understand the device capability description in natural language to determine the functions that the second IoT device can implement. Since the device capability description broadcast by the second IoT device is described in natural language, and the first IoT device has the ability to understand natural language, the first IoT device can accurately understand the functions that the second IoT device can implement, and the first IoT device can subsequently determine whether it can successfully communicate with the second IoT device based on the functions that the second IoT device can implement, or in other words, the first IoT device can subsequently generate natural language instructions that match the functions that the second IoT device can implement based on the functions that the second IoT device can implement, so as to successfully communicate with the second IoT device.

[0108] In one implementation, the broadcast message includes a service type, and whether the second IoT device has a natural language understanding capability can be determined through the service type of the second IoT device.

[0109] Step S22: Send a service query to the second IoT device.

[0110] In this embodiment, a service query is sent to the second IoT device. Specifically, when the first IoT device scans the broadcast message of the second IoT device, a service query is sent to the second IoT device to obtain the connection message of the second IoT device, thereby establishing a communication connection with the second IoT device based on the connection message of the second IoT device.

[0111] In one embodiment, the broadcast message includes a service type; before sending a service query to the second IoT device, it is determined whether the second IoT device has natural language understanding capabilities based on the service type; in response to the second IoT device having natural language understanding capabilities, the service query is sent to the second IoT device.

[0112] Since the first IoT device has the ability to generate natural language, the first IoT device can accurately generate natural language instructions that reflect communication needs (such as control needs, query needs, etc.). If the second IoT device does not have the ability to understand natural language, the second IoT device cannot accurately understand and execute the natural language instructions generated by the first IoT device. Even if the first IoT device establishes a communication connection with the second IoT device that does not have the ability to understand natural language, accurate and effective communication between the two cannot be achieved subsequently. Therefore, it is necessary to establish a communication connection with the second IoT device that has the ability to understand natural language, so that accurate and effective communication can be achieved subsequently.

[0113] Therefore, before sending a service query to the second IoT device, it is necessary to first determine whether it has natural language understanding capabilities, and after determining that the second IoT device has natural language understanding capabilities, send a service query to it so as to obtain the connection message of the second IoT device, thereby establishing a communication connection with the second IoT device with natural language understanding capabilities, and ensuring subsequent accurate and effective communication between the two. In addition, sending a service query to the second IoT device after determining that the second IoT device has natural language understanding capabilities reduces the waste of resources.

[0114] Step S23: receiving a query response fed back by the second IoT device.

[0115] In this embodiment, a query response fed back by the second IoT device is received, wherein the query response includes a connection message of the second IoT device. Specifically, the second IoT device monitors the service query, and when monitoring the service query sent by the first IoT device, generates a query response including a connection message, and feeds it back to the first IoT device, so that the first IoT device obtains the connection message of the second IoT device, thereby establishing a communication connection with the second IoT device based on the connection message of the second IoT device.

[0116] For example, the connection message fed back by the second IoT device includes ""endpoint": "http: / / 192.168.1.100:8080"" and ""websocket": "ws: / / 192.168.1.100:8081"". http: / / 192.168.1.100:8080 and "ws: / / 192.168.1.100:8081" can be regarded as the connection addresses provided by the second IoT device. The first IoT device can subsequently use these two addresses to establish a communication connection with the second IoT device.

[0117] Step S24: Establish a communication connection with the second IoT device based on the connection information.

[0118] In this embodiment, a communication connection is established with the second IoT device based on the connection message. That is, the first IoT device can establish a communication connection with the second IoT device based on the connection information fed back by the second IoT device. Since the first IoT device has the ability to generate natural language and the second IoT device has the ability to understand natural language, accurate and effective communication between the first IoT device and the second IoT device can be achieved.

[0119] In the communication between the first IoT device and the second IoT device, due to various reasons (such as transmission errors, coding problems, device differences, etc.), the instructions may be unclear or ambiguous. Therefore, in one embodiment, when the second IoT device receives a fuzzy instruction, the second IoT device can try to use a fuzzy matching algorithm to identify the true intention of the instruction. Based on the fuzzy matching, the second IoT device can send a confirmation request to the first IoT device, requiring the first IoT device to clarify or correct the instruction, which helps to ensure the accuracy and executability of the instruction.

[0120] When the first IoT device sends multiple conflicting instructions to the second IoT device at the same time, there will be a command conflict. Therefore, in one embodiment, the second IoT device can determine which instruction to execute based on the priority of the instructions, and the priority can be determined based on factors such as the urgency and importance of the instructions. In other embodiments, the second IoT device can monitor the reception of instructions in real time, and once a conflict is found, it will immediately send a conflict notification to the first IoT device and request the control end to resolve it.

[0121] During the communication between the first IoT device and the second IoT device, communication interruption may occur due to network failure, device failure, etc. Therefore, in one embodiment, when communication interruption is detected, the first IoT device and the second IoT device may automatically try to reestablish the connection.

[0122] In a specific implementation, the physical architecture of the first IoT device is: 1) a device that supports physical layer protocols; 2) has natural language generation capabilities; 3) implements the TCP / IP protocol stack; 4) supports mDNS service discovery. The physical architecture of the second IoT device is: 1) a device that supports physical layer protocols; 2) has natural language understanding capabilities; 3) implements the TCP / IP protocol stack; 4) supports mDNS service broadcasting.

[0123] In a specific implementation, the network architecture of the first IoT device and the second IoT device is: 1) supporting AP and STA dual modes; 2) supporting Mesh network expansion; 3) supporting IPv4 / IPv6 dual stack.

[0124] In a specific implementation, the first IoT device and the second IoT device are provided with a connection security mechanism and an application layer security mechanism.

[0125] Connection security mechanisms include network layer security mechanisms and transport layer security mechanisms. Network layer security mechanisms include WPA2 / WPA3 encryption, MAC address filtering, network isolation and traffic monitoring; WPA2 encryption mainly relies on AES and CCMP for encryption, and WPA3 encryption uses more powerful encryption algorithms, such as SAE and 256-bit password hash functions; MAC address filtering is a network security measure that increases network security by limiting the MAC addresses of devices allowed to connect to the network. MAC addresses are unique identifiers for network devices. By setting MAC address filtering, unauthorized IoT devices can be prevented from connecting to the network; network isolation technology is a method of dividing a network into multiple independent areas or subnets to improve network security and performance; traffic monitoring involves real-time monitoring and analysis of network traffic. Through traffic monitoring, abnormal traffic and potential network attacks can be discovered and handled in a timely manner. The transport layer security mechanism includes TLS 1.3 encryption, certificate verification, session management and key negotiation. TLS 1.3 enhances security by removing insecure encryption algorithms and protocol features. It only supports stronger and more secure encryption suites, thereby reducing potential security vulnerabilities. Certificate verification is the process of ensuring the authenticity of the identities of the communicating parties (the first IoT device and the second IoT device), which can prevent man-in-the-middle attacks and other forms of identity fraud. Session management allows the communicating parties (the first IoT device and the second IoT device) to share keys and other state information during the session, thereby improving the efficiency and security of communication. Key negotiation is the process in which the communicating parties (the first IoT device and the second IoT device) negotiate encryption keys when establishing a communication connection. Key negotiation ensures that only the communicating parties can know the encryption keys, thereby ensuring the confidentiality of communication.

[0126] Application layer security mechanisms include access control security mechanisms and data security mechanisms. Access control security mechanisms include token authentication, permission management, operation auditing, and anomaly detection; Token authentication is a token-based identity authentication mechanism, in which users or devices need to provide a valid token to prove their identity before accessing resources or services, which can ensure that only users or devices holding valid tokens can access protected resources or services; Permission management refers to the process of managing and controlling access rights between the first IoT device and the second IoT device, which determines which users or devices can access which resources or services, and which operations they can perform; Operation auditing refers to the process of recording and reviewing the operation behavior between the first IoT device and the second IoT device, which can help system administrators monitor and review the operation behavior of users or devices in order to promptly discover and investigate possible security violations or improper behavior; Anomaly detection refers to the process of real-time monitoring and analysis of the communication or operation behavior between the first IoT device and the second IoT device to discover possible security threats or abnormal behavior. Data security mechanisms include end-to-end encryption, integrity verification, anti-replay protection and sensitive data desensitization; end-to-end encryption is an encryption technology that ensures that data remains encrypted when transmitted between the first IoT device and the second IoT device, and middlemen (such as hackers, service providers or servers hosting communications) cannot view, intercept or tamper with the data; integrity verification is a mechanism to ensure that data has not been tampered with or damaged during transmission or storage. Through integrity verification, the integrity and authenticity of the data can be verified, thereby ensuring the accuracy and reliability of the data; anti-replay protection is a security measure used to prevent attackers from resending previously captured data packets to recipients for fraud or attack. Anti-replay protection can ensure the uniqueness and timeliness of data packets during transmission, thereby preventing attackers from using old data packets to attack; sensitive data desensitization is a data processing technology that reduces the risk of data leakage by blurring, replacing or deleting sensitive data.

[0127] In a specific implementation, exception handling is provided, which includes network exception handling and service exception handling.

[0128] Network exception handling includes disconnection handling and network switching handling. Disconnection handling includes automatic reconnection mechanism, session recovery, state synchronization and offline cache. When the connection between the first IoT device and the second IoT device is disconnected, it will automatically try to re-establish the connection; save the session information on the first IoT device and the second IoT device, and when the connection is re-established, restore the previous operation state according to the saved session information; after the connection is disconnected and re-established, ensure that the state information between the first IoT device and the second IoT device remains consistent, specifically through the state synchronization mechanism, synchronize the latest state information from one end to the other, which can include device status, configuration information, operation records, etc.; when the network connection is unstable or disconnected, temporarily store the data to be transmitted in the local cache, specifically, set an offline cache area on the first IoT device or the second IoT device to store the data to be transmitted, and when the network connection is restored, transmit the data in the cache to the other party.

[0129] Business exception handling includes smooth switching, IP address update, service re-registration and connection reconstruction. Smooth switching refers to ensuring the continuity and stability of communication when switching the network between the first IoT device and the second IoT device, without interrupting the existing connection or minimizing the interruption time; IP address update refers to updating the relevant configuration to ensure the continuity of communication when the IP address of the first IoT device or the second IoT device changes during the network switching process; service re-registration means that after the network switching, the first IoT device or the second IoT device needs to re-register its service information with the service registration center or middleware to ensure that other services can discover and access them; connection reconstruction refers to the process of re-establishing the connection between the first IoT device and the second IoT device after the existing connection is interrupted due to network switching, which usually involves steps such as re-authentication, negotiation of security parameters and restoration of session status.

[0130] Business exception handling includes instruction parsing errors and state synchronization errors. Instruction parsing errors include error prompts, instruction correction, downgrade processing and log records. When the instruction sent by the first IoT device cannot be correctly parsed by the second IoT device, the second IoT device should be able to send a clear error prompt to the first IoT device, which helps the first IoT device to quickly locate the problem and understand the specific reason for the failure of instruction parsing; the first IoT device can correct the sent instruction according to the error prompt to ensure the correctness and effectiveness of the instruction, which helps to reduce communication failures and data transmission problems caused by instruction errors; in some cases, if the instruction parsing error cannot be solved immediately, the second IoT device can adopt a downgrade processing strategy to ensure the basic operation of the system and the integrity of the data, which helps to avoid problems such as system crashes or data loss caused by instruction parsing errors; in the process of handling instruction parsing errors, the error log should be recorded in detail for subsequent analysis and troubleshooting, which helps to track the source, cause and impact of instruction parsing errors and provide strong support for the fundamental solution of the problem. State synchronization errors include retry mechanisms, state verification, forced synchronization and alarm notifications. The retry mechanism means that when the first IoT device is synchronizing its status with the second IoT device, if a synchronization error is detected, the first IoT device will try to resend the synchronization request to ensure that the status information can be correctly synchronized; status verification means that during the synchronization process, the first IoT device and the second IoT device verify the status information to ensure the accuracy and consistency of the status information; forced synchronization means that after a status synchronization error is detected, the first IoT device takes compulsory measures to require the second IoT device to immediately synchronize its status to ensure the consistency of the status information; alarm notification means that after a status synchronization error is detected, the first IoT device sends an alarm message to the administrator or relevant personnel so that timely countermeasures can be taken.

[0131] See also Figure 3 , Figure 3 FIG. 1 is a flow chart of another embodiment of the inter-device communication method provided by the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to the embodiment of FIG. Figure 3 The process sequence shown is limited. Figure 3 As shown, the execution subject of this embodiment is a second IoT device, and this embodiment includes: Step S31: The second IoT device establishes a communication connection with the first IoT device.

[0132] Step S31 is similar to step S11 and will not be described again here.

[0133] Step S32: Receive a first natural language instruction sent by a first IoT device, and understand and execute the first natural language instruction.

[0134] Step S32 is similar to step S13 and will not be described again.

[0135] See also Figure 4 , Figure 4 4 is a schematic diagram of the structure of an embodiment of an electronic device provided by the present application. The electronic device 40 includes a memory 41 and a processor 42 coupled to each other, and the processor 42 is used to execute program instructions stored in the memory 41 to implement the steps of any of the above-mentioned inter-device communication method embodiments. In a specific implementation scenario, the electronic device 40 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 40 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0136] Specifically, the processor 42 is used to control itself and the memory 41 to implement the steps of any of the above-mentioned inter-device communication method embodiments. The processor 42 can also be called a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 42 can be implemented by an integrated circuit chip.

[0137] It should be noted that the electronic device 40 in this embodiment is used to implement the inter-device communication method corresponding to step S11 to step S13 and subsequent steps.

[0138] See also Figure 5 , Figure 5 : is a structural diagram of another embodiment of an electronic device provided by the present application. The electronic device 50 includes a memory 51 and a processor 52 coupled to each other, and the processor 52 is used to execute program instructions stored in the memory 51 to implement the steps of any of the above-mentioned inter-device communication method embodiments. In a specific implementation scenario, the electronic device 50 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 50 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0139] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps of any of the above-mentioned inter-device communication method embodiments. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 52 can be implemented by an integrated circuit chip.

[0140] It should be noted that the electronic device 50 in this embodiment is used to implement the inter-device communication method corresponding to step S31 - step S32 and subsequent steps.

[0141] See also Figure 6 , Figure 6 1 is a schematic diagram of an embodiment of an inter-device communication system provided by the present application. The inter-device communication system 60 includes a first Internet of Things device and a second Internet of Things device, wherein the first Internet of Things device is an electronic device 40 and the second Internet of Things device is an electronic device 50.

[0142] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0143] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for communication between devices, characterized in that: include: A first IoT device establishes communication connections with a plurality of second IoT devices respectively, wherein the first IoT device has a natural language generation capability and the second IoT device has a natural language understanding capability; Based on the current control intention, generate a first natural language instruction to at least one second Internet of Things device; The corresponding first natural language instruction is sent to the at least one second Internet of Things device respectively, so that the at least one second Internet of Things device understands and executes the corresponding first natural language instruction.

2. The method according to claim 1, characterized in that Before generating a first natural language instruction to at least one second IoT device based on the current control intention, the method further includes: Obtaining a device capability description of the second IoT device, and determining functions that can be implemented by the second IoT device based on the device capability description; The generating, based on the current control intention, a first natural language instruction to at least one second IoT device comprises: Based on the current control intention, a first natural language instruction matching the function of the at least one second Internet of Things device is generated.

3. The method according to claim 1, characterized in that The first IoT device establishes communication connections with a plurality of second IoT devices respectively, including: Scanning a broadcast message of the second IoT device; Sending a service query to the second IoT device; receiving a query response fed back by the second IoT device, wherein the query response includes connection information of the second IoT device; Based on the connection information, a communication connection is established with the second Internet of Things device.

4. The method according to claim 3, characterized in that The broadcast message includes a device capability description, where the device capability description is used to determine the functions that can be implemented by the second IoT device; And / or, the broadcast message includes a service type; before sending the service query to the second IoT device, the method further includes: Based on the service type, determining whether the second IoT device has natural language understanding capability; In response to the second IoT device having a natural language understanding capability, sending the service query to the second IoT device is performed.

5. The method according to claim 2 or 4, characterized in that: The device capability description includes functions supported by the device and at least one of a device type, a function parameter range, and a current state; And / or, the device capability description is described in natural language, and the first IoT device has natural language understanding capability and can perform natural language understanding on the device capability description to determine the functions that can be implemented by the second IoT device.

6. The method according to claim 1, characterized in that Before generating a first natural language instruction to at least one second IoT device based on the current control intention, the method further includes: receiving control demand information, wherein the control demand information is input by a user or sent by a third IoT device, and / or the control demand information is in the form of text, voice or image; The control requirement information is analyzed to determine the current control intention.

7. The method according to claim 1, characterized in that After establishing communication connections with the plurality of second IoT devices respectively, the method further includes: Receiving a current state of the second IoT device; Perform at least one of the following steps: save the current state; display the current state; and feed back the current state to a user or a fourth IoT device.

8. The method according to claim 7, characterized in that The receiving the current state of the second IoT device comprises at least one of the following steps: Sending a query request to the second IoT device, receiving first status information sent by the second IoT device, parsing the first status information to obtain the current status, wherein at least one of the query request and the first status information is described in a natural language; Receive second state information described in natural language and sent by the second Internet of Things device, parse the second state information, and obtain the current state, wherein the second state information is sent by the second Internet of Things device when detecting that its own state has changed.

9. The method according to claim 8, characterized in that The sending a query request to the second IoT device includes: Receiving query demand information, wherein the query demand information is input by a user or sent by a fourth IoT device, and / or the query demand information is in the form of text, voice or image; Analyze the query demand information to determine the current query intention; Based on the current query intention, a query request for the second Internet of Things device is generated.

10. The method according to claim 1, characterized in that The first IoT device has a natural language understanding capability, and the second IoT device has a natural language generation capability; After sending the corresponding first natural language instruction to the at least one second Internet of Things device respectively, the method further includes: Receiving a natural language response sent by the at least one second Internet of Things device, wherein the natural language response represents a response result of the second Internet of Things device to the first natural language instruction; Performing natural language understanding on each of the natural language responses respectively to obtain a response result of each of the first natural language instructions; Do at least one of the following: Based on the response result, determining whether to resend the first natural language instruction to the corresponding second Internet of Things device; Based on the response result, natural language feedback information is generated, and the natural language feedback information is fed back to the control demander, wherein the current control intention is determined based on the control demand information provided by the control demander, and the control demander is a user or a third Internet of Things device; Based on the response result, a second natural language instruction for at least one of the second Internet of Things devices is generated, and the corresponding second natural language instruction is sent to at least one of the second Internet of Things devices.

11. The method according to claim 1, characterized in that: The first IoT device has a natural language understanding capability, and at least one of the second IoT devices has a natural language generation capability; The method further comprises: Receiving a third natural language instruction sent by the second Internet of Things device having a natural language generation capability; Performing natural language understanding on the third natural language instruction to obtain an understanding result; An operation corresponding to the understanding result is performed.

12. The method according to claim 1, characterized in that Before sending the corresponding first natural language instruction to the at least one second Internet of Things device respectively, the method further includes: respectively adopting an application layer protocol or a transport layer protocol supported by the at least one second IoT device to encapsulate the corresponding first natural language instruction; And / or, sending the corresponding first natural language instruction to the at least one second IoT device respectively includes: The first natural language instruction corresponding to the at least one second Internet of Things device is sent to the at least one second Internet of Things device respectively through the physical layer protocol supported by the at least one second Internet of Things device.

13. The method according to claim 1, characterized in that The natural language processing task of the first IoT device or the second IoT device is performed by a natural language model, and different natural language processing tasks are selected to be implemented by the local natural language model or the online natural language model, and the online natural language model includes at least one of the natural language model deployed on the cloud device and the natural language model deployed on the edge device; And / or, the natural language processing tasks of the first IoT device and the second IoT device are performed relying on the device domain knowledge graph; And / or, the natural language used between the first IoT device and the second IoT device is a language that can be understood by the receiving end or a language that matches the location; And / or, the method further comprises: During the process of establishing the communication connection or before sending the first natural language instruction, identity authentication is performed with the second Internet of Things device, wherein the establishing the communication connection or the sending of the first natural language instruction is performed when the identity authentication is passed, and at least one interaction data between the first Internet of Things device and the second Internet of Things device during the identity authentication process is described in natural language.

14. The method according to claim 1, characterized in that The first IoT device and the second IoT device are cross-domain devices, or are different devices in a smart home system, or are different devices in an industrial system.

15. A method for communication between devices, characterized in that: include: A second IoT device establishes a communication connection with the first IoT device, wherein the second IoT device has a natural language understanding capability and the first IoT device has a natural language generation capability; Receive a first natural language instruction sent by the first Internet of Things device, and understand and execute the first natural language instruction.

16. The method according to claim 15, characterized in that The second IoT device establishes a communication connection with the first IoT device, including: The second IoT device broadcasts a message; Receiving a service query sent by the first IoT device; A query response including connection information is sent to the first IoT device, so that the first IoT device establishes a communication connection with the second IoT device based on the connection information.

17. The method according to claim 16, characterized in that The broadcast message includes a device capability description, where the device capability description is used to characterize functions that can be implemented by the second IoT device; And / or, the broadcast message includes a service type, so that the first IoT device determines whether the second IoT device has natural language understanding capabilities based on the service type before sending a service query to the second IoT device, and sends the service query to the second IoT device in response to the second IoT device having the natural language understanding capabilities.

18. The method according to claim 15, characterized in that After the second IoT device establishes a communication connection with the first IoT device, at least one of the following steps is also included: receiving a query request sent by the first Internet of Things device, and sending first status information to the first Internet of Things device, so that the first Internet of Things device parses the first status information to obtain a current status of the second Internet of Things device; Sending second status information to the first IoT device so that the first IoT device parses the second status information to obtain the current status of the second IoT device, wherein the second status information is sent by the second IoT device when detecting a change in its own status.

19. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the inter-device communication method as described in any one of claims 1-14.

20. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the inter-device communication method as described in any one of claims 15-18.

21. A device communication system, characterized in that: The system includes a first Internet of Things device and a second Internet of Things device, the first Internet of Things device is the electronic device described in claim 19, and the second Internet of Things device is the electronic device described in claim 20.

Images