Intelligent device control method, device, storage medium and device

Abstract

The present disclosure relates to a method, device, storage medium and equipment for controlling intelligent devices to improve the intelligence of intelligent device control. The method includes: in response to receiving a control instruction initiated to a target device, controlling a target virtual device in a virtual world to execute the control instruction, and obtaining virtual operation information, wherein the target virtual device is a mapping object of a target device in the real world in the virtual world, and the virtual operation information includes the scope of influence of the target virtual device on the virtual world after executing the control instruction; determining the target virtual objects in the virtual world within the scope of influence, and the degree of influence of the control instruction on the target virtual objects; determining the instruction execution strategy according to the degree of influence; and controlling the target device to execute the control instruction according to the instruction execution strategy.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent control technology, and in particular, to an intelligent device control method, device, storage medium and device. Background Art

[0002] With the development of intelligent control technology, various intelligent devices have emerged. In the related technology, different mode experiences can be brought to users through the coordinated operation of multiple intelligent devices. For example, in the "good night mode", the bedroom lights, curtains, and TV devices in the turned-on state can be automatically turned off according to the combination instructions corresponding to the mode. For example, the corresponding lighting equipment can be controlled to turn off according to the light-off instruction issued by the user. However, the method of directly executing the combination instructions corresponding to the mode or other single control instructions is not flexible enough and has a low degree of intelligence. Summary of the invention

[0003] The purpose of the present disclosure is to provide a smart device control method, device, storage medium and device to solve the problems existing in the related art.

[0004] In order to achieve the above-mentioned purpose, a first aspect of an embodiment of the present disclosure provides a smart device control method, the method comprising:

[0005] In response to receiving a control instruction initiated to a target device, controlling a target virtual device in a virtual world to execute the control instruction, and obtaining virtual operation information, wherein the target virtual device is a mapping object of the target device in the real world in the virtual world, and the virtual operation information includes an impact range of the target virtual device on the virtual world after executing the control instruction;

[0006] Determining a target virtual object in the virtual world that is within the influence range, and the degree of influence of the control instruction on the target virtual object;

[0007] Determining the instruction execution strategy according to the degree of impact;

[0008] The target device is controlled to execute the control instruction according to the instruction execution policy.

[0009] Optionally, determining the degree of influence of the control instruction on the target virtual object includes:

[0010] Acquire target object data corresponding to the target virtual object;

[0011] The degree of influence of the control instruction on the target virtual object is determined according to the target object data of the target virtual object, the virtual operation information and the environment information of the virtual world.

[0012] Optionally, determining the influence degree of the control instruction on the target virtual object according to the target object data of the target virtual object, the virtual operation information and the environment information of the virtual world includes:

[0013] The target object data of the target virtual object, the virtual operation information and the environmental information of the virtual world are input into the trained evaluation model to obtain a result output by the trained evaluation model that represents the degree of influence of the control instruction on the target virtual object.

[0014] Optionally, the target virtual object includes a target influence object corresponding to the control instruction, and other virtual objects within the influence range except the target influence object, and accordingly, the influence degree includes a first influence degree on the target influence object and a second influence degree on the other virtual objects;

[0015] Determining the instruction execution strategy according to the impact degree includes:

[0016] When the value of the first impact degree is greater than or equal to a first preset threshold value, and the value of the second impact degree is less than a second preset threshold value, determining the first preset strategy as the instruction execution strategy;

[0017] The first preset policy is one of the candidate policies corresponding to the target device under the control instruction.

[0018] Optionally, determining the instruction execution strategy according to the impact degree includes:

[0019] When the value of the first impact degree is less than the first preset threshold and the value of the second impact degree is less than the second preset threshold, or when the value of the first impact degree is greater than or equal to the first preset threshold and the value of the second impact degree is greater than or equal to the second preset threshold, determine the second preset strategy as the instruction execution strategy;

[0020] The second preset policy is one of the candidate policies corresponding to the target device under the control instruction.

[0021] Optionally, determining the instruction execution strategy according to the impact degree includes:

[0022] When the value of the first impact degree is less than the first preset threshold value, and the value of the second impact degree is greater than or equal to the second preset threshold value, determining a third preset strategy as the instruction execution strategy;

[0023] The third preset strategy is one of the candidate strategies that the target device corresponds to under the control instruction.

[0024] Optionally, when the number of the target virtual objects is 1, determining the instruction execution strategy according to the impact degree includes:

[0025] When the impact value of the impact degree is greater than or equal to a third preset threshold, determining the first preset strategy as the instruction execution strategy;

[0026] When the impact value is less than the third preset threshold, determining the second preset strategy or the third preset strategy as the instruction execution strategy;

[0027] The candidate policies corresponding to the target device under the control instruction include the first preset policy, the second preset policy and the third preset policy.

[0028] Optionally, after controlling the target device to execute the control instruction according to the instruction execution policy, the method further includes:

[0029] The virtual operation information of the target virtual device is updated according to the actual operation information of the target device, so that the virtual operation information of the target virtual device is consistent with the actual operation information of the target device.

[0030] Optionally, the virtual world is a digital twin world, and the method further includes:

[0031] The target device in the real world is mapped to the virtual world through digital twin technology to obtain the target virtual device.

[0032] A second aspect of an embodiment of the present disclosure provides a smart device control device, the device comprising:

[0033] a response module, configured to, in response to receiving a control instruction initiated to a target device, control a target virtual device in a virtual world to execute the control instruction, and obtain virtual operation information, wherein the target virtual device is a mapping object of the target device in the real world in the virtual world, and the virtual operation information includes an impact range of the target virtual device on the virtual world after executing the control instruction;

[0034] A first determination module is used to determine a target virtual object in the virtual world that is within the influence range, and the influence degree of the control instruction on the target virtual object;

[0035] A second determination module, used to determine an instruction execution strategy according to the impact degree;

[0036] A control module is used to control the target device to execute the control instruction according to the instruction execution strategy.

[0037] A third aspect of the present disclosure provides an electronic device, including:

[0038] a memory having a computer program stored thereon;

[0039] A processor is used to execute the computer program in the memory to implement the steps of the method in the first aspect.

[0040] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method in the first aspect when executed by a processor.

[0041] Related technologies usually directly control the device to execute mode combination instructions or single control instructions. However, this ignores the changes in the scene corresponding to the instruction and fails to take into account the objects affected by the instruction in the scene, so that the execution effect of this method cannot be close to real life and has a low level of intelligence. Compared with the method of controlling smart devices in related technologies, the technical solution provided by the present disclosure determines the target virtual objects affected by the control instructions in the virtual world, and the degree of influence of executing the control instructions on the target virtual objects, and determines the instruction execution strategy according to the degree of influence, so that the target device in the real world can be flexibly controlled to execute the control instructions according to the instruction execution strategy. In this way, the execution effect of the control instruction can be closer to real life, avoiding inconvenience or damage to the objects affected by the control instructions, and the degree of intelligence is high.

[0042] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0044] Figure 1 The figure is a flow chart of a smart device control method according to an exemplary embodiment of the present disclosure.

[0045] Figure 2 The present invention is a block diagram of a smart device control apparatus according to an exemplary embodiment of the present invention.

[0046] Figure 3 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0048] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0049] The relevant technology can bring different mode experiences to users through the coordinated operation of multiple smart devices. For example, in the "leave the living room mode", the lights in the living room, the turned-on TV equipment and other electrical appliances can be automatically turned off according to the combination instructions corresponding to the mode. However, if user B is in the "movie viewing mode" in the living room, if user A triggers the "leave the living room mode", then directly controlling the smart device according to the combination instructions corresponding to the "leave the living room mode" will cause inconvenience to user B. It can be seen that the method of directly controlling the target device to execute the mode combination instruction or a single control instruction by the relevant technology cannot observe the situation of the scene corresponding to the instruction, and cannot take into account the objects affected by the instruction in the scene, so that the execution effect of this method cannot be close to real life and the intelligence level is low.

[0050] In view of this, the embodiment of the present disclosure provides a smart device control method, which determines the target virtual object affected by the control instruction in the virtual world, and the degree of influence of executing the control instruction on the target virtual object, and determines the instruction execution strategy according to the degree of influence, so that the target device in the real world can be flexibly controlled to execute the control instruction according to the instruction execution strategy. In this way, the execution effect of the control instruction can be closer to real life, avoiding inconvenience or damage to the object affected by the control instruction, and the degree of intelligence is high.

[0051] It should be noted that the intelligent device control method provided in the embodiment of the present disclosure can be applied to an intelligent device control terminal, which may include a cloud controller and a local controller. Among them, the cloud controller may be a cloud brain set in a cloud server, and the cloud brain may integrate intelligent technology with human assistance (HI, Human Intelligence) to provide efficient and secure cloud intelligent operation services for robots and intelligent devices. The local controller may refer to a local control device, such as a mobile phone, smart phone, laptop computer, digital broadcast receiver, personal digital assistant (PDA), tablet computer (PDA) and other user equipment (User Equipment, UE). Both the cloud controller and the local controller can load a virtual world corresponding to the real world, and the synchronization of the virtual world can be achieved through communication between the two.

[0052] Among them, the virtual world can refer to the digital twin world, which can collect sensor data from the real world and construct the digital twin world through digital twin technology. It is not difficult to understand that the virtual environment information in the digital twin world (that is, the virtual world) is consistent with the environment information in the real world. The virtual environment information can include virtual devices (such as virtual lights, virtual televisions, and virtual air conditioners, etc.), virtual animals and plants, virtual buildings, virtual characters, virtual light, virtual water, virtual roads, and virtual transportation tools mapped from the real world.

[0053] In one embodiment, the virtual environment information in the virtual world can be updated based on the environment information collected in real time by the target device (i.e., the device controlled by the control instruction) and / or the environment information collected by other devices in the real world, so that the virtual environment information in the virtual world is consistent with the environment information in the real world.

[0054] It should be noted that smart devices (which may include target devices and other devices) may refer to devices that can receive and / or share data, such as smart lights, smart home appliances, smart speakers, camera lights and other IoT devices. Smart devices may also refer to smart sensors, such as meter-wave radars, visual cameras, ultrasonic sensors, radar sensors and infrared sensors.

[0055] In addition, the connection between the real world and the virtual world can be established through communication technologies such as wired networks, Wi-Fi, Bluetooth, cameras, ultra-wideband (UWB), ultrasound, lidar, millimeter-wave radar, and infrared to obtain changes in the real world (such as device changes caused by users operating smart devices) and to feed back changes in the virtual world to the real world (such as transmitting control instructions to corresponding devices in the real world).

[0056] Reference Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for controlling a smart device according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the smart device control method may include:

[0057] S101, in response to receiving a control instruction initiated to a target device, controlling a target virtual device in a virtual world to execute the control instruction to obtain virtual operation information.

[0058] The target virtual device is a mapping object of the target device in the real world in the virtual world, and the virtual operation information includes the impact range of the target virtual device on the virtual world after executing the control instruction.

[0059] It should be noted that the control instruction can be initiated by a local user or by a cloud user. The user can initiate the control instruction through voice or through the program software on the terminal device, and the present disclosure does not make specific limitations on this. The target device may refer to the device controlled by the control instruction, and the number of target devices may be one or more. Since the virtual environment information of the virtual world is consistent with the environment information of the real world, and the target virtual device is the mapping object of the target device in the real world in the virtual world, the virtual operation information obtained by simulating the control of the target virtual device in the virtual world to execute the control instruction is consistent with the operation information obtained by the target device in the real world to execute the control instruction. The scope of influence of the target virtual device included in the virtual operation information on the virtual world after executing the control instruction is also consistent with the scope of influence of the target device on the real world after executing the control instruction.

[0060] It is not difficult to understand that the digital twin technology can be used to map the target device in the real world to the virtual world to obtain the target virtual device, and the digital twin technology can be used to simulate the operation information obtained by the target device in the real world executing control instructions to obtain the virtual operation information obtained by the target virtual device executing control instructions in the virtual world.

[0061] S102, determining a target virtual object within an influence range in the virtual world, and the influence degree of the control instruction on the target virtual object.

[0062] S103, determining the instruction execution strategy according to the impact degree.

[0063] Among them, the target virtual object may refer to a virtual device and a virtual character (corresponding to a device and a character in the real world) affected by the control instruction. It is not difficult to understand that when the virtual environment information of the virtual world is consistent with the environment information of the real world, the target virtual object determined to be within the influence range is also consistent with the target object within the influence range in the real world. On this basis, the degree of influence of the control instruction on the target virtual object can be determined, and the instruction execution strategy can be determined according to the degree of influence.

[0064] S104, controlling the target device to execute the control instruction according to the instruction execution policy.

[0065] It should be understood that the target device can be flexibly controlled to execute control instructions according to the instruction execution strategy to improve the flexibility of instruction execution and the intelligence of instruction execution effect, thereby improving user experience.

[0066] The technical solution provided by the present disclosure determines the target virtual object affected by the control instruction in the virtual world, and the degree of influence of the execution of the control instruction on the target virtual object, and determines the instruction execution strategy according to the degree of influence, so that the target device in the real world can be flexibly controlled to execute the control instruction according to the instruction execution strategy. In this way, the execution effect of the control instruction can be closer to real life, avoiding inconvenience or damage to the object affected by the control instruction, and the degree of intelligence is high.

[0067] It should be noted that through digital twin technology, semantic maps can be established in the virtual world, such as establishing an ID for a virtual light in the virtual world (the ID can be used to identify the virtual light). It is also possible to simulate the parameters / attributes, operating range, and functions / roles of devices in the real world, and set operating parameters / attributes, operating range, and operating capabilities for virtual devices in the virtual world.

[0068] Among them, for setting operating parameters / attributes, for example, the operating parameters / attributes of the virtual light may include on / off, brightness adjustment, color temperature adjustment, color adjustment, whether it can be timed, delayed, voice controlled, whether there is button adjustment, whether electric auxiliary operation is required, and remaining power, etc. The operating parameters / attributes of the virtual water cup may include the size of the base, the capacity of the water cup, and whether the water cup can be placed on a platform larger than the base, etc. For setting the operating range, for example, the operating range of the virtual millimeter-wave radar can be set to simulate the range that can be detected by the millimeter-wave radar in the real world. For setting the operating capacity, for example, for the function of a water cup holding water in the real world, the operating capacity of the virtual water cup can be set to the water cup can hold water, and for example, for the function of a lamp that can illuminate in the real world, the operating capacity of the virtual lamp can be set to the lamp can illuminate.

[0069] On this basis, the virtual operation information may include the operation parameters / attributes, operation range, and operation capability of the virtual device. It can be understood that, when there is one target device, the influence range included in the virtual operation information is the operation range of the target device. When there are multiple target devices, the influence range included in the virtual operation information is the union of the operation ranges of the multiple target devices.

[0070] In addition, the virtual operation information may also include fault information, which may indicate whether the virtual device has a fault. When the fault information indicates that the virtual device has a fault (e.g., device aging), it may indicate that the device corresponding to the virtual device in the real world also has a fault risk. In this case, a prompt message may be sent to the user.

[0071] Optionally, in the above step S102, determining the influence degree of the control instruction on the target virtual object may include:

[0072] Obtain target object data corresponding to the target virtual object;

[0073] The degree of influence of the control instruction on the target virtual object is determined according to the target object data of the target virtual object, the virtual operation information and the environment information of the virtual world.

[0074] Where the target virtual object is a virtual character, the target object data may include the historical habits (e.g., historical usage duration), gender, age, preferences, and special matters of the virtual character. Where the target virtual object is a virtual device, the target object data may include virtual operation information of the virtual device.

[0075] It is not difficult to understand that the degree of influence of the control instruction on the target virtual object can be determined based on the target object data of the target virtual object, the virtual operation information and the environment information of the virtual world.

[0076] Optionally, determining the influence degree of the control instruction on the target virtual object according to the target object data, the virtual operation information and the environment information of the virtual world of the target virtual object may include:

[0077] The target object data, virtual operation information and environment information of the virtual world of the target virtual object are input into the trained evaluation model to obtain a result output by the trained evaluation model representing the degree of influence of the control instruction on the target virtual object.

[0078] Among them, the evaluation model can be trained under the training framework of reinforcement learning. For example, sample data such as target object data samples, virtual operation information samples, and environmental information samples of the virtual world can be obtained by simulating the real world, and the evaluation model can be trained in the virtual world based on these sample data. In the training process of reinforcement learning, the training parameters of the evaluation model can be adjusted according to the results of the evaluation model output that characterize the degree of influence of the control instruction on the target virtual object and the label value corresponding to the result. In this way, after multiple iterations of training, a trained evaluation model can be obtained.

[0079] It should also be noted that the operating parameters / attributes of the virtual devices in the virtual world may also include preset pre-conditions / post-conditions. For example, the pre-condition for obtaining the virtual bowl in the virtual cupboard may be to open the virtual cupboard, and the pre-condition for placing the virtual cup is to find a virtual platform larger than the base of the virtual cup. The pre-conditions / post-conditions can assist the evaluation model in evaluating the degree of influence of the control instruction on the target virtual object. For example, when the control instruction represents placing the virtual cup on the platform, if the platform is smaller than the base of the virtual cup, the pre-condition is not met, which may indicate that the execution of the control instruction has a high degree of influence on the virtual cup. In this case, the execution of the control instruction can be stopped, and the user is prompted that the execution of the control instruction may cause damage to the cup.

[0080] Optionally, when there are multiple target virtual objects, the target virtual objects may include the target impact object corresponding to the control instruction, and other virtual objects other than the target impact object within the impact range, and accordingly, the impact degree includes a first impact degree on the target impact object and a second impact degree on the other virtual objects. On this basis, the above step S103 may include:

[0081] When the value of the first impact degree is greater than or equal to the first preset threshold, and the value of the second impact degree is less than the second preset threshold, the first preset strategy is determined as the instruction execution strategy.

[0082] The first preset strategy is one of the candidate strategies corresponding to the target device under the control instruction.

[0083] It should be noted that the first preset threshold and the second preset threshold can be training parameters in the evaluation model training process, and the first preset threshold and the second preset threshold can be adjusted according to the result of the iterative training of the evaluation model. It is not difficult to understand that the target impact object can be the object on which the control instruction is executed (for example, when the control instruction is "good night mode", the target impact object can be a virtual character who expects to fall asleep quietly, and the virtual character can be a virtual initiator of the control instruction, or a virtual recipient of the control instruction). Other virtual objects can be virtual characters and / or virtual devices other than the target impact object within the influence range.

[0084] It should also be noted that the value of the first degree of influence being greater than or equal to the first preset threshold value may indicate that the execution of the control instruction has a high degree of influence on the target affected object, that is, the execution effect of the control instruction meets expectations. Similarly, the value of the second degree of influence being less than the second preset threshold value may indicate that the execution of the control instruction has a low degree of influence on other virtual objects, that is, the possibility of the execution of the control instruction causing inconvenience or damage to other virtual objects is small. In this case, the first preset strategy may be determined as the instruction execution strategy. The first preset strategy may be to directly execute the control instruction.

[0085] Optionally, the above step S103 may further include:

[0086] When the value of the first impact degree is less than the first preset threshold and the value of the second impact degree is less than the second preset threshold, or when the value of the first impact degree is greater than or equal to the first preset threshold and the value of the second impact degree is greater than or equal to the second preset threshold, the second preset strategy is determined as the instruction execution strategy.

[0087] The second preset strategy is one of the candidate strategies corresponding to the target device under the control instruction.

[0088] In one embodiment, the numerical value of the first degree of influence being less than the first preset threshold value may indicate that the degree of influence of the execution control instruction on the target influence object is low, that is, the execution effect of the control instruction does not meet expectations. Wherein not meeting expectations may, for example, mean that when the target influence object is leaving or has left the influence range of the execution control instruction, the execution control instruction cannot act on the target influence object or the degree of action is low. Similarly, the numerical value of the second degree of influence being less than the second preset threshold value may indicate that the degree of influence of the execution control instruction on other virtual objects is low, that is, the possibility of the execution of the control instruction causing inconvenience or damage to other virtual objects is small. In this case, the second preset strategy may be determined as the instruction execution strategy. The second preset strategy may be to postpone the execution of the control instruction or to weaken the execution of the control instruction. In a specific implementation, it may be determined whether the second preset strategy is to postpone the execution of the control instruction or to weaken the execution of the control instruction according to actual conditions, and the present disclosure does not make specific limitations on this.

[0089] For example, when the execution control instruction has a low degree of influence on the target influence object and a low degree of influence on other virtual objects, the execution control instruction can be weakened for the target influence object that is leaving, and when it is determined that the target influence object has left the scope of influence of the execution control instruction, the execution control instruction can be stopped. In addition, the evaluation model can determine that the target influence object that is leaving or has left may return to the scope of influence of the execution control instruction after a preset time period based on the historical habits of the target influence object. In this case, the control instruction can be executed after the preset time period (i.e., the execution of the control instruction can be delayed).

[0090] In another embodiment, a value of the first degree of influence greater than or equal to the first preset threshold value may indicate that the execution of the control instruction has a high degree of influence on the target affected object, that is, the execution effect of the control instruction meets expectations. Similarly, a value of the second degree of influence greater than or equal to the second preset threshold value may indicate that the execution of the control instruction has a high degree of influence on other virtual objects, that is, the execution of the control instruction is likely to cause inconvenience or damage to other virtual objects. In this case, the second preset strategy can be determined as the instruction execution strategy. The second preset strategy may be to postpone the execution of the control instruction or to weaken the execution of the control instruction. In a specific implementation, it can be determined whether the second preset strategy is to postpone the execution of the control instruction or to weaken the execution of the control instruction according to actual conditions, and the present disclosure does not make specific limitations on this.

[0091] For example, when the execution control instruction has a high degree of influence on the target affected object and a high degree of influence on other virtual objects, the evaluation model can determine that other virtual objects may leave the influence range of the execution control instruction after a preset time period based on their historical habits. In this case, the control instruction can be executed after the preset time period (i.e., the execution control instruction can be postponed). The execution control instruction can also be weakened, and the control instruction can be executed normally when it is determined that other virtual objects have left the influence range of the execution control instruction.

[0092] Optionally, the above step S103 may further include:

[0093] When the value of the first impact degree is less than the first preset threshold and the value of the second impact degree is greater than or equal to the second preset threshold, the third preset strategy is determined as the instruction execution strategy.

[0094] The third preset strategy is one of the candidate strategies corresponding to the target device under the control instruction.

[0095] It is understandable that the value of the first degree of influence being less than the first preset threshold value may indicate that the degree of influence of the execution of the control instruction on the target affected object is low, that is, the execution effect of the control instruction does not meet expectations. Similarly, the value of the second degree of influence being greater than or equal to the second preset threshold value may indicate that the degree of influence of the execution of the control instruction on other virtual objects is high, that is, the possibility of the execution of the control instruction causing inconvenience or damage to other virtual objects is high. In this case, the third preset strategy may be determined as the instruction execution strategy. The third preset strategy may be to weaken the execution of the control instruction or not to execute the control instruction.

[0096] For example, when the control instruction represents placing a virtual water cup on a platform (at this time, the target impact object of the target virtual object is the virtual initiator who initiated the control instruction, and other virtual objects include the virtual water cup), if the platform is smaller than the base of the virtual water cup, the precondition is not met. In this case, the value of the first degree of impact may be less than the first preset threshold, and the second degree of impact may be greater than or equal to the second preset threshold. That is, the execution of the control instruction has a low impact on the virtual initiator (the virtual water cup cannot be safely placed as expected), and a high impact on the virtual water cup (the virtual water cup may be damaged). In this case, the control instruction may not be executed or stopped, and the user is prompted that executing the control instruction may cause damage to the water cup.

[0097] For another example, when the execution control instruction has a low degree of influence on the target influence object and a high degree of influence on other virtual objects, the evaluation model can determine that the target influence object that is leaving or has left may return to the influence range of the execution control instruction based on the historical habits of the target influence object, and determine that other virtual objects may leave the influence range of the execution control instruction based on their historical habits. In this case, the execution control instruction can be weakened, and the control instruction can be executed normally when it is determined that the target influence object has returned to the influence range and other virtual objects have left the influence range of the execution control instruction.

[0098] It should also be noted that, although the candidate strategies corresponding to the target device under the control instruction may include the first preset strategy, the second preset strategy and the third preset strategy, different target devices make slightly different adjustments to the candidate strategies corresponding to different control instructions.

[0099] For example, under the control instruction corresponding to the "goodnight mode", the target device may include lights, curtains, and television devices, etc. On this basis, the first preset strategy may be to directly execute the control instruction, that is, to turn off the lights, curtains, and television devices, etc. The second preset strategy may be to delay the execution of the control instruction or to weaken the execution of the control instruction, wherein the delayed execution of the control instruction may refer to turning off the lights, curtains, and television devices, etc. after a preset time (the preset time may be determined according to the actual situation, and the present disclosure does not specifically limit this), or after the affected object (for example, there is an object C that needs lights in the scene of the "goodnight mode") is no longer affected (for example, object C leaves the scene of the "goodnight mode"). Weakening the execution of the control instruction may refer to dimming the brightness of the lights, closing some curtains (for example, closing the gauze curtains when the curtains include blackout curtains and gauze curtains), and lowering the volume of the TV, etc. The third preset strategy may be to weaken the execution or non-execution of the control instruction.

[0100] For example, in the "nursing home mode", the user initiates a water pouring command, and the target devices may include a water dispenser, a water cup, and a water pouring robot. On this basis, the first preset strategy may be to directly execute the control command, that is, to control the water pouring robot to pick up the water cup to the water dispenser to pour water, and deliver the water cup to the user. The second preset strategy may be to delay the execution of the control command or to weaken the execution of the control command, where delaying the execution of the control command may refer to executing the control command after a preset time. Weakening the execution of the control command may refer to raising / lowering the temperature of the water outlet of the water dispenser so that the outlet water temperature is suitable for drinking by the elderly. The third preset strategy may be to weaken the execution or not execute the control command.

[0101] It is not difficult to understand that for the pouring water instruction in the "nursing home mode", the target virtual object may include the target influence object (such as a virtual character who needs to drink water), and other virtual objects (such as a virtual water dispenser). Since the virtual water dispenser is usually provided with a preset water outlet temperature, when the preset water outlet temperature is inconsistent with the water outlet temperature in the "nursing home mode", the value of the first influence degree may be less than the first preset threshold value, and the second influence degree may be greater than or equal to the second preset threshold value. That is, the execution of the control instruction has a low degree of influence on the target influence object (it is not possible to obtain drinking water with a water outlet temperature suitable for the elderly as expected), and a high degree of influence on the virtual water dispenser (the preset water outlet temperature of the virtual water dispenser is inconsistent with the water outlet temperature in the "nursing home mode"). In this case, the execution of the control instruction can be weakened, that is, it means raising / lowering the water outlet temperature of the water dispenser so that the water outlet temperature is suitable for the elderly to drink. It should be understood that the drinking water temperature obtained by directly executing the pouring water instruction in the related technology is usually consistent with the preset water outlet temperature of the water dispenser. Compared with the related art, when the preset water outlet temperature of the water dispenser is high, the weakening execution instruction may refer to lowering the temperature of the water outlet of the water dispenser. When the preset water outlet temperature of the water dispenser is low, the weakening execution instruction may refer to raising the temperature of the water outlet of the water dispenser.

[0102] Optionally, when the number of target virtual objects is 1, the target virtual object may be the virtual initiator of the control instruction, or another virtual object within the influence range (in this case, the virtual initiator is not within the influence range). On this basis, the above step S103 may include:

[0103] When the impact value of the impact degree is greater than or equal to the third preset threshold, determining the first preset strategy as the instruction execution strategy;

[0104] When the impact value is less than the third preset threshold, the second preset strategy or the third preset strategy is determined as the instruction execution strategy.

[0105] The candidate strategies corresponding to the target device under the control instruction include a first preset strategy, a second preset strategy and a third preset strategy.

[0106] It should be noted that the third preset threshold may be a training parameter in the evaluation model training process, and the third preset threshold may be adjusted according to the result of the iterative training of the evaluation model.

[0107] In one embodiment, if the influence value of the influence degree is greater than or equal to the first preset threshold, it can indicate that the influence degree of the execution control instruction on the target virtual object is high, that is, the execution effect of the control instruction meets expectations. In this case, the first preset strategy can be determined as the instruction execution strategy.

[0108] In another embodiment, if the impact value of the impact degree is less than the second preset threshold, it can indicate that the impact degree of executing the control instruction on the target virtual object is low, that is, the execution effect of the control instruction does not meet expectations. In this case, the second preset strategy or the third preset strategy can be determined as the instruction execution strategy. In a specific implementation, the second preset strategy or the third preset strategy can be adopted according to the actual situation, and the present disclosure does not make specific limitations on this.

[0109] Optionally, after controlling the target device to execute the control instruction according to the instruction execution policy, the technical solution provided by the embodiment of the present disclosure may further include:

[0110] The virtual operation information of the target virtual device is updated according to the actual operation information of the target device, so that the virtual operation information of the target virtual device is consistent with the actual operation information of the target device.

[0111] It is not difficult to understand that after controlling the target device in the real world to execute control instructions according to the instruction execution strategy, in order to maintain the consistency between the real world and the virtual world, the virtual operation information of the target virtual device can be updated according to the actual operation information of the target device, so that the virtual operation information of the target virtual device is consistent with the actual operation information of the target device.

[0112] The technical solution provided by the present disclosure determines the target virtual object affected by the control instruction in the virtual world, and the degree of influence of the execution of the control instruction on the target virtual object, and determines the instruction execution strategy according to the degree of influence, so that the target device in the real world can be flexibly controlled to execute the control instruction according to the instruction execution strategy. In this way, the execution effect of the control instruction can be closer to real life, avoiding inconvenience or damage to the object affected by the control instruction, and the degree of intelligence is high.

[0113] Based on the same inventive concept, the present disclosure also provides a smart device control device 100, referring to Figure 2 , Figure 2 1 is a block diagram of an intelligent device control apparatus 100 according to an exemplary embodiment of the present disclosure. The intelligent device control apparatus 100 includes:

[0114] The response module 101 is used to control the target virtual device in the virtual world to execute the control instruction in response to receiving the control instruction initiated to the target device, and obtain virtual operation information, wherein the target virtual device is a mapping object of the target device in the real world in the virtual world, and the virtual operation information includes the impact range of the target virtual device on the virtual world after executing the control instruction;

[0115] A first determination module 102 is used to determine a target virtual object within an influence range in the virtual world, and the degree of influence of the control instruction on the target virtual object;

[0116] A second determination module 103, used to determine the instruction execution strategy according to the impact degree;

[0117] The control module 104 is used to control the target device to execute the control instruction according to the instruction execution policy.

[0118] The technical solution provided by the present disclosure determines the target virtual object affected by the control instruction in the virtual world, and the degree of influence of the execution of the control instruction on the target virtual object, and determines the instruction execution strategy according to the degree of influence, so that the target device in the real world can be flexibly controlled to execute the control instruction according to the instruction execution strategy. In this way, the execution effect of the control instruction can be closer to real life, avoiding inconvenience or damage to the object affected by the control instruction, and the degree of intelligence is high.

[0119] Optionally, the first determining module 102 is further configured to:

[0120] Obtain target object data corresponding to the target virtual object;

[0121] The degree of influence of the control instruction on the target virtual object is determined according to the target object data of the target virtual object, the virtual operation information and the environment information of the virtual world.

[0122] Optionally, the first determining module 102 is further configured to:

[0123] The target object data, virtual operation information and environment information of the virtual world of the target virtual object are input into the trained evaluation model to obtain a result output by the trained evaluation model representing the degree of influence of the control instruction on the target virtual object.

[0124] Optionally, the target virtual object includes a target influence object corresponding to the control instruction, and other virtual objects within the influence range except the target influence object, and accordingly, the influence degree includes a first influence degree on the target influence object and a second influence degree on the other virtual objects;

[0125] The second determining module 103 is further configured to:

[0126] When the value of the first impact degree is greater than or equal to the first preset threshold value, and the value of the second impact degree is less than the second preset threshold value, determining the first preset strategy as the instruction execution strategy;

[0127] The first preset policy is one of the candidate policies corresponding to the target device under the control instruction.

[0128] Optionally, the second determining module 103 is further configured to:

[0129] When the value of the first impact degree is less than the first preset threshold value and the value of the second impact degree is less than the second preset threshold value, or when the value of the first impact degree is greater than or equal to the first preset threshold value and the value of the second impact degree is greater than or equal to the second preset threshold value, the second preset strategy is determined as the instruction execution strategy;

[0130] The second preset policy is one of the candidate policies corresponding to the target device under the control instruction.

[0131] Optionally, the second determining module 103 is further configured to:

[0132] When the value of the first impact degree is less than the first preset threshold value, and the value of the second impact degree is greater than or equal to the second preset threshold value, determining the third preset strategy as the instruction execution strategy;

[0133] The third preset policy is one of the candidate policies corresponding to the target device under the control instruction.

[0134] Optionally, when the number of target virtual objects is 1, the second determining module 103 is further configured to:

[0135] When the impact value of the impact degree is greater than or equal to the third preset threshold, determining the first preset strategy as the instruction execution strategy;

[0136] When the impact value is less than a third preset threshold, determining the second preset strategy or the third preset strategy as the instruction execution strategy;

[0137] The candidate strategies corresponding to the target device under the control instruction include a first preset strategy, a second preset strategy and a third preset strategy.

[0138] Optionally, the intelligent device control apparatus 100 further includes an updating module, which is used to:

[0139] After controlling the target device to execute the control instruction according to the instruction execution policy, the virtual operation information of the target virtual device is updated according to the actual operation information of the target device, so that the virtual operation information of the target virtual device is consistent with the actual operation information of the target device.

[0140] Optionally, the virtual world is a digital twin world, and the intelligent device control device 100 further includes a mapping module, which is used to:

[0141] Through digital twin technology, the target device in the real world is mapped to the virtual world to obtain the target virtual device.

[0142] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0143] Based on the same inventive concept, the present disclosure also provides an electronic device, referring to Figure 3 , Figure 3 FIG. 2 is a block diagram of an electronic device 200 according to an exemplary embodiment of the present disclosure. Figure 3 As shown, the electronic device 200 may include: a processor 201 and a memory 202. The electronic device 200 may also include one or more of a multimedia component 203, an input / output (I / O) interface 204, and a communication component 205.

[0144] The processor 201 is used to control the overall operation of the electronic device 200 to complete all or part of the steps in the above-mentioned intelligent device control method. The memory 202 is used to store various types of data to support the operation of the electronic device 200, and these data may include, for example, instructions for any application or method used to operate on the electronic device 200, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 202 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 203 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 202 or sent through the communication component 205. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 204 provides an interface between the processor 201 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 205 is used for wired or wireless communication between the electronic device 200 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 205 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0145] In an exemplary embodiment, the electronic device 200 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned smart device control method.

[0146] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned smart device control method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 202 including program instructions, and the above-mentioned program instructions can be executed by the processor 201 of the electronic device 200 to complete the above-mentioned smart device control method.

[0147] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned smart device control method when executed by the programmable device.

[0148] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0149] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0150] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for controlling an intelligent device, characterized in that: The method comprises: In response to receiving a control instruction initiated to a target device, controlling a target virtual device in a virtual world to execute the control instruction, and obtaining virtual operation information, wherein the target virtual device is a mapping object of the target device in the real world in the virtual world, and the virtual operation information includes an impact range of the target virtual device on the virtual world after executing the control instruction; Determining a target virtual object in the virtual world that is within the influence range, and the degree of influence of the control instruction on the target virtual object; Determining the instruction execution strategy according to the degree of impact; Controlling the target device to execute the control instruction according to the instruction execution strategy; Wherein, determining the influence degree of the control instruction on the target virtual object includes: Acquire target object data corresponding to the target virtual object; Determining the degree of influence of the control instruction on the target virtual object according to the target object data of the target virtual object, the virtual operation information and the environment information of the virtual world; The determining, according to the target object data of the target virtual object, the virtual operation information, and the environment information of the virtual world, the degree of influence of the control instruction on the target virtual object comprises: Inputting the target object data of the target virtual object, the virtual operation information, and the environment information of the virtual world into the trained evaluation model, and obtaining a result output by the trained evaluation model that represents the degree of influence of the control instruction on the target virtual object; The target virtual object includes the target influence object corresponding to the control instruction, and other virtual objects within the influence range except the target influence object, and accordingly, the influence degree includes a first influence degree on the target influence object and a second influence degree on the other virtual objects; Determining the instruction execution strategy according to the impact degree includes: When the value of the first impact degree is greater than or equal to a first preset threshold value, and the value of the second impact degree is less than a second preset threshold value, determining the first preset strategy as the instruction execution strategy; The first preset policy is one of the candidate policies corresponding to the target device under the control instruction.

2. The method according to claim 1, characterized in that Determining the instruction execution strategy according to the impact degree includes: When the value of the first impact degree is less than the first preset threshold and the value of the second impact degree is less than the second preset threshold, or when the value of the first impact degree is greater than or equal to the first preset threshold and the value of the second impact degree is greater than or equal to the second preset threshold, determine the second preset strategy as the instruction execution strategy; The second preset policy is one of the candidate policies corresponding to the target device under the control instruction.

3. The method according to claim 1, characterized in that: Determining the instruction execution strategy according to the impact degree includes: When the value of the first impact degree is less than the first preset threshold value, and the value of the second impact degree is greater than or equal to the second preset threshold value, determining a third preset strategy as the instruction execution strategy; The third preset strategy is one of the candidate strategies that the target device corresponds to under the control instruction.

4. The method according to claim 1, characterized in that: When the number of the target virtual objects is 1, determining the instruction execution strategy according to the impact degree includes: When the impact value of the impact degree is greater than or equal to a third preset threshold, determining the first preset strategy as the instruction execution strategy; When the impact value is less than the third preset threshold, determining the second preset strategy or the third preset strategy as the instruction execution strategy; The candidate policies corresponding to the target device under the control instruction include the first preset policy, the second preset policy and the third preset policy.

5. The method according to claim 1, characterized in that After controlling the target device to execute the control instruction according to the instruction execution strategy, the method further includes: The virtual operation information of the target virtual device is updated according to the actual operation information of the target device, so that the virtual operation information of the target virtual device is consistent with the actual operation information of the target device.

6. The method according to claim 1, characterized in that The virtual world is a digital twin world, and the method further includes: The target device in the real world is mapped to the virtual world through digital twin technology to obtain the target virtual device.

7. A smart device control device, characterized in that: The device comprises: a response module, configured to, in response to receiving a control instruction initiated to a target device, control a target virtual device in a virtual world to execute the control instruction, and obtain virtual operation information, wherein the target virtual device is a mapping object of the target device in the real world in the virtual world, and the virtual operation information includes an impact range of the target virtual device on the virtual world after executing the control instruction; A first determination module is used to determine a target virtual object in the virtual world that is within the influence range, and the influence degree of the control instruction on the target virtual object; A second determination module, used to determine an instruction execution strategy according to the impact degree; A control module, used for controlling the target device to execute the control instruction according to the instruction execution strategy; The first determining module is further configured to: Acquire target object data corresponding to the target virtual object; Determining the degree of influence of the control instruction on the target virtual object according to the target object data of the target virtual object, the virtual operation information and the environment information of the virtual world; The first determining module is further configured to: Inputting the target object data of the target virtual object, the virtual operation information, and the environment information of the virtual world into the trained evaluation model, and obtaining a result output by the trained evaluation model that represents the degree of influence of the control instruction on the target virtual object; The target virtual object includes the target influence object corresponding to the control instruction, and other virtual objects within the influence range except the target influence object, and accordingly, the influence degree includes a first influence degree on the target influence object and a second influence degree on the other virtual objects; The second determining module is further used for: When the value of the first impact degree is greater than or equal to a first preset threshold value, and the value of the second impact degree is less than a second preset threshold value, determining the first preset strategy as the instruction execution strategy; The first preset policy is one of the candidate policies corresponding to the target device under the control instruction.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

Images