Internet of Things equipment control method and device and AR head-mounted display equipment

Abstract

The invention provides an Internet of Things equipment control method and device using ultra-bandwidth and visual image technology, and AR head-mounted display equipment, and relates to the field of Internet of Things and expansion range. The method comprises the following steps: firstly, transmitting a request signal to each piece of Internet of Things equipment through an Internet of Things module; secondly, receiving response signals returned by the Internet of Things devices, and obtaining the distance between the user and the Internet of Things devices based on the receiving time of the response signals so as to update the position of the user; collecting a dynamic image following the visual field of the user through the camera module, and comparing the appearing picture of the Internet of Things equipment with a corresponding equipment appearance image; and finally, when the position of the user and an image comparison result trigger a preset condition, a range expansion method is adopted to instruct the user whether to perform virtual control on the Internet of Things equipment in a three-dimensional space. The transmittable attribute of the ultra-bandwidth technology is fully utilized, and the visual image technology is combined, so that the Internet of Things equipment is controlled along with the visual field of the user, and the convenience, the accuracy and the usability are improved.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things and its extension, and in particular to an Internet of Things device control method and device using ultra-wideband and visual image technology, and an AR head-mounted display device. Background Art

[0002] The Internet of Things (IoT) refers to connecting any object to the network through information sensing devices according to agreed protocols. Objects exchange and communicate information through information dissemination media to achieve intelligent identification, positioning, tracking, supervision and other functions.

[0003] At present, Ultra Wide Band (UWB) technology is used to achieve control of IoT devices. Specifically, in a specific virtual space, by sending and receiving extremely narrow pulse transmission data at the nanosecond or even sub-nanosecond level, high-precision positioning is achieved, so that the target IoT device can be selected for remote control. However, the control method based only on ultra-wideband technology is too single and the triggering method is not flexible, which causes many inconveniences in practice.

[0004] In view of this, there is an urgent need to provide a new method for controlling IoT devices. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides an Internet of Things device control method, device and AR head-mounted display device using ultra-wideband and visual image technology, which solves the technical problems that the control method based on ultra-wideband technology is too single and the triggering method is inflexible.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] A method for controlling an Internet of Things device using ultra-wideband and visual image technology, comprising:

[0008] Send request signals to various IoT devices through the IoT module;

[0009] Receive a response signal returned by each of the IoT devices, and obtain the distance between the user and each IoT device based on the reception time of the response signal to update the user's location;

[0010] The camera module collects dynamic images that follow the user's field of view, and compares the IoT device images that appear in the dynamic images with the corresponding device appearance images;

[0011] When the user's location and the image comparison result trigger a preset condition, an extended range method is used to ask the user in three-dimensional space whether to virtually control the Internet of Things device.

[0012] Preferably, it also includes:

[0013] Receive a user's selection of no or virtual control of the IoT device and record the event.

[0014] Preferably, when the IoT module transmits a request signal to each IoT device for the first time, it also receives an attribute information package returned by each IoT device; wherein the attribute information package includes at least the device name, IoT identification code, control method and device appearance image.

[0015] Preferably, the Internet of Things module transmits a request signal to each Internet of Things device for the first time, performs security verification at the corresponding Internet of Things device, and receives the attribute information package after the security verification is passed.

[0016] Preferably, when the IoT device screen appears in the dynamic image, the camera module is adjusted to a continuous shooting calibration mode until the IoT device appears completely in the dynamic image.

[0017] Preferably, the trigger preset conditions include:

[0018] The user's position meets the preset requirements for the orientation and angle of the target IoT device; and when the continuously captured dynamic images are compared with the corresponding device appearance images, the number of dynamic images whose similarity continuously exceeds the preset threshold value meets the quantity requirement, and / or the comparison time whose similarity continuously exceeds the preset threshold value meets the duration requirement.

[0019] An Internet of Things device control device using ultra-wideband and visual image technology, comprising:

[0020] A request module, used to transmit a request signal to each IoT device through the IoT module;

[0021] An updating module, configured to receive a response signal returned by each of the IoT devices, and obtain the distance between the user and each IoT device based on the reception time of the response signal to update the user's location;

[0022] A comparison module, used to collect dynamic images following the user's field of view through a camera module, and compare the IoT device screen appearing in the dynamic image with the corresponding device appearance image;

[0023] The request module is used to, when the user's position and the image comparison result trigger a preset condition, adopt an extended range method to request the user in three-dimensional space whether to virtually control the Internet of Things device.

[0024] Preferably, it also includes:

[0025] The recording module is used to receive the user's selection of not performing or performing virtual control on the IoT device and record the event.

[0026] Preferably, when the IoT module transmits a request signal to each IoT device for the first time, it also receives an attribute information package returned by each IoT device; wherein the attribute information package includes at least the device name, IoT identification code, control method and device appearance image.

[0027] Preferably, the Internet of Things module transmits a request signal to each Internet of Things device for the first time, performs security verification at the corresponding Internet of Things device, and receives the attribute information package after the security verification is passed.

[0028] Preferably, when the IoT device screen appears in the dynamic image, the camera module is adjusted to a continuous shooting calibration mode until the IoT device appears completely in the dynamic image.

[0029] Preferably, the trigger preset conditions include:

[0030] The user's position meets the preset requirements for the orientation and angle of the target IoT device; and when the continuously captured dynamic images are compared with the corresponding device appearance images, the number of dynamic images whose similarity continuously exceeds the preset threshold value meets the quantity requirement, and / or the comparison time whose similarity continuously exceeds the preset threshold value meets the duration requirement.

[0031] An AR head mounted display device, characterized by comprising the Internet of Things device control device as described above.

[0032] The present invention provides an Internet of Things device control method, device and AR head mounted display device using ultra-wideband and visual image technology. Compared with the prior art, it has the following beneficial effects:

[0033] In the present invention, firstly, a request signal is transmitted to each IoT device through the IoT module; secondly, a response signal returned by each IoT device is received, and the distance between the user and each IoT device is obtained based on the reception time of the response signal to update the user's location; again, a dynamic image following the user's field of view is collected through the camera module, and the IoT device screen appearing in the dynamic image is compared with the corresponding device appearance image; finally, when the user's location and the image comparison result trigger the preset conditions, the range expansion method is adopted to ask the user whether to virtually control the IoT device in three-dimensional space. By making full use of the transmittable properties of ultra-wideband technology and combining visual image technology, the IoT device can be controlled following the user's field of view, which improves convenience, accuracy and ease of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A block diagram of an Internet of Things device control method using ultra-wideband and visual image technology provided by an embodiment of the present invention;

[0036] Figure 2 A block diagram of another method for controlling an Internet of Things device using ultra-wideband and visual image technology provided by an embodiment of the present invention;

[0037] Figure 3 A structural block diagram of an Internet of Things device control device using ultra-wideband and visual image technology provided by an embodiment of the present invention;

[0038] Figure 4 A structural block diagram of another IoT device control device using ultra-wideband and visual image technology provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The embodiments of the present application provide an IoT device control method, device and AR head-mounted display device that utilize ultra-bandwidth and visual image technology, thereby achieving convenience while providing users with a new user experience, and greatly improving accuracy and ease of use.

[0041] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0042] In the embodiment of the present invention, firstly, a request signal is transmitted to each IoT device through the IoT module; secondly, a response signal returned by each IoT device is received, and the distance between the user and each IoT device is obtained based on the reception time of the response signal to update the user's location; again, a dynamic image following the user's field of view is collected through the camera module, and the IoT device screen appearing in the dynamic image is compared with the corresponding device appearance image; finally, when the user's location and the image comparison result trigger the preset conditions, the range expansion method is adopted to ask the user whether to virtually control the IoT device in three-dimensional space. By making full use of the transmittable properties of ultra-wideband technology and combining visual image technology, the IoT device can be controlled following the user's field of view, which improves convenience, accuracy and ease of use.

[0043] In addition, the embodiment of the present invention also includes receiving a user's choice of not performing or performing virtual control of the IoT device and recording the event. The collected user habits can be used to further adjust the trigger conditions, to a certain extent predict the user's needs and improve the comfort of use.

[0044] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0045] Embodiment 1:

[0046] like Figure 1 As shown, an embodiment of the present invention provides an Internet of Things device control method using ultra-wideband and visual image technology, including:

[0047] S1. Send a request signal to each IoT device through the IoT module;

[0048] S2. Receive a response signal returned by each IoT device, and obtain the distance between the user and each IoT device based on the reception time of the response signal to update the user's location;

[0049] S3, collecting dynamic images following the user's field of view through a camera module, and comparing the IoT device screen appearing in the dynamic image with the corresponding device appearance image;

[0050] S4. When the user's location and the image comparison result trigger a preset condition, an extended range method is used to ask the user in three-dimensional space whether to virtually control the IoT device.

[0051] The embodiments of the present invention make full use of the transmittable properties of ultra-wideband technology and combine it with visual image technology to achieve control of IoT devices following the user's field of view, thereby improving convenience, accuracy and ease of use.

[0052] In an optional embodiment, if Figure 2 As shown, the embodiment of the present invention also includes:

[0053] S5. Receive the user's selection of not controlling or controlling the IoT device virtually, and record the event.

[0054] The user habits collected by the embodiment of the present invention can be used to further adjust the trigger conditions, to a certain extent predict the user's needs, and to improve the comfort of use.

[0055] Next, we will introduce the steps of the above scheme in detail:

[0056] In step S1, a request signal is transmitted to each IoT device through the IoT module.

[0057] In step S2, a response signal returned by each of the IoT devices is received, and the distance between the user and each IoT device is obtained based on the reception time of the response signal to update the user's location.

[0058] It should be noted that if the request signal is not transmitted for the first time through the IoT module in step S1, then each IoT device in step S2 only needs to return a response signal, and obtain the distance between the user and each IoT device based on the reception time of the response signal to update the user's location (the update process is a conventional application of ultra-wideband technology and will not be repeated here).

[0059] When the IoT module transmits the request signal for the first time in step S1, each IoT device needs to return its own attribute information package in step S2 to ensure a stable connection. The attribute information package includes at least the device name, IoT identification code, control method and device appearance image.

[0060] In addition, in step S1, the IoT module transmits a request signal to each IoT device for the first time, and in step S2, the corresponding IoT device is required to perform security verification and return the attribute information package after the security verification is passed to avoid illegal intrusion.

[0061] In step S3, a dynamic image following the user's field of view is collected by a camera module, and the IoT device screen appearing in the dynamic image is compared with the corresponding device appearance image.

[0062] In fact, the camera module following the user's field of view is not always able to capture the IoT device. Therefore, in most cases, the shooting interval of the camera module can be increased (even the camera module is normally in standby mode) to reduce energy consumption.

[0063] When the camera module following the user's field of view captures an IoT device, that is, when the IoT device image begins to appear in the dynamic image, the camera module can be adjusted to a continuous shooting calibration mode until the IoT device appears completely in the dynamic image, so as to facilitate timely comparison with the corresponding device appearance image.

[0064] In addition, the embodiments of the present invention do not limit the specific type of the above-mentioned camera module, and those skilled in the art may select a grayscale camera or an RGB camera, etc. according to actual needs.

[0065] In step S4, when the user's location and the image comparison result trigger a preset condition, an extended range method is used to ask the user in three-dimensional space whether to virtually control the IoT device.

[0066] The trigger preset conditions include:

[0067] The user's location meets the preset orientation and angle requirements of the target IoT device. Based on the description of the previous steps, the continuously shot dynamic images are compared with the corresponding device appearance images, and the dynamic images whose similarity continuously exceeds the preset threshold meet the quantity requirements, and / or the comparison time whose similarity continuously exceeds the preset threshold meets the duration requirements.

[0068] In step S5, a user's selection of not performing virtual control or performing virtual control on the IoT device is received, and the event is recorded.

[0069] For example, when a user does not virtually control an IoT device, the user's selection is received and the event is recorded. At this time, it can be set that within a period of time in the future, based on the same user-described location and image comparison results, the user will no longer be asked whether to virtually control the same IoT device.

[0070] For another example, when a user virtually controls an IoT device, the user's selection is received and the event is recorded, and if the same user's position and image comparison results appear frequently, it can be set that the user needs to virtually control the same IoT device for a period of time in the future.

[0071] In this way, it is possible to predict user needs to a certain extent and improve user comfort.

[0072] Embodiment 2:

[0073] like Figure 3 As shown, an embodiment of the present invention provides an Internet of Things device control device using ultra-wideband and visual image technology, including:

[0074] A request module, used to transmit a request signal to each IoT device through the IoT module;

[0075] An updating module, configured to receive a response signal returned by each of the IoT devices, and obtain the distance between the user and each IoT device based on the reception time of the response signal to update the user's location;

[0076] A comparison module, used to collect dynamic images following the user's field of view through a camera module, and compare the IoT device screen appearing in the dynamic image with the corresponding device appearance image;

[0077] The request module is used to, when the user's position and the image comparison result trigger a preset condition, adopt an extended range method to request the user in three-dimensional space whether to virtually control the Internet of Things device.

[0078] The embodiments of the present invention make full use of the transmittable properties of ultra-wideband technology and combine it with visual image technology to achieve control of IoT devices following the user's field of view, thereby improving convenience, accuracy and ease of use.

[0079] In an optional embodiment, if Figure 4 As shown, the embodiment of the present invention also includes:

[0080] The recording module is used to receive the user's selection of not performing or performing virtual control on the IoT device and record the event.

[0081] The user habits collected by the embodiment of the present invention can be used to further adjust the trigger conditions, to a certain extent predict the user's needs, and to improve the comfort of use.

[0082] Next, we will introduce each module of the above solution in detail:

[0083] In the request module, it is used to send a request signal to each IoT device through the IoT module.

[0084] In the update module, it is used to receive the positioning information returned by each of the IoT devices and update the user's location.

[0085] It should be noted that if the request signal is not sent for the first time through the IoT module in the request module, then each IoT device in the update module only needs to return positioning information to update the user's location (the update process is existing technology and will not be repeated here).

[0086] When the request module transmits the request signal for the first time through the IoT module, each IoT device in the update module needs to return its own attribute information package to ensure a stable connection. The attribute information package includes at least the device name, IoT identification code, control method and device appearance image.

[0087] In addition, the request module transmits a request signal to each IoT device for the first time through the IoT module, and the corresponding IoT device in the update module is also required to undergo security verification, and then return the attribute information package after the security verification is passed to avoid illegal intrusion.

[0088] In the comparison module, a dynamic image following the user's field of view is collected through a camera module, and the IoT device screen appearing in the dynamic image is compared with the corresponding device appearance image.

[0089] In fact, the camera module following the user's field of view is not always able to capture the IoT device. Therefore, in most cases, the shooting interval of the camera module can be increased (even the camera module is normally in standby mode) to reduce energy consumption.

[0090] When the camera module following the user's field of view captures an IoT device, that is, when the IoT device image begins to appear in the dynamic image, the camera module can be adjusted to a continuous shooting calibration mode until the IoT device appears completely in the dynamic image, so as to facilitate timely comparison with the corresponding device appearance image.

[0091] In addition, the embodiments of the present invention do not limit the specific type of the above-mentioned camera module, and those skilled in the art may select a grayscale camera or an RGB camera, etc. according to actual needs.

[0092] In the request module, when the user's position and the image comparison result trigger the preset conditions, an extended range method is used to ask the user in three-dimensional space whether to virtually control the Internet of Things device.

[0093] The trigger preset conditions include:

[0094] The user's location meets the preset orientation and angle requirements of the target IoT device. Based on the description of the previous steps, the continuously shot dynamic images are compared with the corresponding device appearance images, and the dynamic images whose similarity continuously exceeds the preset threshold meet the quantity requirements, and / or the comparison time whose similarity continuously exceeds the preset threshold meets the duration requirements.

[0095] In the recording module, it is used to receive the user's selection of not controlling or controlling the IoT device virtually, and record the event.

[0096] For example, when a user does not virtually control an IoT device, the user's selection is received and the event is recorded. At this time, it can be set that within a period of time in the future, based on the same user-described location and image comparison results, the user will no longer be asked whether to virtually control the same IoT device.

[0097] For another example, when a user virtually controls an IoT device, the user's selection is received and the event is recorded, and if the same user's position and image comparison results appear frequently, it can be set that the user needs to virtually control the same IoT device by default for a period of time in the future.

[0098] In this way, it is possible to predict user needs to a certain extent and improve user comfort.

[0099] Embodiment 3:

[0100] An embodiment of the present invention provides an AR head-mounted display device, including the Internet of Things device control device as described in Example 2.

[0101] It is understandable that since the AR head-mounted display device provided by the embodiment of the present invention includes the above-mentioned Internet of Things device control device, the explanations, examples and beneficial effects of other related contents can refer to the corresponding parts in the Internet of Things device control device, and will not be repeated here.

[0102] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0103] 1. The embodiments of the present invention make full use of the transmittable properties of ultra-wideband technology and combine it with visual image technology to achieve control of IoT devices following the user's field of view, thereby improving convenience, accuracy and ease of use.

[0104] 2. The user habits collected by the embodiment of the present invention can be used to further adjust the trigger conditions, to a certain extent predict the user's needs and improve the comfort of use.

[0105] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0106] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for controlling an Internet of Things device using ultra-wideband and visual image technology, characterized in that: include: Send request signals to various IoT devices through the IoT module; Receive a response signal returned by each of the IoT devices, and obtain the distance between the user and each IoT device based on the reception time of the response signal to update the user's location; The camera module collects dynamic images that follow the user's field of view, and compares the IoT device images that appear in the dynamic images with the corresponding device appearance images; When the user's location and the image comparison result trigger a preset condition, an extended range method is used to ask the user in three-dimensional space whether to virtually control the Internet of Things device.

2. The method for controlling an Internet of Things device according to claim 1, wherein: Also includes: Receive a user's selection of no or virtual control of the IoT device and record the event.

3. The method for controlling an Internet of Things device according to claim 1, wherein: When the IoT module transmits a request signal to each IoT device for the first time, an attribute information package returned by each IoT device is also received; wherein the attribute information package at least includes a device name, an IoT identification code, a control method, and an appearance image of the device.

4. The method for controlling an Internet of Things device according to claim 3, wherein: The Internet of Things module transmits a request signal to each Internet of Things device for the first time, performs security verification at the corresponding Internet of Things device, and receives the attribute information package after the security verification is passed.

5. The method for controlling an Internet of Things device according to claim 1, wherein: When the Internet of Things device screen appears in the dynamic image, the camera module is adjusted to a continuous shooting calibration mode until the Internet of Things device appears completely in the dynamic image.

6. The method for controlling an Internet of Things device according to claim 5, wherein: The trigger preset conditions include: The user's position meets the preset requirements for the orientation and angle of the target IoT device; and when the continuously captured dynamic images are compared with the corresponding device appearance images, the number of dynamic images whose similarity continuously exceeds the preset threshold value meets the quantity requirement, and / or the comparison time whose similarity continuously exceeds the preset threshold value meets the duration requirement.

7. An Internet of Things device control device using ultra-wideband and visual image technology, characterized in that: include: A request module, used to transmit a request signal to each IoT device through the IoT module; An updating module, configured to receive a response signal returned by each of the IoT devices, and obtain the distance between the user and each IoT device based on the reception time of the response signal to update the user's location; A comparison module, used to collect dynamic images following the user's field of view through a camera module, and compare the IoT device screen appearing in the dynamic image with the corresponding device appearance image; The request module is used to, when the user's position and the image comparison result trigger a preset condition, adopt an extended range method to request the user in three-dimensional space whether to virtually control the Internet of Things device.

8. The Internet of Things device control device according to claim 7, characterized in that: Also includes: The recording module is used to receive the user's selection of not performing or performing virtual control on the IoT device and record the event.

9. The Internet of Things device control device according to claim 7, characterized in that: When the IoT module transmits a request signal to each IoT device for the first time, an attribute information package returned by each IoT device is also received; wherein the attribute information package at least includes a device name, an IoT identification code, a control method, and an appearance image of the device.

10. The Internet of Things device control device according to claim 9, characterized in that: The Internet of Things module transmits a request signal to each Internet of Things device for the first time, performs security verification at the corresponding Internet of Things device, and receives the attribute information package after the security verification is passed.

11. The Internet of Things device control device according to claim 7, characterized in that: When the Internet of Things device screen appears in the dynamic image, the camera module is adjusted to a continuous shooting calibration mode until the Internet of Things device appears completely in the dynamic image.

12. The Internet of Things device control device according to claim 11, characterized in that: The trigger preset conditions include: The user's position meets the preset requirements for the orientation and angle of the target IoT device; and when the continuously captured dynamic images are compared with the corresponding device appearance images, the number of dynamic images whose similarity continuously exceeds the preset threshold value meets the quantity requirement, and / or the comparison time whose similarity continuously exceeds the preset threshold value meets the duration requirement.

13. An AR head mounted display device, characterized in that: It comprises the Internet of Things device control device as described in any one of claims 7 to 12.

Images