An automatic control method based on human perception, a first electronic device and a system

Abstract

The application provides a human perception-based automatic control method, a first electronic device and a system, and relates to the field of automatic control. The system comprises a central device, a first electronic device and a second electronic device, the first electronic device comprises a first ultra-wideband module and a millimeter wave radar module, and the second electronic device comprises at least one mobile device. Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human position, the position information of the second electronic device obtained by the central device and the position information of the user, at least one second electronic device automatically performs a preset operation according to the position information of the user and the position information of the second electronic device. The application does not require the user to carry a device, realizes automatic control of IoT devices, and is convenient and fast.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic control, and in particular to an automatic control method based on human perception, a first electronic device and a system. BACKGROUND

[0002] With the rapid development of smart home, more and more users install Internet of Things (IoT) devices (also referred to as IoT devices, smart devices, etc.) in places such as home or office, or even install IoT devices in the whole house (also referred to as whole-house intelligence, whole-house smart home, smart butler, etc.). Users can experience the convenient user experience brought by IoT devices. However, in the prior art, automatic control of IoT devices requires users to carry mobile devices (such as smartphones, wearable devices, etc.) and to achieve automatic control of IoT devices through communication and perception between mobile devices and IoT devices. Sometimes, users forget to carry mobile devices in places such as home or office, and cannot achieve automatic control of IoT devices, thereby bringing inconvenience to users and affecting user experience. SUMMARY

[0003] Based on the above background technology, how to make the automatic control of IoT devices by users more convenient and further improve user experience has become a problem we need to consider.

[0004] To solve the above technical problems, the present application provides an automatic control method based on human perception, a first electronic device and a system. The technical solution of the present application can make IoT devices automatically perform an operation when a user is close to the IoT devices, without any operation by the user or carrying any electronic device by the user, thereby greatly improving user experience. In addition, the IoT devices do not need to be modified in hardware. For example, a smart speaker generally does not have a camera and does not need to install an additional camera.

[0005] In a first aspect, the present application provides a human perception based communication system. The system comprises a hub device, a first electronic device and R second electronic devices; wherein any two of the hub device, the first electronic device and any one of the second electronic devices (the second electronic device is any one of the R second electronic devices) communicate in a wired or wireless manner; the first electronic device comprises a first ultra-wideband module and a millimeter wave radar module; optionally, further comprising an IMU module. The R second electronic devices comprise a mobile device, the mobile device comprising a second ultra-wideband module; R is a positive integer greater than or equal to 1. Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in a whole house coordinate system provided by the hub device; according to the position information of the user and the position information of the R second electronic devices, the hub device controls or notifies at least one of the R second electronic devices to perform a preset operation; in response to the control or notification of the hub device, the at least one second electronic device performs the preset operation.

[0006] In this embodiment, the first electronic device obtains the position of any one of the second electronic devices and the position of any one of the users in the whole house through the first ultra-wideband module and the millimeter wave radar module; the hub device controls or notifies the second electronic device to perform a preset operation according to the position of the user and the position of the second electronic device. In this way, without any operation of the user, and without the user carrying any electronic device, the IoT device (the second electronic device) automatically performs an operation, which is convenient and fast.

[0007] According to the first aspect, based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human position, and the communication between the hub device and the first electronic device, the hub device periodically or in real time obtains the position information of the R second electronic devices and the position information of the user in a whole house coordinate system provided by the hub device. In this way, the hub device can control the second electronic device in real time, and improve the automatic control efficiency of the second electronic device.

[0008] According to the first aspect, or any one of the implementation modes of the above first aspect, when the user is one user, the position information of the user is the position information of the one user.

[0009] According to the first aspect, or any one of the implementation modes of the above first aspect, when the user is a plurality of users, the position information of the user is the center position information of the plurality of users.

[0010] In an implementation mode, the position of the one user is represented by the coordinates of the one user in a coordinate system (such as a whole house coordinate system). The center position of the plurality of users is the average value of the coordinates of the plurality of users.

[0011] According to a first aspect, or any of the implementations of the first aspect, the T second electronic devices of the R second electronic devices support voice wake-up; in response to receiving the wake-up voice, the T second electronic devices respectively send first messages to the hub device; in response to receiving the at least one first message, the hub device obtains position information of the T second electronic devices and position information of the user in a whole-house coordinate system; the hub device determines one device closest to the user from the T second electronic devices; the hub device sends a first indication message to the device closest to the user; in response to receiving the first indication message, the device closest to the user wakes up; wherein T is a positive integer greater than or equal to 1 and less than or equal to R.

[0012] In this way, only one device closest to the user in the whole house is woken up, avoiding multiple devices responding to wake-up at the same time.

[0013] In an implementation, in response to receiving the at least one first message, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system, including: in response to receiving one first message, the hub device obtains the position information of the second electronic device sending the first message and the first room or first area where the second electronic device is located in the whole-house coordinate system; the hub device further obtains the position information of the user in the first room or first area in the whole-house coordinate system. In this implementation, the hub device receives one first message, obtains the first room or first area where the second electronic device corresponding to the first message is located, and obtains the position information of the user in the first room or first area. One device closest to the user in the first room or first area is selected to respond to wake-up.

[0014] In another implementation, in response to receiving the at least one first message, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system, including: in response to receiving multiple first messages within a preset time length, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system. In this implementation, the hub device receives the first messages sent by all (or all capable of sending the first messages) second electronic devices, and determines one device closest to the user from the T second electronic devices to respond to wake-up.

[0015] According to the first aspect, or any of the implementations of the first aspect, based on the communication between the hub device and the T second electronic devices of the R second electronic devices, the hub device pre-obtains that the T second electronic devices support voice wake-up; T is a positive integer greater than or equal to 1 and less than or equal to R; the user is one user or multiple users.

[0016] According to the first aspect, or any one of the implementations of the above first aspect, when the user is a plurality of users, the device closest to the user is the device closest to the center position of the plurality of users. The center position of the plurality of users is the average of the coordinates of the plurality of users.

[0017] According to the first aspect, or any one of the implementations of the above first aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio; the user includes a first user; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; after obtaining that the second device is the closest playing device to the first user, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier of the first video or the first audio and playing progress information; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier of the first video or the first audio and the playing progress information; in response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information.

[0018] In this implementation, in the playing of the first device, the user leaves the first device and approaches the second device, and the automatic switching is automatically continued by the second device, and the first device is automatically stopped. This method enables the user to realize seamless and automatic switching of audio or video playing in the above movement without carrying any device.

[0019] In one implementation, obtaining that the second device is the closest playing device to the first user includes: obtaining that, among the devices that do not play a video or an audio, the second device is the closest playing device to the first user.

[0020] In one implementation, after the second device continues playing the first video or the first audio according to the playing progress information, the second device sends a second feedback message to the hub device, for representing completion of the switching. The hub device learns that the first device stops playing and the second device starts playing according to the second feedback message, and updates and saves the running information of the first device and the second device.

[0021] According to the first aspect, or any of the implementations of the first aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio; the user includes a first user and a second user; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; in response to an operation of the second user, the first device starts playing a second video or a second audio, and sends a second notification message to the hub device; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; after obtaining that the second device is the closest playing device to the second user, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier and the playing progress information of the second video or the second audio; in response to receiving the second notification message, the second device continues playing the second video or the second audio according to the playing progress information.

[0022] In this implementation, in the playing of the first video or the first audio corresponding to the first user by the first device, the second user uses the first device to play the second video or the second audio. According to the second user leaving the first device and approaching the second device, the automatic switching to the second device automatically continues playing the second video or the second audio, and the first device automatically stops playing. This method enables the user to seamlessly and automatically switch the playing of the audio or the video in the above movement without carrying any device, only by leaving the first device and approaching the second device.

[0023] In an implementation, obtaining that the second device is the closest playing device to the second user includes: obtaining that, among the devices that do not play the video or the audio, the second device is the closest playing device to the second user.

[0024] In an implementation, after the second device continues playing the second video or the second audio according to the playing progress information, the second device sends a second feedback message to the hub device, for representing completion of the switching. The hub device learns that the first device stops playing and the second device starts playing according to the second feedback message, and updates and saves the running information of the first device and the second device.

[0025] According to the first aspect, or any of the implementations of the first aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user, the first user is located in the first room or the first area; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; in response to obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier and playing progress information of the first video or the first audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier and the playing progress information of the first video or the first audio; in response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information.

[0026] In this implementation, in the playing of the first device, in response to the user leaving a first room or a first area where the first device is located and entering a second room or a second area where the second device is located, the playing of the first device is automatically stopped and the playing of the second device is automatically continued. This method enables the user to realize seamless and automatic switching of audio or video playing without carrying any device in the movement described above.

[0027] In an implementation, after obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0028] In an implementation, after obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0029] In an implementation, after the second device continues playing the first video or the first audio according to the playing progress information, the second device sends a second feedback message to the hub device, the second feedback message being used to represent completion of the switching. The hub device learns that the first device stops playing and the second device starts playing according to the second feedback message, and updates and saves the running information of the first device and the second device.

[0030] According to the first aspect, or any of the implementations of the first aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user and a second user, the first user and the second user are located in the first room or the first area; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; in response to an operation of the second user, the first device starts playing a second video or a second audio, and sends a second notification message to the hub device; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; in response to obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier and the playing progress information of the second video or the second audio; in response to receiving the second notification message, the second device continues playing the second video or the second audio according to the playing progress information.

[0031] In this implementation, in the process of playing the first video or the first audio corresponding to the first user by the first device, the second user uses the first device to play the second video or the second audio. According to the second user leaving the first room or the first area where the first device is located and entering the second room or the second area where the second device is located, the second device automatically continues playing the second video or the second audio, and the first device automatically stops playing. This method enables the user to seamlessly and automatically switch the playing of the audio or the video without carrying any device in the above-mentioned movement, only by leaving the first device and approaching the second device.

[0032] In one implementation, after obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0033] In one implementation, after obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0034] In an embodiment, after the second device continues to play the second video or the second audio according to the play progress information, the second device sends a second feedback message to the hub device, which represents completion of the switching. The hub device learns from the second feedback message that the first device stops playing and the second device starts playing, and updates the saved running information of the first device and the second device.

[0035] According to the first aspect, or any of the embodiments of the first aspect, the R second electronic devices include a stereo system, the stereo system including the first device and the second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device and the second device are located in the first room or the first area, a first distance between the first device and the user, and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first play message and a second play message to the first device and the second device respectively, the first play message including the first volume information and the second play message including the second volume information; in response to receiving the first play message, the first device plays according to the first volume information; in response to receiving the second play message, the second device plays according to the second volume information.

[0036] In this embodiment, when the two audio playing devices playing the stereo left channel and the stereo right channel, the volume of the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The audio playing device far from the user has a large volume compensation value, so that the user receives the same volume from the two audio playing devices, and the user's experience of listening to stereo audio is improved.

[0037] In an embodiment, the stereo system further includes a third device, the third device playing a stereo auxiliary sound; the stereo system includes the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user; the hub device determines first volume information for the first device, second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first play message, a second play message and a third play message to the first device, the second device and the third device respectively, the third play message including the third volume information; in response to receiving the third play message, the third device plays according to the third volume information.

[0038] According to the first aspect, or any of the implementations of the first aspect, the R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device is located in a first area and the second device is located in a second area, the first area being adjacent to the second area; the hub device further obtains a first distance between the first device and the user and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information and the second playing message comprising the second volume information; the first device plays according to the first volume information in response to receiving the first playing message; the second device plays according to the second volume information in response to receiving the second playing message.

[0039] In this implementation, the two audio playing devices playing the stereo left channel and the stereo right channel are located in adjacent areas, for example, a living room and a dining room are connected together, there are less partitions and barriers between the two, and the spatial independence is weak; the devices in the same stereo system can be located in the living room and the dining room respectively. When the two audio playing devices playing the stereo left channel and the stereo right channel play, the volume of the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The volume compensation value of the audio playing device far from the user is large, so that the user receives the same volume played by the two audio playing devices, and the user's experience of listening to stereo audio is improved.

[0040] In an implementation, the stereo system further comprises a third device, the third device playing a stereo auxiliary sound; the stereo sound comprises the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; the hub device determines first volume information for the first device, second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising the third volume information; the third device plays according to the third volume information in response to receiving the third playing message.

[0041] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device to the user and a second volume attenuation value of the playing volume of the second device to the user; and determines, according to the first volume attenuation value and the second volume attenuation value, the first volume information for the first device and the second volume information for the second device.

[0042] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device to the user and a second volume attenuation value of the playing volume of the second device to the user includes:

[0043] A p = K - DL w + A e ,

[0044] wherein A p is a volume attenuation value; K is a distance attenuation factor, representing volume attenuation caused by distance, related to the distance between the device and the user; DL w is a directivity factor, representing volume attenuation caused by the direction of audio signal emission; A e is an air absorption attenuation, representing volume attenuation caused by air absorption of audio when the audio propagates in the air.

[0045] In this embodiment, the playing volume of the audio playing device is compensated according to the attenuation value in the volume transmission process.

[0046] According to the first aspect, or any one of the embodiments of the first aspect, the R second electronic devices include the first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user and no second user; the central device further obtains that the first device enters a private mode and plays a first video; when detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing and automatically switches to a screen saver; when detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device; in response to receiving the second notification message, the first device automatically resumes playing the first video.

[0047] In this embodiment, under the condition that the device is in the private mode, if the central device detects that a specific category of user (such as a child) enters the room or area where the device is located, the playing is automatically paused and switched to a screen saver; if the central device detects that the specific category of user (such as a child) leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout the process.

[0048] In an embodiment, after the first device automatically stops playing and automatically switches to the screen saver, the first device sends a first feedback message to the hub device; in response to receiving the first feedback message, the hub device learns that the first device has stopped playing and switched to the screen saver.

[0049] In an embodiment, after the first device automatically resumes playing the first video, the first device sends a second feedback message to the hub device; in response to receiving the second feedback message, the hub device learns that the first device has resumed playing.

[0050] According to the first aspect, or any one of the embodiments of the above first aspect, the R second electronic devices include the first device; the hub device learns that the first device is located in the first room or the first area, the first room or the first area has the first user and no second user; the hub device further learns that the first device enters the private mode and plays the first video; in response to detecting that a user other than the first user enters the first room or the first area, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing and automatically switches to the screen saver; in response to detecting that the first room or the first area has no user other than the first user, the hub device sends a second notification message to the first device; in response to receiving the second notification message, the first device automatically resumes playing the first video.

[0051] In this embodiment, under the condition that the device enters the private mode, if the hub device detects that a user other than the first user enters the room or area where the device is located, the playing is automatically paused and the screen saver is automatically switched; if the hub device detects that the user other than the first user leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout.

[0052] In an embodiment, after the first device automatically stops playing and automatically switches to the screen saver, the first device sends a first feedback message to the hub device; in response to receiving the first feedback message, the hub device learns that the first device has stopped playing and switched to the screen saver.

[0053] In an embodiment, after the first device automatically resumes playing the first video, the first device sends a second feedback message to the hub device; in response to receiving the second feedback message, the hub device learns that the first device has resumed playing.

[0054] According to the first aspect, or any one of the implementations of the first aspect, the R second electronic devices include the first device; the hub device obtains that the first device is playing the first video, and obtains the first position of the first device, that the first device is located in the first room or the first area, the second position of the user in the first room or the first area, and the first viewing area corresponding to the first device; in response to detecting that there is no user in the first viewing area within a first preset time length, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device stops playing the first video; in response to detecting that there is a user in the first viewing area within a second preset time length, the hub device sends a second notification message to the first device; in response to receiving the second notification message, the first device resumes playing the first video.

[0055] In this implementation, the hub device obtains the relative positions of the user and the device, and if the user leaves the viewing area of the first device, the first device automatically stops playing or pauses playing; if the user enters the viewing area of the first device, the first device automatically continues playing; no manual operation of the user is required throughout.

[0056] According to the first aspect, or any one of the implementations of the first aspect, in response to detecting that the distance between at least one user and the first device is less than a first preset distance within a third preset time length, the hub device sends a third notification message to the first device; in response to receiving the third notification message, the first device stops playing the first video and outputs a reminder message; in response to detecting that there is no user with a distance less than the first preset distance from the first device and there is a user in the first viewing area within a fourth preset time length, the hub device sends a fourth notification message to the first device; in response to receiving the fourth notification message, the first device resumes playing the first video.

[0057] In this implementation, the hub device obtains the relative positions of the user and the device, and if the user is too close to the first device, the first device automatically stops playing or pauses playing and outputs a reminder message; if the user is in the viewing area of the first device and is not too close to the first device, the first device automatically resumes playing; no manual operation of the user is required throughout.

[0058] According to the first aspect, or any one of the implementations of the first aspect, in response to detecting that there is no user with a distance less than the first preset distance from the first device and there is no user in the first viewing area within a fifth preset time length, the hub device sends a fifth notification message to the first device; in response to receiving the fifth notification message, the first device automatically hibernates.

[0059] In this embodiment, the central device acquires the relative position of the user and the device, and if the user is away from the viewing area of the first device for a long time, the first device automatically hibernates to save power and protect privacy, without manual operation of the user.

[0060] According to the first aspect, or any one of the implementations of the above first aspect, the first electronic device comprises an inertial measurement unit module; the R second electronic devices comprise the first device, and the first device does not comprise an ultra-wideband module; based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the central device and the first electronic device, the central device acquires the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the central device, comprising: the first ultra-wideband module establishes a first coordinate system, based on the position measurement of the mobile device by the first electronic device and the labeling of the first device by the mobile device, the first electronic device acquires a first coordinate of the first device in the first coordinate system; the millimeter wave radar module establishes a second coordinate system, based on the perception of the human body by the first electronic device, the first electronic device acquires a second coordinate of the user in the second coordinate system; through the conversion between the first coordinate system and the second coordinate system, the first electronic device acquires a third coordinate of the second coordinate in the first coordinate system; the first electronic device sends the related coordinates in the first coordinate system to the central device, and the related coordinates comprise the first coordinate and the third coordinate; based on the output of the inertial measurement unit module, in combination with the parallel between the whole-house coordinate system and the geographic coordinate system, the central device converts the related coordinates in the first coordinate system into coordinates in the whole-house coordinate system, and further acquires the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system.

[0061] In this embodiment, for the second electronic device which does not comprise an ultra-wideband module, the first electronic device labels it at least once through the mobile device comprising an ultra-wideband module to acquire its position information.

[0062] In a second aspect, the application provides an automatic control method based on human perception, applied to a communication system based on human perception, which comprises a central device, a first electronic device and R second electronic devices; any two of the central device, the first electronic device and any one of the R second electronic devices communicate in a wired or wireless manner; the first electronic device comprises a first ultra-wideband module and a millimeter wave radar module; the R second electronic devices comprise a mobile device, and the mobile device comprises a second ultra-wideband module; R is a positive integer greater than or equal to 1. The method comprises: based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human position, and the communication between the central device and the first electronic device, the central device obtains the position information of the R second electronic devices and the position information of the user in a whole-house coordinate system provided by the central device; according to the position information of the user and the position information of the R second electronic devices, the central device controls or notifies at least one of the R second electronic devices to perform a preset operation; in response to the control or notification of the central device, the at least one second electronic device performs the preset operation.

[0063] In the method, the first electronic device obtains the position of any one of the second electronic devices and the position of any one of the users in the whole house through the first ultra-wideband module and the millimeter wave radar module; and the central device controls or notifies the second electronic device to perform a preset operation according to the position of the user and the position of the second electronic device. In this way, without any operation of the user or carrying any electronic device by the user, the IoT device (second electronic device) automatically performs an operation, which is convenient and fast.

[0064] According to the second aspect, based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human position, and the communication between the central device and the first electronic device, the central device periodically or in real time obtains the position information of the R second electronic devices and the position information of the user in a whole-house coordinate system provided by the central device. In this way, the central device can control the second electronic device in real time, thereby improving the automatic control efficiency of the second electronic device.

[0065] According to the second aspect, or any one of the implementation manners of the second aspect, when the user is one user, the position information of the user is the position information of the one user.

[0066] According to the second aspect, or any one of the implementation manners of the second aspect, when the user is a plurality of users, the position information of the user is the central position information of the plurality of users.

[0067] In an implementation manner, the position of the one user is represented by the coordinates of the one user in a coordinate system (such as a whole-house coordinate system). The central position of the plurality of users is the average value of the coordinates of the plurality of users.

[0068] According to a second aspect, or any possible implementation mode of the second aspect, T second electronic devices of the R second electronic devices support voice wake-up; in response to receiving the wake-up voice, the T second electronic devices respectively send first messages to the hub device; in response to receiving at least one first message, the hub device obtains position information of the T second electronic devices and position information of the user in a whole-house coordinate system; the hub device determines one device closest to the user from the T second electronic devices; the hub device sends a first indication message to the device closest to the user; in response to receiving the first indication message, the device closest to the user wakes up; wherein T is a positive integer greater than or equal to 1 and less than or equal to R.

[0069] In this way, only one device closest to the user in the whole house is woken up, avoiding multiple devices responding to wake-up at the same time.

[0070] In an implementation mode, in response to receiving at least one first message, the hub device obtains position information of the T second electronic devices and position information of the user in a whole-house coordinate system, comprising: in response to receiving one first message, the hub device obtains position information of one second electronic device sending the first message and a first room or a first area where the second electronic device is located in the whole-house coordinate system; and the hub device further obtains position information of the user in the first room or the first area in the whole-house coordinate system. In this implementation mode, the hub device receives one first message, obtains a first room or a first area where a second electronic device corresponding to the first message is located, and obtains position information of the user in the first room or the first area. One device closest to the user in the first room or the first area is selected to respond to wake-up.

[0071] In another implementation mode, in response to receiving at least one first message, the hub device obtains position information of the T second electronic devices and position information of the user in a whole-house coordinate system, comprising: in response to receiving multiple first messages within a preset time length, the hub device obtains position information of the T second electronic devices and position information of the user in the whole-house coordinate system. In this implementation mode, the hub device receives first messages sent by all (or all capable of sending first messages) second electronic devices, and determines one device closest to the user from the T second electronic devices to respond to wake-up.

[0072] According to the second aspect, or any possible implementation mode of the second aspect, the hub device pre-obtains that T second electronic devices of R second electronic devices support voice wake-up based on communication between the hub device and the T second electronic devices; T is a positive integer greater than or equal to 1 and less than or equal to R; and the user is one user or multiple users.

[0073] According to a second aspect, or any possible implementation mode of the second aspect, when the user is a plurality of users, the device closest to the user is the device closest to the center position of the plurality of users. The center position of the plurality of users is the average of the coordinates of the plurality of users.

[0074] According to the second aspect, or any possible implementation mode of the second aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio; the user includes a first user; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; after obtaining that the second device is the closest playing device to the first user, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier of the first video or the first audio and playing progress information; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier of the first video or the first audio and the playing progress information; in response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information.

[0075] In this implementation mode, in the playing of the first device, the user leaves the first device and approaches the second device, and the automatic switching is automatically continued by the second device, and the first device is automatically stopped. This method enables the user to realize seamless and automatic switching of audio or video playing in the above-mentioned movement without carrying any device, only by leaving the first device and approaching the second device.

[0076] In an implementation mode, obtaining that the second device is the closest playing device to the first user includes: obtaining that, among the devices that do not play a video or an audio, the second device is the closest playing device to the first user.

[0077] In an implementation mode, after the second device continues playing the first video or the first audio according to the playing progress information, the second device sends a second feedback message to the hub device, for representing completion of switching. The hub device learns that the first device is stopped playing and the second device starts playing according to the second feedback message, and updates and saves the running information of the first device and the second device.

[0078] According to a second aspect, or any possible implementation mode of the second aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio; the user includes a first user and a second user; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; in response to an operation of the second user, the first device starts playing a second video or a second audio, and sends a second notification message to the hub device; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; after obtaining that the second device is the nearest playing device to the second user, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier and the playing progress information of the second video or the second audio; in response to receiving the second notification message, the second device continues playing the second video or the second audio according to the playing progress information.

[0079] In this implementation mode, in the process that the first device plays the first video or the first audio corresponding to the first user, the second user uses the first device to play the second video or the second audio. According to the second user leaving the first device and approaching the second device, the automatic switching is performed to automatically continue playing the second video or the second audio by the second device, and the first device automatically stops playing. This method enables the user to realize seamless and automatic switching of audio or video playing in the above-mentioned movement without carrying any device.

[0080] In an implementation mode, obtaining that the second device is the nearest playing device to the second user includes: obtaining that, among the devices that do not play the video or the audio, the second device is the nearest playing device to the second user.

[0081] In an implementation mode, after the second device continues playing the second video or the second audio according to the playing progress information, the second device sends a second feedback message to the hub device, for representing completion of the switching. The hub device learns that the first device stops playing and the second device starts playing according to the second feedback message, and updates and saves the running information of the first device and the second device.

[0082] According to a second aspect, or any possible implementation mode of the second aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user, the first user is located in the first room or the first area; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; when it is acquired that the first user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier and playing progress information of the first video or the first audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier and the playing progress information of the first video or the first audio; in response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information.

[0083] In this implementation mode, in the playing of the first device, when the user leaves the first room or the first area where the first device is located and enters the second room or the second area where the second device is located, the automatic switching to the second device for automatically continuing playing is automatically performed, and the first device automatically stops playing. This method enables the user to realize seamless and automatic switching of audio or video playing without carrying any device in the above movement, only by leaving the first room or the first area and entering the second room or the second area.

[0084] In an implementation mode, after acquiring that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0085] In an implementation mode, after acquiring that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0086] In an implementation mode, after the second device continues playing the first video or the first audio according to the playing progress information, the second device sends a second feedback message to the hub device, for representing completion of the switching. The hub device learns that the first device stops playing and the second device starts playing according to the second feedback message, and updates and saves the running information of the first device and the second device.

[0087] According to a second aspect, or any possible implementation mode of the second aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user and a second user, the first user and the second user are located in the first room or the first area; in response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; in response to an operation of the second user, the first device starts playing a second video or a second audio, and sends a second notification message to the hub device; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; in response to obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identifier and the playing progress information of the second video or the second audio; in response to receiving the second notification message, the second device continues playing the second video or the second audio according to the playing progress information.

[0088] In this implementation mode, in the process that the first device plays the first video or the first audio corresponding to the first user, the second user uses the first device to play the second video or the second audio. According to the second user leaving the first room or the first area where the first device is located and entering the second room or the second area where the second device is located, the second device automatically continues playing the second video or the second audio, and the first device automatically stops playing. This method enables the user to realize seamless and automatic switching of audio or video playing in the above-mentioned movement without carrying any device, only by leaving the first device and approaching the second device.

[0089] In an implementation mode, after obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0090] In an implementation mode, after obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0091] In an embodiment, after the second device continues to play the second video or the second audio according to the play progress information, the second device sends a second feedback message to the hub device, for representing completion of the switching. The hub device learns from the second feedback message that the first device stops playing and the second device starts playing, and updates the saved running information of the first device and the second device.

[0092] According to a second aspect, or any possible implementation of the second aspect, the R second electronic devices comprise a stereo system, the stereo system comprising the first device and the second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device and the second device are located in the first room or the first area, a first distance between the first device and the user, and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first play message and a second play message to the first device and the second device respectively, the first play message comprising the first volume information and the second play message comprising the second volume information; the first device plays according to the first volume information in response to receiving the first play message; the second device plays according to the second volume information in response to receiving the second play message.

[0093] In this embodiment, when the two audio playing devices playing the stereo left channel and the stereo right channel, the volume played by the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The audio playing device far away from the user has a large volume compensation value, so that the user receives the same volume played by the two audio playing devices, and the experience of listening to stereo audio by the user is improved.

[0094] In an embodiment, the stereo system further comprises a third device, the third device playing a stereo auxiliary sound; the stereo system comprises the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user; the hub device determines first volume information for the first device, second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first play message, a second play message and a third play message to the first device, the second device and the third device respectively, the third play message comprising the third volume information; the third device plays according to the third volume information in response to receiving the third play message.

[0095] According to a second aspect, or any possible implementation mode of the second aspect, the R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device is located in a first area and the second device is located in a second area, the first area being adjacent to the second area; the hub device further obtains a first distance between the first device and the user and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information and the second playing message comprising the second volume information; the first device plays according to the first volume information in response to receiving the first playing message; the second device plays according to the second volume information in response to receiving the second playing message.

[0096] In this implementation mode, the two audio playing devices playing the stereo left channel and the stereo right channel are located in adjacent areas, for example, a living room and a dining room are connected together, there are less partitions and barriers between the two, and the spatial independence is weak; the devices in the same stereo system can be located in the living room and the dining room respectively. When the two audio playing devices playing the stereo left channel and the stereo right channel play, the volume of the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The volume compensation value of the audio playing device far from the user is large, so that the user receives the same volume played by the two audio playing devices, and the experience of listening to stereo audio by the user is improved.

[0097] In an implementation mode, the stereo system further comprises a third device playing a stereo auxiliary sound; the stereo sound comprises the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; the hub device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising the third volume information; the third device plays according to the third volume information in response to receiving the third playing message.

[0098] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device to the user and a second volume attenuation value of the playing volume of the second device to the user; and determines, according to the first volume attenuation value and the second volume attenuation value, the first volume information for the first device and the second volume information for the second device.

[0099] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device to the user and a second volume attenuation value of the playing volume of the second device to the user includes:

[0100] A p = K - DL w + A e ,

[0101] wherein A p is a volume attenuation value; K is a distance attenuation factor, representing volume attenuation caused by distance, related to the distance between the device and the user; DL w is a directivity factor, representing volume attenuation caused by the direction of audio signal emission; A e is an air absorption attenuation, representing volume attenuation caused by air absorption of audio when the audio propagates in the air.

[0102] In this embodiment, the playing volume of the audio playing device is compensated according to the attenuation value in the volume transmission process.

[0103] According to a second aspect, or any one of the embodiments of the second aspect, the R second electronic devices include the first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user and no second user; the central device further obtains that the first device enters a private mode and plays a first video; when detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing and automatically switches to a screensaver; when detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device; in response to receiving the second notification message, the first device automatically resumes playing the first video.

[0104] In this embodiment, under the condition that the device is in the private mode, if the central device detects that a specific category of user (such as a child) enters the room or area where the device is located, the playing is automatically paused and the screensaver is automatically switched; if the central device detects that the specific category of user (such as a child) leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout the process.

[0105] In one implementation, after the first device automatically stops playing and automatically switches to the screen saver, the first device sends a first feedback message to the hub device; in response to receiving the first feedback message, the hub device learns that the first device has stopped playing and switched to the screen saver.

[0106] In one implementation, after the first device automatically resumes playing the first video, the first device sends a second feedback message to the hub device; in response to receiving the second feedback message, the hub device learns that the first device has resumed playing.

[0107] According to a second aspect, or any implementation of the second aspect, the R second electronic devices include the first device; the hub device learns that the first device is located in the first room or the first area, the first room or the first area has the first user and no second user; the hub device further learns that the first device enters the private mode and plays the first video; in response to detecting that a user other than the first user enters the first room or the first area, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device automatically stops playing and automatically switches to the screen saver; in response to detecting that the first room or the first area has no user other than the first user, the hub device sends a second notification message to the first device; in response to receiving the second notification message, the first device automatically resumes playing the first video.

[0108] In this implementation, under the condition that the device enters the private mode, if the hub device detects that a user other than the first user enters the room or area where the device is located, the playing is automatically paused and the screen saver is automatically switched; if the hub device detects that the user other than the first user leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout.

[0109] In one implementation, after the first device automatically stops playing and automatically switches to the screen saver, the first device sends a first feedback message to the hub device; in response to receiving the first feedback message, the hub device learns that the first device has stopped playing and switched to the screen saver.

[0110] In one implementation, after the first device automatically resumes playing the first video, the first device sends a second feedback message to the hub device; in response to receiving the second feedback message, the hub device learns that the first device has resumed playing.

[0111] According to a second aspect, or any of the implementations of the second aspect, the R second electronic devices include the first device; the hub device obtains that the first device is playing the first video, and obtains the first position of the first device, that the first device is located in the first room or the first area, the second position of the user in the first room or the first area, and the first viewing area corresponding to the first device; in response to detecting that there is no user in the first viewing area within a first preset time length, the hub device sends a first notification message to the first device; in response to receiving the first notification message, the first device stops playing the first video; in response to detecting that there is a user in the first viewing area within a second preset time length, the hub device sends a second notification message to the first device; in response to receiving the second notification message, the first device resumes playing the first video.

[0112] In this implementation, the hub device obtains the relative positions of the user and the devices, and if the user leaves the viewing area of the first device, the first device automatically stops playing or pauses playing; if the user enters the viewing area of the first device, the first device automatically continues playing; no manual operation of the user is required throughout.

[0113] According to the second aspect, or any of the implementations of the second aspect, in response to detecting that the distance between at least one user and the first device is less than a first preset distance within a third preset time length, the hub device sends a third notification message to the first device; in response to receiving the third notification message, the first device stops playing the first video and outputs a reminder message; in response to detecting that no user has a distance less than the first preset distance from the first device and there is a user in the first viewing area within a fourth preset time length, the hub device sends a fourth notification message to the first device; in response to receiving the fourth notification message, the first device resumes playing the first video.

[0114] In this implementation, the hub device obtains the relative positions of the user and the devices, and if the user is too close to the first device, the first device automatically stops playing or pauses playing and outputs a reminder message; if the user is in the viewing area of the first device and is not too close to the first device, the first device automatically resumes playing; no manual operation of the user is required throughout.

[0115] According to the second aspect, or any of the implementations of the second aspect, in response to detecting that no user has a distance less than the first preset distance from the first device and there is no user in the first viewing area within a fifth preset time length, the hub device sends a fifth notification message to the first device; in response to receiving the fifth notification message, the first device automatically hibernates.

[0116] In this embodiment, the central device acquires the relative position of the user and the device, and if the user is away from the viewing area of the first device for a long time, the first device automatically hibernates to save power and protect privacy, without manual operation of the user.

[0117] According to the second aspect, or any one of the implementations of the second aspect above, the first electronic device comprises an inertial measurement unit module; the R second electronic devices comprise the first device, and the first device does not comprise an ultra-wideband module; based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the position of the human body, and the communication between the central device and the first electronic device, the central device acquires the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the central device, comprising: the first ultra-wideband module establishes a first coordinate system, based on the position measurement of the mobile device by the first electronic device and the labeling of the first device by the mobile device, the first electronic device acquires a first coordinate of the first device in the first coordinate system; the millimeter wave radar module establishes a second coordinate system, based on the perception of the human body by the first electronic device, the first electronic device acquires a second coordinate of the user in the second coordinate system; through the conversion between the first coordinate system and the second coordinate system, the first electronic device acquires a third coordinate of the second coordinate in the first coordinate system; the first electronic device sends the related coordinates in the first coordinate system to the central device, and the related coordinates comprise the first coordinate and the third coordinate; based on the output of the inertial measurement unit module, in combination with the parallel between the whole-house coordinate system and the geographic coordinate system, the central device converts the related coordinates in the first coordinate system into coordinates in the whole-house coordinate system, and further acquires the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system.

[0118] In this embodiment, for the second electronic device which does not comprise an ultra-wideband module, the first electronic device labels it at least once through the mobile device comprising an ultra-wideband module to acquire its position information.

[0119] In a third aspect, the present application provides a hub device. The hub device communicates with any two of the first electronic device, the R second electronic devices and the hub device in a wired or wireless manner; the first electronic device comprises a first ultra-wideband module and a millimeter wave radar module; the R second electronic devices comprise a mobile device, and the mobile device comprises a second ultra-wideband module; R is a positive integer greater than or equal to 1. Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in a whole-house coordinate system provided by the hub device; and based on the position information of the user and the position information of the R second electronic devices, the hub device controls or notifies at least one of the R second electronic devices to perform a preset operation.

[0120] In this embodiment, the hub device controls or notifies the second electronic device to perform a preset operation according to the position of the user and the position of the second electronic device. In this way, without any operation of the user and without carrying any electronic device by the user, the IoT device (the second electronic device) automatically performs an operation, which is convenient and fast.

[0121] According to the third aspect, based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in a whole-house coordinate system provided by the hub device, which comprises: based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device periodically or in real time obtains the position information of the R second electronic devices and the position information of the user in a whole-house coordinate system provided by the hub device. In this way, the hub device can control the second electronic device in real time, and improve the automatic control efficiency of the second electronic device.

[0122] According to the third aspect, or any one of the third aspect, the user is one user, and the position information of the user is the position information of the one user; or the user is multiple users, and the position information of the user is the central position information of the multiple users. In an embodiment, the position of the one user is represented by the coordinates of the one user in a coordinate system (such as a whole-house coordinate system). The central position of the multiple users is the average value of the coordinates of the multiple users.

[0123] According to a third aspect, or any possible implementation mode of the third aspect, the T second electronic devices of the R second electronic devices support voice wake-up; in response to receiving the first message from at least one of the T second electronic devices, the hub device obtains position information of the T second electronic devices and position information of the user in a whole-house coordinate system; the first message is used to indicate that the second electronic device receives a wake-up voice; the hub device determines a second electronic device closest to the user from the T second electronic devices; the hub device sends a first indication message to the second electronic device closest to the user, and the first indication message is used to instruct the second electronic device closest to the user to wake up; wherein T is a positive integer greater than or equal to 1 and less than or equal to R.

[0124] In this way, only one second electronic device closest to the user in the whole house is woken up, and multiple devices are avoided from responding to the wake-up at the same time.

[0125] In an implementation mode, in response to receiving the first message from one of the T second electronic devices, the hub device obtains position information of the second electronic device sending the first message and a first room or a first area where the second electronic device is located in a whole-house coordinate system; and the hub device further obtains position information of the user in the first room or the first area in the whole-house coordinate system. In this implementation mode, the hub device receives one first message, obtains a first room or a first area where the second electronic device corresponding to the first message is located, and obtains position information of the user in the first room or the first area. A second electronic device closest to the user in the first room or the first area is selected to respond to the wake-up.

[0126] In another implementation mode, in response to receiving multiple first messages within a preset time length, the hub device obtains position information of the T second electronic devices and position information of the user in a whole-house coordinate system. In this implementation mode, the hub device receives the first messages sent by all (or all capable of sending the first messages) second electronic devices, and determines a second electronic device closest to the user from the T second electronic devices to respond to the wake-up.

[0127] According to the third aspect, or any possible implementation mode of the third aspect, the hub device pre-obtains that the T second electronic devices of the R second electronic devices support voice wake-up based on communication between the hub device and the T second electronic devices; T is a positive integer greater than or equal to 1 and less than or equal to R; and the user is one user or multiple users.

[0128] According to a third aspect, or any possible implementation mode of the third aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio; the user includes a first user; the hub device receives a first notification message from the first device, the first notification message is used to notify the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; after obtaining that the second device is the closest playing device to the first user, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message includes an identifier and playing progress information of the first video or the first audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the first video or the first audio, and the second notification message includes the identifier and the playing progress information of the first video or the first audio.

[0129] In this implementation mode, in the first device playing, the hub device determines that the user leaves the first device and approaches the second device, and controls switching to continue playing by the second device and stop playing by the first device. This method enables the user to realize seamless and automatic switching of audio or video playing in the above movement without carrying any device, only by leaving the first device and approaching the second device.

[0130] In an implementation mode, obtaining that the second device is the closest playing device to the first user includes: obtaining that, among devices that do not play a video or an audio, the second device is the closest playing device to the first user.

[0131] According to a third aspect, or any possible implementation mode of the third aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio; the user includes a first user and a second user; the hub device receives a first notification message from the first device, the first notification message is used to notify the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; the hub device receives a second notification message from the first device, the second notification message is used to notify the first device to start playing a second video or a second audio; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; after obtaining that the second device is the closest playing device to the second user, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message includes an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the second video or the second audio, and the second notification message includes the identifier and the playing progress information of the second video or the second audio.

[0132] In this implementation mode, in the process that the first device plays the first video or the first audio corresponding to the first user, the second user uses the first device to play the second video or the second audio. The hub device determines that the second user leaves the first device and approaches the second device, and controls switching to continue playing the second video or the second audio by the second device and stop playing by the first device. This method enables the user to realize seamless and automatic switching of audio or video playing in the above-mentioned movement without carrying any device, only by leaving the first device and approaching the second device.

[0133] In an implementation mode, obtaining that the second device is the closest playing device to the second user includes: obtaining that, among the devices that do not play the video or the audio, the second device is the closest playing device to the second user.

[0134] According to a third aspect, or any possible implementation mode of the third aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user, and the first user is located in the first room or the first area; the hub device receives a first notification message from the first device, the first notification message is used to notify the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; in response to obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message includes an identifier and playing progress information of the first video or the first audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the first video or the first audio, and the second notification message includes the identifier and the playing progress information of the first video or the first audio.

[0135] In this implementation mode, in the playing of the first device, the hub device controls switching to continue playing by the second device and stop playing by the first device according to the user leaving the first room or the first area where the first device is located and entering the second room or the second area where the second device is located. This method enables the user to realize seamless and automatic switching of audio or video playing in the above-mentioned movement without carrying any device, only by leaving the first room or the first area and entering the second room or the second area.

[0136] In an implementation mode, after obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0137] In an implementation mode, after obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0138] According to a third aspect, or any possible implementation mode of the third aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user and a second user, the first user and the second user are located in the first room or the first area; the hub device receives a first notification message from the first device, the first notification message is used to inform the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; the hub device receives a second notification message from the first device, the second notification message is used to inform the first device to start playing a second video or a second audio; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; when it is acquired that the second user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message includes an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the second video or the second audio, and the second notification message includes the identifier and the playing progress information of the second video or the second audio.

[0139] In this implementation mode, in the process that the first device plays the first video or the first audio corresponding to the first user, the second user uses the first device to play the second video or the second audio. The hub device controls switching to continue playing the second video or the second audio by the second device and stops playing by the first device according to the second user leaving the first room or the first area where the first device is located and entering the second room or the second area where the second device is located. This method enables the user to realize seamless and automatic switching of audio or video playing without carrying any device in the above-mentioned movement, only by leaving the first device and approaching the second device.

[0140] In an implementation mode, after acquiring that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0141] In an implementation mode, after acquiring that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0142] According to a third aspect, or any possible implementation mode of the third aspect, the R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device and the second device are located in the first room or the first area, a first distance between the first device and the user, and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information and the second playing message comprising the second volume information; the first volume information is used to indicate a playing volume of the first device, and the second volume information is used to indicate a playing volume of the second device.

[0143] In this implementation mode, when two audio playing devices playing the stereo left channel and the stereo right channel, the volume played by the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The audio playing device far away from the user has a large volume compensation value, so that the user receives the same volume played by the two audio playing devices, and the experience of listening to stereo audio is improved.

[0144] In an implementation mode, the stereo system further comprises a third device, the third device playing a stereo auxiliary sound; the stereo system comprises the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user; the hub device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising the third volume information; the third volume information is used to indicate a playing volume of the third device.

[0145] According to a third aspect, or any possible implementation mode of the third aspect, the R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device is located in a first area and the second device is located in a second area, the first area being adjacent to the second area; the hub device further obtains a first distance between the first device and the user and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information and the second playing message comprising the second volume information; the first volume information is used to indicate the playing volume of the first device and the second volume information is used to indicate the playing volume of the second device.

[0146] In this implementation mode, the two audio playing devices playing the stereo left channel and the stereo right channel are located in adjacent areas, for example, a living room and a dining room are connected together, there are less partitions and barriers between the two, and the spatial independence is weak; the devices in the same stereo system can be located in the living room and the dining room respectively. When the two audio playing devices playing the stereo left channel and the stereo right channel play, the volume of the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The volume compensation value of the audio playing device far from the user is large, so that the user receives the same playing volume of the two audio playing devices, and the experience of the user listening to the stereo audio is improved.

[0147] In an implementation mode, the stereo system further comprises a third device playing a stereo auxiliary sound; the stereo sound comprises the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; the hub device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising the third volume information; in response to receiving the third playing message, the third volume information is used to indicate the playing volume of the third device.

[0148] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device reaching the user and a second volume attenuation value of the playing volume of the second device reaching the user; and the central device determines, according to the first volume attenuation value and the second volume attenuation value, the first volume information for the first device and the second volume information for the second device.

[0149] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device reaching the user and a second volume attenuation value of the playing volume of the second device reaching the user includes:

[0150] A p = K - DL w + A e ,

[0151] wherein A p is a volume attenuation value; K is a distance attenuation factor, representing volume attenuation caused by distance, related to the distance between the device and the user; DL w is a directivity factor, representing volume attenuation caused by the direction of audio signal emission; A e is an air absorption attenuation, representing volume attenuation caused by air absorption of audio when the audio propagates in the air.

[0152] In this embodiment, the playing volume of the audio playing device is compensated according to the attenuation value in the volume transmission process.

[0153] According to a third aspect, or any possible implementation of the third aspect, the R second electronic devices include the first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user and no second user; the central device further obtains that the first device enters a private mode and plays a first video; when detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to a screen saver; when detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

[0154] In this embodiment, under the condition that the device opens the private mode, if the central device detects that a specific category of user (such as a child) enters the room or area where the device is located, the playing is automatically paused and switched to a screen saver; if the central device detects that a specific category of user (such as a child) leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout the process.

[0155] In one implementation, in response to receiving the first feedback message from the first device, the hub device learns that the first device has stopped playing and switches to the screen saver.

[0156] In one implementation, in response to receiving the second feedback message from the first device, the hub device learns that the first device has resumed playing.

[0157] According to the third aspect, or any of the implementations of the third aspect, the R second electronic devices include a first device; the hub device learns that the first device is located in a first room or a first area, the first room or the first area has a first user and no second user; the hub device further learns that the first device enters the private mode and plays a first video; in response to detecting that a user other than the first user enters the first room or the first area, the hub device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to the screen saver; in response to detecting that the first room or the first area has no user other than the first user, the hub device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

[0158] In this implementation, under the condition that the device enters the private mode, if the hub device detects that a user other than the first user enters the room or area where the device is located, the playing is automatically paused and switched to the screen saver; if the hub device detects that the user other than the first user leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout.

[0159] According to the third aspect, or any of the implementations of the third aspect, the R second electronic devices include a first device; the hub device learns that the first device plays a first video, the hub device further learns a first position of the first device, the first device is located in a first room or a first area, a second position of a user in the first room or the first area, and a first viewing area corresponding to the first device; in response to detecting that there is no user in the first viewing area within a first preset time length, the hub device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing the first video; in response to detecting that there is a user in the first viewing area within a second preset time length, the hub device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

[0160] In this implementation, the hub device obtains the relative position of the user and the device, if the user leaves the viewing area of the first device, the first device automatically stops playing or pauses playing; if the user enters the viewing area of the first device, the first device automatically continues playing; no manual operation of the user is required throughout.

[0161] According to a third aspect, or any possible implementation mode of the third aspect, in response to detecting that the distance between at least one user and the first device is less than the first preset distance within a third preset time length, the hub device sends a third notification message to the first device; the third notification message is used to instruct the first device to stop playing the first video and output a reminder information; in response to detecting that no user has a distance less than the first preset distance from the first device and a user is in the first viewing area within a fourth preset time length, the hub device sends a fourth notification message to the first device; the fourth notification message is used to instruct the first device to resume playing the first video.

[0162] In this implementation mode, the hub device acquires the relative positions of the user and the device, and controls the first device to stop playing or pause playing and output a reminder information if it is determined that the user is too close to the first device; controls the first device to automatically resume playing if it is determined that the user is in the viewing area of the first device and is not too close to the first device. No manual operation of the user is required throughout.

[0163] According to a third aspect, or any possible implementation mode of the third aspect, in response to detecting that the distance between at least one user and the first device is less than the first preset distance within a third preset time length, the hub device sends a third notification message to the first device; the third notification message is used to instruct the first device to stop playing the first video and output a reminder information; in response to detecting that no user has a distance less than the first preset distance from the first device and a user is in the first viewing area within a fourth preset time length, the hub device sends a fourth notification message to the first device; the fourth notification message is used to instruct the first device to resume playing the first video.

[0164] In this implementation mode, the hub device acquires the relative positions of the user and the device, and controls the first device to stop playing or pause playing and output a reminder information if it is determined that the user is too close to the first device; controls the first device to automatically resume playing if it is determined that the user is in the viewing area of the first device and is not too close to the first device. No manual operation of the user is required throughout.

[0165] In a fourth aspect, the present application provides an automatic control method based on human perception, applied to a hub device, the hub device being in communication with any two of the first electronic device and any one of R second electronic devices in a wired or wireless manner; the first electronic device comprising a first ultra-wideband module and a millimeter wave radar module; the R second electronic devices comprising a mobile device, the mobile device comprising a second ultra-wideband module; R being a positive integer greater than or equal to 1. The method comprises: based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the position of the human body, the communication between the hub device and the first electronic device, the hub device acquiring the position information of the R second electronic devices and the position information of the user in a whole-house coordinate system provided by the hub device; and according to the position information of the user and the position information of the R second electronic devices, the hub device controls or notifies at least one of the R second electronic devices to perform a preset operation.

[0166] In this embodiment, the central device controls or notifies the second electronic device to perform the preset operation according to the position of the user and the position of the second electronic device. In this way, without any operation of the user and without any electronic device carried by the user, the IoT device (the second electronic device) automatically performs an operation, which is convenient and fast.

[0167] According to the fourth aspect, based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human position, and the communication between the central device and the first electronic device, the central device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the central device, including: based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human position, and the communication between the central device and the first electronic device, the central device periodically or in real time obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the central device. In this way, the central device can control the second electronic device in real time, and improve the automatic control efficiency of the second electronic device.

[0168] According to the fourth aspect, or any one of the embodiments of the fourth aspect, when the user is one user, the position information of the user is the position information of the one user; when the user is multiple users, the position information of the user is the central position information of the multiple users. In an embodiment, the position of the one user is represented by the coordinates of the one user in the coordinate system (such as the whole-house coordinate system). The central position of the multiple users is the average of the coordinates of the multiple users.

[0169] According to the fourth aspect, or any one of the embodiments of the fourth aspect, T second electronic devices of the R second electronic devices support voice wake-up; in response to receiving a first message from at least one of the T second electronic devices, the central device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system; the first message is used to indicate that the second electronic device receives the wake-up voice; the central device determines a device closest to the user from the T second electronic devices; the central device sends a first indication message to the device closest to the user, and the first indication message is used to instruct the device closest to the user to wake up; wherein T is a positive integer greater than or equal to 1 and less than or equal to R.

[0170] In this way, only one device closest to the user in the whole house is woken up, avoiding multiple devices responding to wake-up at the same time.

[0171] In an embodiment, in response to receiving the first message from one of the T second electronic devices, the hub device obtains the position information of the one second electronic device sending the first message and the first room or the first area where the one second electronic device is located in the whole-house coordinate system; and the hub device further obtains the position information of the user in the first room or the first area in the whole-house coordinate system. In this embodiment, the hub device receives one first message, obtains the first room or the first area where the second electronic device corresponding to the first message is located, and obtains the position information of the user in the first room or the first area. A device closest to the user in the first room or the first area is selected to respond to the wake-up.

[0172] In another embodiment, in response to receiving a plurality of first messages within a preset time length, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system. In this embodiment, the hub device receives the first messages sent by all (or all capable of sending the first messages) second electronic devices, and determines a device closest to the user from the T second electronic devices to respond to the wake-up.

[0173] According to a fourth aspect, or any possible implementation of the fourth aspect, the hub device pre-obtains that the T second electronic devices support voice wake-up based on the communication between the hub device and the T second electronic devices of the R second electronic devices; T is a positive integer greater than or equal to 1 and less than or equal to R; and the user is one user or multiple users.

[0174] According to the fourth aspect, or any possible implementation of the fourth aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio; the user includes a first user; the hub device receives a first notification message from the first device, the first notification message is used to notify the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; after obtaining that the second device is a closest playing device to the first user, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message includes an identifier and playing progress information of the first video or the first audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the first video or the first audio, and the second notification message includes the identifier and the playing progress information of the first video or the first audio.

[0175] In this embodiment, in the playing of the first device, the hub device determines that the user leaves the first device and approaches the second device, and controls switching to continue playing by the second device, and the first device stops playing. This method enables the user to realize seamless and automatic switching of audio or video playing in the above movement without carrying any device, only by leaving the first device and approaching the second device.

[0176] In one embodiment, obtaining the second device as the playing device closest to the first user comprises: obtaining, from the devices that do not play the video or the audio, the second device as the playing device closest to the first user.

[0177] According to a fourth aspect, or any possible implementation of the fourth aspect above, the R second electronic devices comprise a first device and a second device, the first device and the second device support playing the video or the audio; the user comprises a first user and a second user; the hub device receives a first notification message from the first device, the first notification message is used to notify the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; the hub device receives a second notification message from the first device, the second notification message is used to notify the first device to start playing a second video or a second audio; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; after obtaining the second device as the playing device closest to the second user, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message comprises an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the second video or the second audio, and the second notification message comprises the identifier and the playing progress information of the second video or the second audio.

[0178] In this embodiment, in the playing of the first device of the first video or the first audio corresponding to the first user, the second user uses the first device to play the second video or the second audio. The hub device determines that the second user leaves the first device and approaches the second device, and controls switching to continue playing by the second device, and the first device stops playing. This method enables the user to realize seamless and automatic switching of audio or video playing in the above movement without carrying any device, only by leaving the first device and approaching the second device.

[0179] In one embodiment, obtaining the second device as the playing device closest to the second user comprises: obtaining, from the devices that do not play the video or the audio, the second device as the playing device closest to the second user.

[0180] According to a fourth aspect, or any possible implementation mode of the fourth aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user, and the first user is located in the first room or the first area; the hub device receives a first notification message from the first device, the first notification message is used to notify the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; in response to obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message includes an identifier and playing progress information of the first video or the first audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the first video or the first audio, and the second notification message includes the identifier and the playing progress information of the first video or the first audio.

[0181] In this implementation mode, in the playing of the first device, the hub device controls switching to continue playing by the second device and stop playing by the first device according to the user leaving the first room or the first area where the first device is located and entering the second room or the second area where the second device is located. This method enables the user to realize seamless and automatic switching of audio or video playing in the above-mentioned movement without carrying any device, only by leaving the first room or the first area and entering the second room or the second area.

[0182] In an implementation mode, after obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0183] In an implementation mode, after obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

[0184] According to a fourth aspect, or any possible implementation mode of the fourth aspect, the R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user and a second user, the first user and the second user are located in the first room or the first area; the hub device receives a first notification message from the first device, the first notification message is used to inform the first device to start playing a first video or a first audio; in response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; the hub device receives a second notification message from the first device, the second notification message is used to inform the first device to start playing a second video or a second audio; in response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; when it is acquired that the second user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device, the first notification message is used to instruct the first device to stop playing; the hub device receives a first feedback message from the first device, the first feedback message includes an identifier and playing progress information of the second video or the second audio; in response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message is used to instruct the second device to play the second video or the second audio, and the second notification message includes the identifier and the playing progress information of the second video or the second audio.

[0185] In this implementation mode, in the process that the first device plays the first video or the first audio corresponding to the first user, the second user uses the first device to play the second video or the second audio. The hub device controls switching to continue playing the second video or the second audio by the second device and stops playing by the first device according to the second user leaving the first room or the first area where the first device is located and entering the second room or the second area where the second device is located. This method enables the user to realize seamless and automatic switching of audio or video playing in the above-mentioned movement without carrying any device, only by leaving the first device and approaching the second device.

[0186] In an implementation mode, after acquiring that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0187] In an implementation mode, after acquiring that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

[0188] According to a fourth aspect, or any possible implementation mode of the fourth aspect, the R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device and the second device are located in the first room or the first area, a first distance between the first device and the user, and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information and the second playing message comprising the second volume information; the first volume information is used to indicate a playing volume of the first device, and the second volume information is used to indicate a playing volume of the second device.

[0189] In this implementation mode, when two audio playing devices playing the stereo left channel and the stereo right channel, the volume played by the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The audio playing device far away from the user has a large volume compensation value, so that the user receives the same volume played by the two audio playing devices, and the experience of listening to stereo audio is improved.

[0190] In an implementation mode, the stereo system further comprises a third device playing a stereo auxiliary sound; the stereo system comprises the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user; the hub device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising the third volume information; the third volume information is used to indicate a playing volume of the third device.

[0191] According to a fourth aspect, or any possible implementation mode of the fourth aspect, the R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a stereo left channel and the second device playing a stereo right channel; the hub device obtains that the first device is located in a first area and the second device is located in a second area, the first area being adjacent to the second area; the hub device further obtains a first distance between the first device and the user and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; the hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information and the second playing message comprising the second volume information; the first volume information is used to indicate the playing volume of the first device and the second volume information is used to indicate the playing volume of the second device.

[0192] In this implementation mode, the two audio playing devices playing the stereo left channel and the stereo right channel are located in adjacent areas, for example, a living room and a dining room are connected together, there are less partitions and barriers between the two, and the spatial independence is weak; the devices in the same stereo system can be located in the living room and the dining room respectively. When the two audio playing devices playing the stereo left channel and the stereo right channel play, the volume of the two audio playing devices is automatically compensated according to the distance between the two audio playing devices and the user. The volume compensation value of the audio playing device far from the user is large, so that the user receives the same playing volume of the two audio playing devices, and the experience of the user listening to the stereo audio is improved.

[0193] In an implementation mode, the stereo system further comprises a third device playing a stereo auxiliary sound; the stereo sound comprises the stereo left channel, the stereo right channel and the stereo auxiliary sound; the hub device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; the hub device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising the third volume information; in response to receiving the third playing message, the third volume information is used to indicate the playing volume of the third device.

[0194] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device reaching the user and a second volume attenuation value of the playing volume of the second device reaching the user; and the central device determines, according to the first volume attenuation value and the second volume attenuation value, the first volume information for the first device and the second volume information for the second device.

[0195] In an embodiment, the central device determines, according to the first distance and the second distance, a first volume attenuation value of the playing volume of the first device reaching the user and a second volume attenuation value of the playing volume of the second device reaching the user includes:

[0196] A p = K - DL w + A e ,

[0197] wherein A p is a volume attenuation value; K is a distance attenuation factor, representing volume attenuation caused by distance, related to the distance between the device and the user; DL w is a directivity factor, representing volume attenuation caused by the direction of audio signal emission; A e is an air absorption attenuation, representing volume attenuation caused by air absorption of audio when the audio propagates in the air.

[0198] In this embodiment, the playing volume of the audio playing device is compensated according to the attenuation value in the volume transmission process.

[0199] According to a fourth aspect, or any possible implementation of the fourth aspect, the R second electronic devices include the first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user and no second user; the central device further obtains that the first device enters a private mode and plays a first video; when detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to a screen saver; when detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

[0200] In this embodiment, under the condition that the device enters the private mode, if the central device detects that a specific category of user (such as a child) enters the room or area where the device is located, the playing is automatically paused and the screen saver is automatically switched; if the central device detects that the specific category of user (such as a child) leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout the process.

[0201] In an implementation, in response to receiving the first feedback message from the first device, the hub device learns that the first device has stopped playing and switches to the screen saver.

[0202] In an implementation, in response to receiving the second feedback message from the first device, the hub device learns that the first device has resumed playing.

[0203] According to the fourth aspect, or any of the implementations of the fourth aspect, the R second electronic devices include the first device; the hub device learns that the first device is located in the first room or the first area, the first room or the first area has the first user, and has no second user; the hub device further learns that the first device enters the private mode and plays the first video; in response to detecting that a user other than the first user enters the first room or the first area, the hub device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to the screen saver; in response to detecting that the first room or the first area has no user other than the first user, the hub device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

[0204] In this implementation, under the condition that the device enters the private mode, if the hub device detects that a user other than the first user enters the room or area where the device is located, the playing is automatically paused and switched to the screen saver; if the hub device detects that the user other than the first user leaves the room or area where the device is located, the original video or original audio is automatically switched and the playing is automatically resumed; no manual operation is required throughout.

[0205] According to the fourth aspect, or any of the implementations of the fourth aspect, the R second electronic devices include the first device; the hub device learns that the first device plays the first video, the hub device further learns the first position of the first device, the first device is located in the first room or the first area, the second position of the user in the first room or the first area, and the first viewing area corresponding to the first device; in response to detecting that there is no user in the first viewing area within a first preset time length, the hub device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing the first video; in response to detecting that there is a user in the first viewing area within a second preset time length, the hub device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

[0206] In this implementation, the hub device obtains the relative position of the user and the device, if the user leaves the viewing area of the first device, the first device automatically stops playing or pauses playing; if the user enters the viewing area of the first device, the first device automatically continues playing; no manual operation of the user is required throughout.

[0207] According to a fourth aspect, or any possible implementation mode of the fourth aspect, in response to detecting that the distance between at least one user and the first device is less than the first preset distance within a third preset time length, the hub device sends a third notification message to the first device; the third notification message is used to instruct the first device to stop playing the first video and output a reminder information; in response to detecting that no user has a distance less than the first preset distance from the first device and a user is in the first viewing area within a fourth preset time length, the hub device sends a fourth notification message to the first device; the fourth notification message is used to instruct the first device to resume playing the first video.

[0208] In this implementation mode, the hub device acquires the relative positions of the user and the device, and if it is determined that the user is too close to the first device, the hub device controls the first device to stop playing or pause playing and output a reminder information; if it is determined that the user is in the viewing area of the first device and is not too close to the first device, the hub device controls the first device to automatically resume playing; no manual operation of the user is required throughout.

[0209] According to a fourth aspect, or any possible implementation mode of the fourth aspect, in response to detecting that the distance between at least one user and the first device is less than the first preset distance within a third preset time length, the hub device sends a third notification message to the first device; the third notification message is used to instruct the first device to stop playing the first video and output a reminder information; in response to detecting that no user has a distance less than the first preset distance from the first device and a user is in the first viewing area within a fourth preset time length, the hub device sends a fourth notification message to the first device; the fourth notification message is used to instruct the first device to resume playing the first video.

[0210] In this implementation mode, the hub device acquires the relative positions of the user and the device, and if it is determined that the user is too close to the first device, the hub device controls the first device to stop playing or pause playing and output a reminder information; if it is determined that the user is in the viewing area of the first device and is not too close to the first device, the hub device controls the first device to automatically resume playing; no manual operation of the user is required throughout.

[0211] In this implementation mode, the hub device acquires the relative positions of the user and the device, and if it is determined that the user is too close to the first device, the hub device controls the first device to stop playing or pause playing and output a reminder information; if it is determined that the user is in the viewing area of the first device and is not too close to the first device, the hub device controls the first device to automatically resume playing; no manual operation of the user is required throughout.

[0212] In this implementation mode, the hub device acquires the relative positions of the user and the device, and if it is determined that the user is too close to the first device, the hub device controls the first device to stop playing or pause playing and output a reminder information; if it is determined that the user is in the viewing area of the first device and is not too close to the first device, the hub device controls the first device to automatically resume playing; no manual operation of the user is required throughout.

[0213] The fifth aspect and any possible implementation mode of the fifth aspect correspond to the fourth aspect and any possible implementation mode of the fourth aspect respectively. The technical effects of the fifth aspect and any possible implementation mode of the fifth aspect correspond to the technical effects of the fourth aspect and any possible implementation mode of the fourth aspect, which will not be described herein.

[0214] The sixth aspect and any one of the implementation forms of the sixth aspect correspond to the fourth aspect and any one of the implementation forms of the fourth aspect respectively. The technical effects corresponding to the sixth aspect and any one of the implementation forms of the sixth aspect can refer to the technical effects corresponding to the fourth aspect and any one of the implementation forms of the fourth aspect, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0215] Figure 1 A scene schematic diagram of the automatic control method based on human perception provided by the embodiment of the present application;

[0216] Figure 2 A structure schematic diagram of the first electronic device in the automatic control method provided by the embodiment of the present application;

[0217] Figure 3 A structure schematic diagram of the central device in the automatic control method provided by the embodiment of the present application;

[0218] Figure 4 A structure schematic diagram of the second electronic device in the automatic control method provided by the embodiment of the present application;

[0219] Figure 5A A structure schematic diagram of the ultra-wide band (UWB) module in the first electronic device provided by the present application;

[0220] Figure 5B A structure schematic diagram of the millimeter wave radar module in the first electronic device provided by the embodiment of the present application;

[0221] Figure 6 A structure schematic diagram of the UWB module in the second electronic device provided by the embodiment of the present application;

[0222] Figure 7 A plurality of antenna distribution schematic diagrams of the UWB module in the first electronic device provided by the embodiment of the present application;

[0223] Figure 8 A plurality of antenna distribution schematic diagrams of the millimeter wave radar module in the first electronic device provided by the embodiment of the present application;

[0224] Figure 9 A plurality of establishment methods of the first coordinate system in the case that the UWB module in the first electronic device includes three antennas provided by the embodiment of the present application;

[0225] Figure 10 A schematic diagram of one establishment method of the second coordinate system provided by the embodiment of the present application;

[0226] Figure 11 A schematic diagram of one establishment method of the third coordinate system provided by the embodiment of the present application;

[0227] Figure 12 A principle diagram of coordinate calculation of the second electronic device in the first coordinate system is provided for the embodiments of the present application.

[0228] Figure 13 A diagram of several marking modes of the second electronic device is provided for the embodiments of the present application.

[0229] Figure 14 A diagram of marking modes of the space region is provided for the embodiments of the present application.

[0230] Figure 15 A diagram of pitch angle, azimuth angle and roll angle of the second coordinate system relative to the first coordinate system is provided for the embodiments of the present application.

[0231] Figure 16 A diagram of pitch angle, azimuth angle and roll angle of the third coordinate system relative to the first coordinate system is provided for the embodiments of the present application.

[0232] Figure 17 A diagram of one establishment mode of the fourth coordinate system is provided for the embodiments of the present application.

[0233] Figure 18 A principle diagram of determining distance and radial velocity of the reflection point by the millimeter wave radar is provided for the embodiments of the present application.

[0234] Figure 19 A principle diagram of determining signal direction of the reflection signal of the reflection point by the millimeter wave radar is provided for the embodiments of the present application.

[0235] Figure 20 A principle diagram of determining the coordinate of the user by the first electronic device in the fourth coordinate system is provided for the embodiments of the present application.

[0236] Figure 21 A method diagram of determining the coordinate of the user by the first electronic device in the fourth coordinate system is provided for the embodiments of the present application.

[0237] Figure 22 A method diagram of obtaining the breathing frequency and heartbeat frequency of the user by the millimeter wave radar is provided for the embodiments of the present application.

[0238] Figure 23 A method diagram of determining the human posture of the user by the millimeter wave radar is provided for the embodiments of the present application.

[0239] ​​​​​​Figure 24 A conversion schematic diagram of the first coordinate system and the fourth coordinate system in the first electronic device provided for an embodiment of the present application is shown in FIG. 1A;

[0240] Figure 25 A schematic diagram of one example of establishing a whole-house coordinate system (fifth coordinate system), the first coordinate system, and the sixth coordinate system (geographic coordinate system) provided for an embodiment of the present application is shown in FIG. 2A;

[0241] Figure 26 A flowchart and principle schematic diagram of an automatic control method based on human perception in a whole-house scene provided for an embodiment of the present application is shown in FIG. 3A;

[0242] Figure 27 A flowchart schematic diagram of correcting installation errors of the first electronic device provided for an embodiment of the present application is shown in FIG. 4A;

[0243] Figure 28 A schematic diagram of an ICP algorithm principle provided for an embodiment of the present application is shown in FIG. 5A;

[0244] Figure 29 A schematic diagram of region division and a user interface provided for an embodiment of the present application is shown in FIG. 6A;

[0245] Figure 30A A schematic diagram of a method for waking up an audio playback device provided for an embodiment of the present application is shown in FIG. 7A;

[0246] Figure 30B A schematic diagram of an automatic device waking up method based on human perception in a whole-house scene provided for an embodiment of the present application is shown in FIG. 8A;

[0247] Figure 30C A flowchart schematic diagram of an automatic device waking up method based on human perception in a whole-house scene provided for an embodiment of the present application is shown in FIG. 9A;

[0248] Figure 30D Another schematic diagram of an automatic device waking up method based on human perception in a whole-house scene provided for an embodiment of the present application is shown in FIG. 10A;

[0249] Figure 31 A schematic diagram of an automatic device switching method based on human perception in a whole-house scene provided for an embodiment of the present application is shown in FIG. 11A;

[0250] Figure 32A - Figure 32E A flowchart schematic diagram of an automatic device switching method based on human perception in a whole-house scene provided for an embodiment of the present application is shown in FIG. 12A;

[0251] Figure 33 A human-computer interaction interface schematic diagram involved in an automatic device switching method based on human perception in a whole-house scene provided for an embodiment of the present application is shown in FIG. 13A;

[0252] Figure 34AA scene and principle schematic diagram of the method for automatically adjusting stereo sound based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0253] Figure 34B Another scene and principle schematic diagram of the method for automatically adjusting stereo sound based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0254] Figure 35 A flowchart of the method for automatically adjusting stereo sound based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0255] Figure 36 A scene and principle schematic diagram of the method for automatically playing a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0256] Figure 37A - Figure 37B A flowchart of the method for automatically playing a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0257] Figure 38 A man-machine interaction interface schematic diagram involved in the method for automatically playing a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0258] Figure 39A A schematic diagram of automatically pausing playing, automatically stopping playing or automatically closing a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0259] Figure 39B A schematic diagram of automatically starting playing, automatically resuming playing or automatically opening a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0260] Figure 40A - Figure 40C A flowchart of the method for automatically playing a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0261] Figure 41 A schematic diagram of a method for determining a viewing area in the method for automatically playing a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0262] Figure 42 A schematic diagram of a method for determining a distance between a user and a video playing device in the method for automatically playing a device based on human body perception in a whole-house scene provided by the embodiment of the present application;

[0263] Figure 43 A structural schematic diagram of the first electronic device provided by the embodiment of the present application;

[0264] Figure 44A second electronic device structure schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0265] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing the specific embodiments and are not intended to be limiting on the present application. As used in the specification and the appended claims of the present application, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that “at least one” and “one or more” as used in the embodiments herein indicates one or two or more (including two). The term “and / or” is used to describe the association relationship of the associated objects, which means that there can be three relationships; for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character “ / ” generally represents an “or” relationship between the associated objects.

[0266] In the present specification, the phrase “one embodiment” or “some embodiments” etc. means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the phrases “in one embodiment”, “in some embodiments”, “in other some embodiments”, “in further some embodiments” etc. appearing in different places in the present specification are not necessarily all referring to the same embodiment, but mean “one or more but not all embodiments”, unless otherwise specifically emphasized. The terms “include”, “contain”, “have” and their variants mean “including but not limited to”, unless otherwise specifically emphasized. The term “connection” includes direct connection and indirect connection, unless otherwise specified. “First”, “second” are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0267] In the embodiments of the present application, the words “exemplarily” or “for example” are used to mean as an example, illustration or description. Any embodiment or design scheme described as “exemplarily” or “for example” in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words “exemplarily” or “for example” are used to present the relevant concept in a specific manner.

[0268] I. Overall scenario introduction

[0269] Exemplarily, Figure 1A scene schematic diagram of the automatic control method based on human perception provided by the embodiments of the present application is shown in (a) of FIG. 13. Figure 1 As shown in (a) of FIG. 13, the whole house includes an entrance passage, a kitchen, a dining room, a living room, a balcony, a master bedroom, a secondary bedroom, a bathroom, etc. The whole house is provided with at least one first electronic device. Exemplarily, each room or area includes at least one first electronic device. The whole house is also provided with a second electronic device (such as an IoT device). Specifically, the kitchen is provided with a rice cooker or an electric pressure cooker, a gas device, etc.; the living room is provided with a sound box (such as a smart sound box), a television (such as a smart television, also known as a smart screen, a large screen, etc.), a routing device, etc.; the balcony is provided with a clothes drying rack (such as a smart clothes drying rack, etc.); the dining room is provided with a sweeping robot, etc.; the master bedroom is provided with a television (such as a smart television), a sound box (such as a smart sound box), a floor lamp (such as a smart floor lamp), a routing device, etc.; the secondary bedroom is provided with a table lamp (such as a smart table lamp), a sound box (such as a smart sound box), etc.; the bathroom is provided with a body fat scale, etc.

[0270] The automatic control method based on human perception provided by the embodiments of the present application includes an automatic control method based on human perception in a whole house scenario, and also includes an automatic control method based on human perception in a single room or area. The automatic control method based on human perception provided by the embodiments of the present application is applied to a communication system. Accordingly, the communication system includes a communication system based on human perception in a whole house scenario (which can also be referred to as a whole house intelligent system), and a communication system based on human perception in a single room or area (which can also be referred to as a room intelligent system or an area intelligent system). The communication system includes at least one first electronic device 100 and at least one second electronic device 300. In addition, the communication system can also include a hub device 200.

[0271] The first electronic device 100 is configured to locate the second electronic device 300 and / or a user. The first electronic device 100 can include a sensor. In an example, the first electronic device 100 includes an Ultra-Wide Band (UWB) module and a millimeter wave radar module. The UWB module is configured to locate the second electronic device 300, and the millimeter wave radar module is configured to locate the user.

[0272] UWB technology is a kind of wireless communication technology, which does not use carrier modulation signals, but uses a sequence of energy pulses below nanoseconds or microseconds, and expands the pulses into a frequency range through orthogonal frequency division modulation or direct sequencing. UWB has the characteristics of wide spectrum, high precision, low power consumption, strong anti-multipath ability, high security, and low system complexity, and is mainly used for short-distance and high-rate wireless communication, especially in the field of indoor positioning. Generally speaking, the positioning accuracy of UWB system can reach centimeter level. The UWB system includes a UWB base station and a UWB tag. The UWB base station determines the position (coordinates) of the UWB tag by detecting the distance between the UWB tag and the UWB base station, and the signal direction of the UWB tag; that is, the UWB tag is positioned. Among them, the UWB positioning is based on the UWB coordinate system (also called the first coordinate system).

[0273] In an example, the second electronic device includes a UWB module. The UWB module of the first electronic device 100 implements the UWB base station function, and the UWB module of the second electronic device implements the UWB tag function. By positioning the UWB module of the second electronic device through the UWB module of the first electronic device 100, the positioning of the second electronic device by the first electronic device 100 can be realized.

[0274] In another example, some second electronic devices do not include a UWB module. Generally, the second electronic device including the UWB module is a mobile device (such as a smart phone, a remote controller, etc.). In this way, the second electronic device not including the UWB module can be labeled by the second electronic device including the UWB module, and the positioning of the second electronic device not including the UWB module by the first electronic device 100 can be realized. The specific labeling method will be described in detail later.

[0275] The millimeter wave radar works in the millimeter wave band, and is mainly used for detecting moving objects, with the working frequency band distributed in the 30-300GHz frequency domain (wavelength of 1-10mm). The millimeter wave radar continuously emits (radiates) specific forms of radio electromagnetic signals when working, and receives electromagnetic echo signals reflected by objects, and determines the spatial information of the objects by comparing the differences between the transmitted signals and the received signals. The millimeter wave radar has the characteristics of small size and high spatial resolution; deployed in the room, it can be used to detect the position of the human body (user) in the whole house, physiological characteristics (such as breathing rate, heartbeat rate, etc.), identity category (such as adult, child, etc.) and human body posture (such as standing, sitting, lying, etc.) information. In this way, the first electronic device 100 can position the user through the integrated millimeter wave radar module, and even detect the physiological characteristics, identity category and human body posture information of the user. The specific method will be described in detail later.

[0276] Positioning via millimeter-wave radar is based on the millimeter-wave radar coordinate system (also known as the second coordinate system). The coordinates obtained from positioning in the second coordinate system and the first coordinate system need to be transformed or unified to the same coordinate system. The specific coordinate system transformation methods will be described in detail later.

[0277] It should be noted that this application embodiment uses a first electronic device 100 including a UWB module and a millimeter-wave radar as an example for description. In other embodiments, the first electronic device 100 may only include a UWB module or a millimeter-wave radar. The first electronic device 100 including the UWB module is used for locating the second electronic device 300, and the first electronic device 100 including the millimeter-wave radar is used for locating the user. The above two types of first electronic devices 100 can cooperate to realize the automatic control method based on human perception provided in this application embodiment. This application embodiment is not limited in this respect.

[0278] The first electronic device 100 can acquire the location of the second electronic device 300 (i.e., locate the second electronic device), and it can also acquire the location of users within a room or area (i.e., locate the users). A house includes at least one room or area. If only one first electronic device is installed throughout the house, signal attenuation due to wall obstructions and other factors may occur, meaning one first electronic device cannot cover all areas of the house. Therefore, multiple first electronic devices are typically installed throughout the house. For example, one first electronic device can be installed in each relatively independent space within the house (e.g., living room, bedroom, study, balcony, bathroom, kitchen, hallway, etc.) to locate the second electronic device and users within that independent space; in this way, the second electronic device or user located anywhere in the house can be detected by the first electronic device.

[0279] For example, such as Figure 1 As shown in (a), the signal transmission and reception range of the first electronic device installed in the entrance hallway can cover the entrance hallway. The signal transmission and reception range of the first electronic device installed in the kitchen can cover the kitchen. The signal transmission and reception range of the first electronic device installed in the living room can cover the living room. The signal transmission and reception range of the first electronic device installed in the dining room can cover the dining room. The signal transmission and reception range of the first electronic device installed on the balcony can cover the balcony. The signal transmission and reception range of the first electronic device installed in the master bedroom can cover the master bedroom. The signal transmission and reception range of the first electronic device installed in the bathroom can cover the bathroom. The signal transmission and reception range of the first electronic device installed in the second bedroom can cover the second bedroom. In one example, as... Figure 1As shown in (b), the first electronic device can be installed on the wall of a room or area. In one example, the first electronic device can be installed on the ceiling of the room or area. This reduces signal obstruction by furniture and other objects throughout the room, preventing signal obstruction from reducing the detection accuracy of the first electronic device. In another example, the first electronic device can be installed on the floor of the room or area. Optionally, the first electronic device 100 can exist independently or be integrated with a second electronic device. This application does not limit this. For example, the first electronic device 100 can be integrated with a smart air conditioner into a single device.

[0280] Optionally, some rooms or areas may not require a first electronic device; that is, not all rooms or areas need to have at least one first electronic device. For example, a dining room may not need a first electronic device. The entrance hallway and dining room can share a first electronic device, or the dining room and living room can share a first electronic device.

[0281] It should be noted that, although in Figure 1 In (a), only a smart TV is shown as the second electronic device 300. However, those skilled in the art should know that the second electronic device 300 includes, but is not limited to, smart TVs, smart speakers, smart lighting fixtures (such as ceiling lights, smart table lamps, aromatherapy lamps, etc.), robot vacuum cleaners, body fat scales, smart clothes racks, smart rice cookers, air purifiers, humidifiers, desktop computers, routers, smart sockets, water dispensers, smart refrigerators, smart air conditioners, smart switches, smart door locks, etc. It should be noted that the second electronic device 300 may not be a smart home device, but rather a portable device, such as a personal computer (PC), tablet computer, mobile phone, smart remote control, etc. This application does not limit the specific form of the second electronic device 300.

[0282] The second electronic device 300 and the first electronic device 100 can be connected to the hub device 200 through wired (such as power line communication (PLC)) and / or wireless (such as wireless fidelity (Wi-Fi), Bluetooth, etc.) modes. It can be understood that the connection modes of the second electronic device 300 and the first electronic device 100 to the hub device 200 can be the same or different. For example, the second electronic device 300 and the first electronic device 100 are both connected to the hub device 200 through wireless mode. Alternatively, the second electronic device 300 is connected to the hub device 200 through wireless mode, and the first electronic device 100 is connected to the hub device 200 through wired mode. Alternatively, the smart speaker, smart TV, body fat scale, and robot vacuum cleaner in the second electronic device 300 are connected to the hub device 200 through wireless (such as Wi-Fi) mode, and the smart table lamp, smart clothes hanger, and smart door lock in the second electronic device 300 are connected to the hub device 200 through wired mode (such as PLC). Preferably, the first electronic device 100 and the second electronic device 300 communicate through wireless mode.

[0283] In an example, the first electronic device 100 can upload the position information of the second electronic device 300 obtained by detection, and at least one of the user's position, physiological characteristics, identity category, and human body posture, etc. information to the hub device 200 through wired or wireless mode.

[0284] The hub device 200, also known as hub, central control system, or host, etc., is used to receive the information sent by the first electronic device 100. Optionally, the hub device 200 is also used to build a whole house map, establish a whole house coordinate system, unify the position information obtained by each first electronic device 100 to the whole house coordinate system, and unify the measurement. In this way, the position information of the second electronic device 300 or the user detected and obtained by each first electronic device 100 can be unified into the whole house coordinate system, and the specific position of the second electronic device 300 or the user in the whole house can be determined. The hub device 200 also notifies or controls the second electronic device 300 according to the received information (including but not limited to position information). Accordingly, the conversion of each coordinate system is also involved, which will be described in detail later.

[0285] In one implementation, the central device 200 receives information sent by the first electronic device 100, including the location information of the second electronic device 300 and at least one of the user's location, physiological characteristics, identity category, and human posture. Based on the location information of the second electronic device 300 and at least one of the user's location, physiological characteristics, identity category, and human posture, the central device 200 notifies or controls the second electronic device 300 to execute preset instructions. For example, when a user wakes up a smart speaker via voice, the central device 200 notifies or controls one or more smart speakers closest to the user to be woken up based on the locations of multiple smart speakers throughout the house. For example, when a user moves from one room to another, the central device 200 controls the smart speaker in the room the user left to stop playing audio and controls the smart speaker in the room the user entered to start playing (e.g., resume playback) audio. As another example, the central device 200 controls the playback volume of two smart speakers (each playing the left and right channels of the same audio) based on their distance from the user, ensuring that the volume of the left and right channel audio received by the user is consistent. For example, if a user is watching a video (e.g., the video contains violent content) on a smart TV in a room, and a child is detected entering the room, the central device 200 will control the smart TV to stop playing the video. Alternatively, the central device 200 can notify or control the smart TV to start or stop playing video based on the user's position relative to the smart TV (e.g., distance, orientation).

[0286] Optionally, such as Figure 1 As shown in (a), at least one central device 200 is installed throughout the house. First electronic devices in each room or area can send the detected user's location information, as well as the location information of one or more second electronic devices in the same room or area, to the central device 200. The central device 200 acquires the detection data (including but not limited to location information) from all rooms or areas of the house, and can then notify or control the corresponding second electronic devices in the corresponding rooms or areas when preset conditions are met.

[0287] Optionally, a central device (not shown in the figure) can be installed in each room or area of ​​the house. The first electronic device in each room or area can send the detected user's location information, as well as the location information of one or more second electronic devices in that room or area, to the central device in that room or area. The central device in that room or area then sends the information to the central device 200 for the entire house. The central device 200 for the entire house receives the detection data from each room or area, and can then notify or control the central device in the corresponding room or area when preset conditions are met. The central device in the corresponding room or area then notifies or controls the corresponding second electronic device.

[0288] Optionally, the hub device of each room or each area, the hub device of the whole house can exist separately, or can be integrated with the first electronic device or the second electronic device into one device, or can be integrated with the first electronic device and the second electronic device into one device. The present application does not limit this.

[0289] Optionally, some rooms or areas can not need to be provided with a hub device; that is, not all rooms or areas are provided with at least one hub device. For example, the dining room can not be provided with a hub device. The dining room and the entrance hall share one hub device, or the dining room and the living room share one hub device.

[0290] Optionally, the hub device 200 of the whole house can also assume the function of the hub device of a certain room or area. For example, the hub device 200 of the whole house can also assume the function of the hub device of the living room. In an example, for each room or area other than a certain room or area (for example, the living room), one hub device is provided. When the hub device 200 of the whole house communicates with the second electronic device of each room or area other than the above-mentioned certain room or area (for example, the living room), it still passes through the hub device of each room or area; and when it communicates with the second electronic device of the above-mentioned certain room or area (for example, the living room), it no longer passes through the hub device of the above-mentioned certain room or area (for example, the living room).

[0291] In an example, the communication system further comprises a routing device (such as a router). The routing device is used to connect a local area network or the Internet, and select and set the path of the transmitted signal using a specific protocol. Exemplarily, one or more routers are deployed inside the whole house to form a local area network, or access a local area network or the Internet. The second electronic device 300 or the first electronic device 100 accesses the router, and performs data transmission with the devices in the local area network or the devices in the Internet through the Wi-Fi channel established by the router. In an embodiment, the hub device 200 can be integrated with the routing device into one device. For example, the hub device 200 is integrated with the routing device into a routing device, that is, the routing device has the function of the hub device 200. The routing device can be one or more of the sub-mother routing devices, or can be an independent routing device.

[0292] In an example, the communication system further comprises a gateway. The gateway is also called an inter-network connector or a protocol converter. In an embodiment, the gateway is used to forward the information of the first electronic device 100 to the routing device or the hub device 200. In another embodiment, the function of the hub device 200 can be realized by the gateway.

[0293] In an example, the communication system further comprises a server (e.g., a cloud server). The hub device 200, the routing device or the gateway can send the information received from the first electronic device 100 to the server. Further, the hub device 200, the routing device or the gateway can also send the control information of the hub device 200 to the second electronic device 300 to the server. Further, the hub device 200, the routing device or the gateway can also upload various information generated during the operation of the second electronic device 300 to the server for the user to view.

[0294] In an example, the communication system further comprises one or more input devices (e.g., the input device is a control panel). The control panel displays the human-computer interaction interface of the communication system. The user can view the information of the communication system (e.g., the connection information of each device in the communication system), the operation information of the second electronic device 300 and / or the control information of the hub device 200 to the second electronic device 300, etc. on the human-computer interaction interface. The user can also input control instructions on the human-computer interaction interface by clicking the screen or voice, etc. to control the devices in the communication system.

[0295] The above is only some elaboration of the automatic control method based on human perception provided by the embodiments of the present application. It should be noted that any content in the above-mentioned examples or optional manners can be freely combined, and the combined content is also within the scope of the present application.

[0296] II. Introduction of the hardware structure of the electronic device involved

[0297] For example, Figure 2 A structural schematic diagram of a first electronic device 100 is shown.

[0298] As Figure 2 shown, the first electronic device 100 can include a processor 110, a memory 120, a power management module 130, a power supply 131, a wireless communication module 140, a UWB module 150, a millimeter wave radar module 160, etc.

[0299] It can be understood that Figure 2 The schematic structure does not constitute a specific limitation on the first electronic device 100. In some other embodiments of the present application, the first electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software or a combination of software and hardware.

[0300] The processor 110 can include one or more processing units, which can be independent devices or integrated in one or more processors. For example, the processor 110 is a central processing unit (CPU), which can also be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0301] The memory 120 can be used to store computer executable program codes, including instructions. For example, the memory 120 can also store data processed by the processor 110. In addition, the memory 120 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the first electronic device 100 by running instructions stored in the memory 120 and / or instructions stored in the memory disposed in the processor.

[0302] The power management module 130 is used to receive power input from the power supply 131. The power supply 131 can be a battery or a mains power supply. The power management module 130 receives power supply from the battery and / or the mains power supply to supply power to various components of the first electronic device 100, such as the processor 110, the memory 120, the wireless communication module 140, the UWB module 150, the millimeter wave radar module 160, etc.

[0303] The wireless communication module 140 can provide a wireless communication solution based on wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), ZigBee, and the like, which is applied to the first electronic device 100. The wireless communication module 140 can be one or more devices that integrate at least one communication processing module. The wireless communication module 140 receives electromagnetic waves via an antenna, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 140 can also receive signals to be transmitted from the processor 110, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via an antenna. It should be noted that Figure 4 The number of antennas of the wireless communication module 140, the UWB module 150, and the millimeter wave radar module 160 is only exemplary. It can be understood that the communication module 140, the UWB module 150, and the millimeter wave radar module 160 can include more or fewer antennas, and the embodiments of the present application do not limit this.

[0304] The UWB module 150 can provide a wireless communication solution based on UWB technology, which is applied to the first electronic device 100. Exemplarily, the UWB module 150 is used to implement the functions of the UWB base station described above. In the embodiments of the present application, the UWB base station can locate the UWB tag. Specifically, the time length of the UWB signal flying in the air can be calculated by detecting the UWB signal and combining certain positioning algorithms, and the distance between the UWB tag and the UWB base station is obtained by multiplying the time length by the speed (such as the speed of light) of the UWB signal in the air. In the embodiments of the present application, the UWB base station can also determine the direction of the UWB tag relative to the UWB base station (i.e., the signal direction of the UWB tag) according to the phase difference of the UWB signal sent by the UWB tag to different antennas of the UWB base station. The signal direction includes horizontal direction and vertical direction.

[0305] Exemplarily, Figure 3 A structural schematic diagram of a hub device 200 is shown.

[0306] As Figure 3As shown, the hub device 200 can include a processor 210, a memory 220, a power management module 230, a power supply 231, a wireless communication module 240, and the like.

[0307] It can be understood that, Figure 3 The illustrated structure does not constitute a specific limitation on the hub device 200. In other embodiments of the present application, the hub device 200 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0308] The processor 210 can include one or more processing units, and different processing units can be independent devices or integrated in one or more processors. For example, the processor 210 is a central processing unit (CPU), which can also be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0309] The memory 220 can be used to store computer executable program codes, which include instructions. For example, the memory 220 can also store data processed by the processor 210. In addition, the memory 220 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like. The processor 210 executes various functional applications and data processing of the hub device 200 by running instructions stored in the memory 220 and / or instructions stored in the memory disposed in the processor.

[0310] The power management module 230 is used to receive the input of the power supply 231. The power supply 231 can be a battery or a mains power supply. The power management module 230 receives power supply from the battery and / or the mains power supply to power various components of the hub device 200, such as the processor 210, the memory 220, the wireless communication module 240, and the like.

[0311] The wireless communication module 240 can provide a wireless communication solution including wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), ZigBee, and the like, which are applied to the hub device 200. The wireless communication module 240 can be one or more devices that integrate at least one communication processing module. The wireless communication module 240 receives electromagnetic waves via an antenna, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 210. The wireless communication module 240 can also receive signals to be transmitted from the processor 210, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via the antenna.

[0312] Exemplarily, Figure 4 A structural schematic diagram of a second electronic device 300 is shown.

[0313] As Figure 4 shown, the second electronic device 300 can include a processor 310, a memory 320, a universal serial bus (USB) interface 330, a power module 340, a UWB module 350, a wireless communication module 360, and the like. Optionally, the second electronic device 300 can also include an audio module 370, a loudspeaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a display screen 380, a sensor module 390, and the like.

[0314] It can be understood that, Figure 4 The illustrated structure does not constitute a specific limitation on the second electronic device 300. In some other embodiments of the present application, the second electronic device 300 can include more or fewer components than those shown, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. In addition, Figure 4 The illustrated interface connection relationship between the modules is only illustrative and does not constitute a structural limitation on the second electronic device 300. In some other embodiments of the present application, the second electronic device 300 can also use a different interface connection mode, or a combination of multiple interface connection modes. Figure 4 ​

[0315] The processor 310 can include one or more processing units, for example: the processor 310 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a video codec, a digital signal processor (DSP), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0316] The memory 320 can be used to store computer executable program codes, including instructions. For example, the memory 320 can also store data processed by the processor 310. In addition, the memory 320 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 310 executes various functional applications and data processing of the second electronic device 300 by running the instructions stored in the memory 320 and / or the instructions stored in the memory arranged in the processor.

[0317] The USB interface 330 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 330 can be used to connect a charger to charge the second electronic device 300, and can also be used to transmit data between the second electronic device 300 and a peripheral device.

[0318] The power module 340 is used to supply power to various components of the second electronic device 300, such as the processor 310, the memory 320, etc.

[0319] The UWB module 350 can provide a solution for wireless communication based on UWB technology applied to the second electronic device 300. Exemplarily, the UWB module 350 is used to implement the functions of the UWB tag described above.

[0320] The wireless communication module 360 can provide a wireless communication solution including wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), ZigBee, and the like, which are applied to the second electronic device 300. The wireless communication module 360 can be one or more devices that integrate at least one communication processing module. The wireless communication module 360 receives an electromagnetic wave via an antenna, frequency-modulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 310. The wireless communication module 360 can also receive a signal to be transmitted from the processor 310, frequency-modulate it, amplify it, and radiate it as an electromagnetic wave via an antenna. The wireless communication module 360 can be integrated with the UWB module 350 or separately disposed, and the present application is not limited in this regard.

[0321] The second electronic device 300 can implement an audio function through an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a headphone interface 370D, and an application processor, etc. For example, audio playback, recording, etc.

[0322] The audio module 370 is used to convert digital audio information into an analog audio signal output, and is also used to convert an analog audio input into a digital audio signal. The audio module 370 can also be used to encode and decode an audio signal. In some embodiments, the audio module 370 can be disposed in the processor 310, or part of the functions of the audio module 370 can be disposed in the processor 310.

[0323] The speaker 370A, also known as a "loudspeaker", is used to convert an audio electrical signal into a sound signal. The second electronic device 300 can listen to audio through the speaker 370A.

[0324] The receiver 370B, also known as an "earpiece", is used to convert an audio electrical signal into a sound signal.

[0325] The microphone 370C, also known as a "microphone", "sound transducer", is used to convert a sound signal into an electrical signal. A user can speak into the microphone 370C by placing his mouth close to the microphone 370C to input a sound signal into the microphone 370C.

[0326] The earphone interface 370D is configured to connect a wired earphone. The earphone interface 370D can be a USB interface 330, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0327] The display screen 380 is configured to display images, videos, and the like. The display screen 380 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like.

[0328] Optionally, the sensor module 390 includes an inertial measurement unit (IMU) module, and the like. The IMU module can include a gyroscope, an accelerometer, and the like. The gyroscope and the accelerometer can be configured to determine a motion posture of the second electronic device 300. In some embodiments, the angular velocity of the second electronic device 300 around three axes can be determined by the gyroscope. The accelerometer can be configured to detect the magnitude of the acceleration of the second electronic device 300 in each direction (generally three axes). When the second electronic device 300 is stationary, the magnitude and direction of gravity can be detected. In the embodiments of the present application, the device posture of the second electronic device 300 can be obtained according to the angular velocity and the acceleration measured by the IMU module. Optionally, some second electronic devices can include the IMU module, and some second electronic devices do not include the IMU module.

[0329] Optionally, the second electronic device 300 further includes a filter (such as a Kalman filter). Exemplarily, the output of the IMU module and the output of the UWB module 350 can be superimposed, and the superimposed signal can be input to the Kalman filter for filtering, thereby reducing errors.

[0330] Exemplarily,Figure 5A The structure of the UWB module in the first electronic device provided by the embodiments of the present application is shown. As shown in Figure 5A The UWB module 150 includes a transmitter 1501 and a receiver 1502. The transmitter 1501 and the receiver 1502 can operate independently. The transmitter 1501 includes a data signal generator, a pulse generator, a modulator, a digital-to-analog converter, a power amplifier, a transmitting antenna, and the like. The data signal generator is configured to generate a data signal and to send timing start indication information to the receiver 1502 when starting to generate the data signal. The pulse generator is configured to generate a periodic pulse signal. The digital-to-analog converter is configured to convert a digital signal into an analog signal. The data signal to be transmitted is modulated onto the pulse signal generated by the pulse generator through the modulator, amplified by the power amplifier, and then transmitted by the transmitting antenna. The receiver 1502 includes a receiving antenna, a mixer, a filter, a sampling module, a first processing module, and the like. Any one of the receiving antennas receives a UWB signal (for example, in the form of a pulse sequence), amplifies the received UWB signal through the mixer and the filter, and then converts the UWB signal into a baseband digital signal through the sampling module. The first processing module is configured to process the baseband digital signal to detect the UWB signal. For example, the first processing module calculates the time of flight (ToF) of the UWB signal according to the timing start indication information and the time when the pulse sequence is received, and calculates the distance between the first electronic device 100 and the second electronic device 300 including the UWB module 350 according to the ToF and the speed (for example, the speed of light) of the UWB signal in the air. For another example, the first processing module calculates the direction of the second electronic device 300 including the UWB module 350 according to the phase difference of the pulse sequences received by the multiple receiving antennas. It should be noted that Figure 5A The pulse sequence received by each receiving antenna in the receiver 1502 passes through a set of power amplifiers, mixers, filters, and sampling modules, which represents the processing flow of the pulse sequence. Alternatively, the receiver 1502 can only include a set of power amplifiers, mixers, filters, and sampling modules. In an embodiment, one antenna can realize the functions of one transmitting antenna in the transmitter 1501 and one receiving antenna in the receiver 1502, that is, the transmitting antenna and the receiving antenna are integrated into the same antenna.

[0331] Exemplarily, Figure 5B The structure of the millimeter wave radar module in the first electronic device provided by the embodiments of the present application is shown. As shown in Figure 5BAs shown, the millimeter wave radar module 160 includes a transmitting antenna, a relay switch, a receiving antenna, a waveform generator, a mixer, a filter, and a second processing module, etc. The waveform generator is configured to generate a transmitting signal, such as a linear frequency-modulation continuous wave (LFMCW). The transmitting signal is split by a power divider, and a part of the transmitting signal is amplified by a power amplifier and then transmitted by the relay switch through a selected transmitting antenna. Another part of the transmitting signal is used as a local oscillator, and mixed with a millimeter wave signal received by the receiving antenna through a mixer. The output signal of the mixer is a difference frequency signal, which is filtered and amplified by a filter and then converted into a digital difference frequency signal by a sampling module. The second processing module processes the digital difference frequency signal to detect a target and obtain distance and signal direction information of the target. Figure 5B In some embodiments, n and m are positive integers greater than or equal to 1.

[0332] Exemplarily, Figure 6 The structure of the UWB module in the second electronic device is shown. As shown, Figure 6 The UWB module 350 includes a transmitter 3501 and a receiver 3502. The transmitter 3501 and the receiver 3502 can operate independently. The transmitter 3501 includes a data signal generator, a pulse generator, a modulator, a digital-to-analog converter, a power amplifier, and a transmitting antenna, etc. The data signal generator is configured to generate a data signal. The pulse generator is configured to generate a periodic pulse signal. The digital-to-analog converter is configured to convert a digital signal into an analog signal. The data signal to be transmitted is modulated onto the pulse signal generated by the pulse generator through the modulator, and then amplified by the power amplifier and transmitted by the transmitting antenna. The receiver 3502 includes a receiving antenna, a mixer, a filter, a sampling module, and a processing module, etc. The receiving antenna receives a UWB signal (such as in the form of a pulse sequence), and the received UWB signal is mixed by the mixer, filtered and amplified by the filter, and then converted into a baseband digital signal by the sampling module. The processing module is configured to process the baseband digital signal to detect the UWB signal. In an embodiment, the transmitting antenna in the transmitter 3501 and the receiving antenna in the receiver 3502 can be integrated into the same antenna.

[0333] Exemplarily, Figure 7 The distribution of several antennas in the UWB module in the first electronic device provided by the embodiments of the present application is shown. In the embodiments, Figure 7Fig. 1(a) exemplarily shows two two-antenna structures. One is a transverse type (e.g., horizontal type) antenna structure, and the other is a longitudinal type (e.g., vertical type) antenna structure. Preferably, the distance between antenna 0 and antenna 1 is λ / 2, where λ is the wavelength of the UWB signal. The transverse type antenna structure can be used to measure the transverse direction (e.g., horizontal direction) of the UWB signal, and the longitudinal type antenna structure can be used to measure the longitudinal direction (e.g., vertical direction) of the UWB signal. In one implementation, the transverse direction of the UWB signal can be measured by Figure 7 Fig. 1(a) exemplarily shows two two-antenna structures. One is a transverse type (e.g., horizontal type) antenna structure, and the other is a longitudinal type (e.g., vertical type) antenna structure. Preferably, the distance between antenna 0 and antenna 1 is λ / 2, where λ is the wavelength of the UWB signal. The transverse type antenna structure can be used to measure the transverse direction (e.g., horizontal direction) of the UWB signal, and the longitudinal type antenna structure can be used to measure the longitudinal direction (e.g., vertical direction) of the UWB signal. In one implementation, the transverse direction of the UWB signal can be measured by

[0334] Figure 7 Fig. 1(b) and Figure 7 Fig. 1(c) exemplarily show three-antenna structures. As shown in Figure 7 Fig. 1(b) and Figure 7 Fig. 1(c), the three antennas present an L-shaped (or right-angled triangular) structure relationship. As shown in Figure 7 Fig. 1(b), antenna 0 and antenna 1 are aligned in the transverse direction (e.g., horizontal direction), and antenna 0 and antenna 2 are aligned in the longitudinal direction (e.g., vertical direction), i.e., the plane on which antenna 0, antenna 1 and antenna 2 are located is a longitudinal plane (e.g., longitudinal plane), and the three antennas present an L-shaped distribution relationship in the longitudinal plane. As shown in Figure 7 Fig. 1(c), the plane on which antenna 0, antenna 1 and antenna 2 are located is a transverse plane (e.g., horizontal plane), and the line connecting antenna 0 and antenna 1 (assuming that there is a line connecting antenna 0 and antenna 1) is perpendicular to the line connecting antenna 0 and antenna 2 (assuming that there is a line connecting antenna 0 and antenna 2). That is, antenna 0, antenna 1 and antenna 2 present an L-shaped distribution relationship in the transverse plane. Exemplarily, when antenna 0, antenna 1 and antenna 2 present an L-shaped distribution, the distance between antenna 0 and antenna 1, and the distance between antenna 0 and antenna 2 can be less than or equal to λ / 2, where λ is the wavelength of the UWB signal. The distance between antenna 0 and antenna 1, and the distance between antenna 0 and antenna 2 can be the same or different.

[0335] Figure 7 Fig. 1(d) exemplarily shows some other three-antenna structures. As shown in Figure 7As shown in (d), the three antennas present a triangular (e.g., equilateral triangle, isosceles triangle) structural relationship. For example, the plane on which the antenna 0, the antenna 1 and the antenna 2 are located is a longitudinal plane (e.g., vertical plane), and the antenna 0, the antenna 1 and the antenna 2 present a triangular distribution on the longitudinal plane. For another example, the antenna 0, the antenna 1 and the antenna 2 present a triangular distribution on a transverse plane (e.g., horizontal plane). Exemplarily, when the antenna 0, the antenna 1 and the antenna 2 present a triangular distribution, the distance between any two of the antenna 0, the antenna 1 and the antenna 2 can be less than or equal to λ / 2; where λ is the wavelength of the UWB signal. Moreover, the distance between any two of the antenna 0, the antenna 1 and the antenna 2 can be the same or different. For example, the distance between the antenna 0 and the antenna 1 is λ / 2; the distance between the antenna 0 and the antenna 2 is √3λ / 2.

[0336] It can be understood that more than three antennas are also within the scope of the present application. For example, as shown in (e), the antenna 0, the antenna 1, the antenna 2 and the antenna 3 present a rectangular distribution. Among them, any three of the four antennas present the L-shaped distribution or the triangular distribution described above. Figure 7

[0337] Exemplarily, the first electronic device 100 obtains the transverse direction of the UWB signal according to the phase difference of the UWB signal from the second electronic device 300 arriving at the two transversely distributed antennas of the UWB module 150; and obtains the longitudinal direction of the UWB signal according to the phase difference of the UWB signal from the second electronic device 300 arriving at the two longitudinally distributed antennas of the UWB module 150. Further, the first electronic device 100 obtains the direction of the UWB signal according to the transverse direction and the longitudinal direction.

[0338] In some other examples, the UWB module 150 of the first electronic device 100 can only include one antenna. At this time, three or more first electronic devices 100 are needed to be distributed in an L shape or a triangle shape to jointly obtain the direction of the UWB signal. The specific principle is similar to the above, which will not be described here.

[0339] The embodiments of the present application do not limit the number and distribution of the antennas in the UWB module of the first electronic device 100, as long as the direction of the UWB signal can be obtained.

[0340] Exemplarily, Figure 8 ​This illustration shows several antenna configurations for a millimeter-wave radar module in a first electronic device according to an embodiment of this application. Exemplarily, the transmitting antennas include transmitting antenna 0, transmitting antenna 1, and transmitting antenna 2. The receiving antennas include receiving antenna 0, receiving antenna 1, receiving antenna 2, and receiving antenna 3. The configurations of transmitting antennas 0, 1, and 2, as well as receiving antennas 0, 1, 2, and 3, can be as follows: Figure 8 As shown in (a) or (b). The transmitting antenna transmits electromagnetic signals operating in the millimeter-wave band (such as LFMCW), and the receiving antenna receives the signals reflected by a reflective object (object or human body) from the electromagnetic signals operating in the millimeter-wave band. The millimeter-wave radar module 160 obtains a difference frequency signal based on the transmitted and received signals, and determines the position of the object or human body based on the difference frequency signal.

[0341] like Figure 8 As shown, three transmitting antennas and four receiving antennas are located in the same longitudinal plane (e.g., a vertical plane), with the three transmitting antennas arranged in a triangular pattern on the longitudinal plane. In one example, as... Figure 8 As shown in (a), transmitting antenna 0 and transmitting antenna 2 are located in the same transverse plane (e.g., a horizontal plane), and the four receiving antennas are located on the same transverse line (e.g., a horizontal line). Exemplarily, the distance between any two receiving antennas is equal (e.g., all are λ). L / 2); The distances between transmitting antenna 0 and transmitting antenna 2 are equal (e.g., both are 2λ). L The longitudinal distances between transmitting antenna 1 and transmitting antenna 0, and between transmitting antenna 1 and transmitting antenna 2, are all equal (e.g., both are λ). L / 2). λ L This is the wavelength of the highest frequency of a linearly frequency-modulated continuous signal. In another example, such as... Figure 8 As shown in (b), transmitting antenna 0 and transmitting antenna 2 are located on the same longitudinal line (e.g., a vertical line); the four receiving antennas are located on the same longitudinal line (e.g., a vertical line). The distance between any two receiving antennas is equal (e.g., λ). L / 2); the distances between transmitting antenna 0 and transmitting antenna 2 are equal (e.g., both are 2λ). L The transverse distances between transmitting antenna 1 and transmitting antenna 0, and between transmitting antenna 1 and transmitting antenna 2, are all equal (e.g., both are λ). L / 2). It is understood that the number and distribution of transmitting and / or receiving antennas may be other forms. The embodiments of this application are not limited in this respect.

[0342] The multiple transmitting antennas and multiple receiving antennas are used to accurately measure the direction of the reflected signal, i.e., the direction of arrival of the reflected signal, including the lateral direction of arrival (e.g., horizontal direction of arrival) and the longitudinal direction of arrival (e.g., vertical direction of arrival), and to increase the receiving aperture of the millimeter wave radar as much as possible. The millimeter wave radar module 160 can calculate the lateral direction of arrival of the target according to the phase difference of the multiple receiving antennas in the lateral direction (e.g., horizontal direction) of the reflected signal, and calculate the longitudinal direction of arrival of the target according to the phase difference of the multiple receiving antennas in the longitudinal direction (e.g., vertical direction) of the reflected signal.

[0343] Optionally, the number of transmitting antennas can be more than or less than 3. Optionally, the number of receiving antennas can be more than or less than 4. The present application does not limit this. In an embodiment, the number of transmitting antennas is at least one, and the number of receiving antennas is at least three.

[0344] In an embodiment, the number of transmitting antennas is one, and the number of receiving antennas is three. Among them, the three receiving antennas, receiving antenna 0, receiving antenna 1 and receiving antenna 2, are distributed in a triangular shape. For the convenience of introduction, it is assumed that the connecting line (actually no connecting line) between receiving antenna 0 and receiving antenna 1 is located in the lateral direction, and the connecting line (actually no connecting line) between receiving antenna 0 and receiving antenna 2 is located in the longitudinal direction. In this way, the transmitting signal of the transmitting antenna is reflected by the reflector (object or human body), and the three receiving antennas receive the reflected signal respectively. The millimeter wave radar module 160 can obtain the lateral direction of arrival (e.g., horizontal direction of arrival) of the reflected signal according to the phase difference between the reflected signals received by receiving antenna 0 and receiving antenna 1 respectively, and obtain the longitudinal direction of arrival (e.g., vertical direction of arrival) of the reflected signal according to the phase difference between the reflected signals received by receiving antenna 0 and receiving antenna 2 respectively. Further, the direction of arrival of the reflected signal can be determined according to the lateral direction of arrival and the longitudinal direction of arrival.

[0345] In another embodiment, the number of transmitting antennas is at least two, and the number of receiving antennas is at least two. Taking two transmitting antennas, transmitting antenna 0 and transmitting antenna 1, and two receiving antennas, receiving antenna 0 and receiving antenna 1, as an example. Assume that the line (in fact, no line) between transmitting antenna 0 and transmitting antenna 1 is in the horizontal direction, and the line (in fact, no line) between receiving antenna 0 and receiving antenna 1 is in the vertical direction. After the transmitting signals of the two transmitting antennas are reflected by the reflector (object or human body) respectively, at least one receiving antenna receives the reflected signals. The millimeter wave radar module 160 can calculate the horizontal direction of arrival (for example, horizontal direction of arrival) of the reflected signals (at this time, also referred to as reflected signals) according to the phase difference of the signals respectively transmitted by the two transmitting antennas 0 and 1 to the same receiving antenna. After the transmitting signals of the transmitting antennas are reflected by the reflector (object or human body), the two receiving antennas receive the reflected signals respectively; according to the phase difference between the reflected signals respectively received by the two receiving antennas 0 and 1, the vertical direction of arrival (for example, vertical direction of arrival) of the reflected signals is obtained. Further, the direction of arrival of the reflected signals can be determined according to the horizontal direction of arrival and the vertical direction of arrival.

[0346] In another embodiment, the number of transmitting antennas is at least two, and the number of receiving antennas is at least two. Taking two transmitting antennas, transmitting antenna 0 and transmitting antenna 1, and two receiving antennas, receiving antenna 0 and receiving antenna 1, as an example. Assume that the line (in fact, no line) between transmitting antenna 0 and transmitting antenna 1 is in the vertical direction, and the line (in fact, no line) between receiving antenna 0 and receiving antenna 1 is in the horizontal direction. After the transmitting signals of the two transmitting antennas are reflected by the reflector (object or human body) respectively, at least one receiving antenna receives the reflected signals. The millimeter wave radar module 160 can calculate the vertical direction of arrival (for example, horizontal direction of arrival) of the reflected signals (at this time, also referred to as reflected signals) according to the phase difference of the signals respectively transmitted by the two transmitting antennas 0 and 1 to the same receiving antenna; according to the phase difference between the reflected signals respectively received by the two receiving antennas 0 and 1, the horizontal direction of arrival (for example, horizontal direction of arrival) of the reflected signals is obtained. Further, the direction of arrival of the reflected signals can be determined according to the horizontal direction of arrival and the vertical direction of arrival.

[0347] In another implementation, the number of transmitting antennas is at least three, and the number of receiving antennas is at least one. Taking three transmitting antennas—transmitting antenna 0, transmitting antenna 1, and transmitting antenna 2—and one receiving antenna—receiving antenna 0—as an example, transmitting antennas 0, 1, and 2 are arranged in a triangle. Assume that the line connecting transmitting antenna 0 and 1 (which is not actually a line) is in the transverse direction (e.g., horizontal), and the line connecting transmitting antenna 0 and 2 (which is not actually a line) is in the longitudinal direction. The transmitted signals from transmitting antennas 0, 1, and 2 are reflected by reflective objects (objects or human bodies), and the reflected signals are received by receiving antenna 0. The millimeter-wave radar module 160 can calculate the lateral direction of the transmitted signal (also known as the reflected signal) based on the phase difference between the signals transmitted by transmitting antenna 0 and transmitting antenna 1 reaching the same receiving antenna; and calculate the longitudinal direction of the transmitted signal (also known as the reflected signal) based on the phase difference between the signals transmitted by transmitting antenna 0 and transmitting antenna 2 reaching the same receiving antenna.

[0348] III. Introduction to Positioning Principles

[0349] For ease of explanation, the following section will take the first electronic device 100, which includes a UWB module 150 and a millimeter-wave radar module 160, as an example to introduce the specific positioning principle in detail.

[0350] It should be noted that positioning refers to obtaining location. In this embodiment, location is represented by coordinates in a coordinate system. For example, the location of the first electronic device is represented by the coordinates of the first electronic device, the location of the second electronic device is represented by the coordinates of the second electronic device, and the user's location is represented by the user's coordinates. It is understood that in other embodiments, location may be represented in other ways. This embodiment does not limit this.

[0351] (I) Establishment of the first coordinate system (first electronic device coordinate system)

[0352] To achieve accurate positioning, the UWB module 150 needs to establish a primary coordinate system, using coordinates within this system for precise location. The following section will combine... Figure 9 This section details the process of establishing the first coordinate system. Figure 9 Taking a UWB module comprising three antennas as an example in (a) and (b), the distances between antenna 0 and antenna 1, and between antenna 0 and antenna 2, are preset distances. Let a point on antenna 0 (such as the center point, one end point, etc.) be the origin O. e The line connecting antenna 0 and antenna 1 is taken as X. e The axis is defined, and the direction from antenna 1 to antenna 0 is X.e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. Figure 9 positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. Figure 9 positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. Figure 9 positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1.

[0353] In some examples, for the convenience of calculation and to reduce calculation error, the Z-axis can be arranged on the plumb surface, and the positive direction of the Z-axis is opposite to the direction of gravity. Optionally, the outer surface of the first electronic device 100 can be marked with prompt information for prompting the correct installation or placement manner, so that the Z-axis of the first coordinate system is arranged on the plumb surface, and the positive direction of the Z-axis is opposite to the direction of gravity. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. Figure 9 Figure 9 positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e positive direction of the Z-axis. In the plane where the antenna 0, the antenna 1 and the antenna 2 are located, the straight line perpendicular to the X-axis is the Z-axis, and the positive direction of the Z-axis is the direction from the antenna 0 to the antenna 1. e ​The axis is located in the plumb plane, and Z e The positive direction of the axis is opposite to the direction of gravity. For example, when installing the first electronic device, the user can use a plumb instrument to determine that the arrow on the outer surface of the first electronic device is parallel to the plumb line determined by the plumb instrument, and the arrow direction is upward, so that the Z e The axis is located in the plumb plane, and Z e The positive direction of the axis is opposite to the direction of gravity.

[0354] Alternatively, the first electronic device 100 can include only one UWB module, and the UWB module 150 can include only one antenna. In this way, three first electronic devices 100 need to cooperate with each other to establish a first coordinate system. For the detailed process of establishing the first coordinate system in this case, please refer to the Chinese invention patent application No. 202110872916.6. Here, it is not repeated. It should be pointed out that the entire content of the Chinese invention patent application No. 202110872916.6 is incorporated into the present application, and is within the scope of the present application.

[0355] Exemplarily, also according to the X e The direction of the Y e The direction of the Y e The positive direction of the axis. Among them, the left-handed rectangular coordinate system can be simply referred to as the left-handed system, which is one of the methods for defining a rectangular coordinate system in space. Among them, the positive direction of the X e The positive direction of the Y e The positive direction of the Y e The positive direction of the axis is as follows: put the left hand at the position of the origin, make the thumb, index finger and middle finger form a right angle with each other, the thumb and index finger are in the same plane, and point the thumb to the X e The positive direction of the Y e The positive direction of the Y e The positive direction of the Y

[0356] For ease of description, in the embodiments of the present application, the Y e The positive direction of the Y b The positive direction of the Y t The positive direction of the Y e The positive direction of the Y b The positive direction of the Y t The positive direction of the Y e The positive direction of the Y b The positive direction of the Y t The positive direction of the Y e The positive direction of the Y b The positive direction of the Y tThe positive orientation of the axes is also within the scope of this application. Furthermore, as long as the above rules or methods are met, the names of any two or more axes in any of the first, second, and third coordinate systems can be switched. Of course, other rules can also be used for the positive orientation of the three axes in the first coordinate system, which will not be elaborated here.

[0357] Optionally, the first coordinate system can be automatically established after receiving specific input, or it can be established in advance.

[0358] For example, after the first electronic device 100 is installed according to the prompts, it automatically establishes a first coordinate system when it receives a specific input. This specific input can be user input or non-user input (e.g., receiving a command message from another device such as a remote control).

[0359] For example, after the first electronic device 100 is installed according to the prompts, when the first electronic device 100 receives specific input, it automatically retrieves relevant information from its local storage or a server, thereby retrieving a pre-established first coordinate system. Unless otherwise specified, the server in this application can be a central device in a home or a cloud server.

[0360] It should be noted that the above uses a point on antenna 0 as the origin of the first coordinate system. This is only an example; a point on other antennas (such as antenna 1) can also be the origin of the first coordinate system.

[0361] (II) Establishment of the Second Coordinate System

[0362] For example, Figure 10 The process of establishing the second coordinate system provided in an embodiment of this application is illustrated. For example... Figure 10 As shown in (a), the edge profile of the second electronic device includes four sides: two vertical sides and two horizontal sides. b The center of gravity or center of the second electronic device, containing O b The point is on the axis parallel to the horizontal side of the second electronic device. b Axis, X b The positive direction of the axis points to the right side of the second electronic device; including O b The point is parallel to the vertical side of the second electronic device, and its axis is Y. b axis, Y b The positive direction of the axis points to the top of the second electronic device, and the direction of the second electronic device is Y. b Positive direction of the axis; Z b The axis is perpendicular to X. b axis and Y bThe plane where the axis is located is determined as the positive direction of the Z axis according to the rule of the right-handed orthogonal coordinate system. b The plane where the axis is located is determined as the positive direction of the Z axis according to the rule of the right-handed orthogonal coordinate system. b The plane where the axis is located is determined as the positive direction of the Z axis according to the rule of the right-handed orthogonal coordinate system. b The plane where the axis is located is determined as the positive direction of the Z axis according to the rule of the right-handed orthogonal coordinate system. Figure 10 (b) is Figure 10 The perspective view of the second electronic device in (a).

[0363] It should be noted that, Figure 10 The second coordinate system is only introduced schematically. The second coordinate system can also be defined according to other rules. For example, the coordinate origin O b may also be any point on the second electronic device or any point outside the second electronic device. In addition, the directions of the three axes of the second coordinate system are not limited to Figure 10 the positive direction of the X axis shown in (a) or (b) of the first coordinate system. b the positive direction of the Y axis. b the positive direction of the Z axis. b the positive direction of the Z axis.

[0364] Alternatively, the second coordinate system can be established in advance. For example, when the second electronic device is manufactured, the second coordinate system has been established, and the related information of the second coordinate system is saved locally or on a server; when the second electronic device is started, or when the first electronic device receives a specific input, the second electronic device calls the related information of the second coordinate system from the local or the server.

[0365] (III) Establishment of the third coordinate system

[0366] Exemplarily, Figure 11 the establishment process of the third coordinate system provided by the embodiments of the present application is shown. As shown in Figure 11 , the edge profile of the second electronic device includes four edges: the first edge A0A1, the second edge A1A2, the third edge A2A3, and the fourth edge A3A0. Among them, the first edge A0A1 and the third edge A2A3 are vertical edges, and the second edge A1A2 and the fourth edge A3A0 are horizontal edges. Alternatively, the intersection point of the leftmost edge of the display area of the second electronic device and the lowermost edge of the display area of the second electronic device (i.e., the lower left corner of the display area of the second electronic device) is taken as the coordinate origin O t . The axis containing the O t point and parallel to A3A0 is taken as the X t axis, and the positive direction of the X t axis is the direction from the A0 point to the A3 point; the axis containing the O t point and parallel to A0A1 is taken as the Y t axis, and the positive direction of the Y t axis is the direction from the A0 point to the A1 point; the Zt The axis is perpendicular to X. t axis and Y t The plane containing the axis is used to determine the Z-axis according to the rules of the right-hand rectangular coordinate system. t The positive direction of the axis.

[0367] It should be noted that, Figure 11 The third coordinate system is only illustrated schematically. The third coordinate system can also be defined according to other rules. Optionally, O t It can be the center of the display area of ​​the second electronic device, or any point within the display area of ​​the second electronic device. Furthermore, the positive axes of the third coordinate system are not limited to... Figure 11 X shown t axis, Y t Axis and Z t The positive direction indicated by the axis.

[0368] It should be noted that when the edge outline of the display area of ​​the second electronic device is the edge outline of the second electronic device, point A0 of the second electronic device and point O t Points coincide; when the edge contour of the display area of ​​the second electronic device is not the edge contour of the second electronic device, for example, when there is a border outside the display area of ​​the second electronic device, point A0 of the second electronic device does not coincide with point O. t Points overlap.

[0369] Optionally, the third coordinate system can be pre-established. For example, the third coordinate system is established when the second electronic device leaves the factory, and the relevant information of the third coordinate system is stored locally or on a server; when the second electronic device starts up, or when the second electronic device receives a trigger, the second electronic device calls the third coordinate system from the local machine or server.

[0370] (iv) Coordinate calculation in the first coordinate system

[0371] The following is combined Figure 12 This will specifically explain the calculation principle for positioning the first electronic device 100 relative to the second electronic device 300 in the first coordinate system. Figure 12 In this context, the second electronic device 300 includes a UWB module, and the second electronic device 300 is exemplified by a mobile device (e.g., a smartphone or a remote control). Figure 12 As shown in (a), the first electronic device 100 and the second electronic device 300 are located in the same room or area. The first electronic device 100 has established a first coordinate system, and the second electronic device 300 has established a second coordinate system. Figure 12As shown in (b), the first electronic device 100 and the second electronic device 300 can perform UWB communication, according to which the distance between the second electronic device 300 and the first electronic device 100, the direction of the second electronic device 300 relative to the first electronic device 100 can be determined, so that the coordinates of the second electronic device 300 in the first coordinate system can be determined.

[0372] 1. Measure the distance L between the first electronic device and the second electronic device.

[0373] Exemplarily, as shown in (c), the distance L between the first electronic device and the second electronic device can be obtained in the following manner: Figure 12

[0374] The first electronic device 100 can measure the distance L between the first electronic device 100 and the second electronic device 300 by using a two-way ranging method. The two-way ranging method includes single-sided two-way ranging (SS-TWR) and double-sided two-way ranging (DS-TWR). Here, the DS-TWR is taken as an example to briefly describe the ranging method.

[0375] In the DS-TWR method, the time stamps of the round trip between the two first electronic devices 100 and the second electronic device 300 are recorded, and finally the time of flight is obtained. Although the DS-TWR method increases the response time, it can reduce the ranging error. According to the number of transmitted messages, the double-sided two-way ranging can be divided into two methods: 4-message method and 3-message method.

[0376] Taking the 3-message method as an example, the second electronic device 300 sends a ranging request message (i.e., the first message) and records the sending time T s1 . After the first electronic device 100 receives the request message, it records the receiving time T r1 . The time difference t between T r1 and T s1 is the transmission time of the message between the two devices. The first electronic device 100 takes time T re1 to process the request message. Then, the first electronic device 100 sends a response message (i.e., the second message) and records the sending time T s2 , and the second electronic device 300 receives the response message and records the receiving time T r2 . The time difference t between T r2 and T s2 . The time difference T ro1 from the occurrence of the first message to the reception of the second message by the second electronic device 300 isThe second electronic device 300 processes the response message, and the time consumption is T re2 The second electronic device 300 sends the last message (i.e., the third message), and records the sending time T s3 The first electronic device 100 receives the third message, and records the receiving time T r3 Wherein, T r3 The time difference between T s3 And the time difference between the first electronic device 100 from the beginning of sending the second message to receiving the third message is T ro2 Therefore, the transmission time t of the message between the two devices can be calculated by the following formula (1), and the distance L between the first electronic device 100 and the second electronic device 300 can be calculated according to formula (2).

[0377]

[0378] L = c x t formula (2)

[0379] Wherein, c is the transmission rate of UWB signal in medium. c is generally selected as the speed of light.

[0380] Alternatively, the ranging request message can also be sent by the first electronic device 100; correspondingly, the response message can also be sent by the second electronic device 300; correspondingly, the last message can also be sent by the first electronic device 100. The first electronic device 100 or the second electronic device 300 can calculate L according to formula (1) and formula (2).

[0381] It should be noted that the distance L between the first electronic device 100 and the second electronic device 300 can also be calculated by other ways, which are not limited to the above listed ways.

[0382] 2, Determine the signal direction of the second electronic device.

[0383] The direction of the second electronic device 200 can be represented by the first electronic device 100 measuring the direction of the signal emitted by the second electronic device 200. In this application, the direction of the signal emitted by the second electronic device 200 can be represented by two angles α and β. For example, as shown in Figure 12 (d) of the drawings, α is the angle between the component of the UWB signal emitted by the second electronic device in the X e O e Y e plane of the first coordinate system and the negative axis of the X e axis, and generally α ∈ [0, π]. α can also be understood as follows: assuming that the UWB module of the second electronic device forms a vector to the UWB module of the first electronic device, the vector is in the X e O e Ye Components on the plane and X e The angle between the negative axis and the first coordinate system. β is the angle between the UWB signal emitted by the second electronic device and the Z-axis of the first coordinate system. e The angle between the positive and negative axes, usually β ∈ [0, π]. β can also be understood as: assuming the UWB module of the second electronic device forms a vector with respect to the UWB module of the first electronic device, this vector is perpendicular to Z. e The angle between the positive and negative axes.

[0384] The following example uses three antennas and combines them with two antenna distribution types in the UWB module of the first electronic device—L-shaped and triangular—to illustrate the solution process for α and β.

[0385] In one example, in Figure 12 In (e), the UWB module of the first electronic device adopts an L-shaped three-antenna structure. The first electronic device can determine the included angle α based on the UWB signal received by antenna 1; and can determine the included angle β based on the UWB signal received by antenna 2. Generally speaking, L1 and L2 are much smaller than the distance L between the second electronic device and the first electronic device, so the UWB signals can be considered parallel when they reach the UWB module of the first electronic device; similarly, when the UWB signals reach the UWB module of the first electronic device, at X... e O e Y e Components on a plane can also be considered as parallel. For example... Figure 12 As shown in (e), the UWB signal is represented by two parallel solid lines. The UWB signal in X e O e Y e The components in the plane are represented by two parallel dashed lines; the distance between antenna 0 and antenna 1 is L1, and the distance between antenna 0 and antenna 2 is L2; ​​passing through point O e As O e M1 is perpendicular to the line containing M1N1 and passes through point O. e As O e M2 is perpendicular to the line containing M2N2; where N1 is the UWB signal at X. e O e Y e The line containing the component in the plane and the X e The intersection of the axes, N2 is the intersection of the line containing the UWB signal and the Z-axis. e The intersection of the axes. The UWB signal and the Z-axis. e The angle between the positive and negative axes is β. The UWB signal is in the X... e O e Y e Components on the plane and X eThe angle between the positive and negative axes is α. Preferably, both L1 and L2 are λ / 2, where λ is the wavelength of the UWB signal. The phases of the same UWB signal measured by antennas 0, 1, and 2 are respectively... and The phase difference between antenna 1 and antenna 0 is The phase difference between antenna 2 and antenna 0 is because and All have been measured, therefore All of them can be calculated. Since λ / 2 corresponds to the phase difference π, α and β can be calculated by combining the cosine formula, d1, d2, L1 and L2, according to formula (3) and formula (4).

[0386]

[0387]

[0388] When L1 is λ / 2

[0389] When L2 is λ / 2

[0390] In one example, such as Figure 12 As shown in (f), the UWB module of the first electronic device adopts a triangular three-antenna structure. (This is related to...) Figure 12 Similar to the description in (e), the UWB signal can be considered parallel when it reaches the UWB module of the first electronic device; similarly, the UWB signal, when it reaches the UWB module of the first electronic device, is parallel at X. e O e Y e Components on a plane can also be considered as parallel. For example... Figure 12 As shown in (f), the UWB signal is represented by two parallel solid lines. The UWB signal in X e O e Y e The components in the plane are represented by two parallel dashed lines; the distance between antenna 0 and antenna 1 is L1, and the distance between antenna 0 and antenna 2 in the Z-axis is... e The distance between the projections on the axis is L2; ​​point NO is the distance between antenna 1 and antenna 0 on the X-axis. e The center point on the axis. Passing through point O. e As O e M1 is perpendicular to the line containing M1N1, and N0M2 is perpendicular to the line containing M2N2 through point N0; where N1 is the UWB signal at X e O e Y e The line containing the component in the plane and the X eThe intersection of the axes, N2 is the intersection of the line containing the UWB signal and the Z-axis. e The intersection of the axes.

[0391] UWB signal and Z e The angle between the positive and negative axes is B. The UWB signal is at X... e O e Y e Components on the plane and X e The angle between the negative and positive axes is α. Preferably, both L1 and L2 are λ / 2, where λ is the wavelength of the UWB signal.

[0392] The formula for calculating α is the same as that for formula (3), and will not be repeated here. When calculating B, first calculate the phase of the UWB signal emitted by the second electronic device reaching NO. Phase with antenna 2 The difference, that is

[0393] Then, using formula (5), the path difference d2 between reaching NO and reaching antenna 2 is calculated, and then β is calculated using formula (6), as follows:

[0394]

[0395]

[0396] 3. Calculate the coordinates of the second electronic device in the first coordinate system based on the distance L between the second electronic device and the first electronic device, and the signal direction of the second electronic device.

[0397] like Figure 12 As shown in (d), the first electronic device can calculate the coordinates (x, y) of the second electronic device in the first coordinate system established by the first electronic device based on the distance L between the second electronic device and the first electronic device, and the signal of the second electronic device (angle α and angle β), using formula (7). e y e , z e ),as follows:

[0398]

[0399] The first electronic device 100 and the second electronic device 300 are connected by means such as Figure 12The communication interaction shown in (c) allows the distance L between the first electronic device 100 and the second electronic device 300 to be obtained, and the first electronic device 100 can determine the direction of the UWB signal based on the received UWB signal. Therefore, the first electronic device 100 can obtain the direction and distance of the second electronic device 300 relative to itself, and thus obtain the coordinates of the second electronic device 300 in the first coordinate system.

[0400] In cases where the second electronic device includes a UWB module, the coordinates of the second electronic device in the first coordinate system can be obtained in real time or periodically through communication of UWB signals between the second electronic device and the first electronic device.

[0401] For situations where some secondary electronic devices do not include a UWB module, such as smartphones that include a UWB module but smart speakers or smart air conditioners that do not, there are two possible marking methods.

[0402] Marking method one, such as Figure 13 As shown in (a), the smartphone moves to the smart speaker and obtains the coordinates of the smartphone in the first coordinate system through communication of UWB signals between the smartphone and the first electronic device. These coordinates are then marked as the coordinates of the smart speaker in the first coordinate system.

[0403] Marking method two, such as Figure 13 As shown in (b), first use a smartphone to point at the smart air conditioner at position 1, so that the Y coordinate of the second coordinate system... b The positive axis is aligned with the first point on the smart air conditioner (e.g., the power button). Through UWB signal communication between the smartphone and the first electronic device, the coordinates 1 of position 1 in the first coordinate system are obtained. The smartphone also includes an IMU module, which determines the smartphone's attitude angle 1 at position 1. Based on coordinates 1 and attitude angle 1, the straight line 1 established by the smartphone pointing to the first point of the smart air conditioner from position 1 can be determined. Then, the smartphone is used to point at the smart air conditioner from position 2, making the Y-axis of the second coordinate system... b The positive axis points towards the first point on the smart air conditioner. The coordinates 2 of position 2 in the first coordinate system are obtained through UWB signal communication between the smartphone and the first electronic device. The attitude angle 2 of the smartphone at position 2 is determined by the IMU module. Based on coordinates 2 and attitude angle 2, the straight line 2 established by the smartphone pointing from position 2 towards the second point of the smart air conditioner can be determined. The coordinates of the intersection of straight line 1 and straight line 2 are calculated, which are the coordinates of the smart air conditioner in the first coordinate system.

[0404] It should be noted that the smart speaker or the smart air conditioner is only an illustrative example, and the smart speaker is used to represent a second electronic device that is easy to be touched by a user holding a mobile device such as a smart phone, and the smart air conditioner is used to represent a second electronic device that is not easy to be touched by a user holding a mobile device such as a smart phone.

[0405] For the case that some second electronic devices do not contain a UWB module, for example, a smart phone contains a UWB module, and a smart TV does not contain a UWB module, the smart TV can be marked multiple times by the smart phone containing the UWB module. For example, Figure 13 As shown in (c) of FIG. 1A, the smart phone moves to point A0 of the smart TV to mark the position, and the first electronic device obtains the coordinates of the contour points of the lower left corner of the smart TV. Accordingly, in the above manner, the first electronic device can obtain the coordinates of multiple contour points (for example, three contour points of the lower left corner, the upper left corner, and the lower right corner, etc.) of the smart TV. If points A0, A1, A2, and A3 are marked, the first electronic device can obtain the coordinates of four corner contour points. If points A0, A1, and A2 are marked, the first electronic device can obtain the coordinates of three corner contour points. If points A0 and A2 are marked, the first electronic device can obtain the coordinates of two corner contour points. The present application does not limit which of the four corner contour points are selected, as long as the contour range of the smart TV can be finally obtained. Alternatively, the contour range described above can refer to the contour range of the display area of the smart TV. Alternatively, the display area described above can include the frame of the display screen of the smart TV, or can not include the frame of the display screen of the smart TV.

[0406] Preferably, the smart phone can move to more than three different positions of the display area of the smart TV, and when moving to one position of the display area of the smart TV, based on the input of the user, the coordinates of the smart phone at this time are marked as the coordinates of one position of the display area of the smart TV; and the above process is repeated to mark the coordinates of more than three different positions of the display area of the smart TV. Alternatively, the coordinates of three positions of the display area of the smart TV are marked. Similarly, the coordinates of one position of the front area of the smart TV can be marked. During the marking of more than three different positions of the display area of the smart TV, the pointing direction, orientation, etc. of the smart phone at the more than three different positions do not need to be consistent, that is, the orientation, pointing direction, etc. of the smart TV during the marking are not limited.

[0407] Alternatively, the more than three positions of the display area of the smart TV can be more than three positions (for example, 1 / 2, 1 / 3, etc.) of the edge contour (for example, the horizontal edge contour or the vertical edge contour) of the display area of the smart TV, or can be more than three positions of the central part of the display area of the smart TV.

[0408] Optionally, a second electronic device including a UWB module can not only mark a second electronic device that does not include a UWB module, but also mark a spatial region. For example, it can mark the extent of a three-dimensional spatial region. The following example uses a smartphone including a UWB module.

[0409] For example, such as Figure 14 As shown in (a), the smartphone is placed at four positions A, B, C, and D respectively. Based on the above principle, the first electronic device 100 obtains the coordinates of the four positions A, B, C, and D in the first coordinate system. and The vertical line passing through position A The vertical line passing through position B The vertical line passing through position C and the vertical line passing through position D Enclosing a three-dimensional region. Where z e It can be a preset value or the height of the room or area.

[0410] In another example, such as Figure 14 As shown in (b), the smartphone is placed at each of the eight vertices of a three-dimensional spatial region. Based on the above principle, the first electronic device 100 obtains the coordinates of each of the eight vertices in the first coordinate system, thereby obtaining the coordinate range of the three-dimensional spatial region. For example, if the three-dimensional spatial region is a room, then the coordinate range of the room in the first coordinate system is obtained. The above only uses the vertex positions as an example; the actual area can be determined based on the placement of the smartphone. For example, the smartphone may not be placed at a vertex position, in which case the determined area would be smaller than the entire area of ​​the room.

[0411] (V) Transformation between different coordinate systems based on UWB

[0412] In this application, coordinate transformations between different coordinate systems can be performed using vectors. Specifically, the distance between two points is the same in different coordinate systems, but the direction of the vector formed by the two points may differ in different coordinate systems. For example, to convert the coordinates of point O0 in the first coordinate system to the coordinates of point O2 in the second coordinate system, a vector transformation can be performed. For example, this can be achieved by... For example, converting vectors in this way. The distance (both L) is the same in the first coordinate system and the second coordinate system, but the vectors... The direction expressed in the first coordinate system, and the vector The directions represented by the second coordinate system are different. By obtaining the relative directional changes between the first and second coordinate systems, given a vector... The direction expressed by the first coordinate system can be obtained by the vector The direction expressed by the second coordinate system; in combination with O e Point, O b The coordinates of the point in the first coordinate system, and O b The coordinates of the point in the second coordinate system, the coordinates of O e The coordinates of the point in the second coordinate system.

[0413] The above-mentioned coordinate conversion of the same point in different coordinate systems is only illustrative, and the application does not limit the coordinate conversion manner.

[0414] The relative direction change of different coordinate systems can be expressed by the pitch Angle (yaw) ψ and roll angle (roll) θ between coordinate systems. Among them, the azimuth angle can also be called the yaw angle or the heading angle. For example, the pitch angle, azimuth angle and roll angle of the second coordinate system relative to the first coordinate system, the pitch angle, azimuth angle and roll angle of the third coordinate system relative to the first coordinate system. In order to facilitate the calculation of the pitch angle, azimuth angle and roll angle between the three coordinate systems, it is necessary to first assume that the coordinate origins of the above-mentioned three coordinate systems are all gathered to a point. For example, the coordinate origin O e of the UWB base station is parallelly moved to the coordinate origin O b of the second coordinate system. Correspondingly, the first coordinate system is also moved. The definition of the pitch angle, azimuth angle and roll angle is known to those skilled in the art, which will not be described here.

[0415] Exemplarily, Figure 15 The pitch angle Azimuth angle And roll angle

[0416] The coordinate origin O b of the second coordinate system coincides with the coordinate origin O e of the first coordinate system after parallel movement, the three axes of the second coordinate system are X b Axis, Y b Axis and Z b Axis, and the three axes of the first coordinate system are X e Axis, Y e Axis and Z e Axis. As shown in (a) of FIG. Figure 15 , O e Y b ′ (i.e. O b Y b ′) is the Y b Axis in the X e Oe Y e The projection of the Y Figure 15 axis of the second coordinate system onto the X e Z b ′ (i.e. the projection of the Y b Z b ′) is the Z b axis of the first coordinate system. b O b Z e The projection of the Y

[0417] The pitch angle of the second coordinate system relative to the first coordinate system The angle between the Y b axis of the second coordinate system and the X e O e Y e plane of the first coordinate system. That is, the angle between O b Y b ′ and the Y b axis. When the component of O b Y b on the Z e axis is positive, i.e. lies in the positive Z e axis, O b Y b is positive. e When the component of O e Y b on the Z e axis is negative, i.e. lies in the negative Z e axis, O e Y e is negative. That is, the angle between O b Y b ′ and the Y e axis. When the component of O b Y b ′ on the X e axis is positive, i.e. lies in the positive X e axis, O b Y b ′ is positive. e When the component of O e Y b ′ on the X b axis is negative, i.e. lies in the negative X e axis, O e Y b ′ is negative.

[0419] ​Roll angle of the second coordinate system relative to the first coordinate system Z in the second coordinate system b axis and Y b O e Z e The angle between planes. That is, O b Z b ′ and Z b The angle between the axes. When Z... b The positive axis is in the Y b O e Z e The projection on the plane onto X b The component on the axis, located at X b When the axis is positive, It is positive when Z is positive; b The positive axis is in the Y b O e Z e The projection on the plane onto X b The component on the axis, located at X b When the axis is the negative axis, It is negative.

[0420] Alternatively, when O b Z b ′ in X b O b Y b The projection on the plane, in X b The component on the axis, located at X b When the axis is positive, It is positive; when O b Z b ′ in X b O b Y b The projection on the plane, in X b The component on the axis, located at X b When the axis is the negative axis, It is negative.

[0421] For example, Figure 16 The pitch angle of the third coordinate system relative to the first coordinate system is shown. Azimuth and roll angle

[0422] like Figure 16 As shown, the origin O of the third coordinate system t The origin O of the first coordinate system after parallel translation e Coincident, the three axes of the third coordinate system are X and Y. t Axis, Y t Axis and Z taxis, the three axes of the first coordinate system are X e axis, Y e axis, and Z e axis. As shown in (a) of Fig. Figure 16 O e Y t ′ (i.e. O t Y t ′) is Y t axis in the X e O e Y e plane of the first coordinate system. As shown in (b) of Fig. Figure 16 O e Z t ′ (i.e. O t Z t ′) is Z t axis in the Y t O e Z e plane of the first coordinate system.

[0423] pitch angle of the third coordinate system relative to the first coordinate system angle between Y t axis of the third coordinate system and X e O e Y e plane of the first coordinate system. i.e. the angle between O e Y t ′ (i.e. O t Y t ′) and Y t axis. When the component of O e Y t on Z e axis is positive, O e Y t is located on the positive axis of Z e axis, and e is positive; when the component of O t Y e on Z e axis is negative, O e Y e is located on the negative axis of Z e axis, and t is negative.

[0424] azimuth angle of the third coordinate system relative to the first coordinate system angle between Y t axis of the third coordinate system and X e O e Y e plane of the first coordinate system, and Y e axis of the first coordinate system. i.e. the angle between O e Y t ′ (i.e. O t Y t ′) and Ye The angle between axes. When O e Y t ′ in X e The component on the axis, located at X e When the axis is positive, It is positive; when O e Y t ′ in X e The component on the axis, located at X e When the axis is the negative axis, It is negative.

[0425] Roll angle of the third coordinate system relative to the first coordinate system Z in the third coordinate system t axis and Y t O e Z e The angle between planes. That is, O t Z t ′ and Z t The angle between the axes. When Z... t The positive axis is in the Y t O e Z e The projection on the plane onto X t The component on the axis, located at X t When the axis is positive, It is positive when Z is positive; t The positive axis is in the Y t O e Z e The projection on the plane onto X t The component on the axis, located at X t When the axis is the negative axis, It is negative.

[0426] Alternatively, when O t Z t ′ in X t O t Y t The projection on the plane, in X t The component on the axis, located at X t When the axis is positive, It is positive; when O t Z t ′ in X t O t Y t The projection on the plane, in X t The component on the axis, located at X t When the axis is the negative axis, It is negative.

[0427] The directional change of the third coordinate system relative to the first coordinate system can be expressed using the attitude matrix. To express.

[0428]

[0429] Attitude matrix The above formula (8) is prior art, which can be obtained by those skilled in the art. For example, the attitude matrix in Chapter 1.2.1 of the book "Inertial Navigation" (Beijing: Science Press, ISBN 7-03-016428-8, edited by Qin Yongyuan, first edition in May 2006, first printing in May 2006).

[0430] The above is merely an illustrative example of the transformation of the second and third coordinate systems relative to the first coordinate system. Those skilled in the art should understand that the transformation of other coordinate systems is also based on the above principle, using the same formula, only with changes to the corresponding parameters.

[0431] Optionally, the second electronic device may include an IMU module. Optionally, the IMU module of the second electronic device is first calibrated. That is, the coordinate system on which the pitch angle, azimuth angle, and roll angle output by the IMU module of the second electronic device are based is calibrated to the first coordinate system, or the coordinate system on which the IMU module output by the second electronic device is based is calibrated. Calibration Thus, as the second electronic device moves, the pitch, azimuth, and roll angles output by its IMU module become the pitch, azimuth, and roll angles of the second coordinate system relative to the first coordinate system; or, the IMU module outputting the... By transposing, the directional change of the second coordinate system relative to the first coordinate system can be reflected.

[0432] For example, the second coordinate system of the second electronic device can be parallel to the first coordinate system (e.g., X). b The axis is parallel to X. e axis, Y b The axis is parallel to Y. e Axis, Z b The axis is parallel to Z. e (axis), and the positive directions of the corresponding coordinate axes of the two coordinate systems are the same (e.g., X). b Positive axis and X e The positive axes are the same, Y b Positive axis and Y e The positive axes are the same, Z b Positive axis and Z e (With the same positive axis direction), the pitch, azimuth, and roll angles output by the IMU module of the second electronic device are all set to 0.

[0433] For example, the second coordinate system of the second electronic device can be parallel to the first coordinate system, and the positive directions of all axes of the two coordinate systems are the same. By adjustment, the output of the IMU module of the second electronic device can be made so that...

[0434] (vi) Establishment of the fourth coordinate system (millimeter-wave radar coordinate system)

[0435] The millimeter-wave radar module 160 of the first electronic device 100 is used to implement millimeter-wave radar functions. Multiple antennas in the millimeter-wave radar have distance differences in the lateral direction (e.g., horizontal direction) and / or longitudinal direction (e.g., vertical direction), and the distance differences between the antennas can be used to establish a coordinate system (fourth coordinate system) for the millimeter-wave radar.

[0436] In one example, the millimeter-wave radar module 160 includes three transmitting antennas and four receiving antennas. Exemplarily, such as... Figure 17 As shown, three transmitting antennas and four receiving antennas are located in the same longitudinal plane (e.g., a vertical plane). The three transmitting antennas are arranged in a triangle in the longitudinal plane. Transmitting antenna 0 and transmitting antenna 2 are located in the same transverse plane; the four receiving antennas are located on the same transverse line (e.g., a horizontal line). In one embodiment, a point on receiving antenna 0 (e.g., an endpoint on one side) is taken as the origin O of the fourth coordinate system. m Using the line connecting receiving antenna 0 and receiving antenna 1 as the X-axis of the fourth coordinate system m The axis is defined, and the direction from receiving antenna 1 to receiving antenna 0 is X. m The positive direction of the axis; passing through the origin O. m And perpendicular to X m The straight line on the axis is the Z-axis of the fourth coordinate system. m The axis, and the direction pointing to the zenith is Z. m The positive axis; then, combining the right-hand rectangular coordinate system rule, determine the Y-axis of the fourth coordinate system. m Axis and Y m The positive direction of the axis. Optionally, the outer surface of the first electronic device 100 may be marked with prompts to indicate the correct installation or placement method, so that the three transmitting antennas and four receiving antennas of the millimeter-wave radar module 160 in the first electronic device 100 are located in the same longitudinal plane.

[0437] The naming of the three axes in this fourth coordinate system, as well as the positive directions of the three axes, can also use other definitions, which will not be elaborated here. This application embodiment uses... Figure 17 In the fourth coordinate system shown, X m Axis, Y m Axis and Z m Let's take the shaft as an example for introduction.

[0438] It should be noted that the above takes a point on the receiving antenna 0 as the origin of the fourth coordinate system, which is only exemplary. A point on another antenna (such as receiving antenna 1) can also be the origin of the fourth coordinate system.

[0439] Alternatively, the fourth coordinate system can be established in advance. The installation personnel only need to install the first electronic device 100 according to the requirements. For example, the fourth coordinate system has been established before the first electronic device 100 leaves the factory, and the related information of the fourth coordinate system is stored in the local or server. When the first electronic device 100 starts, or when the first electronic device 100 receives a specific trigger, the first electronic device 100 calls the related information of the fourth coordinate system from the local or server. In the absence of special instructions, the server in this application can be the hub device 200 of the family, or a cloud server.

[0440] Preferably, the outer surface of the first electronic device 100 can only have one label prompt information indicating the installation of the first electronic device. In this way, the transmitting antenna and receiving antenna of the millimeter wave radar module, and the antenna of the UWB module all meet the preset requirements.

[0441] (Seven) Coordinate calculation under the fourth coordinate system

[0442] 1. Determine the distance between the reflection point and the millimeter wave radar module, and the radial velocity of the reflection point.

[0443] (1) Determine the distance between the reflection point and the millimeter wave radar module

[0444] The transmitting antenna of the millimeter wave radar module 160 transmits a signal, which is reflected by the reflection point and received by the receiving antenna of the millimeter wave radar module 160. The frequency of the LFMCW millimeter wave radar transmitted signal increases linearly with time, such a signal is called Chirp signal. Combined with Figure 5B , the millimeter wave radar module 160 receives the Chirp signal through the receiving antenna. The received signal and the local oscillator signal are mixed through a mixer to output a difference frequency signal. After filtering, amplification and sampling by a filter, the difference frequency signal is converted into a digital difference frequency signal.

[0445] Exemplarily, Figure 18 A schematic diagram of the principle of determining the distance and radial velocity of the reflection point by the millimeter wave radar provided by the embodiment of the application is shown. As Figure 18 shown in (a), the solid line is the transmitting signal of the millimeter wave radar module 160, and the dashed line is the receiving signal of the millimeter wave radar module 160. One sweep frequency period Tc of the Chirp signal is usually in the order of microseconds (us), and the frequency modulation rate S0 (i.e. the rate of change of frequency) reaches 10 12The order of magnitude (in Hz / s). In the embodiments of this application, the Chirp signal within one sweep period Tc is called a Chirp signal. It is generally assumed that the spatial position of the target does not change within one sweep period Tc.

[0446] like Figure 18 As shown in (b), within one sweep period Tc, the transmitting antenna transmits a chirp signal. After time τ, the receiving antenna receives the signal reflected back from the reflection point. The frequency difference between the received signal and the transmitted signal is τ*S0. The frequency difference between the received signal and the transmitted signal is the frequency f0 of the difference frequency signal, that is, f0=τ*S0. Wherein, τ=2d / c, d is the distance between the reflection point and the millimeter-wave radar module (which can also be regarded as the first electronic device), and c is the transmission rate of the chirp signal in the air, which is generally chosen as the speed of light. Therefore, the relationship between the distance d of the reflection point and the frequency f0 of the difference frequency signal is shown in formula (9).

[0447] d=f0*c / (2*S0) Formula (9)

[0448] Fourier transform can convert a time-domain signal into a frequency-domain signal. A sine wave in the time domain corresponds to a peak value in the frequency domain, which represents the frequency f0 of the difference frequency signal. For example, as... Figure 18 As shown in (c), the transmitted signal from the millimeter-wave radar module is reflected back as three signals through three reflection points. The millimeter-wave radar module receives these three received signals and acquires the three corresponding difference frequency signals. A fast fourier transform (FFT) is performed on the three difference frequency signals to obtain a range curve (called range FFT), which generates a spectrum with three distinct peaks. Each peak represents a corresponding reflection point. The frequency of the difference frequency signal is obtained by calculating the frequency corresponding to the peak. The distance to the reflection point can be obtained by detecting the frequency of the difference frequency signal.

[0449] Similarly, Doppler FFT is performed on multiple difference frequency signals with different sweep periods Tc at the same reflection point to obtain the phase difference between the multiple difference frequency signals. By detecting the phase difference between the multiple difference frequency signals, the radial velocity of the reflection point can be obtained. For detailed principles, please refer to existing technologies, which will not be elaborated here.

[0450] The millimeter-wave radar module receives the chirp signal and, after mixing, power amplification, and filtering the transmitted and received signals, obtains a difference frequency signal. This difference frequency signal is then converted from analog to digital into a digital difference frequency signal. By detecting the digital difference frequency signal, the distance and radial velocity of the reflection point can be obtained.

[0451] For example, such asFigure 18 As shown in (d), one frame of data of the millimeter wave radar module, i.e., data in one radar scanning period, includes M frequency sweeping periods Tc, and there are N sampling points of the difference frequency signal in each frequency sweeping period Tc.

[0452] The frequency of the difference frequency signal can be obtained by performing one-dimensional range FFT on the digital difference frequency signal in one frequency sweeping period Tc. Thus, the distance of the reflection point can be calculated according to the frequency of the difference frequency signal. The number of points of the range FFT is the number N of sampling points of the difference frequency signal corresponding to the Chirp signal.

[0453] The phase difference of the plurality of digital difference frequency signals can be obtained by performing one-dimensional doppler FFT on the digital difference frequency signals of the same reflection point in a plurality of adjacent frequency sweeping periods Tc. Thus, the radial velocity of the reflection point can be calculated according to the phase difference of the plurality of difference frequency signals. The number of points of the doppler FFT is the number of frequency sweeping periods included in one frame of data.

[0454] The joint operation of the range FFT and the doppler FFT can be considered as two-dimensional FFT on one frame of data. In the embodiments of the present application, one frame of data after two-dimensional FFT processing is referred to as one frame of two-dimensional FFT data. Exemplarily, Figure 18 (e) is a schematic diagram of one frame of two-dimensional FFT data obtained by the millimeter wave radar module. As shown in (e), Figure 18 As shown in (e), there are a plurality of peaks in one frame of two-dimensional FFT data, and each peak indicates that there is a reflection point at the corresponding position. The value of the reflection point in the distance dimension or the velocity dimension is the distance of the reflection point or the radial velocity of the reflection point.

[0455] (2) Determine the signal direction of the reflection signal of the reflection point

[0456] The signal direction of the reflection signal includes a lateral direction (such as a horizontal direction) and a longitudinal direction (such as a vertical direction). The lateral direction of the signal can be represented by an azimuth angle, and the longitudinal direction of the signal can be represented by an elevation angle. In an embodiment, the azimuth angle and the elevation angle can be calculated by the phase difference between the received signals of a plurality of receiving antennas of the millimeter wave radar module.

[0457] Exemplarily, Figure 19 A schematic diagram of the principle of determining the signal direction of the reflection signal of the reflection point by the millimeter wave radar provided in the embodiments of the present application is shown. As shown in (f), Figure 19As shown in (a), the millimeter-wave radar module includes four receiving antennas. After a signal transmitted from the same transmitting antenna is reflected at a reflection point, the phase difference between the reflected signals arriving at any two different receiving antennas can be used by the millimeter-wave radar module to measure the azimuth angle of the reflected signal. The specific method by which the millimeter-wave radar module determines the lateral direction of the reflected signal based on the phase difference between the signals arriving at two adjacent receiving antennas can be found in [reference needed]. Figure 12 The method for calculating angle α in (e) will not be elaborated here.

[0458] In one implementation, the accuracy of the measurement signal direction can be improved by increasing the number of antennas. In one example, the antennas of the millimeter-wave radar module employ... Figure 8 The distribution structure is shown in (a). When transmitting signals, the millimeter-wave radar module can switch the transmitting antenna by changing a relay switch, thus separating the receiving antenna from the signals of different transmitting antennas. For example, as shown in (a)... Figure 19 As shown in (b), when transmitting antenna 0 and transmitting antenna 2 alternately transmit signals, based on the phase difference principle generated by the antenna position difference, the two-transmitter, four-receiver antenna can be equivalent to one-transmitter, eight-receiver antenna. For example, in Figure 19(b), the distance between the receiving antennas is λ. L / 2, the distance between transmitting antenna 0 and transmitting antenna 2 is 2λ L ; where λ L The wavelength is millimeter wave. The signal transmitted by transmitting antenna 2 to receiving antenna 0 can be equivalent to the signal received by receiving antenna 4; the signal transmitted by transmitting antenna 2 to receiving antenna 1 can be equivalent to the signal received by receiving antenna 5; the signal transmitted by transmitting antenna 2 to receiving antenna 2 can be equivalent to the signal received by receiving antenna 6; the signal transmitted by transmitting antenna 2 to receiving antenna 3 can be equivalent to the signal received by receiving antenna 7. For example, in the one-transmit-eight-receive schematic diagram of Figure 19(b), receiving antennas 4, 5, 6, and 7 are equivalent virtual receiving antennas.

[0459] A transmitting antenna with a distance in the longitudinal dimension transmits a signal that is reflected at a reflection point before reaching the receiving antenna. The phase difference between this phase difference and the reflected signal's arrival at the receiving antenna can be used by a millimeter-wave radar module to measure the longitudinal direction of the reflected signal (e.g., revealed by the elevation angle). In one example, the antenna of the millimeter-wave radar module uses... Figure 8 The structure shown in (a) is such that transmitting antenna 1 is separated from transmitting antenna 0 by a distance in the longitudinal dimension, and transmitting antenna 1 and transmitting antenna 2 are also separated by a distance in the longitudinal dimension. The elevation angle of the reflected signal can be determined by comparing the signals received by the same receiving antenna from transmitting antenna 1, transmitting antenna 0, and transmitting antenna 2 respectively. For example, as shown in (a) Figure 19The signals transmitted by the transmitting antenna 0 and received by the receiving antenna 2 and the receiving antenna 3, the signals transmitted by the transmitting antenna 2 and received by the receiving antenna 0 and the receiving antenna 1, and the signals transmitted by the transmitting antenna 1 and received by the receiving antenna 0, the receiving antenna 1, the receiving antenna 2 and the receiving antenna 3 can be compared, and the longitudinal direction of the reflection point (for example, revealed by the elevation angle) can be calculated according to the phase difference of the signals in the longitudinal direction. For example, the phase difference between the signal transmitted by the transmitting antenna 0 and received by the receiving antenna 2 and the signal transmitted by the transmitting antenna 1 and received by the receiving antenna 0 can be obtained, and the elevation angle can be calculated according to the phase difference. For details of calculating the elevation angle according to the phase difference of the received signals, refer to the calculation method of the angle β in (f) of the method for determining the reflection point according to the received signals. Figure 12 The calculation method of the angle β in (f) of the method for determining the reflection point according to the received signals is not repeated here.

[0460] (3) Determining the coordinates of the reflection point in the fourth coordinate system

[0461] The first electronic device can calculate the coordinates of the reflection point in the fourth coordinate system established by the first electronic device according to the distance between the reflection point and the millimeter wave radar and the signal direction (azimuth angle and elevation angle) of the reflection signal by using formula (7).

[0462] (4) Determining the coordinates of the user in the fourth coordinate system

[0463] In some cases, due to the different parts of a large volume of human body that may be dressed in different clothes, have different bone structures, etc., different detection results may be obtained, resulting in multiple reflection points for human body detection, or even uneven distribution. For a large volume of objects, different parts may be made of different materials, have different shapes, etc., resulting in multiple reflection points for object detection, or even uneven distribution. For a human body, since the head, hands and feet of the human body will reflect the transmitted signals of the millimeter wave radar, the millimeter wave radar will detect a human body as multiple reflection points in the detectable range. At this time, the point cloud data of the reflection points can be clustered, that is, the multiple detected reflection points are clustered into one class, and the cluster is determined as an object or a human body.

[0464] Exemplarily, Figure 20 (a) of the method for determining the reflection point according to the received signals is a schematic diagram of the clustering effect of the point cloud data. Figure 20each point in (a) represents a reflection point detected by the millimeter wave radar module, and the three closed curves respectively represent classes converged, and the points outside the three closed curves represent reflection points that are not converged into any one class. In an example, the millimeter wave radar module adopts a clustering algorithm to cluster multiple reflection points into an object or a human body (user), and the coordinates of the object or the human body (user) in the fourth coordinate system can be calculated according to the coordinates of the clustered multiple reflection points. Exemplarily, the coordinates of the object or the human body (user) in the fourth coordinate system can be the coordinates of the center of gravity of the object or the human body in the fourth coordinate system. Exemplarily, as shown in (b) of FIG. 20, Figure 20 the smaller points in (b) represent reflection points detected by the millimeter wave radar, and the largest point is the coordinate point of the human body (user) in the fourth coordinate system. The coordinates of the human body (user) in the fourth coordinate system are denoted as

[0465] Further, the height of the object or the height of the human body (user) can also be calculated according to the height H of the first electronic device from the ground, and the coordinates of the object or the human body (user) in the fourth coordinate system. Exemplarily, the height h of the human body (user) can be calculated by using formula (10) m as follows:

[0466]

[0467] Exemplarily, Figure 21 A method flowchart for determining the coordinates of a user in a fourth coordinate system by a first electronic device is shown. As shown in Figure 21 the method can include:

[0468] S2100, the millimeter wave radar module receives a reflection signal.

[0469] S2101, the millimeter wave radar module performs two-dimensional fast Fourier transform on the digital difference frequency signal.

[0470] The receiving antenna of the millimeter wave radar module receives a reflection signal, obtains a digital difference frequency signal according to the reflection signal, and performs two-dimensional fast Fourier transform on the digital difference frequency signal to obtain two-dimensional FFT data.

[0471] S2102, the millimeter wave radar module obtains the distance and radial velocity of the reflection point from the millimeter wave radar by using a target detection algorithm.

[0472] The millimeter wave radar can use a target detection algorithm to perform target detection on a frame of two-dimensional FFT data to obtain the distance and radial velocity of the target.

[0473] It should be noted that in an indoor environment, multipath effect and clutter interference will occur, and the signal received by the millimeter wave radar includes target reflection signal, background noise and clutter interference, etc. For example, Figure 18 In the test environment of the two-dimensional FFT data of one frame shown in (e), there are moving human bodies at a distance of 1m, 2m and 4m from the millimeter wave radar, respectively, and from Figure 18 As can be seen from (e), in addition to the above-mentioned three peaks at a distance of 1m, 2m and 4m from the millimeter wave radar, there are other reflection signals (background noise and clutter interference, etc.) causing larger peaks. If the reflection signals caused by background noise and clutter interference are detected as reflection points, false alarm will occur. In an implementation method, a constant false alarm rate (CFAR) target detection algorithm can be used to obtain the distance and radial velocity of the reflection point to maintain a constant false alarm rate and improve the target detection accuracy.

[0474] It should be noted that the millimeter wave radar module can use the target detection algorithm in the prior art to obtain the distance and radial velocity of the reflection point according to the two-dimensional FFT data, and the application embodiments do not limit the used target detection algorithm. The specific implementation method of the target detection algorithm can be obtained from the prior art, which will not be described here.

[0475] S2103, the millimeter wave radar module determines the signal direction of the reflection signal.

[0476] Exemplarily, the phase difference method, the sum-difference beam method, the music method and the like can be used to estimate the azimuth angle and the elevation angle. The phase difference method, the sum-difference beam method, the music method and the like can be obtained from the prior art, which will not be described here.

[0477] S2104, the millimeter wave radar module determines the coordinates of the reflection point in the fourth coordinate system.

[0478] The millimeter wave radar determines the coordinates of the reflection point in the fourth coordinate system according to the distance between the reflection point and the millimeter wave radar and the signal direction of the reflection point.

[0479] S2105, the millimeter wave radar module determines the coordinates of the smart device or the user in the fourth coordinate system.

[0480] In an embodiment, the detected reflection points are clustered by using a clustering algorithm to cluster multiple reflection points into an intelligent device or a user. The clustering algorithm includes a partition-based clustering method, a density-based partition method, a model-based partition method, a network-based partition method, etc. For example, common clustering algorithms include a density-based spatial clustering of applications with noise (DBSCAN), a K-Means algorithm, a Birch algorithm, etc. Any clustering algorithm can be used for clustering, which is not limited in the embodiments of the present application.

[0481] The coordinates of the intelligent device or the user in the fourth coordinate system can be calculated according to the average values of the coordinates of the multiple reflection points of the clustered intelligent device or user.

[0482] S2106, the millimeter wave radar module tracks the intelligent device or the user.

[0483] The millimeter wave radar module performs target detection on each frame of received data. Further, after detecting the object (intelligent device) or human body in each frame of data by using a target detection algorithm and a clustering algorithm, the detection results in the current frame and the detection results in the previous frame can be matched one by one by using an association algorithm to achieve tracking of the object or the human body (i.e., to obtain changes in the coordinates of the object or the human body over time). Exemplarily, a tracking algorithm (a front-back frame association algorithm) is to calculate the Euclidean distance (the straight-line distance between two points in space) between two targets in two frames, determine the two targets with the shortest Euclidean distance as the same target, and then link the tracking target by using a Hungarian algorithm.

[0484] Further, in an embodiment, the millimeter wave radar module can determine whether a target is stationary or moving according to the target tracking result. The millimeter wave radar module can also be used to detect the physiological characteristics (such as the breathing rate and the heart rate) of a stationary target. If it is determined that the physiological characteristics of the target meet a set condition (such as the breathing rate being within a preset range and the heart rate being within a preset range), it is determined that the target or the clustered target is a human body (a user); and the user is tracked.

[0485] (Eight) Detection of physiological characteristics, identity categories, and human body postures of a user by the millimeter wave radar module

[0486] The specific method of detecting the physiological characteristics, identity categories, and human body postures of a user by the millimeter wave radar module will be described in detail below with reference to the accompanying drawings.

[0487] (1) Detection of physiological characteristics of a user by the millimeter wave radar module

[0488] The physiological features of the user include the breathing frequency, heartbeat frequency, and the like of the user. When the user is static (the position of the user does not change), the slight displacement of the body of the user caused by breathing and heartbeat can cause a phase change of the reflected signal of the millimeter wave radar module. The breathing frequency and heartbeat frequency of the user can be obtained by detecting the phase change of the reflected signal of the millimeter wave radar module when the user is static.

[0489] In an embodiment, as shown in Figure 22 The method for obtaining the breathing frequency and heartbeat frequency of the user by the millimeter wave radar module can include:

[0490] S2201, extracting phase information.

[0491] In S2101, the Range FFT is performed on each frame of data of the millimeter wave radar, and the frequency of the beat signal can be obtained according to the Range FFT result, that is, the phase of the beat signal. In S2106, the millimeter wave radar performs target tracking on the user, and the change of the position of the user over time can be obtained, that is, the position of the user at a certain time can be obtained.

[0492] If the millimeter wave radar determines that the user is static (for example, the coordinate change of the user is less than a set value within a period of time) according to the target tracking result, the phase of the Range FFT result at the current position of the user is extracted, that is, the phase information of the beat signal is extracted. Exemplarily, the radar scanning period is 100 ms, that is, the period of one frame of data is 100 ms. The phase information of the beat signal is extracted once for each frame of data. The phase information of multiple frames of data is continuously extracted, and the relationship between the phase and the frame number, that is, the relationship between the phase and the time, is obtained; denoted as a vibration signal v(j), where j is the frame number.

[0493] S2202, phase unwrapping.

[0494] The phase is unwrapped to obtain the actual displacement curve. In which, the phase value is defined between [-π, π]. If the phase value calculated in S2201 is greater than π, the phase unwrapping is performed by subtracting 2π from the phase value; if the phase value calculated in S2201 is less than -π, the phase unwrapping is performed by adding 2π to the phase value.

[0495] S2203, calculating the phase difference.

[0496] The phase difference operation is performed on the unwrapped phase by subtracting the continuous phase values, and the phase difference Δv is obtained; in this way, the heartbeat signal can be enhanced and any phase drift can be eliminated. Wherein, Δv(k) = v(k) - v(k-1).

[0497] S2204, band-pass filtering.

[0498] According to the heart rate and the respiration rate, the phase value is filtered by a band-pass filter to distinguish. Exemplarily, the passband range of the band-pass filter is set to 0.8Hz-4Hz, the phase value is filtered, and the heart rate can be detected; the passband range of the band-pass filter is set to 0.1Hz-0.6Hz, the phase value is filtered, and the respiration rate can be detected.

[0499] S2205, range estimation.

[0500] The phase difference signal is subjected to FFT, and the respiration rate and the heart rate in N frames of time are obtained according to the peak value and the harmonic characteristics.

[0501] S2206, decision.

[0502] The respiration rate and the heart rate in a period ...

Claims

1. A human perception based communication system, characterized by, The system comprises a hub device, a first electronic device and R second electronic devices; any two of the hub device, the first electronic device and any one of the R second electronic devices communicate in a wired or wireless manner; the first electronic device comprises a first ultra-wideband module and a millimeter wave radar module; the R second electronic devices comprise a mobile device, and the mobile device comprises a second ultra-wideband module; R is a positive integer greater than or equal to 1; Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device; According to the position information of the user and the position information of the R second electronic devices, the hub device controls or notifies at least one of the R second electronic devices to perform a preset operation; wherein the first electronic device measures the position of the mobile device through the first ultra-wideband module to obtain the position information of the R second electronic devices in the coordinate system provided by the first ultra-wideband module; the first electronic device measures the position of the human body through the millimeter wave radar module to obtain the position information of the user in the coordinate system provided by the millimeter wave radar module; based on the position information of the R second electronic devices in the coordinate system provided by the first ultra-wideband module, the hub device obtains the position information of the R second electronic devices in the whole-house coordinate system provided by the hub device; based on the position information of the user in the coordinate system provided by the millimeter wave radar module, the hub device obtains the position information of the user in the whole-house coordinate system provided by the hub device; In response to the control or notification of the hub device, the at least one second electronic device performs the preset operation; The at least one second electronic device performing the preset operation comprises: The second electronic device closest to the user wakes up; or, The first device automatically stops playing the first video or the first audio, and the second device continues to play the first video or the first audio; the at least one second electronic device comprises the first device and the second device, and the distance between the second device and the user is less than the distance between the first device and the user; or, The first device plays according to first volume information, and the second device plays according to second volume information; the at least one second electronic device comprises the first device and the second device, the first volume information is obtained according to the first distance between the first device and the user, and the second volume information is obtained according to the second distance between the second device and the user; or, The first device in the at least one second electronic device automatically stops playing and automatically switches to a screensaver or automatically resumes playing the first video; or, The first device in the at least one second electronic device stops playing the first video or resumes playing the first video.

2. The communication system of claim 1, wherein, Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device, including: Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device periodically or in real time obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device.

3. The communication system according to claim 1 or 2, characterized by When the user is one user, the position information of the user is the position information of the one user.

4. The communication system according to claim 1 or 2, characterized by When the user is multiple users, the position information of the user is the center position information of the multiple users.

5. The communication system of claim 1, wherein, T second electronic devices of the R second electronic devices support voice wake-up. In response to receiving the wake-up voice, the T second electronic devices respectively send first messages to the hub device. In response to receiving at least one of the first messages, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system. The hub device determines a device closest to the user from the T second electronic devices. The hub device sends a first indication message to the device closest to the user. In response to receiving the first indication message, the device closest to the user wakes up. Wherein, T is a positive integer greater than or equal to 1 and less than or equal to R.

6. The communication system of claim 5, wherein, In response to receiving at least one of the first messages, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system, including: In response to receiving one of the first messages, the hub device obtains the position information of the second electronic device sending the first message and the first room or first area where it is located in the whole-house coordinate system; and the hub device further obtains the position information of the user in the first room or the first area in the whole-house coordinate system.

7. The communication system of claim 6, wherein, Based on the communication between the hub device and T second electronic devices of the R second electronic devices, the hub device pre-obtains that the T second electronic devices support voice wake-up. T is a positive integer greater than or equal to 1 and less than or equal to R; the user is one user or multiple users.

8. The communication system of claim 7, wherein, When the user is multiple users, the device closest to the user is the device closest to the center position of the multiple users.

9. The communication system of claim 1, wherein, The R second electronic devices include a first device and a second device, and the first device and the second device support playing video or audio; the user includes a first user; In response to the operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device. In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio. After obtaining that the second device is the closest playing device to the first user, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device automatically stops playing and sends a first feedback message to the hub device, the first feedback message comprising an identification of the first video or the first audio and playing progress information; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message comprising the identification of the first video or the first audio and the playing progress information; In response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information.

10. The communication system of claim 9, wherein, The obtaining that the second device is the closest playing device to the first user comprises: Among devices that have not played the video or the audio, the second device is the closest playing device to the first user.

11. The communication system of claim 1, wherein, The R second electronic devices comprise a first device and a second device, and the first device and the second device support playing a video or an audio; the user comprises a first user and a second user; In response to an operation of the first user, the first device starts playing a first video or a first audio and sends a first notification message to the hub device; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; In response to an operation of the second user, the first device starts playing a second video or a second audio and sends a second notification message to the hub device; In response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; After obtaining that the second device is the closest playing device to the second user, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device automatically stops playing and sends a first feedback message to the hub device, the first feedback message comprising an identification of the second video or the second audio and playing progress information; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message comprising the identification of the second video or the second audio and the playing progress information; In response to receiving the second notification message, the second device continues playing the second video or the second audio according to the playing progress information.

12. The communication system of claim 11, wherein, The obtaining that the second device is the closest playing device to the second user comprises: Among devices that have not played the video or the audio, the second device is the closest playing device to the second user.

13. The communication system of claim 1, wherein, The R second electronic devices comprise a first device and a second device, and the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user comprises a first user, and the first user is located in the first room or the first area; In response to the operation of the first user, the first device starts playing the first video or the first audio, and sends a first notification message to the hub device; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; After obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user. After obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user among the devices that do not play videos or audios. The R second electronic devices include a first device and a second device, the first device and the second device support playing videos or audios, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the users include a first user and a second user, and the first user and the second user are located in the first room or the first area; In response to the operation of the first user, the first device starts playing the first video or the first audio, and sends a first notification message to the hub device; 14. The communication system of claim 13, wherein, In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; 15. The communication system of claim 14, wherein, In response to the operation of the second user, the first device starts playing the second video or the second audio, and sends a second notification message to the hub device; 16. The communication system of claim 1, wherein, In response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including the identification and playing progress information of the second video or the second audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identification and playing progress information of the first video or the first audio; In response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information. ​ ​ In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message comprising an identification of the second video or the second audio and the play progress information; In response to receiving the second notification message, the second device continues playing the second video or the second audio according to the play progress information.

17. The communication system of claim 16, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play video or audio.

18. The communication system of claim 17, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play video or audio.

19. The communication system of claim 1, wherein, The R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a left channel of stereo sound, and the second device playing a right channel of stereo sound; The hub device obtains that the first device and the second device are located in a first room or a first area, a first distance between the first device and the user, and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; The hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information, and the second playing message comprising the second volume information; In response to receiving the first playing message, the first device plays according to the first volume information; In response to receiving the second playing message, the second device plays according to the second volume information.

20. The communication system of claim 19, wherein, The stereo system further comprises a third device, the third device playing a stereo auxiliary sound; the stereo sound comprises a left channel of stereo sound, a right channel of stereo sound and a stereo auxiliary sound; The hub device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user; The hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance, comprising: the hub device determines first volume information for the first device, second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; The hub device sends a first playing message and a second playing message to the first device and the second device respectively, comprising: the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising third volume information; In response to receiving the third playing message, the third device plays according to the third volume information.

21. The communication system of claim 1, wherein, The R second electronic devices include a stereo system, the stereo system including a first device and a second device, the first device playing a left channel of stereo sound, and the second device playing a right channel of stereo sound. The hub device obtains that the first device is located in a first area, the second device is located in a second area, and the first area is adjacent to the second area; the hub device further obtains a first distance between the first device and the user and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance. The hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message including the first volume information, and the second playing message including the second volume information. In response to receiving the first playing message, the first device plays according to the first volume information. In response to receiving the second playing message, the second device plays according to the second volume information.

22. The communication system of claim 21, wherein, The stereo system further includes a third device, the third device playing a stereo auxiliary sound; the stereo sound includes a left channel of stereo sound, a right channel of stereo sound and a stereo auxiliary sound. The hub device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user. The hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance, including: the hub device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance. The hub device sends a first playing message and a second playing message to the first device and the second device respectively, including: the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message including third volume information. In response to receiving the third playing message, the third device plays according to the third volume information.

23. The communication system of any of claims 19-22, wherein, The hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance, including: The hub device determines a first volume attenuation value of a playing volume of the first device reaching the user and a second volume attenuation value of a playing volume of the second device reaching the user according to the first distance and the second distance. The hub device determines first volume information for the first device and second volume information for the second device according to the first volume attenuation value and the second volume attenuation value.

24. The communication system of claim 23, wherein, The central device determines, according to the first distance and the second distance, a first volume attenuation value of a playing volume of the first device to the user and a second volume attenuation value of a playing volume of the second device to the user, including: A p = K - DL w +A e , wherein A p is a volume decay value; K is a distance decay factor, representing volume decay caused by distance, related to the distance between the device and the user; DL w is a directivity factor, representing volume decay caused by the direction of the audio signal emission; A e is an air absorption decay, representing volume decay caused by air absorption of the audio when propagating in air.

25. The communication system of claim 1, wherein, The R second electronic devices include a first device; the central device acquires that the first device is located in a first room or a first area, the first room or the first area has a first user, and has no second user; and the central device further acquires that the first device enters a private mode and plays a first video. When detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device. In response to receiving the first notification message, the first device automatically stops playing and automatically switches to a screen saver. When detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device. In response to receiving the second notification message, the first device automatically resumes playing the first video.

26. The communication system of claim 25, wherein After the first device automatically stops playing and automatically switches to the screen saver, the first device sends a first feedback message to the central device. In response to receiving the first feedback message, the central device learns that the first device has stopped playing and switched to the screen saver.

27. The communication system of claim 25 or 26, wherein After the first device automatically resumes playing the first video, the first device sends a second feedback message to the central device. In response to receiving the second feedback message, the central device learns that the first device has resumed playing.

28. The communication system of claim 1, wherein, The R second electronic devices include a first device; the central device acquires that the first device is located in a first room or a first area, the first room or the first area has a first user, and has no second user; and the central device further acquires that the first device enters a private mode and plays a first video. When detecting that a user other than the first user enters the first room or the first area, the central device sends a first notification message to the first device. In response to receiving the first notification message, the first device automatically stops playing and automatically switches to a screen saver. When detecting that the first room or the first area has no user other than the first user, the central device sends a second notification message to the first device. In response to receiving the second notification message, the first device automatically resumes playing the first video.

29. The communication system of claim 1, wherein, The R second electronic devices include a first device; the central device acquires that the first device plays a first video, and further acquires a first position of the first device, that the first device is located in a first room or a first area, a second position of a user in the first room or the first area, and a first viewing area corresponding to the first device; In response to detecting that the first viewing area has no user within a first preset time length, the central device sends a first notification message to the first device. In response to receiving the first notification message, the first device stops playing the first video; In response to detecting that there is a user in the first viewing area within a second preset time length, the hub device sends a second notification message to the first device; In response to receiving the second notification message, the first device resumes playing the first video.

30. The communication system of claim 29, wherein, In response to detecting that the distance between at least one user and the first device is less than a first preset distance within a third preset time length, the hub device sends a third notification message to the first device; In response to receiving the third notification message, the first device stops playing the first video and outputs reminder information; In response to detecting that there is no user with a distance less than the first preset distance from the first device within a fourth preset time length, and there is a user in the first viewing area, the hub device sends a fourth notification message to the first device; In response to receiving the fourth notification message, the first device resumes playing the first video.

31. The communication system of claim 30, wherein, In response to detecting that there is no user with a distance less than the first preset distance from the first device within a fifth preset time length, and there is no user in the first viewing area, the hub device sends a fifth notification message to the first device; In response to receiving the fifth notification message, the first device automatically hibernates.

32. The communication system of claim 1, wherein, The first electronic device includes an inertial measurement unit module; the R second electronic devices include a first device, and the first device does not include an ultra-wideband module; Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device, including: The first ultra-wideband module establishes a first coordinate system, and based on the position measurement of the mobile device by the first electronic device and the labeling of the first device by the mobile device, the first electronic device obtains a first coordinate of the first device in the first coordinate system; The millimeter wave radar module establishes a second coordinate system, and based on the perception of the human body by the first electronic device, the first electronic device obtains a second coordinate of the user in the second coordinate system; Through conversion between the first coordinate system and the second coordinate system, the first electronic device obtains a third coordinate of the second coordinate in the first coordinate system; The first electronic device sends the related coordinates in the first coordinate system to the hub device, and the related coordinates include the first coordinate and the third coordinate; Based on the output of the inertial measurement unit module, in combination with the parallel of the whole-house coordinate system and the geographic coordinate system, the hub device converts the related coordinates in the first coordinate system into coordinates in the whole-house coordinate system, and further obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system.

33. A method for automatic control based on human perception, applied to a communication system based on human perception, characterized by, The system comprises a hub device, a first electronic device and R second electronic devices; any two of the hub device, the first electronic device and any one of the R second electronic devices communicate in a wired or wireless manner; the first electronic device comprises a first ultra-wideband module and a millimeter wave radar module; the R second electronic devices comprise a mobile device, and the mobile device comprises a second ultra-wideband module; R is a positive integer greater than or equal to 1; the method comprises: Based on the position measurement of the first electronic device on the mobile device, the measurement and conversion of the human body position, the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device; according to the position information of the user and the position information of the R second electronic devices, the hub device controls or notifies at least one of the R second electronic devices to perform a preset operation; wherein the first electronic device measures the position of the mobile device through the first ultra-wideband module, and obtains the position information of the R second electronic devices in the coordinate system provided by the first ultra-wideband module; the first electronic device measures the position of the human body through the millimeter wave radar module, and obtains the position information of the user in the coordinate system provided by the millimeter wave radar module; based on the position information of the R second electronic devices in the coordinate system provided by the first ultra-wideband module, the hub device obtains the position information of the R second electronic devices in the whole-house coordinate system provided by the hub device; based on the position information of the user in the coordinate system provided by the millimeter wave radar module, the hub device obtains the position information of the user in the whole-house coordinate system provided by the hub device; In response to the control or notification of the hub device, the at least one second electronic device performs the preset operation; The at least one second electronic device performing the preset operation comprises: The second electronic device closest to the user is woken up; or, The first device automatically stops playing the first video or the first audio, and the second device continues to play the first video or the first audio; the at least one second electronic device comprises the first device and the second device, and the distance between the second device and the user is less than the distance between the first device and the user; or, The first device plays according to first volume information, and the second device plays according to second volume information; the at least one second electronic device includes the first device and the second device, the first volume information is obtained according to a first distance between the first device and the user, and the second volume information is obtained according to a second distance between the second device and the user; or, The first device in the at least one second electronic device automatically stops playing and automatically switches to a screen saver or automatically resumes playing the first video; or The first device in the at least one second electronic device stops playing the first video or resumes playing the first video.

34. The method of claim 33, wherein, Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device. Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device periodically or in real time obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device.

35. The method of claim 33 or 34, wherein, When the user is one user, the position information of the user is the position information of the one user; when the user is multiple users, the position information of the user is the central position information of the multiple users.

36. The method of claim 33, wherein, T second electronic devices in the R second electronic devices support voice wake-up. In response to receiving a wake-up voice, the T second electronic devices respectively send a first message to the hub device. In response to receiving at least one first message, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system. The hub device determines a device closest to the user from the T second electronic devices. The hub device sends a first indication message to the device closest to the user. In response to receiving the first indication message, the device closest to the user wakes up. T is a positive integer greater than or equal to 1 and less than or equal to R.

37. The method of claim 36, wherein, In response to receiving at least one first message, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system, including: In response to receiving one first message, the hub device obtains the position information of the second electronic device sending the first message and the first room or first area where the second electronic device is located in the whole-house coordinate system; and the hub device further obtains the position information of the user in the first room or the first area in the whole-house coordinate system.

38. The method of claim 37, wherein, Based on the communication between the hub device and T second electronic devices in the R second electronic devices, the hub device pre-obtains that the T second electronic devices support voice wake-up. T is a positive integer greater than or equal to 1 and less than or equal to R; the user is one user or multiple users.

39. The method of claim 38, wherein, When the user is a plurality of users, the device closest to the user is the device closest to the center position of the plurality of users.

40. The method of claim 33, wherein, The R second electronic devices include a first device and a second device, and the first device and the second device support playing a video or audio; the user includes a first user; In response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; After obtaining that the second device is a playing device closest to the first user, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including identification and playing progress information of the first video or the first audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identification and the playing progress information of the first video or the first audio; In response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information.

41. The method of claim 40, wherein, Obtaining that the second device is a playing device closest to the first user includes: Among devices that do not play a video or audio, the second device is a playing device closest to the first user.

42. The method of claim 33, wherein, The R second electronic devices include a first device and a second device, and the first device and the second device support playing a video or audio; the user includes a first user and a second user; In response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; In response to an operation of the second user, the first device starts playing a second video or a second audio, and sends a second notification message to the hub device; In response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; After obtaining that the second device is a playing device closest to the second user, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message including identification and playing progress information of the second video or the second audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message including the identification and the playing progress information of the second video or the second audio; In response to receiving the second notification message, the second device continues playing the second video or the second audio according to the playing progress information.

43. The method of claim 42, wherein, It is acquired that the second device is the closest playing device to the second user among the devices that do not play the video or the audio. It is acquired that the second device is the closest playing device to the second user among the devices that do not play the video or the audio.

44. The method of claim 33, wherein, The R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user, and the first user is located in the first room or the first area; In response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; After it is acquired that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further sends a second notification message to the second device, the second notification message including an identifier of the first video or the first audio and playing progress information of the first video or the first audio; In response to receiving the second notification message, the second device continues playing the first video or the first audio according to the playing progress information. After it is acquired that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the first user. After it is acquired that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further acquires that the second device is the closest playing device to the first user among the devices that do not play the video or the audio.

45. The method of claim 44, wherein, The R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user and a second user, and the first user and the second user are located in the first room or the first area; 46. The method of claim 45, wherein, In response to an operation of the first user, the first device starts playing a first video or a first audio, and sends a first notification message to the hub device; 47. The method of claim 33, wherein, In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; ​ ​ In response to the operation of the second user, the first device starts playing a second video or a second audio, and sends a second notification message to the hub device; In response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device automatically stops playing, and sends a first feedback message to the hub device, the first feedback message comprising an identifier and playing progress information of the second video or the second audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message comprising the identifier and the playing progress information of the second video or the second audio; In response to receiving the second notification message, the second device continues playing the second video or the second audio according to the playing progress information.

48. The method of claim 47, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play videos or audios.

49. The method of claim 48, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among the devices that do not play videos or audios.

50. The method of claim 33, wherein, The R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a left channel of stereo sound, and the second device playing a right channel of stereo sound; The hub device obtains that the first device and the second device are located in a first room or a first area, a first distance between the first device and the user, and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; The hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information, and the second playing message comprising the second volume information; In response to receiving the first playing message, the first device plays according to the first volume information; In response to receiving the second playing message, the second device plays according to the second volume information.

51. The method of claim 50, wherein, The stereo system further comprises a third device, the third device playing auxiliary sound of stereo sound; the stereo sound comprises the left channel of stereo sound, the right channel of stereo sound, and the auxiliary sound of stereo sound; The hub device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user; The central device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance, including: the central device determines first volume information for the first device, second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; The central device sends a first play message and a second play message to the first device and the second device respectively, including: the central device sends a first play message, a second play message and a third play message to the first device, the second device and the third device respectively, and the third play message includes third volume information; In response to receiving the third play message, the third device plays according to the third volume information.

52. The method of claim 33, wherein, The R second electronic devices include a stereo system, the stereo system includes a first device and a second device, the first device plays a left channel of stereo sound, and the second device plays a right channel of stereo sound; The central device obtains that the first device is located in a first area, the second device is located in a second area, and the first area is adjacent to the second area; the central device further obtains a first distance between the first device and the user and a second distance between the second device and the user; the central device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; The central device sends a first play message and a second play message to the first device and the second device respectively, the first play message includes first volume information, and the second play message includes second volume information; In response to receiving the first play message, the first device plays according to the first volume information; In response to receiving the second play message, the second device plays according to the second volume information.

53. The method of claim 52, wherein, The stereo system further includes a third device, and the third device plays auxiliary sound of stereo sound; the stereo sound includes a left channel of stereo sound, a right channel of stereo sound and auxiliary sound of stereo sound; The central device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; The central device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance, including: the central device determines first volume information for the first device, second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; The central device sends a first play message and a second play message to the first device and the second device respectively, comprising: the central device sends a first play message, a second play message and a third play message to the first device, the second device and the third device respectively, and the third play message comprises third volume information; In response to receiving the third play message, the third device plays according to the third volume information.

54. The method of any of claims 50-53, wherein, The central device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance, comprising: The central device determines a first volume attenuation value of the first device and a second volume attenuation value of the second device according to the first distance and the second distance. The central device determines first volume information for the first device and second volume information for the second device according to the first volume attenuation value and the second volume attenuation value.

55. The method of claim 54, wherein, The central device determines a first volume attenuation value of the first device and a second volume attenuation value of the second device according to the first distance and the second distance, comprising: A p = K - DL w +A e , wherein A p is a volume decay value; K is a distance decay factor representing volume decay caused by distance, which is related to the distance between the device and the user; DL w is a directivity factor representing volume decay caused by the direction of the audio signal emission; A e is an air absorption decay representing volume decay caused by air absorbing the audio when the audio propagates in the air.

56. The method of claim 33, wherein, The R second electronic devices include a first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user, and has no second user; and the central device further obtains that the first device enters a private mode and plays a first video. When detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device. In response to receiving the first notification message, the first device automatically stops playing and automatically switches to a screen saver. When detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device. In response to receiving the second notification message, the first device automatically resumes playing the first video.

57. The method of claim 56, wherein, After the first device automatically stops playing and automatically switches to a screen saver, the first device sends a first feedback message to the central device. In response to receiving the first feedback message, the central device learns that the first device has stopped playing and switched to a screen saver.

58. The method of claim 56 or 57, wherein, After the first device automatically resumes playing the first video, the first device sends a second feedback message to the central device. In response to receiving the second feedback message, the central device learns that the first device has resumed playing.

59. The method of claim 33, wherein, The R second electronic devices include a first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user, and has no second user; and the central device further obtains that the first device enters a private mode and plays a first video. In response to detecting that a user other than the first user enters the first room or the first area, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device automatically stops playing and automatically switches to a screen saver; In response to detecting that the first room or the first area is free of users other than the first user, the hub device sends a second notification message to the first device; In response to receiving the second notification message, the first device automatically resumes playing the first video.

60. The method of claim 33, wherein, The R second electronic devices include a first device; the hub device obtains that the first device is playing a first video, the hub device also obtains a first position of the first device, that the first device is located in a first room or a first area, a second position of a user in the first room or the first area, and a first viewing area corresponding to the first device; In response to detecting that the first viewing area is free of users within a first preset time period, the hub device sends a first notification message to the first device; In response to receiving the first notification message, the first device stops playing the first video; In response to detecting that the first viewing area has a user within a second preset time period, the hub device sends a second notification message to the first device; In response to receiving the second notification message, the first device resumes playing the first video.

61. The method of claim 60, wherein, In response to detecting that a distance between at least one user and the first device is less than a first preset distance within a third preset time period, the hub device sends a third notification message to the first device; In response to receiving the third notification message, the first device stops playing the first video and outputs a reminder message; In response to detecting that no user has a distance less than the first preset distance from the first device within a fourth preset time period, and that the first viewing area has a user, the hub device sends a fourth notification message to the first device; In response to receiving the fourth notification message, the first device resumes playing the first video.

62. The method of claim 61, wherein, In response to detecting that no user has a distance less than the first preset distance from the first device within a fifth preset time period, and that the first viewing area is free of users, the hub device sends a fifth notification message to the first device; In response to receiving the fifth notification message, the first device automatically hibernates.

63. The method of claim 33, wherein, The first electronic device includes an inertial measurement unit module; the R second electronic devices include a first device, and the first device does not include an ultra-wideband module; Based on the position measurement of the first electronic device on the mobile device, the measurement and conversion of the human position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in a whole-house coordinate system provided by the hub device, including: The first UWB module establishes a first coordinate system, and based on the position measurement of the mobile device by the first electronic device and the marking of the first device by the mobile device, the first electronic device obtains a first coordinate of the first device in the first coordinate system; The millimeter wave radar module establishes a second coordinate system, and based on the perception of the human body by the first electronic device, the first electronic device obtains a second coordinate of the user in the second coordinate system; Through the conversion between the first coordinate system and the second coordinate system, the first electronic device obtains a third coordinate of the second coordinate in the first coordinate system; The first electronic device sends the related coordinates in the first coordinate system to the hub device, and the related coordinates include the first coordinate and the third coordinate; Based on the output of the inertial measurement unit module and in combination with the parallel relationship between the whole-house coordinate system and the geographic coordinate system, the hub device converts the related coordinates in the first coordinate system into coordinates in the whole-house coordinate system, and further obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system.

64. A hub device, comprising: Any two of the hub device, the first electronic device, and any one of the R second electronic devices communicate in a wired or wireless manner; the first electronic device includes a first UWB module and a millimeter wave radar module; the R second electronic devices include a mobile device, and the mobile device includes a second UWB module; R is a positive integer greater than or equal to 1; Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device; wherein, based on the position information of the R second electronic devices in the coordinate system provided by the first UWB module, the hub device obtains the position information of the R second electronic devices in the whole-house coordinate system provided by the hub device; based on the position information of the user in the coordinate system provided by the millimeter wave radar module, the hub device obtains the position information of the user in the whole-house coordinate system provided by the hub device; the position information of the R second electronic devices in the coordinate system provided by the first UWB module is obtained by the first electronic device through the position measurement of the mobile device by the first UWB module, and the position information of the user in the coordinate system provided by the millimeter wave radar module is obtained by the first electronic device through the measurement of the human body position by the millimeter wave radar module; According to the position information of the user and the position information of the R second electronic devices, the hub device controls or notifies at least one of the R second electronic devices to perform a preset operation; The at least one second electronic device performing the preset operation includes: The second electronic device closest to the user wakes up; or The second electronic device closest to the user wakes up; or The first device automatically stops playing the first video or the first audio, and the second device continues playing the first video or the first audio; the at least one second electronic device includes the first device and the second device, and the second device is closer to the user than the first device is to the user; or The first device plays according to first volume information, and the second device plays according to second volume information; the at least one second electronic device includes the first device and the second device, the first volume information is obtained according to a first distance between the first device and the user, and the second volume information is obtained according to a second distance between the second device and the user; or The first device in the at least one second electronic device automatically stops playing and automatically switches to a screen saver or automatically resumes playing the first video; or The first device in the at least one second electronic device stops playing the first video or resumes playing the first video.

65. The hub device of claim 64, wherein, Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains position information of the R second electronic devices and position information of the user in a whole-house coordinate system provided by the hub device. Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device periodically or in real time obtains position information of the R second electronic devices and position information of the user in a whole-house coordinate system provided by the hub device.

66. The hub device of claim 64 or 65, wherein, When the user is one user, the position information of the user is the position information of the one user; when the user is multiple users, the position information of the user is the central position information of the multiple users.

67. The hub device of claim 64, wherein, T second electronic devices in the R second electronic devices support voice wake-up; In response to receiving a first message from at least one of the T second electronic devices, the hub device obtains position information of the T second electronic devices and position information of the user in the whole-house coordinate system. The first message is used to indicate that a second electronic device receives a wake-up voice. The hub device determines a device closest to the user from the T second electronic devices. The hub device sends a first indication message to the device closest to the user, and the first indication message is used to instruct the device closest to the user to wake up. T is a positive integer greater than or equal to 1 and less than or equal to R.

68. The hub device of claim 67, wherein, In response to receiving a first message from at least one of the T second electronic devices, the hub device obtains position information of the T second electronic devices and position information of the user in the whole-house coordinate system, including: In response to receiving the first message from one of the T second electronic devices, the hub device acquires position information of the second electronic device sending the first message and a first room or a first area where the second electronic device is located in the whole-house coordinate system; and the hub device further acquires position information of a user in the first room or the first area in the whole-house coordinate system.

69. The hub device of claim 68, wherein, Based on the communication between the hub device and the T second electronic devices of the R second electronic devices, the hub device pre-acquires that the T second electronic devices support voice wake-up. T is a positive integer greater than or equal to 1 and less than or equal to R; and the user is one user or multiple users.

70. The hub device of claim 64, wherein, The R second electronic devices include a first device and a second device, and the first device and the second device support playing video or audio; and the user includes a first user. The hub device receives a first notification message from the first device, and the first notification message is used to notify the first device to start playing a first video or a first audio. In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio. After acquiring that the second device is the closest playing device to the first user, the hub device sends a first notification message to the first device, and the first notification message is used to instruct the first device to stop playing. The hub device receives a first feedback message from the first device, and the first feedback message includes an identifier and playing progress information of the first video or the first audio. In response to receiving the first feedback message, the hub device sends a second notification message to the second device, and the second notification message is used to instruct the second device to play the first video or the first audio, and the second notification message includes the identifier and the playing progress information of the first video or the first audio.

71. The hub device of claim 70, wherein, Acquiring that the second device is the closest playing device to the first user includes: Among devices that do not play video or audio, acquiring that the second device is the closest playing device to the first user.

72. The hub device of claim 64, wherein, The R second electronic devices include a first device and a second device, and the first device and the second device support playing video or audio; and the user includes a first user and a second user. The hub device receives a first notification message from the first device, and the first notification message is used to notify the first device to start playing a first video or a first audio. In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio. The hub device receives a second notification message from the first device, and the second notification message is used to notify the first device to start playing a second video or a second audio. In response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio. After obtaining that the second device is the closest playing device to the second user, the hub device sends a first notification message to the first device, the first notification message being used to instruct the first device to stop playing; The hub device receives a first feedback message from the first device, the first feedback message comprising an identifier and playing progress information of the second video or the second audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message being used to instruct the second device to play the second video or the second audio, the second notification message comprising the identifier and the playing progress information of the second video or the second audio.

73. The hub device of claim 72, wherein, Obtaining that the second device is the closest playing device to the second user comprises: Obtaining that, among devices that do not play video or audio, the second device is the closest playing device to the second user.

74. The hub device of claim 64, wherein, The R second electronic devices comprise a first device and a second device, the first device and the second device supporting playing video or audio, the first device being located in a first room or a first area, and the second device being located in a second room or a second area; the user comprises a first user, the first user being located in the first room or the first area; The hub device receives a first notification message from the first device, the first notification message being used to instruct the first device to stop playing; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; After obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device, the first notification message being used to instruct the first device to stop playing; The hub device receives a first feedback message from the first device, the first feedback message comprising an identifier and playing progress information of the first video or the first audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message being used to instruct the second device to play the first video or the first audio, the second notification message comprising the identifier and the playing progress information of the first video or the first audio.

75. The hub device of claim 74, wherein, After obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the first user.

76. The hub device of claim 75, wherein, After obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that, among devices that do not play video or audio, the second device is the closest playing device to the first user.

77. The hub device of claim 64, wherein, The R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the user includes a first user and a second user, and the first user and the second user are located in the first room or the first area; The hub device receives a first notification message from the first device, and the first notification message is used to inform the first device to start playing a first video or a first audio; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; The hub device receives a second notification message from the first device, and the second notification message is used to inform the first device to start playing a second video or a second audio; In response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device sends a first notification message to the first device, and the first notification message is used to instruct the first device to stop playing; The hub device receives a first feedback message from the first device, and the first feedback message includes identification and playing progress information of the second video or the second audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, and the second notification message is used to instruct the second device to play the second video or the second audio, and the second notification message includes the identification and the playing progress information of the second video or the second audio.

78. The hub device of claim 77, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among devices that do not play a video or an audio.

79. The hub device of claim 78, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among devices that do not play a video or an audio.

80. The hub device of claim 64, wherein, The R second electronic devices include a stereo system, the stereo system includes a first device and a second device, the first device plays a left channel of stereo sound, and the second device plays a right channel of stereo sound; The hub device obtains that the first device and the second device are located in a first room or a first area, a first distance between the first device and the user, and a second distance between the second device and the user; and the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance. The central device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising first volume information, and the second playing message comprising second volume information; the first volume information is used to indicate the playing volume of the first device, and the second volume information is used to indicate the playing volume of the second device.

81. The hub device of claim 80, wherein, The stereo system further comprises a third device playing a stereo auxiliary sound; the stereo sound comprises a stereo left sound, a stereo right sound and a stereo auxiliary sound; The central device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user; The central device determines the first volume information for the first device and the second volume information for the second device according to the first distance and the second distance, comprising: the central device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; The central device sends a first playing message and a second playing message to the first device and the second device respectively, comprising: the central device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message comprising third volume information; the third volume information is used to indicate the playing volume of the third device.

82. The hub device of claim 64, wherein, The R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a stereo left sound, and the second device playing a stereo right sound; The central device obtains that the first device is located in a first area, the second device is located in a second area, and the first area is adjacent to the second area; the central device further obtains a first distance between the first device and the user, and a second distance between the second device and the user; the central device determines the first volume information for the first device and the second volume information for the second device according to the first distance and the second distance; The central device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising first volume information, and the second playing message comprising second volume information; the first volume information is used to indicate the playing volume of the first device, and the second volume information is used to indicate the playing volume of the second device.

83. The hub device of claim 82, wherein, The stereo system further comprises a third device playing a stereo auxiliary sound; the stereo sound comprises a stereo left sound, a stereo right sound and a stereo auxiliary sound; The central device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; The central device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; The central device determines the first volume information for the first device and the second volume information for the second device according to the first distance and the second distance, comprising: the central device determines the first volume information for the first device, the second volume information for the second device and the third volume information for the third device according to the first distance, the second distance and the third distance; The central device sends the first playing message and the second playing message to the first device and the second device respectively, comprising: the central device sends the first playing message, the second playing message and the third playing message to the first device, the second device and the third device respectively, and the third playing message comprises the third volume information; the third volume information is used to indicate the playing volume of the third device.

84. The hub device of any of claims 80-83, wherein, The central device determines the first volume information for the first device and the second volume information for the second device according to the first distance and the second distance, comprising: The central device determines the first volume attenuation value of the playing volume of the first device reaching the user and the second volume attenuation value of the playing volume of the second device reaching the user according to the first distance and the second distance; The central device determines the first volume information for the first device and the second volume information for the second device according to the first volume attenuation value and the second volume attenuation value.

85. The hub device of claim 84, wherein, The central device determines the first volume attenuation value of the playing volume of the first device reaching the user and the second volume attenuation value of the playing volume of the second device reaching the user according to the first distance and the second distance, comprising: A p = K - DL w +A e , wherein A p is a volume decay value; K is a distance decay factor, representing volume decay caused by distance, related to the distance between the device and the user; DL w is a directivity factor, representing volume decay caused by the direction of the audio signal emission; A e is an air absorption decay, representing volume decay caused by air absorption of the audio when the audio propagates in the air.

86. The hub device of claim 64, wherein, The R second electronic devices comprise a first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user and has no second user; the central device further obtains that the first device enters a private mode and plays a first video; When detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to a screen saver; When detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

87. The central device of claim 86, wherein In response to receiving a first feedback message from the first device, the central device learns that the first device has stopped playing and switched to a screen saver.

88. The central device of claim 86 or 87, wherein In response to receiving a second feedback message from the first device, the central device learns that the first device has resumed playing.

89. The hub device of claim 64, wherein, The R second electronic devices include a first device; the hub device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user, and has no second user; the hub device further obtains that the first device enters a private mode and plays a first video; In response to detecting that a user other than the first user enters the first room or the first area, the hub device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to a screensaver; In response to detecting that the first room or the first area has no user other than the first user, the hub device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

90. The hub device of claim 64, wherein, The R second electronic devices include a first device; the hub device obtains that the first device plays a first video, and further obtains a first position of the first device, that the first device is located in a first room or a first area, a second position of a user in the first room or the first area, and a first viewing area corresponding to the first device; In response to detecting that the first viewing area has no user within a first preset time length, the hub device sends a first notification message to the first device; The first notification message is used to instruct the first device to stop playing the first video; In response to detecting that the first viewing area has a user within a second preset time length, the hub device sends a second notification message to the first device; The second notification message is used to instruct the first device to resume playing the first video.

91. The hub device of claim 90, wherein, In response to detecting that a distance between at least one user and the first device is less than a first preset distance within a third preset time length, the hub device sends a third notification message to the first device; the third notification message is used to instruct the first device to stop playing the first video and output a reminder message; In response to detecting that no user has a distance less than the first preset distance from the first device within a fourth preset time length, and that the first viewing area has a user, the hub device sends a fourth notification message to the first device; the fourth notification message is used to instruct the first device to resume playing the first video.

92. The hub device of claim 64, wherein, In response to detecting that no user has a distance less than the first preset distance from the first device within a fifth preset time length, and that the first viewing area has no user, the hub device sends a fifth notification message to the first device; the fifth notification message is used to instruct the first device to automatically hibernate.

93. A human perception-based automatic control method applied to a central device, comprising: The central device communicates with any two of the first electronic device, the R second electronic devices and the central device in wired communication or wireless communication; the first electronic device comprises a first ultra-wideband module and a millimeter wave radar module; the R second electronic devices comprise a mobile device, and the mobile device comprises a second ultra-wideband module; R is a positive integer greater than or equal to 1; the method comprises: Based on the position measurement of the first electronic device on the mobile device, the measurement and conversion of the human body position, the communication between the central device and the first electronic device, the central device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the central device; wherein, based on the position information of the R second electronic devices in the coordinate system provided by the first ultra-wideband module, the central device obtains the position information of the R second electronic devices in the whole-house coordinate system provided by the central device; based on the position information of the user in the coordinate system provided by the millimeter wave radar module, the central device obtains the position information of the user in the whole-house coordinate system provided by the central device; the position information of the R second electronic devices in the coordinate system provided by the first ultra-wideband module is obtained by the first electronic device through the first ultra-wideband module to measure the position of the mobile device, and the position information of the user in the coordinate system provided by the millimeter wave radar module is obtained by the first electronic device through the millimeter wave radar module to measure the position of the human body; According to the position information of the user and the position information of the R second electronic devices, the central device controls or notifies at least one second electronic device of the R second electronic devices to perform a preset operation; The at least one second electronic device performing the preset operation comprises: The second electronic device closest to the user wakes up; or, The first device automatically stops playing the first video or the first audio, and the second device continues to play the first video or the first audio; the at least one second electronic device comprises the first device and the second device, and the distance between the second device and the user is less than the distance between the first device and the user; or, The first device plays according to the first volume information, and the second device plays according to the second volume information; the at least one second electronic device comprises the first device and the second device, the first volume information is obtained according to the first distance between the first device and the user, and the second volume information is obtained according to the second distance between the second device and the user; or, The first device in the at least one second electronic device automatically stops playing and automatically switches to a screensaver or automatically resumes playing the first video; or, The first device in the at least one second electronic device stops playing the first video or resumes playing the first video.

94. The method of claim 93, wherein, Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device, including: Based on the position measurement of the mobile device by the first electronic device, the measurement and conversion of the human body position, and the communication between the hub device and the first electronic device, the hub device periodically or in real time obtains the position information of the R second electronic devices and the position information of the user in the whole-house coordinate system provided by the hub device.

95. The method of claim 93 or 94, wherein, When the user is one user, the position information of the user is the position information of the one user; when the user is multiple users, the position information of the user is the central position information of the multiple users.

96. The method of claim 93, wherein, T second electronic devices in the R second electronic devices support voice wake-up; In response to receiving a first message from at least one of the T second electronic devices, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system; The first message is used to indicate that a second electronic device receives a wake-up voice; The hub device determines a device closest to the user from the T second electronic devices; The hub device sends a first indication message to the device closest to the user, and the first indication message is used to instruct the device closest to the user to wake up; Wherein, T is a positive integer greater than or equal to 1 and less than or equal to R.

97. The method of claim 96, wherein, In response to receiving a first message from at least one of the T second electronic devices, the hub device obtains the position information of the T second electronic devices and the position information of the user in the whole-house coordinate system, including: In response to receiving a first message from one of the T second electronic devices, the hub device obtains the position information of the second electronic device sending the first message and the first room or first area where it is located in the whole-house coordinate system; and the hub device further obtains the position information of the user in the first room or the first area in the whole-house coordinate system.

98. The method of claim 97, wherein, Based on the communication between the hub device and T second electronic devices in the R second electronic devices, the hub device pre-obtains that the T second electronic devices support voice wake-up; T is a positive integer greater than or equal to 1 and less than or equal to R; the user is one user or multiple users.

99. The method of claim 93, wherein, The R second electronic devices include a first device and a second device, and the first device and the second device support playing video or audio; the user includes a first user; The hub device receives a first notification message from the first device, and the first notification message is used to notify the first device to start playing a first video or a first audio; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; After obtaining that the second device is the closest playing device to the first user, the hub device sends a first notification message to the first device, the first notification message being used to instruct the first device to stop playing; The hub device receives a first feedback message from the first device, the first feedback message comprising an identifier and playing progress information of the first video or the first audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message being used to instruct the second device to play the first video or the first audio, the second notification message comprising the identifier and the playing progress information of the first video or the first audio.

100. The method of claim 99, wherein, Obtaining that the second device is the closest playing device to the first user comprises: Among devices that have not played the video or the audio, the second device is the closest playing device to the first user.

101. The method of claim 93, wherein, The R second electronic devices comprise a first device and a second device, the first device and the second device supporting playing of a video or an audio; the users comprise a first user and a second user; The hub device receives a first notification message from the first device, the first notification message being used to instruct the first device to stop playing; In response to receiving the first notification message, the hub device establishes a binding relationship between the first user and the first video or the first audio; The hub device receives a second notification message from the first device, the second notification message being used to instruct the first device to start playing a second video or a second audio; In response to receiving the second notification message, the hub device establishes a binding relationship between the second user and the second video or the second audio; After obtaining that the second device is the closest playing device to the second user, the hub device sends a first notification message to the first device, the first notification message being used to instruct the first device to stop playing; The hub device receives a first feedback message from the first device, the first feedback message comprising an identifier and playing progress information of the second video or the second audio; In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message being used to instruct the second device to play the second video or the second audio, the second notification message comprising the identifier and the playing progress information of the second video or the second audio.

102. The method of claim 101, wherein, Obtaining that the second device is the closest playing device to the second user comprises: Among devices that have not played the video or the audio, the second device is the closest playing device to the second user.

103. The method of claim 93, wherein, The R second electronic devices comprise a first device and a second device, the first device and the second device supporting playing of a video or an audio, the first device being located in a first room or a first area, the second device being located in a second room or a second area; the users comprise a first user, the first user being located in the first room or the first area; The central device receives a first notification message from the first device, the first notification message being used to inform the first device to start playing a first video or a first audio; In response to receiving the first notification message, the central device establishes a binding relationship between the first user and the first video or the first audio; After obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the central device further obtains that the second device is the closest playing device to the first user among the devices that do not play a video or an audio. After obtaining that the first user leaves the first room or the first area and enters the second room or the second area, the central device further obtains that the second device is the closest playing device to the first user among the devices that do not play a video or an audio. The R second electronic devices include a first device and a second device, the first device and the second device support playing a video or an audio, the first device is located in a first room or a first area, and the second device is located in a second room or a second area; the users include a first user and a second user, and the first user and the second user are located in the first room or the first area; 104. The method of claim 103, wherein, The central device receives a first notification message from the first device, the first notification message being used to inform the first device to start playing a first video or a first audio; 105. The method of claim 104, wherein, In response to receiving the first notification message, the central device establishes a binding relationship between the first user and the first video or the first audio; 106. The method of claim 93, wherein, The central device receives a second notification message from the first device, the second notification message being used to inform the first device to start playing a second video or a second audio; In response to receiving the second notification message, the central device establishes a binding relationship between the second user and the second video or the second audio; After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the central device sends a first notification message to the first device, the first notification message being used to instruct the first device to stop playing; The central device receives a first feedback message from the first device, the first feedback message including identification and playing progress information of the second video or the second audio; In response to receiving the first feedback message, the central device sends a second notification message to the second device, the second notification message being used to instruct the second device to play the first video or the first audio, and the second notification message includes the identification and the playing progress information of the first video or the first audio. ​ ​ In response to receiving the first feedback message, the hub device sends a second notification message to the second device, the second notification message being used to instruct the second device to play the second video or the second audio, the second notification message comprising an identification of the second video or the second audio and the playing progress information.

107. The method of claim 106, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among devices that do not play video or audio.

108. The method of claim 107, wherein, After obtaining that the second user leaves the first room or the first area and enters the second room or the second area, the hub device further obtains that the second device is the closest playing device to the second user among devices that do not play video or audio.

109. The method of claim 93, wherein, The R second electronic devices comprise a stereo system, the stereo system comprising a first device and a second device, the first device playing a left channel of stereo sound, and the second device playing a right channel of stereo sound; The hub device obtains that the first device and the second device are located in a first room or a first area, a first distance between the first device and the user, and a second distance between the second device and the user; and the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance. The hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message comprising the first volume information, and the second playing message comprising the second volume information; the first volume information being used to instruct a playing volume of the first device, and the second volume information being used to instruct a playing volume of the second device.

110. The method of claim 109, wherein, The stereo system further comprises a third device, the third device playing a stereo auxiliary sound; the stereo sound comprising a left channel of stereo sound, a right channel of stereo sound, and a stereo auxiliary sound; The hub device further obtains that the third device is located in the first room or the first area, and a third distance between the third device and the user. The hub device determines the first volume information for the first device and the second volume information for the second device according to the first distance and the second distance, comprising: the hub device determines the first volume information for the first device, the second volume information for the second device, and third volume information for the third device according to the first distance, the second distance, and the third distance. The hub device sends the first playing message and the second playing message to the first device and the second device respectively, comprising: the hub device sends a first playing message, a second playing message, and a third playing message to the first device, the second device, and the third device respectively, the third playing message comprising third volume information; the third volume information being used to instruct a playing volume of the third device.

111. The method of claim 93, wherein, The R second electronic devices include a stereo system, the stereo system includes a first device and a second device, the first device plays a stereo left channel, and the second device plays a stereo right channel; The hub device obtains that the first device is located in a first area, the second device is located in a second area, and the first area is adjacent to the second area; the hub device further obtains a first distance between the first device and the user and a second distance between the second device and the user; the hub device determines first volume information for the first device and second volume information for the second device according to the first distance and the second distance; The hub device sends a first playing message and a second playing message to the first device and the second device respectively, the first playing message includes the first volume information, and the second playing message includes the second volume information; the first volume information is used to indicate a playing volume of the first device, and the second volume information is used to indicate a playing volume of the second device.

112. The method of claim 111, wherein, The stereo system further includes a third device, the third device plays a stereo auxiliary sound; the stereo system includes a stereo left channel, a stereo right channel and a stereo auxiliary sound; The hub device further obtains that the third device is located in the first area, the second area or a third area adjacent to the second area, and a third distance between the third device and the user; The hub device determines the first volume information for the first device and the second volume information for the second device according to the first distance and the second distance, including: the hub device determines the first volume information for the first device, the second volume information for the second device and third volume information for the third device according to the first distance, the second distance and the third distance; The hub device sends a first playing message and a second playing message to the first device and the second device respectively, including: the hub device sends a first playing message, a second playing message and a third playing message to the first device, the second device and the third device respectively, the third playing message includes third volume information; the third volume information is used to indicate a playing volume of the third device.

113. The method of any of claims 109-112, wherein, The hub device determines the first volume information for the first device and the second volume information for the second device according to the first distance and the second distance, including: The hub device determines a first volume attenuation value of a playing volume of the first device reaching the user and a second volume attenuation value of a playing volume of the second device reaching the user according to the first distance and the second distance; The hub device determines the first volume information for the first device and the second volume information for the second device according to the first volume attenuation value and the second volume attenuation value.

114. The method of claim 113, wherein, The central device determines, according to the first distance and the second distance, a first volume attenuation value of a playing volume of the first device to the user and a second volume attenuation value of a playing volume of the second device to the user, including: A p = K - DL w + A e , wherein A p is a volume decay value; K is a distance decay factor, representing volume decay caused by distance, related to the distance between the device and the user; DL w is a directivity factor, representing volume decay caused by the direction of the audio signal emission; A e is an air absorption decay, representing volume decay caused by air absorption of the audio when the audio propagates in the air.

115. The method of claim 93, wherein, The R second electronic devices include a first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user, and has no second user; and the central device further obtains that the first device enters a private mode and plays a first video. When detecting that at least one second user enters the first room or the first area, the central device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to a screen saver. When detecting that the first room or the first area has no second user, the central device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

116. The method of claim 115, wherein, In response to receiving a first feedback message from the first device, the central device learns that the first device has stopped playing and switched to the screen saver.

117. The method of claim 115 or 116, wherein, In response to receiving a second feedback message from the first device, the central device learns that the first device has resumed playing.

118. The method of claim 93, wherein, The R second electronic devices include a first device; the central device obtains that the first device is located in a first room or a first area, the first room or the first area has a first user, and has no second user; and the central device further obtains that the first device enters a private mode and plays a first video. When detecting that a user other than the first user enters the first room or the first area, the central device sends a first notification message to the first device; the first notification message is used to instruct the first device to stop playing and switch to a screen saver. When detecting that the first room or the first area has no user other than the first user, the central device sends a second notification message to the first device; the second notification message is used to instruct the first device to resume playing the first video.

119. The method of claim 93, wherein, The R second electronic devices include a first device; the central device obtains that the first device plays a first video, and further obtains a first position of the first device, that the first device is located in a first room or a first area, a second position of a user in the first room or the first area, and a first viewing area corresponding to the first device; In response to detecting that, within a first preset time length, the first viewing area has no user, the central device sends a first notification message to the first device; The first notification message is used to instruct the first device to stop playing the first video; In response to detecting that, within a second preset time length, the first viewing area has a user, the central device sends a second notification message to the first device; The second notification message is used to instruct the first device to resume playing the first video.

120. The method of claim 119, wherein, in response to detecting that the distance between at least one user and the first device is less than a first preset distance within a third preset time period, the hub device sends a third notification message to the first device; the third notification message is used to instruct the first device to stop playing the first video and output a reminder information; in response to detecting that no user is within the first preset distance from the first device and there is a user in the first viewing area within a fourth preset time period, the hub device sends a fourth notification message to the first device; the fourth notification message is used to instruct the first device to resume playing the first video.

121. The method of claim 93, wherein, in response to detecting that no user is within the first preset distance from the first device and there is no user in the first viewing area within a fifth preset time period, the hub device sends a fifth notification message to the first device; the fifth notification message is used to instruct the first device to automatically hibernate.

122. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a computer program, when the computer program runs on the electronic device, makes the electronic device execute the method as claimed in any one of claims 93-121.

Images