Demonstration system, method, graphical user interface and related devices
By introducing augmented reality technology in the conference and utilizing the communication system of master and slave devices, the problems of venue limitation and content privacy in existing conference technologies are solved, and flexible conference venues and efficient content display are achieved.
Patent Information
- Application Number
- CN202210415946.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-20
- Publication Date
- 2025-06-10
- Estimated Expiration
- 2042-04-20
AI Technical Summary
Existing conference technology requires display equipment or projection equipment, resulting in limited meeting venues and the content of the conference is not easy to ensure privacy.
By introducing augmented reality technology in the real environment, using the communication system between the master and slave devices, the master device randomly selects locations to display multimedia content in the real environment, and the slave device obtains and displays content when it is close to the virtual screen to ensure that the content is not easily leaked.
It realizes the freedom to conduct meetings in any place, enhances the user's meeting experience, and enhances the security and privacy of the meeting.
Smart Images

Figure CN116962620B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of terminals and communication technologies, and in particular, to a demonstration system, method, graphical interface, and related devices. Background Art
[0002] With the continuous development of computer technology, people's working methods have undergone innovative changes. For example, when holding a meeting, the speaker can display the meeting content through a display device or a projection device to help each participant better understand the content described by the speaker. However, precisely because the current meeting requires a display device or a projection device to display the meeting content, the meeting venue is greatly restricted. Therefore, how to reduce the venue restriction of the meeting and improve the user's meeting experience is an urgent problem to be solved currently. Summary of the Invention
[0003] This application provides a demonstration system, method, graphical interface, and related devices, which introduce augmented reality into the meeting scenario, enabling the meeting to be carried out without being restricted by the environment and devices, and the meeting content is not easily leaked, thus improving the user's meeting experience.
[0004] In a first aspect, an embodiment of this application provides a demonstration system. This system can be applied to a communication system including a first device and a second device. The system includes: the first device can receive a first operation; in response to the first operation, the first device can display first content; when the second device is within a preset range of a first position, the second device obtains the first content, where the first position is a position determined by the first device in the physical space; the second device can superimpose and display the first content in the image collected by the camera in real time, where the display position of the first content in the image overlaps with the first position in the image; or, the second device can project the first content onto the user's retina through the lens of the second device, so that the first content is superimposed on the first position viewed by the user through the lens.
[0005] Wherein, the first content may include visual content such as videos, pictures, texts, photos, charts, etc., and the content may be presented as two-dimensional planar content and / or three-dimensional stereoscopic content. It can be seen that the multimedia content with various presentation forms can enrich the display effect of the meeting content.
[0006] When implementing the system provided in the first aspect, the first device, that is, the main device carried by the speaker, can randomly select a position in the real environment to display the multimedia content required for the meeting. When the user views the real environment through the second device, that is, the slave device carried by the participant, the user can see the multimedia content superimposed on the position determined by the main device.
[0007] It can be seen that the system enhances the display into the meeting scenario. The user can specify a virtual screen in any venue to display the multimedia content that needs to be presented in the meeting, making the meeting venue not restricted by display devices. The user can simultaneously view the real - world environment and the virtual multimedia content. Moreover, the system enables the slave device to display the multimedia content only when it is close to the virtual screen, ensuring that the slave device cannot view the meeting content when it is not in the meeting venue, avoiding waste of system resources by the slave device, and at the same time making the meeting more secure and private, and not easily leaking the meeting content.
[0008] Combined with the first aspect, in a possible implementation manner, when the second device is within the preset range of the first position and before the second device obtains the first content, the system further includes: the first device sends the indication information of the first content to the server; when the second device is within the preset range of the first position, the second device obtaining the first content specifically includes: the second device receives a second operation and sends the position of the second device to the server; the server determines that the second device is within the preset range of the first position according to the position of the second device and sends the indication information to the second device; the second device obtains the first content according to the indication information.
[0009] The system can involve a master device, a slave device, and a server. The master device can send multimedia content to the slave device through the server. In this way, when the distance between the master device and the slave device is far, the master device can conduct a remote meeting, and the master device and the slave device can use the server for data transfer.
[0010] Combined with the first aspect, in a possible implementation manner, when the second device is within the preset range of the first position, the second device obtaining the multimedia content specifically includes: the second device receives a second operation and sends the position of the second device to the first device; the first device determines that the second device is within the preset range of the first position according to the position of the second device and sends the indication information to the second device; the second device obtains the first content according to the indication information.
[0011] The system can only involve the master device and the slave device. During the meeting, the master device can directly communicate with the slave device, avoiding the forwarding of third - party devices, accelerating data transmission, synchronizing the multimedia content viewed by the speaker and participants as much as possible, and enhancing the user's meeting experience.
[0012] In combination with the first aspect, in a possible implementation, after the first device displays the first content, the system further includes: the first device receives a third operation on the first content, and in response to the third operation, the first device performs a fourth operation on the first content, and the fourth operation includes any one of the following: switching the display screen of the first content, ending the display of the first content, pausing the playback of the first content, or starting the playback of the first content.
[0013] That is to say, in addition to being able to display multimedia content, the master device can also receive user operations to change the displayed multimedia content, improving the operability of the meeting content. Exemplarily, taking the multimedia content as a presentation as an example, the master device can switch the upper and lower slides of the presentation according to user operations, pause or start playing the slides, or end the playback of the slides, etc.
[0014] In combination with the first aspect, in a possible implementation, after the second device superimposes and displays the first content on the image captured by the camera in real time, or after the second device projects the first content onto the user's retina through the lens of the second device, the system further includes: the first device sends first information indicating the third operation to the second device; the second device performs the fourth operation on the first content.
[0015] During the process that the master device follows the user operation to change the displayed multimedia content, the slave device can also follow the user operation to synchronously change the displayed multimedia content. In this way, it can be ensured that during the meeting, the participants can view the multimedia content changed by the presenter.
[0016] In combination with the first aspect, in a possible implementation, the first device sending the first information indicating the third operation to the second device specifically includes: when the second device is within the preset range of the first position, the first device sends the first information indicating the third operation to the second device.
[0017] When the master device changes the displayed multimedia content, the master device can synchronously change the multimedia content displayed on the slave device when it determines that the slave device is within the preset range, which can avoid the situation that the multimedia content displayed on the slave device still changes following the master device after the slave device is far away from the meeting place, increasing the security of the meeting and saving system resources.
[0018] In combination with the first aspect, in a possible implementation, the first position includes one or more positions, and the second device includes one or more devices.
[0019] The first position includes one or more positions, which means that the number of virtual screens is not limited. The master device can determine multiple virtual screens, and the slave devices can view the multimedia content displayed on the multiple virtual screens. Participants can adjust the viewing angle according to their own needs to enhance the fun of the meeting and the viewing effect of users. The second device includes one or more devices, which means that the number of slave devices is not limited. During the meeting, one or more slave devices can view the multimedia content displayed by the master device at one or more positions.
[0020] In combination with the first aspect, in a possible implementation, the first position is the position obtained when the first device or the third device starts positioning, the third device is a device in the physical space that is different from the first device or the second device, or the first position is a position pre-stored by the first device or the server.
[0021] That is to say, when the main device determines the position of the virtual screen, it can directly start positioning and determine the position of the main device as the position of the virtual screen. In this way, the main device can arbitrarily select a position in the real environment to display multimedia content, or the main device can determine the position of other devices as the position of the virtual screen. In this way, when the main device has no positioning capability or is inconvenient to move, the main device can specify the position of other devices in the real environment as the position of the virtual screen, or the main device can directly select the position of the virtual screen from one or more pre-stored positions, which can reduce the trouble of real-time positioning of the main device, and when the meeting is a remote meeting, the main device can also easily and quickly determine the position of the virtual screen.
[0022] In combination with the first aspect, in a possible implementation, before the second device displays the first content in an image captured in real time by the camera, or before the second device projects the first content onto the user's retina through a lens of the second device, the system further includes: the second device displays first prompt information, and the first prompt information is used to instruct the second device to deflect in a first direction, and the first direction is pointed by the second device to the first position.
[0023] When a participant views multimedia content through a slave device, since the participant may not know the location of the virtual screen before viewing the multimedia content, the slave device can output prompt information to prompt the participant how to move the slave device, so that the participant can quickly find the location of the virtual screen and view the multimedia content as soon as possible.
[0024] In combination with the first aspect, in a possible implementation, the first device and the second device are in the same physical space, or the first device and the second device are in different physical spaces.
[0025] It can be seen that when the master device and the slave device are in the same physical space, the meeting initiated by the master device can be an on-site meeting. In this way, the participants can not only view the multimedia content, but also see the speech actions of the speaker, etc. When the master device and the slave device are in different physical spaces, the meeting initiated by the master device can be a remote meeting. In this way, the holding of the meeting can be unrestricted by the physical distance between the master device and the slave device, and the master device can remotely display the multimedia content. Additionally, further, when there are multiple slave devices, these slave devices can also be located in the same or different physical spaces. The participants can gather together or be dispersed in different places, weakening the restriction of the meeting venue and providing convenience for users to hold meetings.
[0026] In a second aspect, an embodiment of the present application provides a demonstration method. This method is applied to a first device and includes: The first device can receive a first operation; in response to the first operation, the first device can display first content; when the second device is within a preset range of a first position, the first device can send the first content to the second device, where the first position is a position determined by the first device in the physical space.
[0027] Among them, the first content can include visual content such as videos, pictures, texts, photos, charts, etc., and the content can be presented as two-dimensional planar content and / or three-dimensional stereoscopic content. It can be seen that the multimedia content with various presentation forms can enrich the display effect of the meeting content.
[0028] When implementing the method provided in the second aspect, the first device, that is, the master device carried by the speaker, can randomly select a position in the real environment to display the multimedia content required for the meeting. In this way, the user does not need to consider whether the meeting venue is equipped with a display device for presenting the multimedia content for the meeting members to view. The user can specify a virtual screen in any venue to display the multimedia content required for the meeting. Moreover, the second device, that is, the slave device carried by the participant, can only obtain the multimedia content when it is close to the virtual screen, ensuring that the slave device cannot view the meeting content when it is not in the meeting venue, avoiding waste of system resources by the slave device, and at the same time making the meeting more secure and private, and not easily leaking the meeting content.
[0029] In combination with the second aspect, in a possible implementation manner, when the second device is within the preset range of the first position, the first device sends the first content to the second device, which specifically includes: the first device sends the indication information of the first content to the server, so that when the second position is within the preset range of the first position, the server sends the indication information to the second device, where the indication information is used for the second device to obtain the first content; or, the first device obtains the position of the second device; the first device determines that the second device is within the preset range of the first position according to the position of the second device, and sends the indication information to the second device.
[0030] That is to say, during the meeting process, the master device can achieve indirect communication with the slave device through the server, and send the multimedia content to the slave device through the server. In this way, when the distance between the master device and the slave device is relatively far, the master device can use the server for data transfer to successfully carry out a remote meeting. Or, the master device can achieve direct communication with the slave device, avoid the forwarding of a third-party device, speed up data transmission, and synchronize the multimedia content viewed by the speaker and participants as much as possible to enhance the user's meeting experience.
[0031] In combination with the second aspect, in a possible implementation manner, after the first device displays the first content, the method further includes: the first device receives a third operation on the first content, and in response to the third operation, the first device performs a fourth operation on the first content, and the fourth operation includes any one of the following: switching the display screen of the first content, ending the display of the first content, pausing the playback of the first content, or starting the playback of the first content.
[0032] That is to say, in addition to being able to display multimedia content, the master device can also receive user operations to change the displayed multimedia content, enhancing the operability of the meeting content. Exemplarily, taking the multimedia content as a presentation as an example, the master device can switch the upper and lower slides of the presentation according to user operations, pause or start playing the slides, or end the playback of the slides, etc.
[0033] In combination with the second aspect, in a possible implementation manner, the method further includes: the first device sends the first information indicating the third operation to the second device.
[0034] That is to say, during the process of the master device following the user operation to change the displayed multimedia content, the master device can also inform the slave device of the change process so that the slave device can synchronously change the displayed multimedia content.
[0035] In combination with the second aspect, in a possible implementation, the first device sends the first information indicating the third operation to the second device, specifically including: when the second device is located within a preset range of the first position, the first device sends the first information indicating the third operation to the second device.
[0036] When the master device changes the displayed multimedia content, the master device can inform the slave device of the change process when determining that the slave device is located within a preset range, thereby increasing the security of the conference and saving system resources.
[0037] In combination with the second aspect, in a possible implementation, the first location includes one or more locations, and the second device includes one or more devices.
[0038] The first position includes one or more positions, which means that the number of virtual screens is not limited. The master device can determine multiple virtual screens, and the slave devices can view the multimedia content displayed on the multiple virtual screens. Participants can adjust the viewing angle according to their own needs to enhance the fun of the meeting. The second device includes one or more devices, which means that the number of slave devices is not limited. During the meeting, one or more slave devices can view the multimedia content displayed by the master device at one or more positions.
[0039] In combination with the second aspect, in a possible implementation, the first position is the position obtained when the first device or the third device starts positioning, the third device is a device in the physical space that is different from the first device or the second device, or the first position is a position pre-stored by the first device or the server.
[0040] That is to say, when the main device determines the position of the virtual screen, it can directly start positioning and determine the position of the main device as the position of the virtual screen. In this way, the main device can arbitrarily select a position in the real environment to display multimedia content, or the main device can determine the position of other devices as the position of the virtual screen. In this way, when the main device has no positioning capability or is inconvenient to move, the main device can specify the position of other devices in the real environment as the position of the virtual screen, or the main device can directly select the position of the virtual screen from one or more pre-stored positions, which can reduce the trouble of real-time positioning of the main device, and when the meeting is a remote meeting, the main device can also easily and quickly determine the position of the virtual screen.
[0041] In combination with the second aspect, in a possible implementation, the first device and the second device are in the same physical space, or the first device and the second device are in different physical spaces.
[0042] It can be seen that when the master device and the slave device are in the same physical space, the meeting initiated by the master device can be an on-site meeting. In this way, participants can not only see the multimedia content, but also see the speech actions of the speaker and hear the voice of the speaker, etc. When the master device and the slave device are in different physical spaces, the meeting initiated by the master device can be a remote meeting. In this way, the holding of the meeting can be unrestricted by the physical distance between the master device and the slave device, and the master device can remotely display the multimedia content. Additionally, further, when there are multiple slave devices, these slave devices can also be located in the same or different physical spaces, and participants can gather together or be dispersed in different places, weakening the limitation of the meeting venue and providing convenience for users to hold meetings.
[0043] In a third aspect, an embodiment of the present application provides a demonstration method, which is applied to a second device. The method includes: when the second device is within a preset range of a first position, the second device can obtain first content for display by a first device, where the first position is a position determined by the first device in the physical space; the second device can superimpose and display the first content in an image captured by the camera in real time, where the display position of the first content in the image overlaps with the first position in the image; or, the second device can project the first content onto the retina of the user through the lens of the second device, so that the first content is superimposed on the first position viewed by the user through the lens.
[0044] Among them, the first content may include visual content such as videos, pictures, texts, photos, charts, etc., and the content may be presented as two-dimensional planar content and / or three-dimensional stereoscopic content. It can be seen that the multimedia content with various presentation forms can enrich the display effect of the meeting content.
[0045] When implementing the method provided in the third aspect, the first device, that is, the master device carried by the speaker, can randomly select a position in the real environment for displaying the multimedia content required for the meeting. When the user views the real environment through the second device, that is, the slave device carried by the participant, the user can see the multimedia content superimposed on the virtual screen determined by the master device. In this way, the meeting held by the user can be unrestricted by the environment and devices. Participants can view the multimedia content displayed in the real environment through the slave device, realizing the application of augmented reality in the meeting scenario and providing a new meeting form for the holding of the meeting. At the same time, this method enables the slave device to display the multimedia content only when it is close to the virtual screen, ensuring that the slave device cannot view the meeting content when it is not in the meeting venue, avoiding waste of system resources by the slave device, and also making the meeting more secure and private, and not easily leaking the meeting content.
[0046] Combined with the third aspect, in a possible implementation manner, when the second device is within a preset range of the first position, the second device obtains the first content, specifically including: the second device receives a second operation and sends the position of the second device to the server; when the second device is within the preset range of the first position, the second device obtains the indication information sent by the server; or, the second device receives a second operation and sends the position of the second device to the first device; when the second device is within the preset range of the first position, the second device obtains the indication information sent by the first device; the second device obtains the first content according to the indication information.
[0047] That is to say, when the slave device is close to the virtual screen, the slave device can directly obtain the multimedia content from the master device, or indirectly obtain the multimedia content determined by the master device through the server's data transfer. Among them, the direct communication between the slave device and the master device can avoid the forwarding of third-party devices, accelerate data transmission, and synchronize the multimedia content viewed by the presenter and participants as much as possible. By means of server transfer, remote communication can still be achieved when the slave device and the master device are far apart, so that the progress of the meeting is not restricted by the distance between the master and slave devices.
[0048] Combined with the third aspect, in a possible implementation manner, after the second device superimposes and displays the first content in the image captured by the camera in real time, or after the second device projects the first content onto the user's retina through the lens of the second device, the method further includes: the second device obtains the first information sent by the first device indicating a third operation, where the third operation is an operation on the first content received by the first device; the second device performs a fourth operation on the first content, and the fourth operation includes any one of the following: switching the display screen of the first content, ending the display of the first content, pausing the playback of the first content, or starting the playback of the first content.
[0049] That is to say, in the process of the master device controlling the multimedia content according to the user operation, the master device can inform the slave device of the user operation, and the slave device can control the multimedia content displayed on the slave device according to the user operation. Exemplarily, taking a presentation as an example of a multimedia content, the slave device can switch the upper and lower pages of the slides of the presentation, pause or start playing the slides, or end the playback of the slides, etc., according to the user operation.
[0050] Combined with the third aspect, in a possible implementation manner, the second device obtains the first information sent by the first device, specifically including: when the second device is within a preset range of the first position, the second device obtains the first information sent by the first device.
[0051] That is to say, only when the slave device is close to the virtual screen can the slave device control the multimedia content according to the user's operations on the master device. This can avoid the multimedia content displayed on the slave device still changing with the master device after the slave device is away from the meeting place, thereby increasing the security of the meeting and saving system resources.
[0052] In combination with the third aspect, in a possible implementation, the first location includes one or more locations, and the second device includes one or more devices.
[0053] The first position includes one or more positions, which means that the number of virtual screens is not limited. The master device can determine multiple virtual screens, and the slave devices can view the multimedia content displayed on the multiple virtual screens. Participants can adjust the viewing angle according to their own needs to enhance the fun of the meeting. The second device includes one or more devices, which means that the number of slave devices is not limited. During the meeting, one or more slave devices can view the multimedia content displayed by the master device at one or more positions.
[0054] In combination with the third aspect, in a possible implementation, the first position is the position obtained when the first device or the third device starts positioning, the third device is a device in the physical space that is different from the first device or the second device, or the first position is a position pre-stored by the first device or the server.
[0055] That is to say, when the main device determines the position of the virtual screen, it can directly start positioning and determine the position of the main device as the position of the virtual screen. In this way, the main device can arbitrarily select a position in the real environment to display multimedia content, or the main device can determine the position of other devices as the position of the virtual screen. In this way, when the main device has no positioning capability or is inconvenient to move, the main device can specify the position of other devices in the real environment as the position of the virtual screen, or the main device can directly select the position of the virtual screen from one or more pre-stored positions, which can reduce the trouble of real-time positioning of the main device, and when the meeting is a remote meeting, the main device can also easily and quickly determine the position of the virtual screen.
[0056] In conjunction with the third aspect, in a possible implementation manner, before the second device overlays and displays the first content in an image captured in real time by a camera, or before the second device projects the first content onto a retina of a user through a lens of the second device, the method further includes:
[0057] The second device displays first prompt information, where the first prompt information is used to instruct the second device to deflect toward a first direction, where the first direction is pointed by the second device to the first position.
[0058] When a participant views multimedia content through a slave device, since the participant may not know the position of the virtual screen before viewing the multimedia content, the slave device can output a prompt message to prompt the participant how to move the slave device, facilitating the participant to quickly locate the position of the virtual screen and view the multimedia content as soon as possible.
[0059] Combined with the third aspect, in a possible implementation manner, the first device and the second device are in the same physical space, or the first device and the second device are in different physical spaces.
[0060] It can be seen that when the master device and the slave device are in the same physical space, the meeting initiated by the master device can be an on-site meeting. In this way, the participants can not only see the multimedia content but also see the speaker's speech actions, etc. When the master device and the slave device are in different physical spaces, the meeting initiated by the master device can be a remote meeting. In this way, the holding of the meeting can be unrestricted by the physical distance between the master device and the slave device, and the master device can remotely display the multimedia content. Additionally, further, when there are multiple slave devices, these slave devices can also be located in the same or different physical spaces, and the participants can gather together or be dispersed in different places, weakening the limitation of the meeting venue and facilitating the users to hold meetings.
[0061] In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method described in the second aspect or any one of the implementation manners of the second aspect.
[0062] In a fifth aspect, an embodiment of the present application provides an electronic device, including a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method described in the third aspect or any one of the implementation manners of the third aspect.
[0063] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, when the instructions run on an electronic device, the electronic device executes the method described in the second aspect or any one of the implementation manners of the second aspect, the third aspect or any one of the implementation manners of the third aspect.
[0064] In a seventh aspect, an embodiment of the present application provides a computer program product, when the computer program product runs on a computer, the computer executes the method described in the second aspect or any one of the implementation manners of the second aspect, the third aspect or any one of the implementation manners of the third aspect.
[0065] By implementing the technical method provided in the embodiments of the present application, augmented reality can be introduced into the meeting scenario. Whether it is a remote meeting or an on-site meeting, the meeting venue is not restricted by the environment and equipment. Users can specify a location in the real environment and, when viewing the real environment through a device they carry, such as a mobile phone, AR glasses, etc., view the multimedia content displayed at the specified location. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 FIG. is a schematic diagram of a scenario provided by an embodiment of the present application;
[0067] Figure 2 FIG. is a schematic structural diagram of a communication system provided by an embodiment of the present application;
[0068] Figures 3A - 3F 、 Figures 4A - 4D FIGS. are some user interfaces provided by an embodiment of the present application;
[0069] Figure 5 FIG. is a schematic diagram of the layer overlay principle provided by an embodiment of the present application;
[0070] Figure 6 FIG. is a schematic flowchart of a demonstration method provided by an embodiment of the present application;
[0071] Figure 7 FIG. is a schematic flowchart of a demonstration method provided by an embodiment of the present application;
[0072] Figure 8 FIG. is a schematic structural diagram of an octree provided by an embodiment of the present application;
[0073] Figure 9 FIG. is a schematic flowchart of a demonstration method provided by an embodiment of the present application;
[0074] Figure 10 FIG. is a schematic flowchart of another demonstration method provided by an embodiment of the present application;
[0075] Figure 11 FIG. is a schematic structural diagram of a demonstration device provided by an embodiment of the present application;
[0076] Figure 12 FIG. is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application;
[0077] Figure 13 FIG. is a schematic software structure diagram of an electronic device provided by an embodiment of the present application;
[0078] Figure 14 FIG. is a schematic hardware structure diagram of another electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0079] The technical solutions in the embodiments of the present application will be clearly and elaborately described below with reference to the accompanying drawings. Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B. The "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality" means two or more than two.
[0080] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more than two.
[0081] The term "user interface (UI)" in the following embodiments of the present application is a media interface for interaction and information exchange between an application program or an operating system and a user, and it realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is source code written in specific computer languages such as Java and Extensible Markup Language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content recognizable by the user. The common manifestation form of the user interface is the graphical user interface (GUI), which refers to the user interface related to computer operations displayed in a graphical manner. It can be visual interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and Widgets displayed on the display screen of an electronic device.
[0082] The embodiments of the present application provide a demonstration method. This method involves a master device and a slave device. The master device can obtain multimedia content and determine the position of the virtual screen for playing the multimedia content in the real environment. In response to the operation of the user to start a meeting, when the distance between the slave device and the virtual screen is less than or equal to a preset value, the slave device obtains the multimedia content and displays the multimedia content at a specified position on the display screen according to the position of the virtual screen. At this time, when the user views the multimedia content through the slave device, the multimedia content is superimposed on the position of the virtual screen determined by the master device in the real environment.
[0083] It can be seen that this method integrates augmented reality into the meeting scenario, enabling the meeting venue to be unrestricted by display devices. Users can specify a virtual screen in any venue to display the multimedia content that needs to be presented in the meeting, allowing users to simultaneously view the real-world environment and the virtual multimedia content.
[0084] Among them, the multimedia content is the meeting content that needs to be presented to the participants in the meeting, which can include but is not limited to: visual content such as videos, pictures, texts, photos, charts, etc., and this content can be presented as two-dimensional planar content and / or three-dimensional stereoscopic content. It can be seen that the multimedia content with various presentation forms can enrich the display effect of the meeting content.
[0085] The main device can be an electronic device such as a mobile phone, a computer, a tablet, etc. The meeting speaker can start the meeting, display the multimedia content, and control the playback, pause, switching, etc. of the multimedia content through this main device.
[0086] The slave device can be a device with a camera and a display screen such as a mobile phone, a computer, a tablet, etc. In this way, the slave device can superimpose the multimedia content on the image captured by the camera in real time for the participants to view the real-world environment and the multimedia content simultaneously through the display screen. Or, the slave device can also be an augmented reality (AR) device, including AR glasses, AR head-mounted devices, etc. In this way, when the participants view the real-world environment through the AR device, they can see the multimedia content displayed on the virtual screen specified in the real-world environment. In this way, although there is no display device or projection device in the real-world environment to uniformly display the multimedia content that all participants need to view, the main device can, by setting the position of the virtual screen, enable users to view the multimedia content through their respective slave devices, giving users a feeling that the multimedia content is displayed in the real world, providing a brand-new meeting form for the development of the meeting.
[0087] Generally speaking, the method provided by the embodiments of the present application enables the meetings held by users to be unrestricted by the environment and devices. For example, the presenter and participants can each carry only one mobile phone. The presenter can randomly select a position in the real environment to display the multimedia content required for the meeting. The participants can view the real environment captured by the camera and the multimedia content provided by the presenter through the mobile phone. Moreover, the multimedia content can include two-dimensional and three-dimensional content, introducing virtual reality and mixed reality into the meeting scenario, providing richer meeting content for users, making the meeting more vivid and interesting, and enhancing the user's meeting experience. Further, this method enables the slave device to display the multimedia content only when it is relatively close to the virtual screen, ensuring that the slave device cannot view the meeting content when it is not in the meeting venue, avoiding waste of system resources by the slave device, and at the same time making the meeting more secure and private, and not easily leaking the meeting content.
[0088] To better understand this solution, first, a simple description is given of the application scenarios involved in the demonstration method provided by the embodiments of the present application.
[0089] Figure 1 FIG. shows the application scenarios involved in the demonstration method provided by the embodiments of the present application. As Figure 1 shown, the application scenario may refer to a meeting scenario including a master device 100 and a slave device 200. In this meeting scenario, the user using the master device 100 can be called the presenter, and the user using the slave device 200 can be called the participant. Among them, the presenter can provide multimedia content to each participant and explain the meeting content to each participant.
[0090] In addition, in the real environment shown in this application scenario, there is also an area 1, which can be a blackboard, a wall surface, etc. in the real environment. The presenter can, through the master device 100, designate this area 1 as the virtual screen. During the process of the presenter explaining the meeting, each participant can view the multimedia content provided by the presenter displayed in area 1 through the slave device 200. Exemplarily, the slave device 200 can be a mobile phone as Figure 1 shown. When the participant faces the camera of the slave device 200 towards area 1, the content displayed on the slave device 200 can be a user interface 10 as Figure 1 shown. The user interface 10 displays the image captured by the camera in real time, including the presenter and area 1. In addition, the multimedia content provided by the presenter is also superimposed and displayed on area 1.
[0091] Moreover, when the participant moves the slave device 200 so that in the image captured by the camera, area 1 moves to the left, the multimedia content will also move to the left synchronously following the movement of area 1 and always remain within area 1, thus giving the participant an effect that the multimedia content is displayed in area 1 in the real environment.
[0092] In addition, when the participant views the multimedia content through the slave device 200, the participant can adjust the orientation and / or focal length of the camera so that the content displayed on the slave device 200 can include both the multimedia content and the speaker at the same time. In this way, the participant can view the speaker and the multimedia content presented by the speaker through the slave device 200 at the same time, avoiding the participant from frequently switching perspectives to view the speaker and the multimedia content. Alternatively, the participant can also adjust the orientation and / or focal length of the camera so that the content displayed on the slave device 200 only includes the multimedia content. In this way, when the participant needs to carefully view the multimedia content, the participant can obtain a browsing perspective only for the multimedia content, which helps the participant to view the multimedia content more clearly and obtain the detailed content of the multimedia content.
[0093] By comparing Area 1 in the actual environment with Area 1 in the user interface 10, it can be seen that although the multimedia content is not displayed in Area 1 in the actual environment, the participant can view the multimedia content displayed in Area 1 through the slave device 200. That is to say, there is no need to set up a display device or a screen mirroring device in Area 1, and the participant can also view the multimedia content in Area 1 through a virtual screen. Then, during an actual meeting, the speaker can carry out the meeting quickly anytime and anywhere without considering the limitations of the meeting venue, providing a more vivid and interesting meeting experience for the user.
[0094] It can be understood that the application scenarios of the demonstration method provided by the embodiments of the present application are not limited to the above Figure 1 described application scenarios. For example, the demonstration method can also be applied in a teaching scenario where a teacher provides courseware for students to view during teaching, or it can also be applied in a promotional scenario where a promoter provides product materials for customers to view when promoting products, or it can also be applied in an exhibition scenario where the organizer provides exhibit introductions for exhibits for visitors to view at an exhibition, etc. The embodiments of the present application do not limit the application scenarios of the demonstration method.
[0095] Figure 2 Fig. shows the communication system 1000 provided by the embodiments of the present application.
[0096] As Figure 2 shown, the communication system 1000 may include: a master device 100 and slave devices 200. Among them, the number of master devices 100 may be one, and the number of slave devices 200 may be one or more.
[0097] The master device 100 can obtain multimedia content and determine the position of the virtual screen for playing the multimedia content in the real environment. Additionally, the master device 100 can also receive an operation by the user to start a meeting. In response to this operation, it displays the multimedia content, and when the distance between the slave device 200 and the virtual screen is less than or equal to a preset value, the master device 100 can send the displayed multimedia content to the slave device 200. Further, the master device 100 can also receive an operation by the user on the multimedia content, triggering operations such as playing, pausing, ending, or switching of the multimedia content, and the master device 100 can also send this operation to the slave device 200 to trigger the slave device 200 to play, pause, end, or switch the multimedia content.
[0098] The slave device 200 can receive an operation by the user to join a meeting, send the position information of the slave device to the master device 100, receive the multimedia content and the position information of the virtual screen sent by the master device 100, and display the multimedia content at a specified position on the display screen according to the position of the virtual screen. Additionally, the slave device 200 can also receive indication information of an operation on the multimedia content displayed on the master device 100 and synchronously change the displayed multimedia content.
[0099] In some embodiments, the communication system 1000 may further include a server 300. The server 300 can be used to relay communication information between the master device 100 and the slave device 200, including multimedia content, position information of the virtual screen, etc. The server 300 can also be used to store data, including multimedia content, position information of the virtual screen, etc. In this way, the master device 100 and the slave device 200 can obtain multimedia content through the server 300. The master device 100 can also select a position information from the position information of the virtual screen stored in the server 300 as the position information of the virtual screen for presenting multimedia content in the current meeting. Additionally, the server 300 can also judge whether the slave device 200 is within a preset range according to the position information of the virtual screen and the position information of the slave device 200. When the slave device 200 is within the preset range, the server 300 sends the multimedia content or the indication information of the operation for changing the multimedia content received by the master device 100 to the slave device 200 so that the slave device 200 can display the multimedia content and the changed multimedia content.
[0100] The following Figures 3A - 3F 、 Figures 4A - 4D introduces some user interfaces provided by the embodiments of the present application. Among them, Figures 3A - 3F are some user interfaces provided on the master device 100, Figures 4A - 4D are some user interfaces provided on the slave device 200. It should be noted that Figures 3A - 3F 、 Figures 4A - 4DTaking the master device 100 as a computer and the slave device 200 as a mobile phone as an example for introduction, in other embodiments of the present application, the master device 100 and the slave device 200 may also be other electronic devices, which are not limited herein.
[0101] Figure 3A The user interface 1-1 involved in triggering the playback of multimedia content on the master device 100 is shown. As Figure 3A shown, the user interface 1-1 may include: a menu bar 111, a content display area 112, and an add icon 113. Among them:
[0102] The menu bar 111 may include one or more options. These one or more options can receive user operations and display different application pages in the user interface 1-1. Exemplarily, these one or more options may include: a home page option, a meeting option, an application option, a local option, etc. Among them, the home page option can be used to trigger the display of one or more multimedia contents, the meeting option can be used to trigger joining a meeting, obtaining and displaying multimedia contents output by other devices, the application option can be used to trigger the display of one or more application functions, and the local option can be used to trigger the display of relevant information of the local account, including the user name, user avatar, etc. Exemplarily, Figure 3A the shown user interface 1-1 can be the application page displayed by the master device 100 when the home page option is in the selected state.
[0103] The content display area 112 is used to display options corresponding to one or more multimedia contents. These one or more multimedia contents can be classified according to "recent", "local", and "cloud". Among them, the "recent" multimedia content may refer to the multimedia content most recently browsed in the master device 100, the "local" multimedia content may be the multimedia content locally stored in the master device 100, and the "cloud" multimedia content may be the multimedia content stored in the cloud server. Exemplarily, the content display area 112 may include a first content 112A, which corresponds to a multimedia content. The name of the multimedia content: "Presentation 1" may be displayed in the first content 112A. In addition, the first content 112A may further include: a play icon 1121A, a favorite icon 1122A, an upload icon 1123A, and a delete icon 1124A.
[0104] Among them, the play icon 1121A can be used to trigger the playback of the multimedia content corresponding to the first content 112A. Specifically, the master device 100 can detect a user operation acting on the play icon 1121A, and in response to this operation, the master device 100 triggers the start of a meeting and displays the multimedia content. The favorite icon 1122A can be used to classify the multimedia content corresponding to the first content 112A into the favorite bar, and the user can find the multimedia content in the favorite bar. The upload icon 1123A can be used to upload the multimedia content corresponding to the first content 112A to the cloud server. When the master device 100 deletes the multimedia content corresponding to the first content 112A from the local, the master device 100 can retrieve the multimedia content again from the cloud server. Additionally, exemplarily, when the multimedia content corresponding to the first content 112A has been uploaded to the cloud server, the icon color of the upload icon 1123A can be light, and the master device 100 can no longer respond to a user operation acting on the upload icon 1123A. The delete icon 1124A can be used to delete the first content 112A. Here, deleting the first content 112A can refer to deleting the multimedia content corresponding to the first content 112A, or it can refer to only deleting the record of the multimedia content and not displaying the first content 112A, while the multimedia content is still stored in the master device 100.
[0105] The add icon 113 can be used to trigger the addition of multimedia content. Among them, adding multimedia content can include obtaining multimedia content from other devices or creating multimedia content on the master device 100. When the add icon 113 can be used to trigger the creation of multimedia content on the master device 100, the master device 100 can provide multiple elements such as videos, pictures, texts, photos, graphics, and icons, as well as operation commands for adjusting these elements, such as moving, rotating, deforming, etc., so that the user can create multimedia content by adding, adjusting, and combining these elements.
[0106] As Figure 3A shown, when the master device 100 receives a user operation acting on the play icon 1121A, for example, a click operation where the cursor acts on the play icon 1121A, in response to this operation, the master device 100 starts a meeting and displays, in the user interface 1-1, a first prompt message 114 as Figure 3B shown, and this first prompt message 114 is used to prompt the user to determine the position of the virtual screen.
[0107] In some embodiments, when the multimedia content corresponding to the first content 112A is stored in the cloud server, when the master device 100 receives a user operation acting on the play icon 1121A, the master device 100 can first download and save the multimedia content to the local, and then start the playback of the multimedia content, or alternatively, the master device 100 can play the multimedia content online.
[0108] As shown Figure 3B in the figure, the first prompt message 114 may include: a first option 114A and a second option 114B. Among them:
[0109] The first option 114A can be used to trigger positioning to obtain the location information of the master device 100. The master device 100 can use this location information as the location information of the virtual screen, so that when the user views the multimedia content through the slave device 200, the display position of the multimedia content in the real environment is located at the position of the virtual screen. For example, when the master device 100 starts positioning close to the blackboard, when the user views the multimedia content through the slave device 200, the multimedia content is located on the blackboard.
[0110] The second option 114B can be used to trigger the selection of one of the existing positions as the position where the virtual screen is located. The existing positions can be one or more positions pre-stored by the master device 100, or one or more positions stored in the cloud server. For example, when the master device 100 has previously set the position of the virtual screen in Conference Room A, when the master device 100 starts a meeting in Conference Room A again, the corresponding position in Conference Room A stored in history can be directly used as the position of the virtual screen this time. Another example is that the cloud server can pre-store some available positions, such as the positions in Conference Room A, Conference Room B, and Conference Room C. These positions can be the positions used by the master device and / or other devices when holding meetings in these venues. At this time, the master device 100 can also directly select one of the positions for use. This can reduce the user's operations and help the user start the meeting as soon as possible.
[0111] It can be understood that in addition to the master device 100 starting positioning or selecting a position from the existing positions to determine the position of the virtual screen, other ways to determine the position of the virtual screen can also be included. For example, the master device 100 can send a positioning request to other devices, request to obtain the location information of other devices, and use the location of this device as the location of the virtual screen. The embodiments of the present application do not limit the way for the master device to determine the position of the virtual screen.
[0112] As shown Figure 3B in the figure, when the master device 100 receives a user operation acting on the first option 114A, such as a click operation where the cursor acts on the first option 114A, in response to this operation, the master device 100 starts the positioning function, such as GPS positioning, to obtain the longitude, latitude, and altitude of the current position where the master device 100 is located.
[0113] It can be understood that before the main device 100 activates the positioning function, the speaker can move the main device 100 to the position selected as the virtual screen and then activate the positioning, so as to determine the position of the virtual screen according to the current position of the main device 100. Alternatively, when the main device 100 is not easily movable or does not have a positioning function, the speaker can directly select a position from the existing positions as the position of the virtual screen, reducing the trouble of manual positioning for the user.
[0114] During the positioning process, the main device 100 can update the first prompt message 114 to the second prompt message 115 as shown in Figure 3C The second prompt message 115 is used to prompt the user that positioning is in progress, please wait until the positioning is completed. After that, the main device 100 can update the second prompt message 115 to the third prompt message 116 as shown in Figure 3D The third prompt message 116 is used to prompt the user that the positioning has been completed and obtain the current position of the main device 100.
[0115] Optionally, when the positioning speed of the main device 100 is relatively fast, the main device 100 may not display the second prompt message 115 as shown in Figure 3C After the main device 100 receives the user operation for the first option 114A and completes the positioning, it directly displays the third prompt message 116 as shown in Figure 3D
[0116] As shown in Figure 3D The third prompt message 116 may include a first option 116A and a second option 116B. Among them:
[0117] The first option 116A can be used to trigger the upload of the currently collected position information to the cloud server. The second option 116B can be used to trigger the display of multimedia content and send the multimedia content to the slave device 200 for display.
[0118] In some embodiments, in addition to determining the position of the virtual screen, the main device 100 can also determine the size of the virtual screen, that is, the display size of the multimedia content. Among them, the size of the virtual screen can be a preset size, such as 1 meter in height and 3 meters in width. Or, the size of the virtual screen can be set by the speaker. In other words, the main device 100 can adjust the size of the virtual screen according to the user operation. Or, the main device 100 can automatically determine the size of the virtual screen according to the spatial size of the current environment. The embodiments of the present application do not limit the method for determining the size of the virtual screen.
[0119] As shown in Figure 3D As shown, when the host device 100 receives a user operation on the second option 116B, for example, a click operation where the cursor acts on the second option 116B, in response to this operation, the host device 100 displays a user interface 1-2 as shown in Figure 3E for displaying multimedia content.
[0120] As shown in Figure 3E , the user interface 1-2 may include: a display area 121 and a function selection area 122. Among them:
[0121] The display area 121 is used to display multimedia content. The function selection area 122 may include one or more function icons, and these function icons can receive user operations to trigger and control the multimedia content displayed in the display area 121 to perform different operations. Exemplarily, the function selection area 122 may include a first icon 122A, a second icon 122B, a third icon 122C, and a fourth icon 122D. Among them, the first icon 122A can be used to switch to the previous page of the multimedia content, the second icon 122B can be used to pause the multimedia content currently displayed in the display area 121, the third icon 122C can be used to switch to the next page of the multimedia content, and the fourth icon 122D can be used to end the display of the multimedia content.
[0122] As shown in Figure 3E , when the host device 100 receives a user operation on the third icon 122C, for example, a click operation where the cursor acts on the third icon 122C, in response to this operation, the host device 100 switches to the next page of the multimedia content and displays a user interface 1-2 as shown in Figure 3F .
[0123] Comparing Figure 3E and Figure 3F , Figure 3E the content displayed in the user interface 1-2 shown is the "January - April Statistical Table", Figure 3F the content displayed in the user interface 1-2 shown is the "May - August Statistical Table". Among them, Figure 3E the content shown in Figure 3F and the content shown in
[0124] can be different parts of the same multimedia content, and the host device 100 can switch to display different parts of the multimedia content by switching the upper and lower pages of the multimedia content. Figures 3A - 3FIt can be seen that the master device 100 can determine the location of the virtual screen for playing multimedia content and start a meeting by acquiring and displaying the multimedia content. Moreover, during the process of displaying the multimedia content, the master device 100 can also trigger the control of the multimedia content according to user operations. For example, switch to the previous page of the multimedia content, switch to the next page of the multimedia content, pause the currently playing multimedia content, end the display of the multimedia content, and so on.
[0125] Figure 4A Figure 2-1 shows the user interface on the slave device 200 for triggering the joining of a meeting and viewing multimedia content. As Figure 4A shown, the user interface 2-1 may include: a menu bar 211, a status bar 212, a first control 213, and a second control 214. Among them:
[0126] The menu bar 211 may include one or more options. These one or more options can receive user operations and display different application pages in the user interface 1-1. Exemplarily, these one or more options may include: a home page option, a meeting option, an application option, a local option, and so on. Among them, the home page option can be used to trigger the display of one or more multimedia contents, the meeting option can be used to trigger the joining of a meeting, acquire and display the multimedia contents output by other devices, the application option can be used to trigger the display of one or more application functions, and the local option can be used to trigger the display of relevant information of the local account, including the user name, user avatar, and so on. Exemplarily, Figure 4A the shown user interface 2-1 can be the application interface displayed by the slave device 200 when the meeting option is in the selected state.
[0127] The status bar 212 may include one or more signal strength indicators of mobile communication signals, one or more signal strength indicators of wireless fidelity (WiFi) signals, a battery status indicator, and a time indicator.
[0128] The first control 213 can be used to trigger the joining of a meeting. Specifically, the slave device 200 can receive a user operation acting on the first control 213, and in response to this operation, the slave device 200 displays the multimedia content provided by the master device 100.
[0129] The second control 214 can be used to trigger the creation of a meeting. When a user operation acting on the second control 214 is received from the device 200, the device 200 can become the initiator of the meeting. The initiator can invite other members to join a meeting together. A member in the meeting can select multimedia content, determine the position of the virtual screen for playing the multimedia content, and display the multimedia content to other members in the meeting. At this time, the meeting member who selects the multimedia content and determines the position of the virtual screen is the speaker of the meeting, and the other members are the participants of the meeting.
[0130] As Figure 4A shown, when a user operation acting on the first control 213 is received from the device 200, for example, a click operation, in response to this operation, the device 200 displays a prompt message 215 as Figure 4B shown in the user interface 2-1. The prompt message 215 is used to prompt the user that the camera is currently being turned on.
[0131] As Figure 4C shown, after the device 200 turns on the camera (for example, the rear camera), the device 200 can collect an image through the camera and superimpose and display multimedia content at the position of the virtual screen included in the image. Enabling the user to see both the real environment and the multimedia content through the display screen of the device 200. Additionally, Figure 4C the multimedia content displayed in Figure 3E can be the multimedia content currently displayed on the master device 100, that is, the multimedia content as
[0132] shown. Optionally, after the device 200 turns on the camera, the device 200 can also output a prompt message to prompt the user how to move the device 200 so that the device 200 is aligned with the position of the virtual screen, ensuring that when the user views the multimedia content through the device 200, the display position of the multimedia content can be quickly found. For example, the device 200 can display an arrow on the display screen. When the device 200 displays an arrow pointing to the left, it indicates to the user to move the device 200 to the left, causing the camera of the device 200 to move a certain distance to the left. When the device 200 displays an arrow pointing to the right, it indicates to the user to move the device 200 to the right, causing the camera of the device 200 to move a certain distance to the right.
[0133] Optionally, after the device 200 turns on the camera, the device 200 can also receive a user operation to adjust the focal length of the camera, zooming in or out the viewing angle of the camera, thereby zooming in or out the multimedia content displayed on the device 200. Among them, the user can zoom in the focal length to view the details of the multimedia content, or the user can zoom out the focal length to view more content in addition to the multimedia content at the same time, such as the actions of the speaker located next to the virtual screen, etc.
[0134] During the process of the slave device 200 displaying multimedia content, when the master device 100 controls the multimedia content and switches to the next page of the multimedia content, that is, when displaying Figure 3F the multimedia content shown, the slave device 200 can synchronously display the changed multimedia content, and the changed multimedia content can be Figure 4D the content shown in the user interface 2-2 shown.
[0135] From Figures 4A - 4D it can be seen that the slave device 200 can receive the operation of the user triggering to join the meeting, trigger to turn on the camera, and superimpose and display the multimedia content on the image captured by the camera in real time. In this way, the user can achieve the effect of watching the meeting by watching the content displayed on the slave device 200. Moreover, during the process that the master device 100 triggers and controls the multimedia content to perform different operations according to the user operation, for example, switching to the previous page of the multimedia content, switching to the next page of the multimedia content, pausing the currently played multimedia content, ending the display of the multimedia content, etc., the slave device 200 can also respond to the user operation and control the multimedia content displayed in the slave device 200 to perform the same operations, such as switching to the previous page of the multimedia content, switching to the next page of the multimedia content, pausing the currently played multimedia content, ending the display of the multimedia content, etc.
[0136] Figure 5 shows the layer superposition principle involved when the slave device 200 simultaneously displays the image captured by the camera and the multimedia content.
[0137] As Figure 5 shown, the content displayed in layer 1 can be the image captured by the camera of the slave device 200 in real time, and layer 2 is used to display the multimedia content received by the slave device 200. The slave device 200 can superimpose layer 1 and layer 2 to obtain image 1, and both the image captured by the camera in real time and the multimedia content are displayed in image 1, and image 1 is the content displayed on the display screen of the slave device 200.
[0138] In addition, when the slave device 200 controls the multimedia content to perform different operations according to the operation of the user on the master device 100, the slave device 200 can respond to the operation and change the content displayed in layer 2. For example, switch to display different parts of the multimedia content, pause the content currently displayed in layer 2, end the display of the content displayed in layer 2, etc.
[0139] It should be noted that Figure 5Taking the slave device 200 as a mobile phone as an example, the principle of the slave device 200 simultaneously superimposing and displaying the real environment and multimedia content is shown. When the slave device 200 is an AR device, the content shown in layer 1 should be the picture in the real environment observed by the naked eyes of the user wearing the AR device, and the content shown in layer 2 is the multimedia content displayed by the AR device, or the multimedia content projected by the AR device through the lens onto the retina of the user. The image 1 can be the content that can be seen after the picture in the real environment obtained by the user's eyes and the multimedia content are superimposed on the retina.
[0140] In some embodiments, the way the master device 100 displays multimedia content can also be the same as the way the slave device 200 displays multimedia content, that is, while displaying multimedia content, the user can view the real environment through the master device 100. In this way, during a meeting, the experience of the user viewing multimedia content through the master device 100 is the same as that through the slave device 200.
[0141] The following is based on Figure 1 the application scenario shown, Figure 2 the communication system 1000 shown, Figures 3A - 3F the user interface implemented on the master device 100, and Figures 4A - 4D the user interface implemented on the slave device 200, to describe in detail the demonstration method provided by the embodiments of the present application.
[0142] In the embodiments of the present application, the demonstration method involves the master device and the slave device, or further involves a server. Among them, the master device can be used to initiate a meeting, the slave device can join the meeting, and the server is an optional device that can play the role of relaying the communication information between the master device and the slave device, performing distance judgment, and storing data. When the demonstration method does not involve a server, the master device and the slave device can be in a relatively close environment at the same time, and the master device and the slave device conduct a meeting through direct communication, which can reduce the trouble of data relaying and reduce the delay of the slave device outputting multimedia content. When the demonstration method involves a server, the master device can initiate a meeting remotely. Even if the master device is not near the slave device, the master device can provide multimedia content for the slave device, so that the slave device can remotely view the content shared by the master device. In this way, the meeting can be unrestricted by distance, ensuring that the slave device can view the content shared by the master device regardless of the distance from the master device, and enabling the meeting to be carried out without being restricted by the geographical location of the meeting members.
[0143] In the following two embodiments, Embodiment 1 describes a demonstration method involving the master device, the slave device, and the server, and Embodiment 2 describes a demonstration method involving only the master device and the slave device.
[0144] (1) Embodiment 1
[0145] The following is combined with Figures 6 - 9 to introduce a demonstration method provided by an embodiment of the present application.
[0146] This demonstration method can be divided into three stages: the master device initiates a meeting, the slave device joins the meeting, and the master device changes the meeting content. Among them, the first stage mainly involves the interaction between the master device and the server. Specifically, please refer to Figure 6 the process schematic diagram shown in Figure 7 The second stage mainly involves the interaction between the server and the slave device. Specifically, please refer to Figure 9 the process schematic diagram shown in
[0147] Stage 1: The master device initiates a meeting
[0148] As Figure 6 shown, Stage 1 may include:
[0149] S101. The master device acquires multimedia content.
[0150] The master device may refer to the aforementioned master device 100. The master device may be a device used by the speaker. The master device may determine the multimedia content to be displayed in the meeting according to the speaker's operation, determine the position of the virtual screen where the multimedia content is to be displayed, and trigger the start of the meeting, etc.
[0151] The multimedia content may refer to the content that the speaker needs to display during the meeting to assist in explaining the meeting. For example, in the scenario of a work meeting, the multimedia content may be the company materials that need to be displayed when the company members explain the meeting. Another example is that in the scenario of a classroom meeting, the multimedia content may be the classroom materials used by the teacher during teaching. In the embodiments of the present application, the multimedia content may also be referred to as the first content.
[0152] Among them, the multimedia content may include, but is not limited to: visual content such as videos, pictures, texts, photos, charts, etc. And this content may be presented as two-dimensional plane content, and / or three-dimensional stereoscopic content. Exemplarily, the multimedia content may be presented as a presentation. The presentation may include one or more slides. Each slide may display content such as videos, pictures, texts, photos, charts, etc. At the same time, audio, animations, etc. may also be added. That is to say, the multimedia content may contain multiple contents to be displayed. The user can complete the switching of the multimedia content by switching the content corresponding to the page number, and present all the content included in the multimedia content to the user.
[0153] The ways for the master device to obtain multimedia content may include but are not limited to the following two:
[0154] 1) The master device obtains multimedia content from local
[0155] The multimedia content may be the content stored locally by the master device, and the master device can directly obtain the multimedia content from the storage space. Alternatively, the master device may be installed with an application for producing the multimedia content, and the master device can produce multimedia content through the application according to user operations.
[0156] 2) The master device obtains multimedia content from the server
[0157] The server may store one or more multimedia contents, and these multimedia contents may be sourced from the master device and / or other devices. When the master device needs to display multimedia content, it can find and download it from the server to the local. In this way, it is possible to avoid the multimedia content occupying the storage space of the master device when it is unnecessary.
[0158] In some embodiments, the master device may display options corresponding to one or more multimedia contents, and the master device can select the multimedia content to be displayed this time from one or more multimedia contents according to user operations. Exemplarily, referring to Figure 3A , options corresponding to one or more multimedia contents may be displayed in the content display area 112.
[0159] S102. The master device receives a first operation to play multimedia content.
[0160] In some embodiments, the master device may display one or more play options, with one play option corresponding to one multimedia content. The master device may receive an operation acting on one of the play options, and in response to this operation, trigger the playback of the multimedia content corresponding to the play option.
[0161] Exemplarily, the first operation may refer to Figure 3A the user operation acting on the play icon 1121A as shown in
[0162] It can be understood that the embodiments of the present application do not limit the sequence of steps S101 and S102. In some embodiments, step S101 may be executed prior to step S102. In this case, the master device may first obtain the multimedia content to be played, and then receive the play operation of the user on the multimedia content. In other embodiments, step S102 may be executed prior to step S101. In this case, the master device may first receive the play operation of the user on the multimedia content, triggering the acquisition of the multimedia content, for example, obtaining it from the server.
[0163] S103. The master device obtains the position information of the virtual screen.
[0164] The virtual screen may refer to a fictitious screen for displaying the multimedia content. The master device may specify a location in the real environment as the location of the virtual screen. In other words, the location of the virtual screen is the location in the physical space determined by the master device. In this way, when the user views the real environment through the slave device, the multimedia content displayed at the location can be viewed. In the embodiment of the present application, the location of the virtual screen may also be referred to as the first location.
[0165] The location information may include longitude and latitude and altitude in a geodetic coordinate system. For example, the location information may be represented as a three-dimensional coordinate in an XYZ coordinate system, wherein the vector of the X axis may be used to represent the magnitude of longitude, the vector of the Y axis may be used to represent the magnitude of latitude, and the vector of the Z axis may be used to represent the magnitude of altitude.
[0166] The host device can obtain the location information of the virtual screen in the following three ways:
[0167] 1) The main device starts positioning and obtains the location information of the virtual screen
[0168] That is, the master device can start positioning and determine the current position of the master device as the position of the virtual screen. For example, when determining the position of the virtual screen, the presenter can move the master device near the blackboard, start positioning, and determine the current position of the master device as the position of the virtual screen. In this way, when the participant views the multimedia content through the slave device, the multimedia content can be viewed on the blackboard.
[0169] 2) The host device determines the location information of the virtual screen from the locally stored location information
[0170] The location information stored locally on the master device can be the location information obtained when the master device historically starts positioning, and this location information can be the location information of the virtual screen determined by the historical master device. In this way, the master device does not need to start positioning to determine the location of the virtual screen every time it starts a meeting, which speeds up the speed of starting a meeting on the master device, and when it is inconvenient for the master device to move, it can also start a meeting quickly and conveniently.
[0171] 3) The main device obtains the location information of the virtual screen from the server
[0172] The server may store one or more positions for the user to select, and the host device may select a position from the one or more positions stored in the server as the position of the virtual screen. In this way, when the host device does not have a positioning function or is inconvenient to move, a position may be directly selected from the positions stored in the server as the position of the virtual screen.
[0173] That is to say, the master device can directly initiate positioning or select a position from known positions to determine the position of the virtual screen. It can be understood that the ways for the master device to obtain the position information of the virtual screen are not limited to the above three. For example, the master device can also send a positioning request to other devices (such as a third device), obtain the positions of other devices, and use such positions as the position of the virtual screen.
[0174] In some embodiments, the master device can also determine the size of the virtual screen, including the width and height of the virtual screen. Among them, the size of the virtual screen can be a preset size, or the master device can determine the size of the virtual screen according to user operations, or the master device can automatically adjust the size of the virtual screen according to the size of the real environment, and so on.
[0175] It should be noted that the demonstration method provided by the embodiments of the present application can be applied to meeting scenarios such as on-site meetings and remote meetings. Among them, an on-site meeting means that the speaker and the participants gather together to hold a meeting. The master device and the slave devices can be located in the same physical space. The speaker can determine the virtual screen in the physical space. The participants can view the multimedia content located on the virtual screen through the slave devices and simultaneously observe the actions of the speaker and hear the voice of the speaker. At this time, the master device can determine the position of the virtual screen through the above three methods. A remote meeting means that the speaker and the participants hold a meeting in different places. The master device and the slave devices can be located in different physical spaces. The speaker can remotely determine the virtual screen near the location of the participants. The participants can view the multimedia content located on the virtual screen through the slave devices. Further, the participants can also remotely obtain the voice of the speaker collected on the master device through the slave devices. At this time, the master device can determine the position of the virtual screen through the latter two methods above.
[0176] In some embodiments, the master device can also determine the shape of the virtual screen, and the virtual screen can be in the shape of a rectangle, a square, a circle, a heart, etc., providing a more rich and interesting viewing effect for users.
[0177] In some embodiments, the number of virtual screens can be one or more. That is to say, the master device can determine one or more positions for displaying multimedia content. In this way, the participants can adjust the viewing angle according to their own needs, enhancing the interest of the meeting and the viewing effect of the users.
[0178] It can be understood that the embodiments of the present application do not limit the execution order of step S103. Step S103 can be executed before or after step S101, or before or after step S102. That is to say, the master device can start the meeting first and then determine the position of the virtual screen, or can determine the position of the virtual screen first and then start the meeting.
[0179] S104. The master device plays the multimedia content in response to the first operation.
[0180] The master device may display a playback interface in response to the operation of playing the multimedia content, and display the multimedia content in the playback interface. For example, when the multimedia content is a presentation, one of the slides of the presentation may be displayed in the current playback interface.
[0181] Exemplarily, the playback interface may refer to the user interface 1-2 as Figure 3E shown.
[0182] S105. The master device sends the indication information of the multimedia content to the server.
[0183] The indication information sent by the master device to the server includes the following two cases:
[0184] 1) The multimedia content is the content obtained locally by the master device
[0185] When the multimedia content is the content obtained locally by the master device, the indication information sent by the master device to the server may include the multimedia content and the position information of the virtual screen. After obtaining the multimedia content, the server may send the multimedia content to the slave device for the slave device to display the multimedia content, or the server may also store the multimedia content so that the master device or other devices can obtain the multimedia content from the server when holding a meeting next time.
[0186] 2) The multimedia content is the content obtained by the master device from the server
[0187] When the multimedia content is the content obtained by the master device from the server, the indication information sent by the master device to the server may include the identifier of the multimedia content and the position information of the virtual screen.
[0188] Exemplarily, the indication information sent by the master device to the server may include the structure of the multimedia content Entity:
[0189]
[0190] Among them, Latitude represents the longitude of the position where the virtual screen is located, Longitude represents the dimension of the position where the virtual screen is located, Altitude represents the altitude of the position where the virtual screen is located, ResourceId represents the identifier of the multimedia content, ResourceUri represents the download address of the multimedia content, or the acquisition method of the multimedia content, such as HTTP download or FTP download, etc.
[0191] Phase Two: The slave device joins the meeting
[0192] As Figure 7 shown, stage two may include:
[0193] S201. Receive a second operation of joining a meeting from the device.
[0194] The device may refer to the slave device 200 mentioned above. The device may be a device used by the participant. The master device may join the meeting according to the operation of the participant. The participant may view the multimedia content provided by the master device through the slave device.
[0195] Exemplarily, the second operation may Figure 4A be the user operation acting on the first control 213 as shown.
[0196] In some embodiments, in addition to the slave device triggering to join the meeting according to the user's operation, the slave device may also trigger to join the meeting after receiving the indication information of starting the meeting sent by the master device. In other words, after the master device receives the first operation of playing multimedia content and sends the indication information of the multimedia content to the server, the server may send a request for obtaining the location information of the slave device to the slave device. Here, the slave device may refer to a device that has an association with the master device in advance, such as establishing a communication connection, or being in the same account group, etc. That is to say, when there is an association relationship between the master device and the slave device in advance, the master device may automatically pull the slave device into the meeting according to the location of the slave device when starting the meeting.
[0197] S202. The slave device sends the location information of the slave device to the server.
[0198] In response to the operation of joining the meeting received by the slave device, the slave device starts positioning and sends the location information of the slave device to the server, so that the server can determine whether to send the multimedia content to the slave device according to the position of the slave device and the virtual screen, thereby ensuring that the slave device can only display the multimedia content provided by the master device when it is relatively close to the virtual screen.
[0199] Similar to the location information of the virtual screen, the location information of the slave device may include longitude, latitude and altitude in the geodetic coordinate system. Regarding the location information of the slave device, reference may be made to the location information of the virtual screen mentioned above, which will not be elaborated here.
[0200] S203. The server determines whether the slave device is within a preset range according to the indication information of the multimedia content and the location information of the slave device.
[0201] Specifically, the server may determine whether the slave device is within the preset range of the location of the virtual screen according to the location information of the virtual screen included in the indication information of the multimedia content and the location information of the slave device.
[0202] In a specific implementation, the server can use an octree to determine whether a slave device is located within a preset range. Among them, an octree is a tree-like data structure used to describe a three-dimensional space. The server can use the octree to implement a nearest neighbor search for the location of the virtual screen, and quickly screen out slave devices that are relatively close to the virtual screen. That is to say, when the server obtains the location information of multiple slave devices, the server can use the octree to quickly screen out slave devices whose distance from the virtual screen is less than or equal to a preset value from multiple slave devices.
[0203] When the server uses an octree to determine whether a slave device is located within a preset range, it can specifically include the following steps:
[0204] Step 1: Process the data
[0205] Since the location information obtained when the device uses GPS positioning is the location information in the geodetic coordinate system, the server can convert the location information of the virtual screen and the slave device into the geocentric coordinate system, also known as the Cartesian rectangular coordinate system. Exemplarily, the server can perform coordinate conversion according to the WGS84 standard.
[0206] Specifically, the server can convert the location coordinates (Latitude, Longitude, Altitude) in the geodetic coordinate system into the location coordinates (x, y, z) in the geocentric coordinate system. Exemplarily, assume that the location coordinates of the virtual screen in the geodetic coordinate system are (Latitude0, Longitude0, Altitude0), and the converted location coordinates are (x0, y0, z0). There is a slave device A with location coordinates (Latitude1, Longitude1, Altitude1) in the geodetic coordinate system, and its converted location coordinates are (x1, y1, z1). There is a slave device B with location coordinates (Latitude2, Longitude2, Altitude2) in the geodetic coordinate system, and its converted location coordinates are (x2, y2, z2).
[0207] Step 2: Determine the octree parameters
[0208] The server can use the following formulas 1 and 2 to determine the number of levels level of the octree, and each node of the octree, and the coordinates of this node are (col_x, col_y, col_z).
[0209] level = [S / s] + 1 Formula 1
[0210] col_x = [x / (s * level)], col_y = [y / (s * level)], col_z = [z / (s * level)] Formula 2
[0211] Among them, the function y = [x] is a rounding function, where y is equal to the largest integer not exceeding x.
[0212] S represents a preset range, unit: meter. In other words, slave devices within S meters from the virtual screen can view the multimedia content. s represents the minimum interval of the octree, unit: meter. S can be determined by the presenter according to the size of the meeting venue, or S can be a fixed value, such as 3 meters. The embodiments of the present application do not limit the value of S. s is determined according to the system performance of the server. For example, when the system performance is good, s can take a smaller value; when the system performance is poor, s can take a larger value.
[0213] (x, y, z) is used to represent the position coordinates of the virtual screen or the slave device in the Cartesian rectangular coordinate system. Using Formula 2, the position coordinates of the virtual screen or the slave device can be mapped to the nodes of the octree, and the coordinates of this node are (col_x, col_y, col_z).
[0214] Exemplarily, assume that the position coordinates of the converted virtual screen are (x0, y0, z0), then the coordinates of the octree node after the virtual screen is mapped are (col_x0, col_y0, col_z0). Assume that there is a slave device A, and its converted position coordinates are (x1, y1, z1), and the coordinates of its mapped octree node are (col_x1, col_y1, col_z1). There is a slave device B, and its converted position coordinates are (x2, y2, z2), and the coordinates of its mapped octree node are (col_x2, col_y2, col_z2).
[0215] That is to say, the server can use Formula 1 and Formula 2 to map the positions where the virtual screen and the slave device are located to the nodes of the octree.
[0216] Step 3: Screen slave devices according to the octree parameters
[0217] Among them, the node mapped by the position coordinates of the virtual screen can be the central node O (col_x0, col_y0, col_z0) of the octree.
[0218] After that, the coordinates of 26 nodes around the central node O can be determined by ((col_x0 ± S, col_y0 ± S, col_z0 ± S)). Exemplarily, as Figure 8 shown, point O is the central node of the octree, and K1 - K26 are the 26 nodes around point O. The server can use these 26 nodes to screen out the slave devices that initially meet the requirements. Specifically, when the node mapped by the slave device is within these 26 nodes, the slave device initially meets the requirements.
[0219] Step 4: Further screen the slave devices using the Euclidean distance
[0220] After screening out the slave devices that initially meet the requirements, the server can further determine whether the Euclidean distance between the screened-out slave device and the virtual screen is less than S. If so, it means that the distance between the slave device and the virtual screen is less than or equal to the preset value; otherwise, the slave device is outside the preset range of the virtual screen.
[0221] As can be seen from Steps 1 to 4, the server can map the positions of the virtual screen and the slave devices into the octree, and initially screen out the slave devices that meet the requirements by determining whether the node where the slave device is located is within the specified range of the node where the virtual screen is located, that is, the 26 nodes around the node where the virtual screen is located. Then, the Euclidean distance between the slave devices that initially meet the requirements and the virtual screen is calculated to finally screen out the slave devices whose distance from the virtual screen is less than or equal to the preset value.
[0222] Compared with the server directly calculating the Euclidean distance between all slave devices and the virtual screen, initially screening out some slave devices through the octree, and then calculating the Euclidean distance between this part of the slave devices and the virtual screen to finally determine the slave devices within the preset range, its operation speed is faster and the time spent is less, enabling the slave devices to join the meeting as soon as possible and enhancing the meeting experience of the participants.
[0223] It can be understood that the server can also determine whether the slave device is within the preset range in other ways. For example, the server can also directly determine whether the slave device is within the preset range through the Euclidean distance between the slave device and the virtual screen. Another example is that the server can determine whether the slave device is within the preset range according to whether the signal strength of the slave device is within the threshold range. In addition, not limited to the octree mentioned above, the server can also use the quadtree to screen the slave devices and quickly determine whether the slave device is within the preset range. The embodiments of the present application do not limit the method for determining whether the slave device is within the preset range.
[0224] In the embodiments of the present application, the server can play the roles of storing, forwarding data, and judging distance. Among them, storing data means that the server can be used to store the positions of multimedia content and / or virtual screens. Forwarding data means that the server can be used to forward the information sent by the master device to the slave device. For example, the indication information of the multimedia content, etc. Judging distance means that the server can be used to judge whether the distance between the slave device and the virtual screen is less than or equal to a preset value. Among them, the embodiments of the present application do not limit the number of servers. Exemplarily, the server for storing the positions of multimedia content and / or virtual screens can be a storage server, and the server for judging whether the distance between the slave device and the virtual screen is less than or equal to the preset value can be a management server. The storage server and the management server can be different servers, or the storage server and the management server can be the same server.
[0225] S204. When the slave device is within the preset range, the server sends the multimedia content and the position information of the virtual screen to the slave device.
[0226] When the slave device is within the preset range of the position where the virtual screen is located, the server can send the multimedia content and the position information of the virtual screen to the slave device, so that the slave device can display the multimedia content at the specified position. Otherwise, the server does not send the multimedia content and the position information of the virtual screen to the slave device. This can ensure that the slave device can only view the multimedia content within the meeting venue, increasing the security of the meeting.
[0227] In some embodiments, the server can send the indication information of the multimedia content sent by the master device to the slave device, and the slave device can obtain the multimedia content and the position information of the virtual screen according to the indication information.
[0228] S205. The slave device turns on the camera.
[0229] When the slave device is within the preset range, the slave device can turn on the camera to collect images in real time and present the images to the user for the user to view the real environment through the slave device.
[0230] When the slave device turns on the camera, the slave device can display the images collected by the camera in real time. In addition, the slave device can display a prompt message (such as the first prompt message), and the prompt message indicates the direction in which the slave device needs to deflect, and this direction is the direction from the position where the slave device is located to the position where the virtual screen is located. This is because before the participant views the multimedia content, the participant may not know the position of the virtual screen, and the slave device can guide the user on how to move the slave device so that the camera of the slave device can point to the position where the virtual screen is located, facilitating the participant to view the multimedia content as soon as possible.
[0231] S206. The slave device superimposes and displays multimedia content on the image captured in real time by the camera according to the position information of the virtual screen.
[0232] After obtaining the position of the virtual screen from the slave device, the slave device can superimpose multimedia content on the image captured in real time by the camera according to the relative position relationship between the slave device and the virtual screen. As can be seen from the display screen of the slave device, the position where the multimedia content is located is at the position of the virtual screen determined by the master device. That is to say, the display position of the multimedia content in this image overlaps with the position of the virtual screen in the image. For example, when the presenter designates the position of the blackboard as the position of the virtual screen, when the image captured in real time by the slave device contains the blackboard, the slave device can superimpose and display multimedia content at the position of the blackboard in this image, thus giving the user a visual experience that multimedia content is being played on the blackboard.
[0233] It should be noted that when the slave device is an AR device, step S205 is an optional step, and the slave device can not superimpose multimedia content on the image captured in real time by the camera, but according to the position information of the virtual screen, project the multimedia content onto the user's retina through the lens of the slave device, so that the multimedia content is superimposed on the position of the virtual screen where the user views through the lens of the AR device. In this way, when the user views the real environment through the AR device, the multimedia content is superimposed on the position of the virtual screen in the real environment.
[0234] Specifically, the AR device can include a light-transmitting lens through which the user can directly see the real environment. In addition, the AR device can also include a projector that can project the multimedia content and then project it through the lens onto the human eye retina to be superimposed with the real environment directly seen by the user. In the superimposed image, the multimedia content is displayed at the position of the specified virtual screen in the real environment. Among them, the slave device can determine the projection angle and position of the multimedia content according to the relative position and relative angle between the slave device and the virtual screen in the actual physical space, so that in the image observed by the user's human eye, the multimedia content is superimposed on the position of the virtual screen.
[0235] Phase Three: The master device changes the meeting content
[0236] As Figure 9 shown, Phase Three can include:
[0237] S301. The master device receives a third operation on the multimedia content.
[0238] In some embodiments, the master device can control the playback, pause, switching, and ending of multimedia content according to user operations. For example, the master device can receive a user operation to switch multimedia content and change the display screen of the multimedia content, that is, display the next page of the multimedia content.
[0239] Exemplarily, the third operation can refer to a user operation on the third icon 122C as Figure 3E shown.
[0240] S302. In response to the third operation, the master device performs a fourth operation on the multimedia content.
[0241] When the third operation received by the master device is a pause operation, the master device can respond to this operation and pause the playback of the multimedia content. When the third operation received by the master device is an end operation, the master device can respond to this operation and end the display of the multimedia content. When the third operation received by the master device is a switching operation, the master device can respond to this operation and switch the display screen of the multimedia content. When the third operation received by the master device is a playback operation, the master device can respond to this operation and start playing the multimedia content.
[0242] Exemplarily, referring to Figures 3E - 3F , Figure 3E shows the previous page content of the multimedia content, Figure 3F shows the next page content of the multimedia content. When the master device detects a user operation on the third icon 122C as Figure 3E shown, the content displayed by the master device can be switched from Figure 3E to Figure 3F shown.
[0243] S303. The master device sends the information indicating the third operation to the server.
[0244] The master device sends the information indicating the third operation to the server so that the master device can send the information indicating the third operation to the slave device through the server, so that the slave device can synchronously control the slave device to perform the fourth operation on the multimedia content according to the operation performed by the user on the master device, and control the multimedia content to complete switching, ending, playing, pausing, etc.
[0245] In some embodiments, during the process in which the master device sends the information indicating the third operation to the server, the master device may also send information such as the page number of the multimedia content and the identifier of the multimedia content to the server, so that the server can send the page number of the multimedia content and the identifier of the multimedia content to the slave device. The slave device can determine the multimedia content targeted by the third operation according to the identifier of the multimedia content. Further, the slave device can determine the specific content in the multimedia content targeted by the third operation according to the page number. For example, assuming that the page number is 4, when the third operation acting on the multimedia content received by the master device is used to complete the switching of the multimedia content, the slave device can control the multimedia content to switch from the content corresponding to page 4 to the content corresponding to page 5 according to the page number and the third operation.
[0246] It can be understood that the embodiments of the present application do not limit the execution order of step S302 and step S303. The master device may first send the information indicating the third operation to the server, and then perform the fourth operation on the multimedia content, or, after performing the fourth operation on the multimedia content, send the information indicating the third operation to the server, or, the master device may perform the fourth operation on the multimedia content while sending the information indicating the third operation to the server.
[0247] S304. The server sends a request for obtaining location information to the slave device.
[0248] After receiving the information sent by the master device, the server may trigger the slave device to perform positioning to obtain the current location information of the slave device. Ensure that when the distance between the slave device and the virtual screen is less than or equal to the preset value, the multimedia content displayed on the slave device can be synchronously controlled according to the operations of the master device on the multimedia content.
[0249] S305. The slave device sends the location information of the slave device to the server.
[0250] The slave device can start the positioning function according to the obtained request, obtain the current location information of the slave device, and send the current location information of the slave device to the server.
[0251] S306. The server determines whether the slave device is located within a preset range according to the location information of the virtual screen and the location information of the slave device.
[0252] Specifically, the server can determine whether the slave device is located within the preset range of the location of the virtual screen according to the positional relationship between the virtual screen and the slave device. Among them, the server can use an octree to determine whether the slave device is located within the preset range. For the specific description of this part, reference can be made to the relevant content of step S203 described above.
[0253] Compared with the location information of the slave device obtained by the server in step S203, if the location of the slave device obtained in step S306 has changed, the server can recalculate the node coordinates in the octree to which the location of the slave device is mapped, so as to re-determine whether the location of the slave device is within the preset range. On the contrary, if the location of the slave device has not changed, the server can directly obtain whether the slave device is within the preset range according to the result determined in step S203.
[0254] S307. When the slave device is within the preset range, the server sends the information indicating the third operation to the slave device.
[0255] When the slave device is within the preset range, it means that the slave device is still within the meeting venue. The slave device can synchronously trigger the control of the multimedia content displayed on the slave device according to the control of the multimedia content on the master device, that is, obtain the information indicating the operation received by the master device sent by the server.
[0256] On the contrary, if the slave device is not within the preset range, the server may not send the information indicating the third operation to the slave device. In this way, when the slave device is far away from the meeting venue, the multimedia content displayed on the slave device will not change following the operation of the multimedia content on the main device by the speaker, thus avoiding the slave device from still displaying the meeting content after being far away from the meeting venue and saving system resources.
[0257] S308. In response to the third operation, the slave device performs a fourth operation on the image captured in real time by the camera for the multimedia content.
[0258] In response to the third operation of the multimedia content on the master device by the speaker, the slave device can synchronously change the multimedia content displayed on the slave device, such as pausing the multimedia content, playing the multimedia content, ending the multimedia content, or switching the multimedia content, etc. For the specific fourth operation performed on the slave device, reference can be made to the relevant description in step S302, which will not be elaborated here.
[0259] Exemplarily, referring to Figures 4C - 4D , in the multimedia content displayed on the slave device, the multimedia content can follow the switching operation on the master device by the user, from Figure 4C the previous page content shown in Figure 4D switch to the next page content shown in
[0260] (2) Embodiment 2
[0261] Figure 10 Shows a schematic flowchart of another presentation method provided by an embodiment of the present application.
[0262] As Figure 10As shown, the method involves a master device and a slave device, and the method includes:
[0263] Phase 1 (S401 - S404): The master device initiates a meeting
[0264] S401. The master device obtains multimedia content.
[0265] S402. The master device receives a first operation to play the multimedia content.
[0266] S403. The master device obtains the position information of the virtual screen.
[0267] S404. In response to the first operation, the master device plays the multimedia content.
[0268] Among them, steps S401 - S404 and Figure 6 steps S101 - S104 in are the same, and the relevant description can be referred to.
[0269] Phase 2 (S405 - S410): The slave device joins the meeting
[0270] S405. The slave device receives a second operation to join the meeting.
[0271] S406. The slave device sends the position information of the slave device to the master device.
[0272] S407. The master device determines whether the slave device is within a preset range according to the position information of the virtual screen and the position information of the slave device.
[0273] S408. When the slave device is within the preset range, the master device sends the multimedia content and the position information of the virtual screen to the slave device.
[0274] S409. The slave device turns on the camera.
[0275] S410. The slave device superimposes and displays the multimedia content on the image captured in real time by the camera according to the position information of the virtual screen.
[0276] Among them, steps S405, S409, S410 and Figure 7 steps S201, S205, S206 in are the same, and the relevant description can be referred to. Steps S406 - S408 and Figure 7 steps S202 - S204 in are similar, and the difference lies in Figure 10The slave device in this case sends the location information of the slave device to the master device, and the master device determines whether the slave device is within a preset range. If so, the master device sends the multimedia content and the location information of the virtual screen to the slave device, or the master device sends the indication information of the multimedia content to the slave device, and the slave device obtains the multimedia content according to the indication information, such as downloading the multimedia content from the server. At this time, it is not necessary for the master device to send the multimedia content and the location information of the virtual screen to the server, and Figure 7 In this case, the slave device sends the location information of the slave device to the server, and the server determines whether the slave device is within a preset range. If so, the server sends the multimedia content provided by the master device and the location information of the virtual screen to the slave device. For the specific implementation of steps S406 - S408, reference can be made to the relevant descriptions of steps S202 - S204, which will not be elaborated here.
[0277] Phase Three (S411 - S417): The master device changes the meeting content
[0278] S411. The master device receives a third operation on the multimedia content.
[0279] S412. The master device sends a request for obtaining location information to the slave device.
[0280] S413. The slave device sends the location information of the slave device to the master device.
[0281] S414. The master device determines whether the slave device is within a preset range according to the location information of the virtual screen and the location information of the slave device.
[0282] S415. In response to the third operation, the master device performs a fourth operation on the multimedia content.
[0283] S416. When the slave device is within the preset range, the master device sends the information indicating the third operation to the slave device.
[0284] S417. In response to the third operation, the slave device performs a fourth operation on the multimedia content on the image captured by the camera in real time.
[0285] Among them, steps S411, S416, S417 and Figure 9 steps S301, S303, S308 in Figure 9 are the same, and reference can be made to the relevant descriptions. Steps S412 - S415 and Figure 10The master device directly sends a request for obtaining the location information of the slave device to the slave device, and the slave device also directly sends the location information to the master device. The master device determines whether the slave device is within a preset range based on the location information of the virtual screen and the location information of the slave device. If so, the master device directly sends the indication information of the third operation to the slave device, and there is no process in which the master device sends the indication information to the server and then the server forwards it to the slave device. And Figure 9 In this case, the master device sends the indication information to the server, and the server sends a request for obtaining the location information to the slave device, obtains the location information of the slave device, and determines whether the slave device is within the preset range through the server. If so, the server sends the indication information of the third operation to the slave device. For the specific implementation of steps S412 - S415, reference can be made to the relevant descriptions of steps S304 - S307, which will not be elaborated here.
[0286] It can be understood that the demonstration method involving the master device and the slave device provided in the embodiments of the present application can also be referred to as a demonstration system. The demonstration system includes the communication process between the master device and the slave device, or further includes the communication process between the server and the master device and the slave device. The embodiments of the present application do not limit this name.
[0287] Next, Figure 11 the demonstration device provided in the embodiments of the present application will be introduced.
[0288] As Figure 11 shown, the demonstration device may include: an acquisition module 001, a positioning module 002, a storage module 003, an arithmetic module 004, a superimposing module 005, and a display module 006. Among them:
[0289] The acquisition module 001 can be used to acquire multimedia content. This includes acquiring multimedia content from the local storage space or downloading the multimedia content from the server. In addition, the acquisition module 001 can also send the multimedia content to the storage module 003 for the storage module 003 to store the multimedia content.
[0290] The positioning module 002 can be used to activate the positioning function and obtain the location information of the current location, which may include the location information of the virtual screen and the location information of the slave device. In addition, the positioning module 002 can also send the location information to the storage module 003 for the storage module 003 to store the location information.
[0291] The storage module 003 can be used to store the location information and multimedia content.
[0292] The arithmetic module 004 can be used to determine whether the distance between the slave device and the virtual screen is less than or equal to a preset value based on the location information of the virtual screen and the location information of the slave device.
[0293] The superimposing module 005 can be used to superimpose multimedia content on the image captured in real time by the camera of the slave device when the distance between the slave device and the virtual screen is less than or equal to a preset value.
[0294] The display module 006 can be used to implement the display of multimedia content on the master device and the slave device. Different from the master device, in addition to displaying multimedia content, the slave device can also display the image captured in real time by the camera, and the multimedia content is displayed at a specified position in the image. In addition, in the master device, the display module 006 can also receive operations on the multimedia content, triggering changes to the multimedia content displayed on the master device and the slave device. For example, triggering the switching of the previous and next pages of the displayed multimedia content, pausing or ending the playback of the multimedia content, and so on.
[0295] It should be noted that in Embodiment 1, the acquisition module 001, the positioning module 002, and the display module 006 can be modules included in both the master device and the slave device, and the superimposing module 005 is a module included in the slave device. In the master device, the acquisition module 001 can be used to acquire multimedia content, the positioning module 002 can be used to determine the position of the virtual screen, and the display module 006 can be used to display multimedia content. In the slave device, the acquisition module 001 can be used to acquire multimedia content, the positioning module 002 can be used to start positioning and send the position information of the slave device to the server, the superimposing module 005 can be used to superimpose multimedia content on the image captured by the camera, and the display module 006 can be used to display the superimposed image. The storage module 003 and the operation module 004 can be modules included in the server. In the server, the storage module 003 can be used to store multimedia content and the position of the virtual screen, and the operation module 004 can be used to determine whether the distance between the slave device and the virtual screen is less than the preset value. In Embodiment 2, different from Embodiment 1, the storage module 003 and the operation module 004 can be modules included in the master device. In the master device, the storage module 003 can be used to store multimedia content and the position of the virtual screen, and the operation module 004 can be used to determine whether the distance between the slave device and the virtual screen is less than the preset value. For the parts not mentioned in the demonstration device, reference can be made to the foregoing content, which will not be elaborated here.
[0296] Figure 12 The schematic hardware structure diagram of the electronic device 400 is shown.
[0297] The electronic device 400 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.
[0298] The electronic device 400 may be the main device mentioned above or a slave device.
[0299] The electronic device 400 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0300] It can be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 400. In other embodiments of the present application, the electronic device 400 may include more or fewer components than those illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0301] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.
[0302] The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.
[0303] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0304] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 can receive the charging input from a wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 can receive the wireless charging input through the wireless charging coil of the electronic device 400. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.
[0305] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives the inputs from the battery 142 and / or the charging management module 140 and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as the battery capacity, the number of battery charge cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be provided in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be provided in the same device.
[0306] The wireless communication function of the electronic device 400 can be implemented by antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.
[0307] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 400 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0308] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 400. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves through antenna 1 and radiate it out. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be disposed in the same device.
[0309] The modulation and demodulation processor can include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to speaker 170A, receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor can be an independent device. In some other embodiments, the modulation and demodulation processor can be independent of the processor 110 and be disposed in the same device as the mobile communication module 150 or other functional modules.
[0310] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 400, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.
[0311] In some embodiments, antenna 1 of electronic device 400 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, such that electronic device 400 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).
[0312] Electronic device 400 implements a display function through a GPU, display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
[0313] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be made of an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniLED, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N display screens 194, where N is a positive integer greater than 1.
[0314] The electronic device 400 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.
[0315] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera photosensitive element. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image through algorithms. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.
[0316] The camera 193 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the electronic device 400 may include one or N cameras 193, where N is a positive integer greater than 1.
[0317] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 400 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.
[0318] The video codec is used to compress or decompress digital videos. The electronic device 400 can support one or more video codecs. In this way, the electronic device 400 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0319] The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 400 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.
[0320] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).
[0321] The random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally called DDR5 SDRAM), etc.; the non-volatile memory may include disk storage devices, flash memory.
[0322] Flash memory can be classified into NOR Flash, NAND Flash, 3D NAND Flash, etc. according to the operating principle, and can be classified into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the number of potential levels of storage cells. According to the storage specification, it can be classified into universal flash storage (UFS), embedded multi media Card (eMMC), etc.
[0323] The random access memory can be directly read and written by the processor 110, and can be used to store the operating system or executable programs of other running programs (such as machine instructions), and can also be used to store data of users and application programs, etc.
[0324] The non-volatile memory can also store executable programs and data of users and application programs, etc., and can be pre-loaded into the random access memory for the processor 110 to directly read and write.
[0325] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 400. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external non-volatile memory.
[0326] The electronic device 400 can implement audio functions through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, etc. For example, music playback, recording, etc.
[0327] The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert analog audio input into a digital audio signal. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.
[0328] The speaker 170A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 400 can listen to music or hands-free calls through the speaker 170A.
[0329] The receiver 170B, also known as the "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device 400 answers a call or a voice message, the voice can be listened to by bringing the receiver 170B close to the human ear.
[0330] The microphone 170C, also known as the "microphone" or "transmitter", is used to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak by bringing the mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 400 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 400 may be provided with two microphones 170C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 400 may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, implement a directional recording function, etc.
[0331] The headphone jack 170D is used to connect a wired headphone. The headphone jack 170D may be a USB interface 130, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.
[0332] The pressure sensor 180A is used to sense a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 400 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 400 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 400 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations with the same touch position but different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.
[0333] The gyro sensor 180B can be used to determine the motion posture of the electronic device 400. In some embodiments, the angular velocity of the electronic device 400 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 400 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 400 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.
[0334] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 400 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
[0335] The magnetic sensor 180D includes a Hall sensor. The electronic device 400 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 400 is a flip phone, the electronic device 400 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.
[0336] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 400 in all directions (generally three axes). When the electronic device 400 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.
[0337] The distance sensor 180F is used to measure the distance. The electronic device 400 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 400 can use the distance sensor 180F to measure the distance to achieve fast focusing.
[0338] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 400 emits infrared light outward through the light emitting diode. The electronic device 400 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 400. When insufficient reflected light is detected, the electronic device 400 can determine that there is no object near the electronic device 400. The electronic device 400 can use the proximity light sensor 180G to detect that the user holds the electronic device 400 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.
[0339] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 400 can adaptively adjust the brightness of the display screen 194 according to the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance during photography. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 400 is in a pocket to prevent accidental touch.
[0340] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 400 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering of incoming calls, etc.
[0341] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 400 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 400 reduces the performance of the processor near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 400 heats the battery 142 to prevent abnormal shutdown of the electronic device 400 caused by low temperature. In some other embodiments, when the temperature is lower than yet another threshold, the electronic device 400 boosts the output voltage of the battery 142 to prevent abnormal shutdown caused by low temperature.
[0342] The touch sensor 180K, also known as a "touch control device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch control screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 400, at a different position from the display screen 194.
[0343] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals of the vibrating bone mass of the human vocal part. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulsation signals. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone to form a bone conduction earphone. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M to implement voice functions. The application processor can parse out heart rate information based on the blood pressure pulsation signals acquired by the bone conduction sensor 180M to implement heart rate detection functions.
[0344] The button 190 includes a power-on button, volume buttons, etc. The button 190 can be a mechanical button or a touch button. The electronic device 400 can receive button inputs and generate key signal inputs related to the user settings and function controls of the electronic device 400.
[0345] The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations for different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. For touch operations on different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminder, receiving messages, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.
[0346] The indicator 192 can be an indicator light and can be used to indicate the charging status, power change, and can also be used to indicate messages, missed calls, notifications, etc.
[0347] The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation from the electronic device 400. The electronic device 400 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 400 interacts with the network through the SIM card to achieve functions such as calls and data communication. In some embodiments, the electronic device 400 uses an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 400 and cannot be separated from the electronic device 400.
[0348] In the embodiments of the present application, when the electronic device 400 is the main device 100:
[0349] The mobile communication module 150 or the wireless communication module 160 can be used to obtain multimedia content and the position information of the virtual screen, send the indication information of the multimedia content and the position information of the virtual screen to other devices, such as a server or a slave device, and after receiving an operation on the multimedia content, send the information indicating the operation to other devices.
[0350] The display screen 194 can be used to display multimedia content and the relevant user interfaces involved when starting a meeting. For the specific description of this user interface, reference can be made to the foregoing Figures 3A - 3FThe relevant content is not elaborated here.
[0351] The internal memory 121 can be used to store multimedia content, as well as virtual screens and the location information of the device.
[0352] In the embodiments of the present application, when the electronic device 400 is a slave device 200:
[0353] The mobile communication module 150 or the wireless communication module 160 can be used to obtain the location information of the slave device, and send the location information of the slave device to other devices, such as a server or a master device, receive the multimedia content and the location information of the virtual screen sent by other devices, and the information indicating the operations acting on the multimedia content received on the master device.
[0354] The camera 193 can be used to collect images of the real environment in real time.
[0355] The display screen 194 can be used to display multimedia content, as well as relevant user interfaces involved when joining a meeting. For the specific description of this user interface, reference can be made to the foregoing Figures 4A - 4D The relevant content is not elaborated here.
[0356] The electronic device can be a portable terminal device equipped with iOS, Android, Microsoft or other operating systems, such as a mobile phone, a tablet computer, a wearable device, etc., and can also be a non-portable terminal device such as a laptop computer with a touch-sensitive surface or a touch panel, or a desktop computer with a touch-sensitive surface or a touch panel. The software system of the electronic device 400 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present invention, taking the Android system with a layered architecture as an example, the software structure of the electronic device 400 is exemplarily described.
[0357] Figure 13 It is the software structure block diagram of the electronic device 400 in the embodiments of the present invention.
[0358] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom are the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer.
[0359] The application layer can include a series of application packages.
[0360] Such as Figure 13 As shown, the application packages can include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, etc.
[0361] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.
[0362] As Figure 13 shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.
[0363] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.
[0364] The content provider is used to store and obtain data, and make this data accessible to applications. The data may include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc.
[0365] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a short message notification icon may include a view for displaying text and a view for displaying pictures.
[0366] The phone manager is used to provide the communication function of the electronic device 400. For example, the management of call states (including connection, disconnection, etc.).
[0367] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, and so on.
[0368] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages, which can disappear automatically after a short stay without user interaction. For example, the notification manager is used to inform that the download is completed, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or a scroll bar text, such as the notification of a background-running application, and can also be a notification that appears on the screen in the form of a dialogue window. For example, prompting text information in the status bar, emitting a prompt sound, the electronic device vibrating, the indicator light flashing, etc.
[0369] Android Runtime includes a core library and a virtual machine. Android runtime is responsible for the scheduling and management of the Android system.
[0370] The core library contains two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core library of Android.
[0371] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to manage the object life cycle, stack management, thread management, security and exception management, as well as garbage collection and other functions.
[0372] The system library can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), etc.
[0373] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
[0374] The media library supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
[0375] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.
[0376] The 2D graphics engine is a graphics engine for 2D drawing.
[0377] The kernel layer is the layer between the hardware and the software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a sensor driver.
[0378] Figure 14 The hardware structure diagram of the electronic device 500 provided in the embodiment of the present application is shown.
[0379] The electronic device 500 can be Figure 1 The server 300 in the communication system 1000 shown.
[0380] As Figure 14 shown, the electronic device 500 may include: one or more processors 201, a memory 202, a communication interface 203, a transmitter 205, a receiver 206, a coupler 207, and an antenna 208. These components can be connected through a bus 204 or other means. Figure 14 Taking the connection through the bus as an example. Among them:
[0381] The communication interface 203 can be used for the electronic device 500 to communicate with other communication devices, such as the electronic device 400. Specifically, the communication interface 203 can be a 3G communication interface, a Long-Term Evolution (LTE) (4G) communication interface, a 5G communication interface, a WLAN communication interface, a WAN communication interface, and so on. Not limited to wireless communication interfaces, the electronic device 500 can also be configured with a wired communication interface 203 to support wired communication. For example, the backhaul link between the electronic device 500 and other servers can be a wired communication connection.
[0382] In some embodiments of the present application, the transmitter 205 and the receiver 206 can be regarded as a wireless modem. The transmitter 205 can be used to perform transmission processing on the signals output by the processor 201. The receiver 206 can be used to receive signals. In the electronic device 500, the number of the transmitter 205 and the receiver 206 can both be one or more. The antenna 208 can be used to convert the electromagnetic energy in the transmission line into electromagnetic waves in free space, or convert the electromagnetic waves in free space into electromagnetic energy in the transmission line. The coupler 207 can be used to divide the mobile communication signal into multiple paths and distribute them to multiple receivers 206. It can be understood that the antenna 208 of the electronic device 500 can be implemented as a massive antenna array.
[0383] In some embodiments, the transmitter 205 can be used to send multimedia content and the position information of the virtual screen to the slave device, or send a request for obtaining position information to the slave device, indicating the information of the operation received by the master device. The receiver 206 can be used to receive multimedia content or the indication information of multimedia content, or the position information of the slave device.
[0384] The memory 202 is coupled to the processor 201 and is used to store various software programs and / or multiple sets of instructions. Specifically, the memory 202 can include high-speed random access memory, and can also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.
[0385] The memory 202 can store an operating system (hereinafter referred to as the system), such as embedded operating systems like uCOS, VxWorks, RTLinux, etc.
[0386] In some embodiments, the memory 202 can be used to store multimedia content, virtual screens, the position information of the slave device, and so on.
[0387] In the embodiments of the present application, the processor 201 can be used to read and execute computer-readable instructions. Specifically, the processor 201 can be used to call the program stored in the memory 202, such as the implementation program of the configuration access policy provided by one or more embodiments of the present application on the side of the electronic device 500, and execute the instructions included in the program.
[0388] In some embodiments, the processor 201 may be configured to calculate whether the distance between the slave device and the virtual screen is less than or equal to a preset value according to the position information of the virtual screen and the position information of the slave device.
[0389] It should be noted that Figure 14 The illustrated electronic device 500 is merely one implementation manner of the embodiments of the present application. In practical applications, the electronic device 500 may further include more or fewer components, which are not limited herein.
[0390] It should be understood that the steps in the above method embodiments may be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The method steps disclosed in combination with the embodiments of the present application may be directly embodied as being executed and completed by the hardware processor, or executed and completed by the combination of the hardware and software modules in the processor.
[0391] The present application further provides an electronic device, which may include: a memory and a processor. Among them, the memory may be used to store a computer program; the processor may be used to call the computer program in the memory so that the electronic device executes the methods executed by the electronic device 400 or the electronic device 500 in any one of the above embodiments.
[0392] The present application further provides a chip system, the chip system includes at least one processor, and is used to implement the functions involved in the methods executed by the electronic device 400 or the electronic device 500 in any one of the above embodiments.
[0393] In a possible design, the chip system further includes a memory, and the memory is used to store program instructions and data, and the memory is located inside or outside the processor.
[0394] The chip system may be composed of chips, or may include chips and other discrete devices.
[0395] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor, which is implemented by reading the software code stored in the memory.
[0396] Optionally, there may also be one or more memories in the chip system. The memory may be integrated with the processor or may be separately provided from the processor. The embodiments of the present application do not limit this. Exemplarily, the memory may be a non-transitory processor, such as a read-only memory (ROM). It may be integrated with the processor on the same chip or may be separately provided on different chips. The embodiments of the present application do not specifically limit the type of the memory and the setting manner of the memory and the processor.
[0397] Exemplarily, the chip system may be a field programmable gate array (FPGA), may be an application specific integrated circuit (ASIC), may also be a system on chip (SoC), may also be a central processing unit (CPU), may also be a network processor (NP), may also be a digital signal processing circuit (DSP), may also be a micro controller unit (MCU), may also be a programmable logic device (PLD), or other integrated chips.
[0398] The present application also provides a computer program product, which includes: a computer program (which may also be referred to as code or instruction). When the computer program is run, it causes the computer to execute the method performed by any one of the electronic devices 400 or 500 in any of the above embodiments.
[0399] The present application also provides a computer-readable storage medium, which stores a computer program (which may also be referred to as code or instruction). When the computer program is run, it causes the computer to execute the method performed by any one of the electronic devices 400 or 500 in any of the above embodiments.
[0400] It should be understood that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by the hardware decoding processor, or executed by a combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.
[0401] In addition, the embodiments of the present application further provide a device. The device can specifically be a component or a module. The device may include one or more processors and a memory connected thereto. Among them, the memory is used to store a computer program. When the computer program is executed by one or more processors, the device is caused to execute the methods in the above method embodiments.
[0402] Among them, the device, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be elaborated here.
[0403] The various embodiments of the present application can be combined arbitrarily to achieve different technical effects.
[0404] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk (SSD)), etc.
[0405] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware with a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The foregoing storage medium includes various media that can store program codes, such as ROM or random access memory RAM, magnetic disks, or optical discs.
[0406] In summary, the above descriptions are only embodiments of the technical solutions of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made according to the disclosure of the present invention shall be included within the protection scope of the present invention.
Claims
1. A demonstration system, characterized in that, the system is applied to a communication system including a first device and a second device, where: the first device receives a first operation; in response to the first operation, the first device displays first content; when the second device is within a preset range of a first position, the second device acquires the first content, and the first position is a position in the physical space determined by the first device; the second device superimposes and displays the first content on an image captured by the camera in real time, where the display position of the first content in the image overlaps with the first position in the image; or, the second device projects the first content onto the retina of the user through the lens of the second device, so that the first content is superimposed on the first position viewed by the user through the lens; after the first device displays the first content: the first device receives a third operation on the first content; in response to the third operation, the first device performs a fourth operation on the first content, and the fourth operation includes any one of the following: switching the display screen of the first content, ending the display of the first content, pausing the playback of the first content, or starting the playback of the first content; after the second device superimposes and displays the first content on an image captured by the camera in real time, or after the second device projects the first content onto the retina of the user through the lens of the second device: the second device acquires first information sent by the first device indicating the third operation; the second device, after acquiring the first information, performs the fourth operation on the first content.
2. The system according to claim 1, characterized in that, when the second device is within a preset range of the first position, before the second device acquires the first content, the system further includes: the first device sends indication information of the first content to the server; when the second device is within a preset range of the first position, the second device acquiring the first content specifically includes: the second device receives a second operation and sends the position of the second device to the server; the server determines that the second device is within a preset range of the first position according to the position of the second device, and sends the indication information to the second device; the second device acquires the first content according to the indication information.
3. The system according to claim 1, characterized in that, when the second device is within a preset range of the first position, the second device acquiring the first content specifically includes: the second device receives a second operation and sends the position of the second device to the first device; the first device determines that the second device is within a preset range of the first position according to the position of the second device, and sends indication information to the second device; the second device acquires the first content according to the indication information.
4. The system according to claim 1, wherein, the first device sending first information indicating the third operation to the second device specifically includes: when the second device is within a preset range of the first position, the first device sends first information indicating the third operation to the second device.
5. The system according to any one of claims 1-4, wherein, the first position includes one or more positions, and the second device includes one or more devices.
6. The system according to any one of claims 1-4, wherein, the first position is a position obtained when the first device or the third device starts positioning, the third device being a device different from the first device or the second device in the physical space, or the first position is a position pre-stored in the first device or the server.
7. The system according to any one of claims 1-4, wherein, before the second device superimposes and displays the first content in an image captured in real time by the camera, or before the second device projects the first content onto the retina of the user through the lens of the second device, the system further includes: the second device displays a first prompt message for instructing the second device to deflect in a first direction, the first direction being from the second device to the first position.
8. The system according to any one of claims 1-4, wherein, the first device and the second device are in the same physical space, or the first device and the second device are in different physical spaces.
9. The system according to any one of claims 1-4, wherein, the first content includes multimedia content of a meeting.
10. The system according to any one of claims 1-4, wherein, the second device includes a first control and a second control, the first control being used to trigger an operation of joining a meeting, and the second control being used to trigger an operation of creating a meeting.
11. The system according to any one of claims 1-4, wherein, the second device further adjusts the position of the camera to adjust the viewing angle of the first content, and / or adjusts the focal length of the camera to magnify or reduce the first content.
12. The system according to any one of claims 1-4, wherein, the first content includes any one of visual contents such as video, picture, text, photo, and chart, and the first content is two-dimensional plane content, and / or three-dimensional stereoscopic content.
13. A presentation method, wherein, the method is applied to a first device, and the method includes: the first device receives a first operation; in response to the first operation, the first device displays first content; Wherein, the first content is acquired by the second device when the second device is within a preset range of the first position. The first position is a position determined by the first device in the physical space, and the first content is superimposed and displayed in the image captured in real time by the camera of the second device. The display position of the first content in the image overlaps with the first position in the image; or, the second device projects the first content onto the retina of the user through the lens of the second device, so that the first content is superimposed on the first position viewed by the user through the lens. After the first device displays the first content, the method further includes: The first device receives a third operation on the first content; In response to the third operation, the first device performs a fourth operation on the first content. The fourth operation includes any one of the following: switching the display screen of the first content, ending the display of the first content, pausing the playback of the first content, or starting the playback of the first content; The first device sends a first message indicating the third operation, and the first message is used to trigger the second device to perform the fourth operation on the first content.
14. The method according to claim 13, wherein, the method further includes: The first device sends the indication information of the first content to the server, so that when the second device is within the preset range of the first position, the server sends the indication information to the second device, and the indication information is used for the second device to acquire the first content; or, The first device acquires the position of the second device; the first device determines that the second device is within the preset range of the first position according to the position of the second device, and sends the indication information to the second device.
15. The method according to claim 13, wherein, The first device sending the first message indicating the third operation specifically includes: When the second device is within the preset range of the first position, the first device sends the first message indicating the third operation to the second device.
16. The method according to any one of claims 13-15, wherein, The first position includes one or more positions, and the second device includes one or more devices.
17. The method according to any one of claims 13-15, wherein, The first position is the position acquired when the first device or the third device starts positioning. The third device is a device different from the first device or the second device in the physical space, or the first position is a position pre-stored by the first device or the server.
18. The method according to any one of claims 13-15, wherein, The first device and the second device are in the same physical space, or the first device and the second device are in different physical spaces.
19. A demonstration method, wherein, The method is applied to a second device, and the method includes: When the second device is within a preset range of a first position, the second device obtains first content for display by a first device, where the first position is a position in a physical space determined by the first device; The second device superimposes and displays the first content in an image captured by a camera in real time, where a display position of the first content in the image overlaps with the first position in the image; or, the second device projects the first content onto a user's retina through a lens of the second device, such that the first content is superimposed on the first position that the user views through the lens; After the second device superimposes and displays the first content in an image captured by the camera in real time, or after the second device projects the first content onto the user's retina through the lens of the second device, the method further includes: The second device obtains first information indicating a third operation sent by the first device, where the third operation is an operation on the first content received by the first device, and the first device responds to the third operation to perform a fourth operation on the first content, and the fourth operation includes any one of the following: switching a display screen of the first content, ending display of the first content, pausing playback of the first content, or starting playback of the first content; After obtaining the first information, the second device performs the fourth operation on the first content.
20. The method according to claim 19, wherein, when the second device is within a preset range of the first position, the second device obtaining the first content specifically includes: The second device receives a second operation and sends the position of the second device to a server; when the second device is within a preset range of the first position, the second device obtains indication information sent by the server, and the indication information is used for the second device to obtain the first content; or, The second device receives a second operation and sends the position of the second device to the first device; when the second device is within a preset range of the first position, the second device obtains indication information sent by the first device; The second device obtains the first content according to the indication information.
21. The method according to claim 19, wherein, the second device obtaining the first information sent by the first device specifically includes: When the second device is within a preset range of the first position, the second device obtains the first information sent by the first device.
22. The method according to any one of claims 20 - 21, wherein, the first position includes one or more positions, and the second device includes one or more devices.
23. The method according to any one of claims 20 - 21, wherein, The first position is the position obtained when the first device or the third device starts positioning, where the third device is a device different from the first device or the second device in the physical space, or the first position is a position pre-stored by the first device or the server.
24. The method according to any one of claims 20-21, wherein, before the second device superimposes and displays the first content in the image captured by the camera in real time, or before the second device projects the first content onto the retina of the user through the lens of the second device, the method further includes: The second device displays a first prompt message for instructing the second device to deflect in a first direction, where the first direction points from the second device to the first position.
25. The method according to any one of claims 20-21, wherein, The first device and the second device are in the same physical space, or the first device and the second device are in different physical spaces.
26. An electronic device, wherein, comprising a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method according to any one of claims 13 to 18.
27. An electronic device, wherein, comprising a memory, one or more processors, and one or more programs; when the one or more processors execute the one or more programs, the electronic device implements the method according to any one of claims 19 to 25.
28. A computer-readable storage medium, comprising instructions, wherein, when the instructions run on an electronic device, the electronic device executes the method according to any one of claims 13 to 18; or when the instructions run on an electronic device, the electronic device executes the method according to any one of claims 19 to 25.
29. A computer program product, wherein, when the computer program product runs on a computer, the computer executes the method according to any one of claims 13 to 18; or when the computer program product runs on a computer, the computer executes the method according to any one of claims 19 to 25.
Citation Information
Patent Citations
Content display property management
CN111226189A
Content sharing system and terminal
WO2021172137A1