A system configured to share presentation data, a method of sharing a display image, and a device for sharing presentation data
The content sharing system addresses the challenge of identifying and prioritizing content by using an automated process to monitor applications, generate presentation data, and transmit it securely to a presentation device, resulting in an enhanced and secure user experience.
Patent Information
- Application Number
- JP2024564672
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2022-05-03
- Publication Date
- 2025-05-27
AI Technical Summary
Existing content sharing systems face challenges in efficiently identifying and prioritizing relevant content for presentation, leading to potential exposure of personal or confidential information and a suboptimal user experience.
The system employs an automated process to monitor applications, identify high-priority content, generate presentation data, convert it into a presentation signal, and transmit it to a presentation device, all while ensuring user experience is context-sensitive and applicable.
This approach enables efficient and secure sharing of prioritized content, enhancing the user experience by reducing the need for manual content selection and minimizing the risk of information leakage.
Smart Images

Figure 2025516280000001_ABST
Abstract
Description
Technical Field
[0001] The disclosure of the present invention generally relates to a content sharing system, and more particularly to a content sharing system for determining content with priority.
Background Art
[0002] As the demand for remote communication increases, there is a decrease in people's social communication. It is necessary to improve the communication characteristics of the content sharing system. In this way, the online communication and people's social communication can be brought closer to each other by a bridge between them.
Summary of the Invention
[0003] To overcome the drawbacks, the present invention provides a reduction or elimination of the above problems.
[0004] To enable a more detailed understanding of the above features of the present disclosure, a more specific description of the present disclosure briefly summarized above can be obtained by referring to the embodiments, some of which are shown in the accompanying drawings. However, it should be noted that the accompanying drawings show only typical embodiments of the present disclosure and should not be considered as limiting its scope, since the present disclosure can recognize other equally effective embodiments.
Brief Description of the Drawings
[0005]
Figure 1A
Figure 1B
Figure 1C
Figure 1D
Figure 2A
Figure 2B
Figure 2C
Figure 2D
Figure 3
Figure 4a-4e
Figure 5
Figure 6a
Figure 6b
Figure 6c
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Best Mode for Carrying Out the Invention
[0006] Hereinafter, the present disclosure will be more fully described with reference to the accompanying drawings showing exemplary embodiments of the present disclosure. However, the present disclosure may be embodied in many different forms and should not be construed as limited to the exemplary embodiments described herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Like reference numerals refer to like elements throughout.
[0007] The terms used herein are for the purpose of describing particular exemplary embodiments only and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes", "including", "has" and "having", when used herein, specify the presence of the stated features, regions, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, regions, integers, steps, operations, elements, components, and / or groups thereof.
[0008] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art and the present disclosure, and should not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
[0009] While presenting using a communication application, the processing device is configured to send content such as an image to the presentation device. The user of the processing device then needs to select specific content to be shared with the presentation device.
[0010] Alternatively, to select specific content, the user can share the desktop of the processing device with multiple windows. In this case, some windows of the processing device that are considered as content not related to or inappropriate for the ongoing presentation are displayed. Further, the user will be exposed to the risk of leakage of personal information and confidential information.
[0011] In FIGS. 1-11 and their corresponding descriptions, embodiments of the present disclosure configured to provide an automated process for identifying relevant content, prioritizing and filtering relevant content, fitting content relevant to optimal presentation, and sending it to the presentation device are shown. Further, in embodiments of the present disclosure, specific actions of the user are not required while still providing a user experience that is applicable and context-sensitive.
[0012] Figures 1A to 1D show the arrangement of devices in the content sharing system 1 according to an embodiment of the present disclosure. The content sharing system 1 includes a plurality of processing devices 10a, 10b, and 10c, and a presentation device 20. The devices in the content sharing system 1 are connected via a network. Any one of the processing devices 10a, 10b, and 10c generates at least one presentation signal. The presentation signal may include at least one data of an audio signal or a presentation image. In some embodiments, at least one presentation signal is transmitted to the presentation device 20. In some embodiments, the processing devices 10a, 10b, and 10c are located in the vicinity of the presentation device 20. Further, as shown in FIG. 1B, a plurality of processing devices 10d, 10e, and 10f other than the processing devices 10a, 10b, and 10c are remotely arranged and may be connected to the content sharing system 1 via a communication application.
[0013] In some embodiments, the content sharing system 1 further includes a base node 30 configured to receive and process a presentation signal. In an exemplary embodiment, as shown in FIG. 1A, the content sharing system 1 further includes at least one base unit 21 electrically coupled to the presentation device 20, or a display unit D1 configured to perform the functions of the base node 30 of the content sharing system 1. Here, the display unit D1 may be a presentation device 20 configured only for display. In another exemplary embodiment, the base unit 21 is further coupled to at least one of a webcam or camera, a speaker, and a microphone. In a further embodiment, referring to FIG. 1C, the presentation device 20 can further include a base node 30 built within the presentation device 20. The presentation device 20 can serve as a base node of the content sharing system 1. In a further exemplary embodiment, as shown in FIG. 1D, the presentation device 20 is connected to the base node 30 of the content sharing system 1 via a wireless network connection. In an exemplary embodiment, the base node 30 can be installed in a server that can be accessed via a network connection. In some embodiments, the presentation device 20 is equipped with configurable characteristics that can be controlled via device control data. Accordingly, the base node 30 may be present in different forms or locations, but can be used as a connection between any of the processing devices 10a-10C and the presentation device 20.
[0014] Figures 2A-2D illustrate different arrangements of devices in content sharing system 1 according to another embodiment of the present disclosure. In some embodiments, content sharing system 1 further includes a peripheral device 11 electrically coupled to processing device 10a. Peripheral device 11 may further be connected to base node 30 of content sharing system 1 via a network. In some other embodiments, peripheral device 11 further pairs with presentation device 20 via base node 30. Peripheral device 11 is configured to initiate or perform sharing of signal presentation between processing device 10a and presentation device 20. In some embodiments, peripheral device 11 transmits a presentation signal from processing device 10a to base unit 21 of presentation device 20, where base unit 21 is electrically coupled to presentation device 20 and performs the functions of base node 30 of content sharing system 1. In some other embodiments, referring to Figures 2A-2D, peripheral device 11 transmits a pairing signal to presentation device 20 via base node 30, while transceiver 105 of processing device 10a is configured to transmit a presentation signal from processing device 10a to presentation device 20 via base node 30. In a further embodiment, as shown in Figure 2B, a plurality of peripheral devices 11 are correspondingly coupled to processing devices 10a, 10b, and 10c. In some embodiments, the presentation signal further includes data having at least one of position, resolution, and size information. In some embodiments, presentation device 20 includes a processor 210 and a transceiver 211 configured to perform the functions of a base node of content sharing system 1. In a further embodiment, presentation device 20 is further coupled to at least one of a webcam or camera, a speaker, and a microphone.
[0015] In some embodiments, as shown in FIGS. 2A and 2B, the base node 30 is in the form of a base unit 21 that connects to or pairs with the presentation device 20. In some embodiments, as shown in FIG. 2C, a peripheral device 11 may be further connected to the base node 30 of the content sharing system 1, and the base node 30 may be built into the presentation device 20. The presentation device 20 can receive a presentation signal via the base node 30. In an exemplary embodiment, a presentation device having a processor (such as the presentation device 20) can function as the base node 30 of the content sharing system 1. In a further exemplary embodiment, as shown in FIG. 2D, the peripheral device 11 can be further connected to the base node 30 of the content sharing system 1, and the base node 30 can be installed in a server that can be accessed via a network connection. Similarly, the presentation device 20 may receive a presentation signal via the base node 30.
[0016] Figure 3 shows a system block diagram of a content sharing system according to an embodiment of the present disclosure. The processing device 10 can be, for example, a computer, laptop, mobile phone, smartphone, tablet, PDA, etc. Such a device has non-volatile storage, a processor, memory, a display, a networking interface, etc. The processing device 10 includes a processor 100, a memory 100a coupled to the processor 100, and a transceiver 105 coupled to the processor 100. A plurality of computer-executable instructions are stored in the memory 100a. The plurality of computer-executable instructions are configured to execute a method of sharing presentation data, including steps of monitoring applications of the processing device 10, identifying content of high-priority applications, generating presentation data of the content, converting the presentation data to form a presentation signal, and transmitting the presentation signal to a presentation device 20, where the transceiver 105 coupled to the processor 100 is configured to transmit the presentation signal to the presentation device 20. The presentation device 20 can be, for example, a computer monitor, television, display panel, in-room conferencing device, speaker device, virtual reality headset, augmented reality headset, mixed reality headset, audio headphones, tactile feedback device, etc., or a combination thereof. Elements of the presentation data include at least one of visual, auditory, tactile, device control data, metadata, and other information, or a combination thereof. In some embodiments, the applications include any and / or all of the applications running in the foreground and background of the operating system of the processing device 10. The content further includes any audio file, video file, content file received from another processing device 10. The window of the application may or may not be displayed on the display 100b of the processing device 10. In some other embodiments, a communication application can be launched or executed in the processing device.In a further embodiment, the communication application is further configured to receive the content of the application from another processing device or an application of another processing device. The presentation data of the content may be a representation of the entire window that is resized and included within the window display of the communication application.
[0017] In some embodiments, as shown in FIG. 3, a plurality of computer-executable instructions are executed by modules of the processor 100. The modules of the processor 100 include a picker 101, an analyzer 102, a mixer 103, and an encoder 104. The picker 101 is configured to monitor an application SW operating on the operating system OS of the processing device 10. In some embodiments, when the communication application is launched, at least a portion of the application SW can be continuously monitored in the background of the communication application. The analyzer 102 is coupled to the picker 101. The analyzer 102 is configured to identify the content of the application SW having a priority. The mixer 103 is coupled to the picker 101 and the analyzer 102. The mixer 103 is configured to obtain the presentation data of the content identified as having a priority and create presentation data for encoding. The encoder 104 is coupled to the mixer 103. The encoder 104 is configured to convert the presentation data into a presentation signal. The transceiver 105 is coupled to the encoder 104 of the processor 100. The transceiver 105 is configured to transmit the presentation signal to the presentation device 20.
[0018] In some embodiments, the trigger marker is used by the analyzer 102 to identify the content of the prioritized application. The trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information. In further embodiments, the analyzer 102 can detect multiple trigger markers. The first trigger marker can be used to identify file content with priority, and the second trigger marker can be used to identify media content with priority. The file content can include at least one of a specific window, a region of the screen, the full screen, a file (e.g., a PowerPoint® file), or network collaborative content (e.g., a file on OneDrive®, a file on a networked WhiteBoard application). In further embodiments, the file content can further include at least one of media files that are pre-recorded and stored as a single file. The media files of the file content can be video files, audio files, or a combination of both. The media content can include streaming video or a representation image of the user of the processing device. In some embodiments, the media content can be a specific window, a region of the screen, or the full screen. In further embodiments, the media content is content that changes continuously over time. In some other embodiments, media files, such as communication applications, can further include the content of the chat room, where the data input from the processing device is displayed as media content. In some embodiments, the trigger marker can be the movement of the participant, the facial expression, or the change in the sound source. If there is a lack of a trigger marker or the trigger marker is no longer detected, the content associated with the trigger marker may be removed from the presentation signal.
[0019] Figure 8 shows that the communication application AW of the processing device 10a is launched according to an embodiment of the present disclosure. The first content SW1 may be file content displayed in a sub-window of the communication application. The plurality of second contents SW2, SW3, and SW4 may be media content displayed in a sub-window of the communication application. The first content SW1 and the second content SW2 of the communication application AW may have priorities and are detected by the content sharing system 1. During operation, a part of the communication application AW may be assigned by the content sharing system as a reference marker IC. The reference marker IC may be at least one icon attached to the communication application AW. In the exemplary embodiment shown in Figure 8, the reference marker IC includes three icons fixed at the corners of the communication application AW. The presentation data of the content is displayed in the communication application AW in a grid-based layout. And the reference marker IC is used to track the presentation data of the content including the second contents SW2, SW3, SW4. The second content SW2 having a priority is identified by the relationship between the reference marker IC on the communication application AW and the trigger marker TR. The reference marker may be set as a reference point for determining the coordinate position of the trigger marker on the window of the communication application. In the exemplary embodiment, as shown in Figure 8, the trigger marker TR may be a frame surrounding the second content SW2. Therefore, the position of the trigger marker TR within the window provides the positioning coordinates of the second content SW2 within the window. Further, the reference marker IC is further configured to determine events such as the start of a communication application (i.e., an online meeting).
[0020] In some embodiments, the position of an image on a grid-based layout changes based on the size of the window of the communication application, the number of presentation data of the content displayed on the application window, or the shape of the application window. The position or coordinates of the reference marker on the window of the communication application remain relatively the same.
[0021] In some embodiments, the analyzer 102 determines an application identifier used to associate an application monitored on the processing device with an application programming interface. The analyzer 102 employs an associated application programming interface to determine the associated application content. The application programming interface can communicate directly with the application monitored on the processing device by using a local application programming interface (API), adopting a remote API for cloud-enabled applications, or adopting other communication means recognized by those skilled in the art. In some embodiments, the priority of the associated application content is determined by associating metadata information determined via the application programming interface. In some embodiments, the priority of the associated application content is determined by a classifier algorithm.
[0022] In some embodiments, referring to FIG. 1A, the presentation device 20 can be further coupled to a base unit 21 that includes a base node 30. In some embodiments, referring to FIG. 1C, the presentation device 20 can further include a base node 30 constructed in the presentation device 20. The presentation device 20 can serve as a base node of the content sharing system 1. In some other embodiments, as shown in FIG. 1D, the base node 30 may be disposed in a cloud server having network connections with the presentation device 20 and the processing devices 10a, 10b, and 10c. In some further embodiments, as shown in FIG. 1A, the presentation device 20 includes a display unit D1 and a base unit 21 coupled to the display unit D1. The base unit 21 functions as a base node of the presentation device 20.
[0023] The base node is configured to receive a presentation signal and convert a plurality of element data of the presentation signal into unit data. In an exemplary embodiment, the presentation signal can be converted into an image to be displayed on the display unit D1. As shown in FIG. 3, the base node includes a transceiver 211 and a decoder 212 electrically coupled to the display unit D1. In some embodiments, the decoder 212 is further configured to edit an image to be displayed on the display unit D1. The editing of the image may be performed automatically. The transceiver 211 is configured to receive a presentation signal from the processing device 10 via a network. The decoder 212 is configured to receive the presentation signal from the transceiver 211 and decode the presentation signal into a format that can be displayed on the display unit D1.
[0024] In some other embodiments, the communication application can be launched or executed on a processing device. In further embodiments, the communication application is further configured to receive application content from another processing device or an application of another processing device. The presentation data of the content may be a representation of the entire window that is resized and included within the window display of the communication application.
[0025] Figures 4(a) to 4(e) show the arrangements of five different display images according to embodiments of the present disclosure. In some embodiments, the display unit D1 is configured to receive one presentation image when only one content of the communication application has priority. Figure 4(a) shows the display unit D1 that displays the decoded first image M1. In some embodiments, the first image M1 corresponds to the first content SW1 in Figure 8. In some other embodiments, the first image M1 corresponds to any one of the second contents SW2, SW3, and SW4 in Figure 8. In other words, the first image M1 is configured to display the image of the content having the highest priority or only priority. Figures 4(b) and 4(c) show the display unit D1 that displays the decoded first image M1 and the second image M2. In some embodiments, the first image M1 corresponds to the first content SW1 in Figure 8, and the second image M2 corresponds to any one of the second contents SW2, SW3, and SW4 in Figure 8. In some other embodiments, the first image M1 corresponds to any one of the second contents SW2, SW3, and SW4 in Figure 8, and the second image M2 corresponds to the first content SW1 in Figure 8. As shown in Figures 4(b) and 4(c), the first image M1 may be an image of a sub-window having a high priority, and the second image M2 may be an image of the content having a low priority. Thereby, the content having a low priority has an image that is displayed at a smaller scale compared to the content having a higher priority. Similar to Figures 4(b) and 4(c), Figures 4(d) and 4(e) show the second image M2 at a smaller scale compared to the first image. However, as shown in Figures 4(d) and 4(e), the aspect ratio of the original image from the processing device is maintained. In some other embodiments, the size of the second image M2 may change over time according to the activity shown in the client window. In further embodiments, the presentation signal further includes metadata configured to determine the sizing of the displayed image.Editing the represented image includes at least operations on the aspect ratio of the content, trimming, watermarking, alpha blending, or editing. Further, the editing of the content may include combinations of text by multiple images, icons, overlays, merges, etc. Metadata includes information corresponding to the type of content included in the image to determine the sizing of the image on the display. In a further embodiment, the sizing of the image may be assigned by the user of the processing device. In a further embodiment, the processing device assigns the sizing of the display image before sending it to the presentation device. The processing device may determine the size of the display image according to the type of content. In a further embodiment, the base node can be configured to analyze the image received by the base node and determine the type of content corresponding to the image. And the sizing of the display image is determined according to the type of content. And the sizing of the display image is determined according to the type of content.
[0026] In some embodiments, the mixer 103 determines an arrangement as shown in FIGS. 4(a) to 4(e), and the determination can be based on the application content of the application with priority, the arrangement of the display unit, the context of the display unit, the context of the user, the local or remote context of the communication session, the physical environment, the rules of the business logic, the rules of the conversation, preferences, or combinations thereof. The determination from the mixer 103 is further encoded into the presentation signal by the encoder 105. Also, the transceiver 105 transmits the presentation signal to the transceiver 211 of the presentation device 20.
[0027] In some other embodiments, the decoder 212 of the presentation device 20 decodes the received presentation signal. The decoder 212 determines an arrangement as shown in FIGS. 4(a) to 4(e), and in that determination, it can be based on the application content of the application with priority, the arrangement of the display unit, the context of the display unit, the context of the user, the local or remote context of the communication session, the physical environment, the rules of the business logic, the rules of the conversation, preferences, or a combination thereof.
[0028] FIG. 5 shows a system block diagram of a content sharing system according to another embodiment of the present disclosure. The processing device 10 can be, for example, a computer, a laptop, a mobile phone, a smartphone, a tablet, a PDA, etc. Such a device has non-volatile storage, a processor, a memory, a display, a networking interface, etc. The processing device 10 includes a processor 100, a memory 100a coupled to the processor 100, and a transceiver 105 coupled to the processor 100. A plurality of computer-executable instructions are stored in the memory 100a. The plurality of computer-executable instructions are configured to execute a method of sharing presentation data including steps of monitoring the applications of the processing device 10, identifying the content of the application with priority, generating presentation data of the content, converting the presentation data to form a presentation signal, and transmitting the presentation signal to the display units D1, D2,....., DN, where the transceiver 105 coupled to the processor 100 is configured to transmit the presentation signal to the display units D1, D2,....., DN. In some embodiments, the content may or may not be displayed on the display of the processing device.
[0029] In some embodiments, the plurality of computer-executable instructions are executed by modules of the processor 100. The modules of the processor 100 include a picker 101, an analyzer 102, a mixer 103, and an encoder 104. The picker 101 is configured to monitor a sub-window SW that operates on the operating system OS of the processing device 10. In some embodiments, when a communication application is launched, content can be continuously received and monitored by the communication application. However, in some embodiments, a portion of the content received from the processing device is not displayed on the application window. The analyzer 102 is coupled to the picker 101. The analyzer 102 is configured to identify content SW of the communication application having a priority. The mixer 103 is coupled to the picker 101 and the analyzer 102 and is configured to obtain presentation data of the content SW identified as having a priority and create presentation data for encoding. The encoder 104 is coupled to the mixer 103 and is configured to convert the presentation data into a presentation signal. In some embodiments, when generating the presentation signal, data regarding the sizing of the presentation data is already included within the presentation signal. The transceiver 105 is coupled to the encoder 104 of the processor 100 and is configured to transmit the presentation signal to display units D1, D2,....., DN.
[0030] In some embodiments, the trigger marker is used by the analyzer 102 to identify content SW with priority. The trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information. In further embodiments, the analyzer 102 can detect a plurality of trigger markers. The first trigger marker can be used to identify file content with priority, and the second trigger marker can be used to identify media content. The file content can include at least one of a specific window, a region of the screen, the full screen, a file (e.g., a PowerPoint file), or network collaboration content (e.g., a file on OneDrive, a file on a networked WhiteBoard application). The media content can include a streaming video or a rendered image of a user of the processing device. In some embodiments, the media content can be a specific window, a region of the screen, or the full screen. Further, the media content can be dynamic content. In some embodiments, the trigger marker is the movement of a participant, a facial expression, or a change in the sound source.
[0031] FIG. 8 shows that the communication application AW of the processing device 10a is activated according to an embodiment of the present invention disclosure. In an exemplary embodiment, the communication application AW may be the communication application 1. The first content SW1 may be file content. The plurality of second contents SW2, SW3, SW4 may be media content of a processing device linked to the communication application. The first content SW1 and the second content SW2 of the communication application AW may have priorities and are detected by the content sharing system 1. During operation, a part of the communication application AW may be assigned as the reference marker IC. The reference marker IC may be at least one icon attached to the communication application AW. In the exemplary embodiment shown in FIG. 8, the reference marker IC includes three icons fixed to the corner of the communication application AW. The presentation data of the content is displayed in the communication application AW in a grid-based layout. And the reference marker IC is used to track the presentation data of the content including the second contents SW2, SW3, SW4. The second content SW2 having a priority is identified by the relationship between the reference marker IC on the communication application AW and the trigger marker TR. In the exemplary embodiment shown in FIG. 8, the trigger marker TR may be a frame surrounding the second content SW2. Further, the reference marker IC is further configured to determine an event such as the start of a communication application (i.e., an online meeting). In some other embodiments, the communication application can be launched or executed in the processing device. In a further embodiment, the communication application is further configured to receive the content of the application from another processing device or an application of another processing device. The presentation data of the content may be the representation of the entire window that is resized and included in the window display of the communication application.
[0032] In some embodiments, the position of an image on a grid-based layout varies according to the size of the window of the communication application, the number of sub-windows on the display on the application window, or the shape of the application window. The position of the reference marker on the communication application remains the same.
[0033] In some embodiments, a base node 30 coupled to a plurality of display units D1, D2,....., DN is configured to receive a presentation signal and convert the presentation signal into a plurality of images to be displayed on the display units D1, D2,....., DN. As shown in FIG. 5, the base node 30 includes a transceiver 211, a decoder 212 coupled to the transceiver 211, and a splitter 213 electrically coupled to the decoder 212 and the display units D1, D2,....., DN. The transceiver 211 is configured to receive a presentation signal from the processing device 10 via a network. The decoder 212 is configured to receive the presentation signal from the transceiver 211 and decode the presentation signal into a file format that can be read by the splitter 213. The splitter 213 is configured to determine the number of display units D1, D2,....., DN on which the decoded presentation signal is to be displayed. Further, the splitter 213 is configured to determine the number of images included in the presentation signal. In some embodiments, the presentation signal includes only one image and can be divided among the plurality of display units D1, D2,....., DN. In some other embodiments, the presentation signal includes a plurality of images and can be displayed on the corresponding plurality of display units D1, D2,....., DN.
[0034] Figures 6(a) to 6(c) show the arrangements of three different display images according to other embodiments of the present disclosure. In some embodiments, the display unit D1 is configured to receive an image of the first content of a communication application having priority, and the display unit D2 is configured to receive an image of the second content of a communication application having priority. Figure 6(a) shows the display unit D1 for displaying the decoded first image M1 and the display unit D2 for displaying the decoded second image M2. In some other embodiments, when only one content has priority, the image of the content is divided and displayed on the display unit D1 and the display unit D2. Figure 6(b) shows the display unit D1 for displaying the decoded first image portion M1a and the display unit D2 for displaying the decoded second image portion M1b. Figure 6(c) shows the display unit D1 for displaying the decoded first image M1 and the display unit D2 for displaying the decoded second image M2 and the third image M3. In other embodiments, the decoded second image M2 and the third image M3 displayed on the display unit D2 may be in different arrangements. The different arrangements of the decoded second image M2 and the third image M3 can refer to the different arrangements of the first image M1 and the second image M2 of the display unit D1 as shown in Figures 4(b) to 4(e).
[0035] In some embodiments, as shown in FIG. 7, the content sharing system has M base units 21a, 21b, and M rows of presentation devices. Each row of presentation devices has N presentation devices, and thus the content sharing system has M * N presentation devices. After the pairing process of the base units 21a, 21b is completed, one of the base units 21a is assigned as the master base unit 21a, and the other is assigned as the slave base unit 21b. The master base unit 21a can be further electrically coupled to one row of the display units D11~D1N, and each slave base unit 21b can be further paired with the corresponding rows of the display units D21~D2N,..., DM1~DMN. Referring further to FIG. 5, each of the base units 21a, 21b includes a transceiver 211, a decoder 212, and a splitter 213 electrically coupled to the corresponding display units D11~DMN. Taking the master base unit 21a as an example, the transceiver 211 of the master base unit 21a is configured to receive a presentation signal from the processing device 10 via the network in cooperation with the paired master base unit 21a. The decoder 212 of the master base unit 21a is configured to receive the presentation signal and decode the presentation signal into a file format that can be read by the splitter 213 of the master base unit 21a. The splitter 213 of the master base unit 21a is configured to determine the number of the N display units D11, D12,....., D1N for displaying the decoded presentation signal. Further, the splitter 213 is configured to determine the number of images included in the presentation signal. In some embodiments, the presentation signal includes only one image and can be split among the plurality of display units D11, D12,....., D1N. In some other embodiments, the presentation signal includes a plurality of images and can be displayed on the corresponding plurality of display units D11, D12,....., D1N accordingly.
[0036] In some embodiments, mixer 103 determines an arrangement as shown in FIGS. 6(a) - 6(c) and FIG. 7. The determination may be based on application content of an application having a priority, arrangement of display units, context of the display units, context of the user, local or remote context of a communication session, physical environment, rules of business logic, rules of conversation, preferences, or a combination thereof. In further embodiments, decoder 212 determines an arrangement as shown in FIGS. 6(a) - 6(c) and FIG. 7. The determination may be based on application content of an application having a priority, arrangement of display units, context of the display units, context of the user, local or remote context of a communication session, physical environment, rules of business logic, rules of conversation, preferences, or a combination thereof.
[0037] In some embodiments, transceiver 211 receives a presentation signal from a plurality of sources, and decoder 212 may determine an arrangement based on the presentation signal, arrangement of display units, local or remote context of a communication session, context of the room, rules of business logic, rules of conversation, preferences, or a combination thereof.
[0038] In some embodiments, the content sharing system shown in FIG. 7 has an arbitrary number of presentation devices for each base node.
[0039] FIG. 9 shows a flowchart of the operation of a processing device according to an embodiment of the present disclosure. In some embodiments, the content sharing system can be executed in the background of the operating system of the processing device. In S1001, the content sharing system monitors the applications running in the operating system. In S1002, the content sharing system shares the image of the content with the presentation device. In an exemplary embodiment, the shared image may present the image of the content SW1 in FIG. 8. In S1003, the content sharing system determines whether the communication application has been launched. In some other embodiments, if the communication application has already been launched, the content sharing system determines whether to share the content with the presentation device. If the communication application has not been launched, the content sharing system can prompt the user to launch the communication application via a message window or an icon as shown in S1004. In some other embodiments, the user may be prompted to share the content with the communication application. When the communication application is launched, in S1005, the content sharing system determines whether there is content that has received a priority by detecting a trigger marker. In some embodiments, the content monitored for the trigger marker may be media content. The priority of the content can be further determined using the trigger marker. That is, the trigger markers corresponding to the first content and the second content are compared and used to determine the priorities of the first content and the second content. In some embodiments, only one content receives a priority. In some other embodiments, multiple contents receive priorities simultaneously. In S1006, a presentation image of the content having a priority is generated. In S1007, the image is encoded so that the presentation image is converted into a presentation signal. If there are multiple presentation data to be transmitted, the images can be combined to form composite image data. The method of forming the composite image date includes alpha blending.In some other embodiments, the plurality of images are arranged in a sequential or pipelined manner to form the image data for transmission. Then, in S1008, a presentation signal is transmitted to the presentation device. In some embodiments, the presentation signal is transmitted to the presentation device. In some embodiments, the presentation signal is transmitted to the presentation device via the base unit. The presentation signal can be transmitted via a wired network or a wireless network.
[0040] FIG. 10 shows a flowchart of the operation of a base node according to an embodiment of the present disclosure. In S2001, the base node receives a presentation signal. In S2002, the presentation signal is decoded. In S2003, it is determined whether there are a plurality of images included in the presentation signal. If there are a plurality of images in the presentation signal, in S2004, the images are separated from each other. In S2005, the number of presentation devices to be used for displaying the content of the presentation signal is determined. In some embodiments, only one presentation device is available for display. In some other embodiments, there are more available presentation devices than are required to display the images. In some other embodiments, all of the presentation devices are used to display the images. In S2007, the size and / or resolution of the images are determined as described in FIGS. 4(a) to 4(e). In addition, the size of the images on the presentation device is determined by the priority of the first content and the second sub-window. If there are a plurality of presentation devices to be used, in S2006, the designation of the images is determined as described in FIGS. 6(a) to 6(c). In S2008, the images are displayed accordingly.
[0041] FIG. 11 shows a flowchart of the operation of a base node according to an embodiment of the present disclosure. In S3001, the base node receives a presentation signal. In S3002, the presentation signal is decoded. In S3003, it is determined whether there are multiple presentation data elements included in the presentation signal. By presenting data elements, data such as visual data, audio data, tactile data, device control data, and metadata can be represented. If there are multiple presentation data elements in the presentation signal, in S3004, the presentation data elements are separated from each other. In S3005, the number of presentation devices used to present the content of the presentation signal is considered. In some embodiments, only one presentation device is available for presentation. In some other embodiments, there are more available presentation devices than are required to display the content. In some other embodiments, all of the presentation devices are used to display the content. In S3007, the conversion of the presentation data elements is determined. In the case of image data, this can include the size, resolution, and other enhancements or adaptations of the image. In the case of audio data, this can include the sampling rate of the audio data, noise reduction, and enhancements or adaptations of the audio for that audio data. The conversion can be adapted depending on the capabilities of the associated presentation device, the context in which the user, system, and device operate, and / or the type, content, and priority of the presentation data element. This conversion can be adapted depending on the priority of the presentation data element. If there are multiple presentation devices to be used, in S3006, the presentation data elements are divided among the presentation devices. The division can assign the presentation data elements to a single presentation device or divide the presentation data elements among multiple presentation devices. This division can be adapted depending on the capabilities of the associated presentation device, the context in which the user, system, and device operate, and / or the type, content, and priority of the presentation data element. In S3008, the presentation data elements are converted and presented.
[0042] Accordingly, one aspect of the present disclosure provides a system configured to share presentation data between a presentation device and a processing device. The system includes a processor, a memory, and a transceiver. The memory is coupled to a processor into which computer-executable instructions are loaded, and the plurality of computer-executable instructions execute a method comprising monitoring an application of the processing device, identifying content of a prioritized application, generating presentation data of the content, converting the presentation data to form a presentation signal, and transmitting the presentation signal to a base node connected to the presentation device. The transceiver is coupled to a processor configured to transmit the presentation signal to the presentation device.
[0043] In some embodiments, the application includes a communication application configured to receive content of the application from another processing device.
[0044] In some embodiments, the method further comprises identifying a reference marker corresponding to the application and identifying a trigger marker corresponding to the content.
[0045] In some embodiments, the reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information, and the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
[0046] In some embodiments, the reference marker is further configured to determine an event of the application.
[0047] In some embodiments, the prioritized content is identified by a relationship between the reference marker and the trigger marker.
[0048] In some embodiments, the method further comprises identifying a trigger marker corresponding to the content, where the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
[0049] In some embodiments, the system further comprises a base unit as a base node connected to the presentation device and configured to receive a presentation signal from the processing device.
[0050] In some embodiments, the system further comprises a peripheral device electrically coupled to the processing device and paired with the base unit, where the presentation signal is transmitted between the peripheral device and the base unit.
[0051] In some embodiments, the base unit is configured to transmit the presentation signal to the presentation device and another presentation device.
[0052] In some embodiments, the method further comprises extracting a plurality of element data from the presentation signal by the base node, combining the plurality of element data by the base node to form unit data, and providing the unit data to the presentation device by the base node.
[0053] In some embodiments, the plurality of element data is image data, the unit data is a display image, and the step of providing the unit data to the presentation device includes displaying the display image on a display unit of the presentation device.
[0054] In some embodiments, the method further comprises separating a plurality of element data from the presentation signal by the base node, providing the plurality of element data on the presentation device by the base node, and editing the plurality of element data provided to the presentation device by the base node.
[0055] In some embodiments, the method further comprises a step of combining a plurality of element data by a base node to form unit data.
[0056] In some embodiments, the method further comprises a step of comparing contents having priorities and a step of determining the priority levels of the contents, wherein the plurality of element data are edited according to the priority levels of the contents.
[0057] In some embodiments, the plurality of element data are image data.
[0058] In some embodiments, the presentation signal includes metadata configured to identify the type of the content.
[0059] Accordingly, another aspect of the present disclosure provides a method for sharing a display image. The method comprises monitoring an application of a processing device, identifying the content of the application having a priority, generating presentation data of the content, converting the presentation data to form a presentation signal, and transmitting the presentation signal to a base node connected to a presentation device.
[0060] In some embodiments, the application includes a communication application, and the communication application is configured to receive the content of the application from another processing device.
[0061] In some embodiments, the method further comprises identifying a reference marker corresponding to the application and identifying a trigger marker corresponding to the content.
[0062] In some embodiments, the reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information, and the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
[0063] In some embodiments, the reference marker is further configured to determine events of the application.
[0064] In some embodiments, the content with priority is identified by the relationship between the reference marker and the trigger marker.
[0065] In some embodiments, the method further comprises identifying a trigger marker corresponding to the content, where the trigger marker includes at least one of icon, metadata, audio data, text, coordinates, and pixel information.
[0066] In some embodiments, the method further comprises transmitting a presentation signal as a base node between the processing device and a base unit connected to the presentation device.
[0067] In some embodiments, the method further comprises transmitting a presentation signal between a peripheral device coupled to the processing device and the base unit.
[0068] In some embodiments, the base unit is configured to transmit presentation data to the presentation device and another presentation device.
[0069] In some embodiments, the method further comprises extracting a plurality of element data from the presentation signal by the base node, combining the plurality of element data by the base node to form unit data, and providing the unit data to the presentation device by the base node.
[0070] In some embodiments, the plurality of element data is image data, the unit data is a display image, and the step of providing the unit data to the presentation device includes displaying the display image on a display unit of the presentation device.
[0071] In some embodiments, the method further comprises separating, by a base node, a plurality of element data from a presentation signal, providing, by the base node, the plurality of element data on a presentation device, and editing, by the base node, the plurality of element data provided to the presentation device.
[0072] In some embodiments, the method further comprises combining a plurality of element data to form unit data.
[0073] In some embodiments, the method further comprises comparing content having priorities and determining a priority level of the content, wherein the plurality of element data is edited according to the priority level of the content.
[0074] In some embodiments, the plurality of element data is image data.
[0075] In some embodiments, the presentation signal includes an identifier configured to identify a type of content.
[0076] Accordingly, another aspect of the present disclosure provides a method for sharing presentation data. The method comprises monitoring a priority of content of a communication application, wherein the content includes content of an application of a processing device linked via the communication application, generating presentation data of the content having priorities, converting the presentation data to form a presentation signal, and transmitting the presentation signal to a presentation device.
[0077] In some embodiments, the method further comprises monitoring a trigger marker to determine content having priorities, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
[0078] In some embodiments, the content with priority includes first content and second content, and the first content and the second content are different types of content.
[0079] In some embodiments, the presentation signal further includes an identifier configured to distinguish the first content from the second content.
[0080] In some embodiments, the method further comprises identifying a reference marker on a communication application and identifying a trigger marker for the second content.
[0081] In some embodiments, the reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information, and the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
[0082] In some embodiments, the reference marker is further configured to determine an event of the communication application.
[0083] In some embodiments, the element data of the first content and the element data of the second content are included in a display window of the communication application.
[0084] In some embodiments, the element data is image data, and the size, shape, and position of the element data of the second content are configured to change within the display window while the position of the reference marker on the display window remains the same.
[0085] In some embodiments, the second content is identified by the relationship between the reference marker and the trigger marker on the display window.
[0086] In some embodiments, the presentation signal is transmitted to the presentation device via a base unit coupled to the presentation device.
[0087] In some embodiments, the presentation signal is transmitted to the base unit via a peripheral device.
[0088] In some embodiments, the method further comprises the steps of extracting, by the base unit, a plurality of element data from the presentation signal, and transmitting the plurality of element data accordingly to a presentation device and another presentation device coupled to the base unit.
[0089] In some embodiments, the plurality of element data is image data.
[0090] In some embodiments, the presentation signal includes a composite image file of presentation data of the content.
[0091] Accordingly, another aspect of the present disclosure provides a method for sharing presentation data. The method comprises connecting, by a processing device, a content sharing system of the processing device to a communication application, monitoring, by the content sharing system, the content of the communication application, determining, by the content sharing system, the priority of the content, encoding, by the content sharing system, at least one element data of the content having a priority to form presentation data, converting, by the content sharing system, the presentation data to form a presentation signal, and presenting, by the content sharing system, the presentation signal to a presentation device.
[0092] In some embodiments, the content includes at least media content and file content, the file content includes at least one of a specific application window, a region of a screen, a full screen, a file, and network collaboration content, and the media content includes a streaming video of a user of the communication application.
[0093] In some embodiments, the method further comprises combining the element data of the content included in the presentation data by a base node corresponding to the presentation device.
[0094] In some embodiments, the method further comprises combining the element data of the content by a content sharing system to form presentation data.
[0095] In some embodiments, the method further comprises monitoring a reference marker and a trigger marker by a content sharing system, where the reference marker is configured to indicate an event of a communication window, and where the trigger marker is configured to indicate content having a priority.
[0096] In some embodiments, the reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information, and the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
[0097] In some embodiments, the method comprises separating the presentation data into a plurality of sub-element data by a base node corresponding to the presentation device, and transmitting the plurality of sub-element data to a display unit of the presentation device accordingly by the base node, where each of the plurality of sub-element data corresponds to one content.
[0098] In some embodiments, the method further comprises separating the presentation data of one content into a plurality of sub-element data by a base node corresponding to the presentation device, and transmitting the sub-element data to a display unit of the presentation device accordingly by the base node.
[0099] In some embodiments, the sub-element data includes image data.
[0100] In some embodiments, the element data includes image data.
[0101] Those skilled in the art will readily appreciate that numerous modifications and changes can be made to the apparatus and methods while retaining the teachings of the present invention. Accordingly, the above disclosure should be construed as being limited only by the details of the appended claims.
Claims
1. A system configured to share presentation data between a presentation device and a processing device, a processor, a memory coupled to the processor, a transceiver coupled to the processor and comprising, wherein the memory has computer-executable instructions loaded thereon, and the plurality of computer-executable instructions monitor the application of the processing device, identify the content of the application having a priority, generate the presentation data of the content, convert the presentation data to form a presentation signal, and transmit the presentation signal to a base node connected to the presentation device and execute a method including, wherein the transceiver is configured to transmit the presentation signal to the presentation device.
2. The system according to claim 1, wherein the application includes a communication application, and the communication application is configured to receive the content of the application from another processing device.
3. The method further includes, identifying a reference marker corresponding to the application, identifying a trigger marker corresponding to the content and including,
4. The reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information, The system according to claim 3, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
5. The system according to claim 3, wherein the reference marker is further configured to determine an event of the application.
6. The system according to claim 3, wherein the content having the priority is identified by a relationship between the reference marker and the trigger marker.
7. The method further includes a step of identifying a trigger marker corresponding to the content, The system according to claim 1, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
8. The system according to claim 1, further comprising a base unit as the base node connected to the presentation device and configured to receive the presentation signal from the processing device.
9. Further comprising a peripheral device electrically connected to the processing device and paired with the base unit, The system according to claim 8, wherein the presentation signal is transmitted between the peripheral device and the base unit.
10. The system according to claim 8, wherein the base unit is configured to transmit the presentation signal to the presentation device and another presentation device.
11. The method further comprises: Extracting a plurality of element data from the presentation signal by the base node; Combining the plurality of element data by the base node to form unit data; Providing the unit data to the presentation device by the base node The system according to claim 1, comprising.
12. The plurality of element data is image data, The unit data is a display image, The system according to claim 11, wherein the step of providing the unit data to the presentation device includes displaying the display image on a display unit of the presentation device.
13. The method further comprises: Separating a plurality of element data from the presentation signal by the base node; Providing the plurality of element data on the presentation device by the base node; Editing the plurality of element data provided to the presentation device by the base node The system according to claim 1, comprising.
14. The system according to claim 13, wherein the method further comprises forming unit data by combining the plurality of element data by the base node.
15. The method further comprises: Comparing the content having a priority; Determining a priority level of the content Including, The system according to claim 13, wherein the plurality of element data is edited according to the priority level of the content.
16. The system according to any one of claims 13 to 15, wherein the plurality of element data is image data.
17. The system according to claim 1, wherein the presentation signal includes metadata configured to identify the type of the content.
18. A method for sharing a display image, comprising: Monitoring an application of the processing device; Identifying content of the application having the priority; The step of generating presentation data of the content; The step of converting the presentation data to form a presentation signal; The step of transmitting the presentation signal to a base node connected to the presentation device A method comprising.
19. The method according to claim 18, wherein the application includes a communication application, and the communication application is configured to receive application content from another processing device.
20. The step of identifying a reference marker corresponding to the application; The step of identifying a trigger marker corresponding to the content The method according to claim 18, further comprising.
21. The reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information, The method according to claim 20, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
22. The method according to claim 20, wherein the reference marker is further configured to determine an event of the application.
23. The method according to claim 20, wherein the content having the priority is identified by a relationship between the reference marker and the trigger marker.
24. The method according to claim 18, further comprising the step of identifying a trigger marker corresponding to the content, The method according to claim 18, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
25. The method according to claim 18, further comprising the step of transmitting the presentation signal as the base node between the processing device and a base unit connected to the presentation device.
26. The method according to claim 25, further comprising the step of transmitting the presentation signal between a peripheral device coupled to the processing device and the base unit.
27. The step of extracting a plurality of element data from the presentation signal by the base node; The step of combining the plurality of element data by the base node to form unit data; The step of providing the unit data to the presentation device by the base node The method of claim 18, further comprising **Claim 29** wherein the plurality of element data is image data, the unit data is a display image, and the step of providing the unit data to the presentation device includes the step of displaying the display image on a display unit of the presentation device, the method of claim 28. **Claim 30** The method further comprises separating, by the base node, a plurality of element data from the presentation signal; providing, by the base node, the plurality of element data onto the presentation device; and editing, by the base node, the plurality of element data provided to the presentation device the method of claim 18. **Claim 31** The method of claim 30, further comprising forming unit data by combining the plurality of element data. **Claim 32** comparing the content having the priority; and determining a priority level of the content further comprising, wherein the plurality of element data is edited according to the priority level of the content, the method of claim 30. **Claim 33** The system according to any one of claims 30 to 32, wherein the plurality of element data is image data. **Claim 34** The method of claim 18, wherein the presentation signal includes an identifier configured to identify a type of the content. **Claim 35** A method of sharing presentation data, comprising: monitoring a priority of content of a communication application, the content including content of an application of a processing device linked via the communication application; generating presentation data of the content having the priority; converting the presentation data to form a presentation signal; and transmitting the presentation signal to a presentation device the method. **Claim 36** The method of claim 35, further comprising monitoring a trigger marker to determine the content having the priority, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information. **Claim 37** The content having the priority includes first content and second content, wherein the first content and the second content are different types of content, the method of claim 35. **Claim 38** The method of claim 37, wherein the prompt signal further includes an identifier configured to distinguish between the first content and the second content.
39. Identifying a reference marker on the communication application; Identifying a trigger marker for the second content; The method of claim 37, further comprising:
40. The reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information; The method of claim 39, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
41. The method of claim 39, wherein the reference marker is further configured to determine an event of the communication application.
42. The method of claim 39, wherein the element data of the first content and the second content is included in a display window of the communication application.
43. The element data is image data; The size, shape, and position of the element data of the second content are configured to change within the display window; The method of claim 42, wherein the position of the reference marker on the display window remains the same.
44. The method of claim 42, wherein the second content is identified by a relationship between the reference marker and the trigger marker on the display window.
45. The method of claim 37, wherein the prompt signal is transmitted to the prompting device via a base unit coupled to the prompting device.
46. The method of claim 45, wherein the prompt signal is transmitted to the base unit via a peripheral device.
47. Extracting a plurality of element data from the prompt signal by the base unit; Transmitting the plurality of element data to the prompting device and another prompting device coupled to the base unit accordingly; The method of claim 45, further comprising:
48. The method of claim 47, wherein the plurality of element data is image data.
49. The method of claim 37, wherein the prompt signal includes a composite image file of the prompt data of the content.
50. A method for sharing prompt data, comprising: Connecting a content sharing system of a processing device to a communication application; The step of monitoring the content of the communication application by the content sharing system; The step of determining the priority of the content by the content sharing system; The step of encoding at least one element data of the content having the priority by the content sharing system to form the presentation data; The step of converting the presentation data by the content sharing system to form a presentation signal; The step of transmitting the presentation signal to a presentation device by the content sharing system A method comprising.
51. The content includes at least media content and file content, The file content includes at least one of a specific application window, a screen area, a full screen, a file, and network collaboration content, The method according to claim 50, wherein the media content includes a streaming video of a user of the communication application.
52. The method according to claim 50, further comprising the step of combining the element data of the content included in the presentation data by a base node corresponding to the presentation device.
53. The method according to claim 50, further comprising the step of combining the element data of the content by the content sharing system to form the presentation data.
54. The content sharing system further includes a step of monitoring a reference marker and a trigger marker, The reference marker is configured to indicate an event of the communication window, The method according to claim 50, wherein the trigger marker is configured to indicate content having the priority.
55. The reference marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information, The method according to claim 54, wherein the trigger marker includes at least one of an icon, metadata, audio data, text, coordinates, and pixel information.
56. The step of separating the presentation data into a plurality of sub-element data by a base node corresponding to the presentation device; The step of transmitting the plurality of sub-element data to a display unit of the presentation device accordingly by the base node Further comprising, The method of claim 50, wherein each of the plurality of sub-element data corresponds to one piece of content. **Claim 57** Separating, by a base node corresponding to the presentation device, the presentation data of one piece of content into a plurality of sub-element data; Transmitting, by the base node, the sub-element data to a display unit of the presentation device accordingly; The method of claim 50, further comprising the above. **Claim 58** The method according to any one of claims 56 and 57, wherein the sub-element data includes image data. **Claim 59** The method of claim 50, wherein the element data includes image data.