Methods and systems for dynamic projection area selection in a multi-user environment

Abstract

Disclosed herein is a dynamic projection area determination method (1600). The method (1600) includes receiving (1602) sensor data and assigning (1604) a unique identification (ID) to each user. Further, generating (1606) a field of view (FOV) for each user to provide an optimal viewing angle for each user. Further, determining (1608) one or more parameters associated with the one or more users based on the received sensor data. Further, creating (1610) a real-time engagement profile for each of the one or more users by fusing the one or more parameters. Further, calculating (1612) a target position and a size of content projection of the projection area based on the FOV and the user interest of each of the one or more users. Further, adjusting (1614) a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

Description

Description Title of Invention :METHODS AND SYSTEMS FOR DYNAMIC PROJECTION AREA SELECTION IN A MULTI-USER ENVIRONMENTTechnical Field

[0001] The present disclosure relates to projection devices, and more particularly, to a method and a system for dynamic projection area selection in a multi-use environment.Background Art

[0002] Projector technology has been recently upscaled for a wide range of applications from conference rooms to classrooms and from public theaters to home theaters, for displaying content over a screen. The projector technology often presents limitations for user engagement and content delivery. Currently, projectors provide limited comfortable and engaging viewing experience for users due to fixed viewing experience, and various viewing angles experience discomfort due to neck strain. The limitations lead to reduced engagement, inefficient information delivery, and limited accessibility. For example, a one-size-fits-all approach may lead to disinterest and a passive viewing experience. Important details might be lost for viewers at certain angles or those seeking specific information. Viewers with visual impairments or needing magnified details have no way to adjust the content.

[0003] Traditional projection systems rely on static displays, projecting the same information onto a single surface for all viewers. There are some existing technologies that attempt to address multi -projection. For instance, Figure 1A illustrates a multi -projector system 100, in accordance with a conventional technique. The multi -projector system involves the use of multiple projectors to create a larger and more immersive image. However, the use of multiple projectors becomes evidently expensive, complex to set up, and requires precise alignment. Figure IB illustrates an interactive projector 102, in accordance with a conventional technique. The interactive projector allows for some user interaction, like highlighting or annotating content. However, the interactive projector often requires specialized software and hardware, and interaction can be limited. Figure 1C illustrates head-mounted displays (HMDs) 104, in accordance with a conventional technique. The HMDs offer a personalized viewing experience. However, the HMDs are bulky, expensive, and cause user fatigue.

[0004] The traditional projection system offers a variety of projecting techniques to address user convenience, however, there are numerous disadvantages and limitations associated with the traditional projection systems. For instance, users at different locations experience varying levels of comfort and information accessibility due to the fixed view and static content which offer a single, immovable image for all viewers. This limits the user experience, especially in scenarios with diverse needs or large audiences. Further, the users at oblique angles experience neck strain or struggle to see the complete image and thereby difficulty seeing the entire projected image. Also, the users closer to a projector might face information overload, while those further away might lack crucial details. The users are forced to adjust their position, hindering the overall viewing experience. Additionally, there is limited user control, for example, users have limited control over the projected content, forcing the users to adjust their own positions, potentially causing discomfort, to optimize their viewing experience. Also, verbal request adjustments to the content, potentially disrupt the presentation flow.

[0005] Therefore, in view of the above-mentioned problems, it is advantageous to provide an improved system and method that can overcome the above-mentioned problems and limitations associated with projection area selection in a multi-user environment.Solution to Problem

[0006] This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention. This summary is neither intended to identify key or essential inventive concepts of the invention nor is it intended for determining the scope of the invention.

[0007] According to an embodiment of the present disclosure, a dynamic projection area determination method is disclosed. The method includes receiving sensor data indicative of at least one of orientation, location, and viewing preference of one or more users with respect to a projection area. Further, the method includes assigning a unique identification (ID) to each of the one or more users based on received sensor data. Further, the method includes generating a field of view (FOV) for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user. The method further comprises determining one or more parameters associated with the one or more users based on the received sensor data, the assigned unique ID, and the generated FOV. The one or more parameters are indicative of a state, an attention, and a role of the corresponding user. The method further comprises creating a real-time engagement profile for each of the one or more users by fusing the one or more parameters. The creating comprises assigning weights to the real-time engagement profile to indicate user interest in viewing content on the projected area. Further, the method includes calculating a target position and a size of content projection of the projection area based on the FOV and the user interest of each of the one or more users. Further, the method includes adjusting a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0008] In one or more embodiments, a dynamic projection area determination system is disclosed. The system includes a memory and a processor communicatively coupled with the memory. The processor is configured to receive sensor data indicative of at least one of orientation, location, and viewing preference of one or more users with respect to a projection area. Further, the processor is configured to assign a unique identification (ID) to each of the one or more users based on received sensor data. Further, the processor is configured to generate a field of view (FOV) for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user. The processor is configured to determine one or more parameters associated with the one or more users based on the received sensor data, the assigned unique ID, and the generated FOV. The one or more parameters are indicative of a state, an attention, and a role of the corresponding user. The processor is configured to create a real-time engagement profile for each of the one or more users by fusing the one or more parameters. The processor, in order to create, is configured to assign weights to the real-time engagement profile to indicate user interest in viewing content on the projected area. Further, the processor is configured to calculate a target position and a size of content projection of the projection area based on the FOV and the user interest of each of the one or more users. Further, the processor is configured to adjust a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0009] To further clarify the advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawing. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting its scope. The invention will be described and explained with additional specificity and detail with the accompanying drawings.

[0010] In an embodiment, an electronic apparatus includes at least one processor including processing circuitry, and memory storing instructions, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic apparatus to obtain sensing data related with one or more users with respect to a projection area, identify a field of view (FOV) for each of the one or more users based on the sensing data, obtain a target position and a size of content projection of the projection area based on the FOV, and obtain a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0011] The instructions, when executed by the at least one processor individually or collectively, may cause the electronic device to output an image through the projecting device based on the position and the orientation of the projecting device.

[0012] The sensing data may include at least one of orientation information, location information, and viewing preference information corresponding to each of the one or more users with respect to the projection area.

[0013] The instructions, when executed by the at least one processor individually or collectively, may cause the electronic device to obtain a unique identification (ID) corresponding to each of the one or more users based on sensing data, identify at least one parameter associated with the one or more users based on the sensing data, the assigned unique ID, and the FOV, and obtain the target position and the size of content projection of the projection area based on the FOV and the at least one parameter.

[0014] The at least one parameter may include at least one of state information, attention information or role information corresponding to each of the one or more users.

[0015] The instructions, when executed by the at least one processor individually or collectively, may cause the electronic device to obtain a real-time engagement profile for each of the one or more users based on the at least one parameter, and obtain the target position and the size of content projection of the projection area based on the FOV and the real-time engagement profile.

[0016] The instructions, when executed by the at least one processor individually or collectively, may cause the electronic device to obtain the real-time engagement profile by assigning weights to indicate user interest in viewing content on the projected area.

[0017] The instructions, when executed by the at least one processor individually or collectively, may cause the electronic device to obtain the sensing data including a three- dimensional (3D) point cloud representation of a multi-user environment, estimating a user gaze direction based on the sensing data, assigning initial priorities during content projection based on the user gaze direction estimated for each of the one or more users, and identify the at least one parameter based on the initial priorities.

[0018] The instructions, when executed by the at least one processor individually or collectively, may cause the electronic device to identify a user of the one or more users from an input frame, wherein the input frame corresponds to an image of the user captured by an imaging device; segregate each user from the input frame based on an encoder-decoder-based image segmentation task; extract one or more discriminating features of each user, wherein the one or more discriminating features correspond to physical attributes associated with the user; and obtain the unique ID based on the extracted one or more discriminating features.

[0019] The instructions, when executed by the at least one processor individually or collectively, may cause the electronic device to align projection in a preferred direction in which the user is interested in watching content based on a user command comprising one of a voice-based command or a gesture-based command, wherein in case of the voice-based command amplifying and de-noising an audio signal received from the user; converting the amplified audio signal into text information; and feeding the text information to a largelanguage model (LLM) framework to evaluate context and predict the preferred direction, wherein in case of the gesture-based command segmenting out body semantics from an input frame, wherein the body semantics correspond to a hand movement of the user; identifying key featuring points from the segmented body semantics; and generating a vector by performing global coordinate mapping on the identified key featuring points.

[0020] In an embodiment, a method of controlling an electronic apparatus, the method comprising: obtaining sensing data related with one or more users with respect to a projection area, identifying a field of view (FOV) for each of the one or more users based on the sensing data, obtaining a target position and a size of content projection of the projection area based on the FOV, and obtaining a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0021] The method may include outputting an image through the projecting device based on the position and the orientation of the projecting device.

[0022] The sensing data may include at least one of orientation information, location information, and viewing preference information corresponding to each of the one or more users with respect to the projection area.

[0023] The method may include obtaining a unique identification (ID) corresponding to each of the one or more users based on sensing data, identifying at least one parameter associated with the one or more users based on the sensing data, the assigned unique ID, and the FOV, and obtaining the target position and the size of content projection of the projection area based on the FOV and the at least one parameter.

[0024] The at least one parameter may include at least one of state information, attention information or role information corresponding to each of the one or more users.Brief Description of Drawings

[0025] These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0026] Figures 1A-1C illustrate different projector system, in accordance with the conventional techniques;

[0027] Figure 2A illustrates a block diagram of a system for dynamic proj ection area selection in a multi-user environment, in accordance with an embodiment of the present disclosure;

[0028] Figure 2B illustrates a block diagram depicting different modules of the system, in accordance with an embodiment of the present disclosure;

[0029] Figures 3A-3B illustrate exemplary scenarios depicting a user profile creation, in accordance with an embodiment of the present disclosure;

[0030] Figure 4 illustrates a flow diagram depicting voice-based command identification, in accordance with an embodiment of the present disclosure;

[0031] Figures 5 illustrate a flow diagram depicting gesture-based direction identification, in accordance with an embodiment of the present disclosure;

[0032] Figure 6 illustrates an exemplary scenario depicting the identification of the preferred wall area, in accordance with an embodiment of the present disclosure;

[0033] Figure 7 illustrates a flowchart depicting steps for estimation of user head pose, in accordance with an embodiment of the present disclosure;

[0034] Figure 8 illustrates another flowchart depicting an overall flow of individual field of view (FOV), in accordance with an embodiment of the present disclosure;

[0035] Figure 9 illustrates an exemplary scenario depicting a calculation of the individual FOV, in accordance with an embodiment of the present disclosure;

[0036] Figure 10 illustrates a flow diagram depicting a user movability score generation, in accordance with an embodiment of the present disclosure;

[0037] Figure 11 illustrates a flowchart depicting generation of attention level through a convolutional neural network (CNN), in accordance with an embodiment of the present disclosure;

[0038] Figure 12 illustrates an exemplary scenario depicting user role identification and attention level identification, in accordance with an embodiment of the present disclosure;

[0039] Figure 13A illustrates a flowchart depicting generation of context-aware priority, in accordance with an embodiment of the present disclosure;

[0040] Figure 13B illustrates an exemplary scenario depicting the generation of context- aware priority, in accordance with an embodiment of the present disclosure;

[0041] Figure 14A illustrates a flowchart depicting generation of a final FOV for the dynamic user-aware projection area, in accordance with an embodiment of the present disclosure;

[0042] Figure 14B illustrates an exemplary scenario depicting the final FOV, in accordance with an embodiment of the present disclosure;

[0043] Figures 15 A-l 5C illustrate a schematic depicting the conversion of the individual FOV into a grid of pixels to generate a final area of projection, in accordance with an embodiment of the present disclosure;

[0044] Figure 16 illustrates a flowchart depicting a method for dynamic projection area selection in the multi-use environment, in accordance with an embodiment of the present disclosure; and

[0045] Figures 17A-17D illustrate different exemplary scenarios of the dynamic projection area selection in the multi-use environment, in accordance with an embodiment of the present disclosure.

[0046] Figures 18 illustrate an embodiment of the electronic apparatus.

[0047] Further, skilled artisans will appreciate those elements in the drawings are illustrated for simplicity and may not have necessarily been drawn to scale. For example, the flow charts illustrate the method in terms of the most prominent steps involved to help and improve understanding of aspects of the present disclosure. Furthermore, in terms of the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.Description of Embodiments

[0048] It should be understood at the outset that although illustrative implementations of the embodiments of the present disclosure are illustrated below, the present invention may be implemented using any number of techniques, whether currently known or in existence. The present disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary design and implementation illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.

[0049] The term “some”, “one or more embodiment”, “one or more example embodiments”, as used herein is defined as “one, or more than one, or all.” Accordingly, the terms “one,” “more than one,” “more than one, but not all” or “all” would all fall under the definition of “some.” The term “some embodiments” may refer to one embodiment, several embodiments, or to all embodiments. Accordingly, the term “some embodiments” is defined as meaning “one embodiment, or more than one embodiment, or all embodiments.”

[0050] The terminology and structure employed herein are for describing, teaching, and illuminating some embodiments and their specific features and elements and do not limit, restrict, or reduce the spirit and scope of the claims or their equivalents.

[0051] More specifically, any terms used herein such as but not limited to “includes,” “comprises”, “has”, “have”, and grammatical variants thereof do not specify an exact limitation or restriction and certainly do not exclude the possible addition of one or more features or elements, unless otherwise stated, and must not be taken to exclude the possible removal of one or more of the listed features and elements, unless otherwise stated with the limiting language “must comprise” or “needs to include.”

[0052] Whether or not a certain feature or element was limited to being used only once, either way, it may still be referred to as “one or more features”, “one or more elements”, “at least one feature” or “at least one element.” Furthermore, the use of the terms “one or more” or “at least one” feature or element does not preclude there being none of that feature or element unless otherwise specified by limiting language such as “there needs to be one or more ...” or “one or more element is required.”

[0053] The terms “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” used in the various embodiments of the present disclosure includes any and all combinations of words enumerated with it. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” means (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0054] Although the term such as “first” and “second” used in various embodiments of the present disclosure may modify various elements of various embodiments, these terms do not limit the corresponding elements. For example, these terms do not limit an order and / or importance of the corresponding elements. These terms may be used for the purpose of distinguishing one element from another element. For example, a first user device and a second user device all indicate user devices and may indicate different user devices. For example, a first element may be named a second element without departing from the scope of right of various embodiments of the present disclosure, and similarly, a second element may be named a first element.

[0055] The expression “configured to (or set to)” used in various embodiments of the present disclosure may be replaced with “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of’ according to the situation. The term “configured to (set to)” does not necessarily mean “specifically designed to” as hardware. Instead, the expression “apparatus configured to . . . ” may mean that the apparatus is “capable of . . . ” along with other devices or parts in a certain situation. For example, “a processor configured to (set to) perform A, B, and C” may be a dedicated processor, for example, an embedded processor, for performing a corresponding operation, or a generic- purpose processor, for example, a Central Processing Unit (CPU) or an application processor (AP), capable of performing a corresponding operation by executing one or more software programs stored in a memory device.

[0056] A term “module” used in the present document may imply a unit including, for example, one of hardware, software, and firmware or a combination of two or more of them. The “module” may be interchangeably used with a term such as a unit, a logic, a logical block, a component, a circuit, and the like. The “module” may be a minimum unit of an integrally constituted component or may be a part thereof. The “module” may be a minimum unit for performing one or more functions or may be a part thereof. The “module” may be mechanically or electrically implemented. For example, the “module” of the present disclosure may include at least one of an Application-Specific Integrated Circuit (ASIC) chip, a Field-ProgrammableGate Arrays (FPGAs), and a programmable-logic device, which are known or will be developed, and which perform certain operations.

[0057] Unless otherwise defined, all terms, and especially any technical and / or scientific terms, used herein may be taken to have the same meaning as commonly understood by one having ordinary skill in the art.

[0058] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein.

[0059] As is traditional in the field, embodiments may be described and illustrated in terms of modules that carry out a described function or functions. These modules, which may be referred to herein as units or blocks or the like, or may include blocks or units, are physically implemented by analog or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, or the like, and may optionally be driven by firmware and software. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like. The circuits constituting a block may be implemented by dedicated hardware, by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block. Each block of the embodiments may be physically separated into two or more interacting and discrete blocks without departing from the scope of the invention. Likewise, the blocks of the embodiments may be physically combined into more complex blocks without departing from the scope of the invention.

[0060] The accompanying drawings are used to help easily understand various technical features and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. As such, the present disclosure should be construed to extend to any alterations, equivalents, and substitutes in addition to those which are particularly set out in the accompanying drawings. Although the terms first, second, third, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally only used to distinguish one element from another.

[0061] Embodiments of the present disclosure will be described below in detail with reference to the accompanying drawings.

[0062] The objective of the present disclosure is to provide a dynamic projection area selection in a multi-user environment by utilizing sensor data, user profiles, and real-time engagement analysis. The sensor data, user profiles real-time engagement analysis, and individual field of view (FOV) to create a context-aware weighting system. The context- aware weighting system assigns priority scores to different regions within each user’s FOV, considering the preferences and behavior of the individuals. The present disclosure discloses a priority-driven merging algorithm that aggregates user-specific weights and optimizes the selection of projection area by considering all user’s priorities and potential constraints. The present disclosure provides a personalized and dynamic projection experience compared to existing systems.

[0063] For the sake of clarity, the first digit of a reference numeral of each component of the present disclosure is indicative of the Figure number, in which the corresponding component is shown. For example, reference numerals starting with digit “1” are shown at least in Figure 1. Similarly, reference numerals starting with digit “2” are shown at least in Figure 2.

[0064] Figure 2 A illustrates a block diagram of a system 200 for dynamic projection area selection in a multi-user environment, in accordance with an embodiment of the present disclosure. Figure 2B illustrates a block diagram depicting different modules of the system 200, in accordance with an embodiment of the present disclosure.

[0065] In an embodiment, the system 200 may be implemented over a user equipment (UE) 202 via a remote server 222. In another embodiment, the system 200 may be implemented over the remote server 222. In one embodiment, the UE 202 may be communicatively coupled to the server 222 through a network 220. In one exemplary embodiment, the UE 202 may include a projector or a projecting device, a smartphone with a projector, etc.

[0066] The UE 202 may include a processor 204, a display 206, a memory, a communication interface 216, and input / output ports 218. The memory 208 may include an operating system 210, a database 212, and modules 214. In an embodiment, the UE 202 may act as an interface for one or more operations performed through the system 200. The UE 202 may be a projector device or a robotic projector or other kind of projecting device. The UE 202 may include a plurality of sensors 224 mounted over the UE 202 to detect and capture data about one or more users and environments.

[0067] Further, the processor 204 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor 204 may be configured to fetch and execute computer-readable instructions and data stored in the memory 208 and / or the modules 214. The modules 214 may include an sensor data and user profiling acquisition module 226, a user profile management module 228, a personalized field of view (FOV) generation module 230, a user engagement and context-aware weighting module 232, a dynamic user aware projection area optimization module 234, and a robot pose adjustment and content projection module 236, as illustrated in Figure 2B. At this time, the processor 204 may be a general- purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, and an Al-dedicated processor such as a neural processing unit (NPU). The processor 204 may control the processing of input data in accordance with a predefined operating rule or artificial intelligence (Al) model stored in the non-volatile memory and the volatile memory, i.e., the memory 208. The predefined operating rule or artificial intelligence model is provided through training or learning. Further, the processor 204 may be operatively coupled to each of the memory, the UO Interface. The processor 204 may be configured to process, execute, or perform a plurality of operations described herein.

[0068] The memory 208 may include any non-transitory computer-readable medium known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic random-access memory (DRAM), and / or non-volatile memory, such as read-only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. The memory 208 is communicatively coupled with the processor to store processing instructions for completing the process. Further, the memory 208 may include the operating system 210 for performing one or more tasks of the system, as performed by an operating system 210 in a computing domain. The memory 208 is operable to store instructions executable by the processor 204.

[0069] As discussed, the UE 202 may include the processor 204, e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 204 may be a component in a variety of systems. The processor 204 may be one or more general processors, digital signal processors, application-specific integrated circuits, field- programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now known or laterdeveloped devices for analyzing and processing data. The processor 204 may implement a software program, such as code generated manually (i.e., programmed).

[0070] As mentioned above, the UE 202 may include the memory 208, such as a memory 208 that can communicate via a bus. The memory 208 may include, but is not limited to, computer- readable storage media such as various types of volatile and non-volatile storage media, including, but not limited to, random access memory, read-only memory, programmable readonly memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media and the like. In one example, memory 208 includes a cache or random-access memory for the processor 204. In alternative examples, the memory 208 is separate from the processor 204, such as a cache memory of a processor, the system memory, or other memory. The memory 208 may be an external storage device or database for storing data. The memory 208 is operable to store instructions 206 executable by the processor 204. The functions, acts or tasks illustrated in the figures or described may be performed by the programmed processor 204 for executing the instructions 206 stored in the memory 208. The functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro-code and the like, operating alone or in combination. Likewise, processing strategies may include multiprocessing, multitasking, parallel processing and the like.

[0071] As shown, the UE 202 may or may not further include the display 206, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a flat panel display, a solid- state display, a cathode ray tube (CRT), a projector, a printer or other now known or later developed display device for outputting determined information. The display 206 may act as an interface for the user to see the functioning of the processor 204, or specifically as an interface with the software stored in the memory 208.

[0072] The present invention contemplates a computer-readable medium that includes memory 208 having executable instructions responsive to a propagated signal so that a device connected to the network 220 can communicate voice, video, audio, images, or any other data over the network 220. Further, the instructions 208 may be transmitted or received over the network 220 via the communication interface or port 218. The communication interface 216 may be a part of the processor 204 or maybe a separate component. The communication interface 216 may be created in software or maybe a physical connection in hardware. The communication interface 216 may be configured to connect with the network 220, external media, the display 210, or any other components in UE 202, or combinations thereof. The connection with the network 220 may be a physical connection, such as a wired Ethernet connection, or may be established wirelessly as discussed later. Likewise, the additional connections with other components of the UE 202 may be physical or may be established wirelessly.

[0073] The network 220 may include wired networks, wireless networks, Ethernet AVB networks, or combinations thereof. The wireless network may be a cellular telephone network, an 802.11, 802.16, 802.20, 802. IQ, or WiMax network. Further, the network 220 may be a public network, such as the Internet, a private network, such as an intranet, or combinations thereof, and may utilize a variety of networking protocols now available or later developed including, but not limited to, TCP / IP based networking protocols. The UE 202 may not be limited to operation with any particular standards and protocols. For example, standards for Internet and other packet-switched network transmissions (e.g., TCP / IP, UDP / IP, HTML, and HTTP) may be used.

[0074] In an embodiment, the system 200 offers a dynamic projection area selection in a multi-user environment. The system 200 may utilize sensor data, user profiles, and real- timeuser engagement analysis to generate a merged projection area optimized for the best user experience. As mentioned earlier, the modules 214 may include the sensor data and user profiling acquisition module 226, the user profile management module 228, the personalized field of view (FOV) generation module 230, the user engagement and context- aware weighting module 232, the dynamic user aware projection area optimization module 234, and the robot pose adjustment and content projection module 236.

[0075] The sensor data and user profiling acquisition module 226 may be configured to acquire sensor data through the plurality of sensors 224 by capturing the environment and user locations. The plurality of sensors 224 may include a LiDAR sensor, a camera, etc, and the sensor data may include LiDAR scans, and camera images, The sensor data and user profiling acquisition module 226 may be configured to retrieve user profiles stored, including predefined roles of users, for example, a presenter and an audience, and preferences like viewing angles.

[0076] In the user profile management module 228, individual users may be identified and tracked from the sensor data. The user profile management module 228 may be configured to analyze user association techniques linked from the sensor data with specific user profiles and may be configured to monitor user positions and movements. The system 200 may analyze user preference and user interaction.

[0077] Further, the personalized FOV generation module 230 may be configured to generate a preliminary FOV for each user. The preliminary FOV for each user may represent a viewable area of each user. Further, the personalized FOV generation module 230 may generate head pose estimation by analyzing the sensor data in a projection environment to estimate the head pose of each user. The personalized FOV generation module 230 may generate a personalized FOV from the user location and orientation data which define initial user FOVs. The personalized FOV generation module 230 may perform FOV refinement from the sensor data by excluding obstructed areas from the FOV.

[0078] Further, the user engagement and context-aware weighting module 232 may prioritize regions within user FOVs for optimal projection. The user engagement and context-aware weighting module 232 may generate user movability analysis, user attention analysis, user role identification, and adaptive user engagement weighing. The user movability analysis may include user movement patterns, for example, head movement, fidgeting, etc. The user movement patterns may be analyzed for user mobility and adjust priority for a stable viewing area.

[0079] The user attention analysis may include analyzing the sensor data to estimate gaze direction and attention level. Further, the user attention analysis may include prioritized areas within FOV that may likely be actively viewed by users to maximize the effectiveness of projected content. The user role identification may be assessed from profiles and real- time behavior, for example, body language. Further, the adaptive user engagement weighing may include assigning adaptive weights to the user's two-dimensional (2D) FOV based on the analysis.

[0080] Further, the dynamic user-aware projection area optimization module 234 may be configured to combine the weighted 2D FOVs of multiple users into a single projection area. The dynamic user-aware projection area optimization module 234 may prioritize regions with higher weights across user FOVs for optimal content visibility. The robot pose adjustment and content projection module 236 may refine the robot position and orientation, i.e., the projector. The robot pose adjustment and content projection module 236 may keep the merged projection area as a target to ensure optimal projection alignment. The robot pose adjustment and content projection module 236 may project desired content onto the merged projection area as definedby the dynamic user-aware projection area optimization module 234. The modules 214 may be described in further detail in conjunction with Figures 3 A-l 17D.

[0081] Further, the system 200, through the processor 204, may be configured to perform one or more operations for selecting the projection area in the multi-user environment. The processor 204 may be configured to receive the sensor data indicative of at least one of the orientation, location, and viewing preferences of one or more users with respect to a projection area. In an embodiment, the processor 204 in order to receive the sensor data, may capture a three-dimensional (3D) point cloud representation of the multi-user environment. Further, the processor 204 may determine the at least one of orientation, the location, and the viewing preference of the one or more users with respect to the projection area based on the 3D point cloud representation. In this embodiment, the processor 204 may further be configured to estimate a user gaze direction based on the at least one of orientation, the location, and the viewing preference of the one or more users with respect to the projection area, for user identification. Further, the processor 204 may assign initial priorities during content projection based on the user gaze direction estimated for each of the one or more users. The one or more users interact through voice commands to provide feedback and update the viewing preferences in real-time.

[0082] The processor 204 may be configured to assign a unique identification (ID) to each of the one or more users based on received sensor data. In an embodiment, the processor 204, in order to assign the unique ID, may be configured to identify a user of the one or more users from an input frame 304, as illustrated in Figure 3B. The input frame 304 corresponds to an image of the user captured by an imaging device. Further, the processor 204 may be configured to segregate each user from the input frame 304 based on an encoder-decoder-based image segmentation task. In this embodiment, the processor 204 may extract one or more discriminating features 302 of each user. The one or more discriminating features 302 correspond to physical attributes associated with the user. Finally, the processor 204 may assign the unique ID based on the extracted one or more discriminating features 302.

[0083] In an embodiment, the processor 204 may be configured to align projection in a preferred direction in which the user is interested in watching content based on a user command comprising one of a voice-based command or a gesture-based command. In case of the voicebased command, the processor 204 may amplify and de-noise an audio signal received from the user. Further, the processor 204 may convert the amplified and audio signal into text information. Further, the processor 204 may feed the text information to a large language model (LLM) framework to evaluate context and predict the preferred direction. In the case of the gesture-based command, the processor 204 may segment out body semantics from the input frame 304. The body semantics correspond to a hand movement of the user. Further, the processor 204 may identify key featuring points from the segmented body semantics. Further, the processor 204 may generate a vector by performing global coordinate mapping on the identified key featuring points.

[0084] In the above embodiment, the processor 204 may identify projection planes of the projection area where the content is projected. The identified projection planes are stored in a database during mapping. Further, the processor 204 may calculate an array of the projection planes by performing geometrical transformation to a global coordinate system in the preferred direction.

[0085] The processor 204 may be configured to generate the FOV for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user. In an embodiment, the processor 204, in order to generate the FOV for each user, may be configured to build a deep learning (DL) model by training on the sensor data indicative of images of the one or more users. Further, the processor204 may extract features from head region of each of the one or more users from the images of the one or more users. Further, the processor 204 may map the extracted features to head pose angles of each of the one or more users, wherein the head pose angles are indicative of roll, pitch, and yaw. Therefore, the processor 204 may generate the FOV for each of the one or more users by calculating a probability distribution of each head pose angle with respect to an actual angle value.

[0086] In the above embodiment, the processor 204 may transform the head pose angles of each user into global coordinates to detect the projection plane. Furthermore, the processor 204 may select the projection plane along the preferred direction based on the global coordinates. Further, the processor 204 may transform the position of the user into the global coordinate to determine an absolute distance between the projection plane and the user.

[0087] The processor 204 may be configured to determine one or more parameters associated with the one or more users based on the received sensor data, the assigned unique ID, and the generated FOV. The one or more parameters are indicative of a state, an attention, and a role of the corresponding user. The processor 204 may be configured to create a real-time engagement profile for each of the one or more users by fusing the one or more parameters. The processor 204 in order to create the real-time engagement profile, may be configure to assign weights to the real-time engagement profile to indicate user interest in viewing content on the projected area. The processor 204 may be configured to calculate a target position and a size of content projection of the projection area based on the FOV and the user interest of each of the one or more users. The processor 204 may be configured to adjust a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0088] In an embodiment, coordinates of each of the one or more users in the input frame 304 may be identified and a centroid of each user in the input frame 304 mya be calculated. Further, a state of the user may be determined by creating a bounding box with variable dimensions based on identified coordinates and calculated centroid of the user. The bounding box corresponds to objects of the variable dimensions associated with the user and present within the input frame 304. Further, a movability score may be assigned to each user of the one or more users based on the state of the user.

[0089] According to the above embodiment, a convolutional recurrent neural network (CRNN) model may be built by training the CRNN model on a dataset of facial expressions of the one or more users from the input frame 304 to identify attention levels of the one or more users. The dataset of facial expressions is indicative of eye gaze, brow furrowing, and mouth movements. Further, relative weights may be assigned by sorting the facial expressions based on the attention levels and the relative priorities of the one or more users may be calculated based on the relative weights.

[0090] According to the above embodiment, the one or more users may be identified from the input frame 304 through an object detection module. Also, different activities performed by the one or more users may be detected. Further, the role of each user may be identified based on the detected activities of the one or more users.

[0091] In an embodiment, a user engagement matrix may be generated based on the one or more parameters. The one or more parameters correspond to a mobility score, an attention score, and relative role importance. Further, engagement weighing score for each user may be calculated based on the user engagement matrix. Further, relative user engagement scores may be calculated collectively to avoid overshooting or undershooting by normalizing user priorities on a linear scale.

[0092] In an embodiment, the projection area may be divided into an array of pixels based on the FOV and priority of each of the one or more users. Further, weight to each user may beassigned based on the priority. Further, weight of each pixel of the array of pixels may be calculated based on overlapping FOVs. Further, a target position and size of content on the projection area may be determined. Successively, a projectable area with a portion on the projection area may be calculated. The projectable area indicates a combined FOV of the one or more users on the projection area.

[0093] In an embodiment, a collision-free trajectory may be generated for the projecting device by calculating the position and orientation of the projecting device with respect to the projection area.

[0094] Figures 3A-3B illustrate exemplary scenarios 300 depicting a user profile creation, in accordance with an embodiment of the present disclosure.

[0095] In an embodiment, prior to the user profile creation, the sensor data and user profile acquisition module 226 may perform sensor data acquisition and user profile acquisition. Stated differently, the sensor data and user profile acquisition module 226 collects sensor data and user profile. The sensor data and user profile acquisition module 226 may utilize the plurality of sensors 224, for example, LiDAR sensor and camera, mounted on a movable projector to capture data about users and environment. The LiDAR sensor may capture three-dimensional (3D) point cloud representation of the environment.

[0096] The sensor data may be used to identify user locations, and determine user postures, user movements, and environment details. The camera may capture real-time video data that may be used for user identification, through facial recognition or other techniques. The realtime video data may be utilized for estimating user gaze direction through different eyetracking algorithms.

[0097] The sensor data and user profiling acquisition module 226 may acquire the user profile by gathering user profile information to personalize the viewing experience. The user profile information may be obtained from different sources. For example, the user profile information may be obtained from a pre-defined user database which may contain pre-existing user information such as roles, i.e., presenter, audience member. The pre- existing user information may be used to assign initial priorities during content projection.

[0098] Further, the user profile information may be obtained through a user input application (App). The one or more users may download and utilize a dedicated mobile App to provide their preferences, for example, preferred viewing angles, sitting position, and manage user profiles. Further, the user profile information may be obtained through voice commands that may allow the one or more users to interact through voice commands to provide feedback or update their preferences in real-time.

[0099] The user profile management module 228 may be configured to generate a preferred area / direction of projection along with the unique IDs of the one or more users. The user profile management module 228 may consider the voice-based command, the gesture-based command and the sensor data, i.e., RGB-D camera data, as input. The user profile management module 228 may be configured to identify each of the one or more users as an individual and may assign each user a user ID. The user profile management module 228 may retrieve preferred area of projection of each user. The user profile management module 228 may be segregated into user profile creation and retrieval, user interaction analysis, and user preference analysis.

[0100] In the user interaction analysis, user inputs such as the voice-based command, the gesture-based command may be analyzed and may be provided to the user engagement and context-aware weighting module 232 for further analysis. The user engagement and context- aware weighting module 232 will be described in greater detail in later embodiments.

[0101] The user interaction analysis may identify discriminatory features of users in the input frame 304 and may provide each user the unique user ID for user tracking and user reidentification purpose. Further, the mapping of discriminatory features to user IDs may bestored into a database for future reference. In the user preference analysis, based on inputs from one or more users, projection areas in the preferred directions of the one or more users may be identified. The output of the user profile management module 228 may be sent to the personalized FOV generation module 230.

[0102] The sensor data and user profile acquisition module 226 may be configured for the user profile creation. The sensor data and user profile acquisition module 226 may consider the RGB-D image as input and generate a database containing user characteristics and associated unique IDs. The sensor data and user profile acquisition module 226 may identify and map the unique IDs of the one or more users. The sensor data and user profile acquisition module 226 may perform an encoder-decoder-based image segmentation to segment out each user of the one or more users as an individual.

[0103] Further, the one or more discriminating features 302 of each individual may be extracted. As mentioned earlier, the one or more discriminating features 302 may include physical attributes like facial distribution such as distance between nose and mouth, size of eyelids, forehead, and other characteristics, height, torso width, skin complexion, and color of clothes. The sensor data and user profile acquisition module 226 may assign each individual the unique ID based on the one or more discriminating features 302. In an embodiment, the current position of the individual in the input frame 304 as bounding boxes may be stored along with the unique ID.

[0104] Figure 4 illustrates a flow diagram 400 depicting voice-based command identification, in accordance with an embodiment of the present disclosure.

[0105] As mentioned earlier, in the user interaction analysis, user inputs such as the voicebased command, and the gesture-based command may be analyzed. The sensor data and user profile acquisition module 226 may generate the preferred direction of projection for each user by considering voice-based commands and the sensor data, for example, the RGB-D camera. The sensor data and user profile acquisition module 226 may identify user- interaction mode, which in this case, may be the voice-based command. In an embodiment, the sensor data and user profile acquisition module 226 may generate the direction in which the user may be interested in watching content.

[0106] In the case of the user interaction of voice-based command, an audio signal of the user may be collected. Further, the audio signal may be amplified and de-noised through an operation amplifier (Op-Amp) and band-pass filters 402.

[0107] Further, the amplified and processed voice signal is sent to a speechT5 model 404 to convert the amplified and processed voice signal to the appropriate text. Further, the text may be transmitted to a large language model (LLM) framework 406 for understanding the context and predicting the direction of projection.

[0108] Figures 5 illustrate a flow diagram 500 depicting the gesture-based command identification by the sensor data and user profile acquisition module 226, in accordance with an embodiment of the present disclosure.

[0109] The sensor data and user profile acquisition module 226 may identify user- interaction mode, which in this case, may be the gesture-based command.

[0110] In the case of gesture-based user interaction, an image processing-based technique may be used. At first, the gesture of the user or the input frame 304 may be fed to an encoder- decoder-based semantic extraction module 502. The encoder-decoder-based semantic extraction module 502 may segment hand semantics 504 out of the input frame

[0111] 304. The hand semantics 504 may be hand coordinates extracted from a semantic mask.

[0112] After extracting the hand semantics 504 out of the input frame 304, the key featuring points 506 may be identified through a convex hull algorithm. Further, based on the keyfeaturing points 506, a direction vector may be generated by performing a global coordinate mapping 508.

[0113] Figure 6 illustrates an exemplary scenario 600 depicting the identification of the preferred wall area, in accordance with an embodiment of the present disclosure.

[0114] The sensor data and user profile acquisition module 226 may generate a projectable plane in the projection direction based on the preferred direction of each user.

[0115] The sensor data and user profile acquisition module 226 may determine the proj ection planes where content may be projected based on the preferred direction of the user. In an embodiment, all possible projection planes may be identified and stored in a database (DB) during mapping or exploration. Further, by performing a simple geometrical transformation to a global coordinate system, the preferred projection plane of each user may be evaluated. Further, an array of preferred projection planes may be sent to the personalized FOV generation module 230 to calculate individual FOV.

[0116] Figure 7 illustrates a flowchart 700 depicting steps for estimation of user head pose, in accordance with an embodiment of the present disclosure.

[0117] The personalized field of view (FOV) generation module 230 may be configured to generate a personalized FOV for each user within a projection environment. The personalized field of view (FOV) generation module 230 may generate individual user FOV by considering the sensor data, i.e., LiDAR sensor and camera, the user profile, and the preference data, as input.

[0118] The personalized FOV generation module 230 may utilize the sensor data to capture user position and orientation. Further, the personalized FOV generation module 230 may employ geometric calculations to determine the optimal viewing angle for each user.

[0119] Further, the personalized FOV generation module 230 may integrate the available user profile data to personalize FOVs based on the preferences. Further, the personalized FOV generation module 230 may maximize user comfort and engagement by ensuring content visibility within a natural gaze area of each user. The personalized FOV generation module 230 may be configured for the user's head pose estimation, as illustrated by the flowchart 700 in Figure 7, and individual FOV generation as illustrated by a flowchart 800 in Figure 8. In an embodiment, the refined individual FOV generated by the personalized FOV generation module 230 may be utilized by the dynamic user-aware projection area optimization module 234.

[0120] In an embodiment, the user head pose estimation may utilize deep learning to estimate individual head pose or user head pose based on the sensor data. The estimated individual head pose may be utilized for generating personalized FOV accurately for each user. The user head pose estimation may generate the individual head pose with angles of roll, pitch, and yaw by considering the sensor data, i.e., camera image and LiDAR scan.

[0121] At step 702, the camera image may be preprocessed to resize the image. For example, the captured image is preprocessed for image resizing (224X224) and normalized in a range of 0-1.

[0122] At 704, the preprocessed image may be fed to a deep neural network (DNN model for generating a raw model prediction. For example, ResNet-50 pre-trained with a head dataset of labeled images is used as the DNN model for head pose estimation. ResNet-50’ s convolutional layers may extract features from the users head region in the camera image / processed image. Further, the final layers of the ResNet-50, also referred to as fully connected layers, may take extracted features and map them to the head pose angles of the user, i.e., yaw, pitch, roll. Further, an output layer, e.g., softmax activation layer, may generate a probability distribution for each pose angle

[0123] At step 706, the raw model prediction may be fed for post-processing to generate the final head pose. For example, during the post-processing probabilities from the softmax activation layer may be converted into actual angle values.

[0124] Figure 8 illustrates another flowchart 800 depicting an overall flow of generating the individual FOV, in accordance with an embodiment of the present disclosure.

[0125] The individual FOV generation may include updating the user profile with the FOV based on preference or the head pose. The user's head pose data, i.e., head pose angles of roll, pitch, yaw, and cartesian coordinates (X, Y, Z) along with the user preference direction may be used as input.

[0126] The individual FOV generation may utilize user position and orientation data to calculate an optimal viewing angle for the projected content. The individual FOV generation may ensure that the projected content falls within a natural FOV of the user.

[0127] At step 802, the head pose angle, i.e., yaw, of each user may be transformed into a global coordinate to find out the projection plane in the user preference direction.

[0128] At step 804, based on the transformed head pose angle, the projection plane coordinate may be determined. For example, when the user has having preferred direction then the projection plane may be selected in that direction.

[0129] At step 806, the distance between the head and projection plane may be calculated. For example, user position (X, Y, Z) may be transformed into the global coordinate to find out the absolute distance between the projection plane and the head.

[0130] At step 808, the ecliptic FOV of the user on the projection plane may be calculated. Assuming, the centroid of the projection plane as Xp, Yp, Zp. The absolute distance between the head and the projection plane may be calculated from the following equation (first group) (1):0131]0132] At step 810, the rectangular FOV of the user may be calculated on the projection plate.For example, considering standard user has a horizontal FOV 0 of approximately 120 degrees and a vertical FOV 0 of approximately 60 degrees.

[0133] Figure 9 illustrates an exemplary scenario 900 depicting a calculation of the individual FOV, in accordance with an embodiment of the present disclosure.

[0134] The user head positioning at (X, Y, Z), may have the FOV projection onto the projection plane, such as vertical FOV range dv & horizontal FOV range dh may be calculated from the following equations (first group) (2) and (3):

[0137] The dv and dh may be the semi-minor and semi-major axes of the ecliptic FOV projection onto the wall, and the equation (first group) over plane may be as follow (4):0138]0139] Therefore, a rectangle having maximum area within the ecliptic FOV region may have the horizontal arm as dhN2 and vertical arm as dv / 2, having the same center as that of the eclipse.

[0140] Further, applying simple geometry the comers of the rectangle may be calculated as mentioned in the following equation (first group) (5):

[0141]

[0142] where (x, y) is the centroid of the eclipse.

[0143] The FOVs of individual users may be sent to the dynamic user-aware projection area optimization module 234 to generate for calculating a final common FOV.

[0144] Figure 10 illustrates a flow diagram 1000 depicting a user movability score generation, in accordance with an embodiment of the present disclosure.

[0145] As mentioned earlier, the user engagement and context-aware weighting module 232 may generate user movability analysis, user attention analysis, user role identification, and adaptive user engagement weighing. The user engagement and context-aware weighting module 232 may generate common FOV of the one or more users present in the input frame 304. The user engagement and context-aware weighting module 232 may consider the unique IDs, user-preferred directions, and sensor data, i.e., RGB images and depth images as input.

[0146] In the user preference identification, the user engagement and context-aware weighting module 232 may take the preferred direction of each user as input and generate the projectable areas in the direction of the personalized FOV generation module 230.

[0147] In the user role identification, the user engagement and context-aware weighting module 232 may identify the role of each user, for example, if the user is a teacher or a presenter a student, and so on, based on RGB images.

[0148] The user engagement and context-aware weighting module 232, in the user movability analysis, may check how freely the user may change or adjust his / her position. Further, the user engagement and context-aware weighting module 232 may generate scores for one or more users based on movability.

[0149] The user engagement and context-aware weighting module 232, in the user attention analysis, may detect the attention level of each user, and based on the attention level the user engagement and context-aware weighting module 232 may provide a score.

[0150] The user engagement and context-aware weighting module 232, in the context- aware priority weighting, may calculate the priority of each user based on roles, attentions, and movability of the one or more users. The user engagement and context-aware weighting module 232 based on the priority, may calculate the common FOV.

[0151] The user engagement and context-aware weighting module 232 in the priority- driven FOV merging, may consider individual FOVs of the one or more users and the respective priority orders of the one or more users. The user engagement and context-aware weighting module 232 may finally generate the common FOV of the one or more users. The user engagement and context-aware weighting module 232 is described in greater detail in conjunction with Figures 10-13B.

[0152] In the illustrated embodiment, the user engagement and context-aware weighting module 232 may generate the user movability analysis and may assign the score based on the movability of the one or more users in the input frame 304. In an embodiment, the movability indicates how easily the one or more users may change the states / positions and adjust to watch the projected content. For example, if a person is standing or sitting on a light plastic chair, he / she may easily rotate or change his / her position if needed, whereas, if a person is sitting on a sofa or bed, it is hard for him / her to adjust his / her current state.

[0153] Therefore, the one or more users with lesser movability may be assigned higher priority during determining common FOV.

[0154] At step 1002, user coordinates in the input frame 304 may be identified.

[0155] At step 1004, the centroid of each user in the input frame 304 may be calculated.

[0156] At step 1006, bounding boxes (BB) with variable dimensions may be created as long as the state of the user may be identified, i.e., any object like a sofa, chair, or bed may be observed that may be attached to the user. In case the BB crosses a threshold, and no such object(s) may be found then the user may be considered free / standing.

[0157] At step 1008, based on the state of the user and the object on which the user may be sitting or lying (if any), the object may be classified into certain pre-defined classes. The predefined class may include but is not limited to tools, plastic chairs, steel chairs, recliners, sofas, beds, and so on. Further, based on the classification, each user may be given or assigned a movability score.

[0158] Figure 11 illustrates a flowchart 1100 depicting generation of attention level through a convolutional neural network (CNN), in accordance with an embodiment of the present disclosure.

[0159] The user engagement and context-aware weighting module 232 may be configured to generate the user attention analysis or user attention level.

[0160] The attention levels of the one or more users may be identified based on the facial expressions. Further, the user engagement and context-aware weighting module 232 based on the attention levels, the one or more users may be sorted and may be assigned relative weights, which may be further used to calculate the relative priorities of the one or more users.

[0161] As mentioned earlier, the CRNN model may be built by training on a dataset of facial expressions of the one or more users from the input frame 304 to identify the attention levels of the one or more users. In an embodiment, a convolutional neural network (CNN) may be utilized for learning spatial hierarchies of features, i.e., in this case, eye gaze, brow furrowing, mouth movements, and so on, from the input frame 304.

[0162] Further, training the CNN with the dataset of different attention levels, the CNN may classify images of the one or more users from the input frame 304 containing facial expressions into specific attention states. In an embodiment, a recurrent neural network (RNN) may be employed to capture temporal dependencies in facial expressions overtime, which may provide more context for attention-level inference. Therefore, spatial features extracted from the CNN may be fed to the RNN for temporal feature extraction.

[0163] Further, the temporal features may be concatenated to the spatial features extracted using the CNN, which may be further fed to a fully connected neural network (FCNN) for final attention level identification.

[0164] Figure 12 illustrates an exemplary scenario 1200 depicting user role identification and attention level identification, in accordance with an embodiment of the present disclosure.

[0165] The user engagement and context-aware weighting module 232 may be configured to generate the user role identification. The user engagement and context-aware weighting module 232 may role of each user existing in the input frame 304 may be identified. The userengagement and context-aware weighting module 232 may consider the RGB frame as input and an array of roles of each user existing in the RGB frame may be an output.

[0166] The user engagement and context-aware weighting module 232 may utilize the object detection module Yolov8 to identify the one or more users from the input frame 304. Further, each user may be identified as the individual to find out their activities.

[0167] Further, a generative adversarial network (GAN) based human activity recognition (HAR) may be used to determine the activities of individual persons. Further, based on the determined activities, roles are identified. For instance, if user A is presenting a project and user B is watching the project being presented by the user A, then user A will be referred to as the presenter and user B will be referred to as the audience.

[0168] Figure 13A illustrates a flowchart 1300 depicting the generation of context-aware priority, in accordance with an embodiment of the present disclosure. Figure 13B illustrates an exemplary scenario 1308 depicting the generation of context-aware priority, in accordance with an embodiment of the present disclosure.

[0169] The user engagement and context-aware weighting module 232 may be configured to generate a normalized engagement weight for the one or more parameters. The normalized engagement weight may correspond to the calculation of the relative user engagement scores to avoid overshooting or undershooting. The one or more parameters, like, user attention score, mobility score, and role importance, may be considered as input by the user engagement and context-aware weighting module 232. The user engagement and context-aware weighting module 232 may generate an array of users and their respective priorities. The user engagement and context-aware weighting module 232 may consider user features and environment information to determine relative user priorities.

[0170] At step 1302, the one or more parameters, i.e., the mobility score, the attention score, and the relative-role-importance, for the one or more users may be gathered or fused to generate the user engagement matrix. The user engagement and context-aware weighting module 232 may based on the user priorities, generate context-aware priority. As illustrated, user priorities, like, priority 1, priority 2, ..., etc., may be grouped and segregated. Further, based on the relative role importance priorities may be arranged in the user-engagement matrix.

[0171] In an embodiment, at least three characteristic user features may be extracted in the following equation (second group) (1):

[0172]

[0173] Where idi is the user ID assigned to ith user, mi, ai and ri are the mobility score, the attention score, and the relative-role-importance of ith user.

[0174] Further, one-to-one data representation as mentioned above may be reduced to one- to-many representation in the following equation (second group) (2):

[0175] 0176’ Further, a user-information-matrix of size nX3 may be generated, where n is the number of users and represents the dimension of information obtained, i.e., mobility, attention and role-importance, for each user. Therefore, the user-information matrix may be illustrated as follows in equation (second group) (3):

[0177] 0178] Where U is the user-information matrix where-

[0179] mi = mobility of ith user, ai = attention of ith user, ri = relative-role-importance of ith

[0180] user.

[0181] At step 1304, the engagement weighing-score of each user may be calculated based on the user-featuring-matrix and engagement matrix through dynamic engagement weightage calculation.

[0182] The user-engagement score may be calculated from the user-information matrix and relative weighing matrix. The relative-weighing-matrix may be a one-dimensional matrix having 3 elements, which may signify the relative importance of the one or more parameters, i.e., mobility, attention, and role, for example, as illustrated in the following equation (second group) (4):

[0183]

[0184] The relative-weighing-matrix may be represented by the following equation (second group) (5):

[0185]

[0186] Where, wm, waand wr are relative weights of the one or more parameters, i.e., mobility score, attention score and relative-role-importance.

[0187] Further, from r and U relative user-priorities may be calculated. Where superscript

[0188] T represents the transpose of a matrix.

[0189] The relative weighing matrix P is given by following equation (second group)

[0191] At step 1306, the user-priorities may be normalized to 1 to calculate relative userengagement scores collectively and to avoid any overshoots or undershoots. Further, the userpriorities may be transmitted to the dynamic user aware projection area optimization module 234 to generate for calculating a final common FOV. The steps 1302-1306 will be described in further detail in the following embodiments.

[0192] In an embodiment, the engagement score of the one or more users may be normalized to calculate relative priorities. It will be apparent that normalization may be important to reduce sensitivity to outliers, enhance convergence, and to avoid any kind of bias in the data.

[0193] In the above embodiment, the minimum-maximum normalization may be used for normalizing an entire series. In an exemplary embodiment, the minimum-maximum normalization may preserve the original shape of the distribution while scaling the values to a consistent range.

[0194] The normalization may be performed through iteration through the series of data and find the minimum value Xmin. Further, iteration through the series of data and finding the maximum value Xmax may be performed. Successively, iteration through the series of data may be done and the scaled value for each sample in the series may be calculated as below in the following equation (second group) (6):

[0195]

[0196] where Xi and Xi are the minimum-maximum normalized and original values of the series.

[0197] Figure 14A illustrates a flowchart 1400 depicting the generation of the final FOV for the dynamic user-aware projection area, in accordance with an embodiment of the present disclosure. Figure 14B illustrates an exemplary scenario 1412 depicting the final FOV, in accordance with an embodiment of the present disclosure.

[0198] The dynamic user-aware projection area optimization module 234 may optimize the projection area for multiple user projection systems to ensure an optimal user experience by considering individual needs and environmental constraints. The dynamic user-aware projection area optimization module 234 may utilize a heuristic method to identify the projection area to maximize visibility based on users’ engagement. The dynamic user-awareprojection area optimization module 234 may consider user profile data, user FOV region on the wall, and user priority, as input. Further, the dynamic user- aware projection area optimization module 234 may generate an optimized projection area, i.e., the target position and size of the content projection.

[0199] At step 1402, the wall may be divided into an array of pixels, by receiving the user profile data, user FOV region on the wall, and user priority. At step 1404, weights may be assigned to each user based on the user priority.

[0200] At step 1406, the weight of the array of pixels may be calculated based on overlapping FOVs. At step 1408, the optimal area of projection may be evaluated. At step 1410, a rectangle of the maximum area within the optimal FOV may be calculated to generate the final FOV, as illustrated in Figure 14B.

[0201] Figures 15A-15C illustrate different schematics 1500, 1502, 1504 depicting the conversion of the individual FOV into a grid of pixels to generate a final area of projection, in accordance with an embodiment of the present disclosure.

[0202] The dynamic user-aware projection area optimization module 234 may convert the FOV for each user into a grid of pixels. The dynamic user-aware projection area optimization module 234 may assign the weight to each pixel based on the user priority.

[0203] The dynamic user-aware projection area optimization module 234 may consider individual user priority scores, individual FOV coordinates, and pre-defined grid resolution, as input and may generate a final area of projection as output.

[0204] As illustrated in Figure 15 A, the dynamic user-aware projection area optimization module 234 may first divide the projection wall into a grid of pixels of certain pre-defined gird resolution. Further, individual FOV may be mapped in the wall coordinate, to identify which pixel of the wall belongs to which individual FOV. Further, a pixel may fall under multiple FOVs of the one or more users.

[0205] Further, each user may be assigned a certain weightage based on relative priorities of each user using the following equation (second group) (7):

[0206]

[0207] Where wi is the weightage given to ith user and ri is the relative rank among all other users and n is the total number of users present.

[0208] As illustrated in Figure 15B, the dynamic user-aware projection area optimization module 234 may calculate the weight of each pixel, which may be the weighted sum of all the overlapping users’ FOVs, normalized to 1. The following equation (second group) (8) may indicate the calculation of the weight of each pixel.

[0209]

[0210] Further, pixels having weights less than a certain threshold £ may be filtered out and the remaining pixels may be considered for final projection.

[0211] Further, the final pixels which may be persisting may be given a true tag of +1 and discarded pixels are given -1 tag, from the following equation (second group) (9):

[0212]

[0213] Further, the final projectable area or optimal FOV may be calculated. The final projectable area may be arbitrary and non-uniform shaped.

[0214] As illustrated in Figure 15C, the dynamic user-aware projection area optimization module 234 may find the maximum area rectangle that may completely fit within the optimal FOV.

[0215] In an embodiment, a simple heuristic-based approach may be used to find out possible rectangles of maximum area.

[0216] At first, corners of the optimal FOV region may be found. Further, for each corner, extend arms which form the corner infinitely. Further, intersection points of all such extended arms may be found. Further, areas of all possible rectangles may be calculated considering any 4 points among intersection points and corners at a time.

[0217] Further, rectangles whose width and height may not be parallel to the X and Y axis respectively may be discarded. Further, the rectangle of the maximum area as the final projection plane or combined FOV may be provided.

[0218] The robot pose adjustment and content projection module 236 may be configured to adjust the orientation and position of the robot pose and to align with the dynamically calculated projection area as mentioned earlier. Further, the robot poses adjustment and content projection module 236 may be configured to project the desired content on to the designated area in real time.

[0219] The robot poses adjustment and content projection module 236 may consider optimized projection area, content, i.e., image, video, etc., data to be projected, as input and may be configured to generate a robot time projection as output.

[0220] The robot poses adjustment and content projection module 236 may utilize the robot’s kinematics model to calculate the necessary pose required to achieve the desired projection area. Further, path planning algorithms may be used to generate a collision-free and efficient trajectory for a robot to reach a target position.

[0221] The robot poses adjustment and content projection module 236 may generate motor commands from trajectory information for motion control. Further, a control system may ensure that the robot / projector executes the planned trajectory smoothly, reaching the desired pose. The robot control system may ensure that the robot executes the planned trajectory reaching the desired pose.

[0222] Simultaneously, content, i.e., image, video etc., data may be fed into a projection system for content projection. The projected contents on dynamically calculated projection area help in user convenience and optimum content viewing with minimum discomfort to user.

[0223] Figure 16 illustrates a flowchart depicting a method 1600 for dynamic projection area selection in the multi-use environment, in accordance with an embodiment of the present disclosure.

[0224] At step 1602, the method 1600 may include receiving the sensor data indicative of at least one of orientation, location, and viewing preference of one or more users with respect to a projection area.

[0225] At step 1604, the method 1600 may assigning the unique ID to each of the one or more users based on received sensor data.

[0226] At step 1606, the method 1600 may generating the FOV for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user.

[0227] At step 1608, the method 1600 may determining the one or more parameters associated with the one or more users based on the received sensor data, the assigned unique ID, and the generated FOV. The one or more parameters are indicative of a state, an attention, and a role of the corresponding user.

[0228] At step 1610, the method 1600 may creating a real-time engagement profile for each of the one or more users by fusing the one or more parameters.

[0229] At step 1612, the method 1600 may calculating the target position and the size of content projection of the projection area based on the FOV and the user interest of each of the one or more users.

[0230] At step 1614, the method 1600 may adjusting the position and the orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0231] In an embodiment, a dynamic projection area determination method (1600) may include receiving (1602) sensor data indicative of at least one of orientation, location, and viewing preference of one or more users with respect to a projection area; assigning (1604) a unique identification (ID) to each of the one or more users based on received sensor data; generating (1606) a field of view (FOV) for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user; determining (1608) one or more parameters associated with the one or more users based on the received sensor data, the assigned unique ID, and the generated FOV, wherein the one or more parameters are indicative of a state, an attention, and a role of the corresponding user; creating (1610) a real-time engagement profile for each of the one or more users by fusing the one or more parameters, wherein creating comprises assigning weights to the real-time engagement profile to indicate user interest in viewing content on the projected area; calculating (1612) a target position and a size of content projection of the projection area based on the FOV and the user interest of each of the one or more users; and adjusting (1614) a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0232] The receiving the sensor data indicative of the at least one of orientation, location, and viewing preference of the one or more users with respect to the projection area further comprises includes capturing a three-dimensional (3D) point cloud representation of a multiuser environment; determining the at least one of orientation, the location, and the viewing preference of the one or more users with respect to the projection area based on the 3D point cloud representation; estimating a user gaze direction based on the at least one of orientation, the location, and the viewing preference of the one or more users with respect to the projection area, for user identification; and assigning initial priorities during content projection based on the user gaze direction estimated for each of the one or more users, wherein the one or more users interact through voice commands to provide feedback and update the viewing preferences in real-time.

[0233] The assigning the unique identification (ID) to each of the one or more users based on received sensor data further includes identifying a user of the one or more users from an input frame, wherein the input frame corresponds to an image of the user captured by an imaging device; segregating each user from the input frame based on an encoder-decoder- based image segmentation task; extracting one or more discriminating features of each user, wherein the one or more discriminating features correspond to physical attributes associated with the user; and assigning the unique ID based on the extracted one or more discriminating features.

[0234] The method (1600) includes aligning projection in a preferred direction in which the user is interested in watching content based on a user command comprising one of a voicebased command or a gesture-based command, wherein in case of the voice-based command: amplifying and de-noising an audio signal received from the user; converting the amplified and audio signal into text information; and feeding the text information to a large language model (LLM) framework to evaluate context and predict the preferred direction, wherein in case of the gesture-based command: segmenting out body semantics from an input frame, wherein the body semantics correspond to a hand movement of the user; identifying key featuring points from the segmented body semantics; and generating a vector by performing global coordinate mapping on the identified key featuring points.

[0235] The method (1600) includes identifying projection planes of the projection area where the content is projected, wherein the identified projection planes are stored in a database during mapping; and calculating an array of the projection planes by performing geometrical transformation to a global coordinate system in the preferred direction.

[0236] The generating the FOV for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user, further includes building a deep learning (DL) model by training on the sensor data indicative of images of the one or more users; extracting features from head region of each of the one or more users from the images of the one or more users; mapping the extracted features to head pose angles of each of the one or more users, wherein the head pose angles are indicative of roll, pitch, and yaw; and generating the FOV for each of the one or more users by calculating a probability distribution of each head pose angle with respect to an actual angle value.

[0237] The method (1600) includes transforming the head pose angles of each user into global coordinates to detect the projection plane; selecting the projection plane along the preferred direction based on the global coordinates; and transforming the position of the user into the global coordinate to determine an absolute distance between the projection plane and the user.

[0238] The method (1600) includes identifying coordinates of each of the one or more users in an input frame and calculating a centroid of each user in the input frame; determining a state of the user by creating a bounding box with variable dimensions based on identified coordinates and calculated centroid of the user, wherein the bounding box corresponds to objects of the variable dimensions associated with the user and present within the input frame; and assigning a movability score to each user of the one or more users based on the state of the user.

[0239] The method (1600) includes building a convolutional recurrent neural network (CRNN) model by training on a dataset of facial expressions of the one or more users from the input frame to identify attention levels of the one or more users, wherein the dataset of facial expressions is indicative of eye gaze, brow furrowing, and mouth movements; and

[0240] assigning relative weights by sorting the facial expressions based on the attention levels and calculating the relative priorities of the one or more users based on the relative weights.

[0241] The method (1600) includes identifying the one or more users from the input frame through an object detection module, and detecting different activities performed by the one or more users; and identifying the role of each user based on the detected activities of the one or more users.

[0242] The method (1600) includes generating a user engagement matrix based on the one or more parameters, wherein the one or more parameters correspond to a mobility score, an attention score, and relative role importance; calculating engagement weighing score for each user based on the user engagement matrix; and calculating relative user engagement scorescollectively to avoid overshooting or undershooting by normalizing user priorities on a linear scale.

[0243] The method (1600) includes dividing the projection area into an array of pixels based on the FOV and priority of each of the one or more users; assigning weight to each user based on the priority; calculating weight of each pixel of the array of pixels based on overlapping FOVs; determining a target position and size of content on the projection area; and calculating a projectable area with a portion on the projection area, wherein the projectable area indicates a combined FOV of the one or more users on the projection area.

[0244] The method (1600) includes generating a collision-free trajectory for the projecting device by calculating the position and orientation of the projecting device with respect to the projection area.

[0245] In an embodiment, a dynamic projection area determination system (200), comprising: a memory; a processor communicatively coupled with the memory and the processor may receive sensor data indicative of at least one of orientation, location, and viewing preference of one or more users with respect to a projection area; assign a unique identification (ID) to each of the one or more users based on received sensor data; generate a field of view (FOV) for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user; determine one or more parameters associated with the one or more users based on the received sensor data, the assigned unique ID, and the generated FOV, wherein the one or more parameters are indicative of a state, an attention, and a role of the corresponding user; create a real-time engagement profile for each of the one or more users by fusing the one or more parameters, wherein creating comprises assigning weights to the real-time engagement profile to indicate user interest in viewing content on the projected area; calculate a target position and a size of content projection of the projection area based on the FOV and the user interest of each of the one or more users; and adjust a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0246] The processor to, receive the sensor data indicative of the at least one of orientation, location, and viewing preference of the one or more users with respect to the projection area, may capture a three-dimensional (3D) point cloud representation of a multi-user environment; determine the at least one of orientation, the location, and the viewing preference of the one or more users with respect to the projection area based on the 3D point cloud representation; estimate a user gaze direction based on the at least one of orientation, the location, and the viewing preference of the one or more users with respect to the projection area, for user identification; and assign initial priorities during content projection based on the user gaze direction estimated for each of the one or more users, wherein the one or more users interact through voice commands to provide feedback and update the viewing preferences in real-time.

[0247] The processor to assign the unique identification (ID) to each of the one or more users based on received sensor data, may identify a user of the one or more users from an input frame, wherein the input frame corresponds to an image of the user captured by an imaging device; segregate each user from the input frame based on an encoder-decoder- based image segmentation task; extract one or more discriminating features of each user, wherein the one or more discriminating features correspond to physical attributes associated with the user; and assign the unique ID based on the extracted one or more discriminating features.

[0248] The processor may align projection in a preferred direction in which the user is interested in watching content based on a user command comprising one of a voice-based command or a gesture-based command, wherein in case of the voice-based command: amplify and de-noising an audio signal received from the user; convert the amplified and audio signal into text information; and feed the text information to a large language model (LLM)framework to evaluate context and predict the preferred direction, wherein in case of the gesture-based command: segment out body semantics from an input frame, wherein the body semantics correspond to a hand movement of the user; identify key featuring points from the segmented body semantics; and generate a vector by performing global coordinate mapping on the identified key featuring points.

[0249] The processor may identify projection planes of the projection area where the content is projected, wherein the identified projection planes are stored in a database during mapping; and calculate an array of the projection planes by performing geometrical transformation to a global coordinate system in the preferred direction.

[0250] The processor, to generate the FOV for each of the one or more users based on the received at least one of orientation, location, and viewing preference to provide an optimal viewing angle for each user, may build a deep learning (DL) model by training on the sensor data indicative of images of the one or more users; extract features from head region of each of the one or more users from the images of the one or more users; map the extracted features to head pose angles of each of the one or more users, wherein the head pose angles are indicative of roll, pitch, and yaw; and generate the FOV for each of the one or more users by calculating a probability distribution of each head pose angle with respect to an actual angle value.

[0251] The processor may transform the head pose angles of each user into global coordinates to detect the projection plane; select the projection plane along the preferred direction based on the global coordinates; and transform the position of the user into the global coordinate to determine an absolute distance between the projection plane and the user.

[0252] The processor may identify coordinates of each of the one or more users in an input frame and calculate a centroid of each user in the input frame; determine a state of the user by creating a bounding box with variable dimensions based on identified coordinates and calculated centroid of the user, wherein the bounding box corresponds to objects of the variable dimensions associated with the user and present within the input frame; and assign a movability score to each user of the one or more users based on the state of the user.

[0253] The processor may build a convolutional recurrent neural network (CRNN) model by training on a dataset of facial expressions of the one or more users from the input frame to identify attention levels of the one or more users, wherein the dataset of facial expressions is indicative of eye gaze, brow furrowing, and mouth movements; and assign relative weights by sorting the facial expressions based on the attention levels and calculating the relative priorities of the one or more users based on the relative weights.

[0254] The processor may identify the one or more users from the input frame through an object detection module, and detecting different activities performed by the one or more users; and identify the role of each user based on the detected activities of the one or more users.

[0255] The processor may generate a user engagement matrix based on the one or more parameters, wherein the one or more parameters correspond to a mobility score, an attention score, and relative role importance; calculate engagement weighing score for each user based on the user engagement matrix; and calculate relative user engagement scores collectively to avoid overshooting or undershooting by normalizing user priorities on a linear scale.

[0256] The processor may divide the projection area into an array of pixels based on the FOV and priority of each of the one or more users; assign weight to each user based on the priority; calculate weight of each pixel of the array of pixels based on overlapping FOVs; determine a target position and size of content on the projection area; and calculate a projectable area with a portion on the projection area, wherein the projectable area indicates a combined FOV of the one or more users on the projection area.

[0257] The processor may generate a collision-free trajectory for the projecting device by calculating the position and orientation of the projecting device with respect to the projection area.

[0258] Figures 17A-17D illustrate different exemplary scenarios of the dynamic projection area selection in the multi-use environment, in accordance with an embodiment of the present disclosure.

[0259] As illustrated in Figure 17A, in a conventional example 1702, a family, sitting at different places in the hall to enjoy a movie. Different members are sitting at different places in the room, causing each of them to have different FOVs. Now the projector projects on the wall at a fixed place, which can be inconvenient to some of the members. Members undergoing inconvenience, change their position to align their FOV to the projection plane.

[0260] In an exemplary embodiment 1704 of the present disclosure, the projector may find out the best projection plane considering FOVs and user importance such that users don’t have to change their position to enjoy the show. For the home environment sitting arrangement changes frequently. So this patent has a strong use case for a home environment.

[0261] As illustrated in Figure 17B, in a conventional example 1704, during exhibitions like CES, presenter face challenges with fixed projector setups that require manual adjustment to show their content on designated areas of the wall or screen. Due to this, if audience attention is toward the presenter, the content remains static and leads to disengagement. The audience might struggle to watch content if doesn’t adapt to the optimal line of sight. Presenters need to manually adjust their position and projected content. Moreover, multiple showcases require multiple projectors / di splays to show content, complicating the setup and limiting flexibility. If the presenter wants to showcase another product at a different location, another projector or display is needed.

[0262] In an exemplary embodiment 1706 of the present disclosure, the robotic projector moves to optimal position based on where the presenter is and where the audience is focusing to ensure the content is visible and aligned to the presenter location due to presenter preference and role priority. If the presenter needs to showcase a different product it just needs to move that location and ask the robot to come here and project. This dynamic adjustment keeps the audience engaged.

[0263] As illustrated in Figure 17C, in a conventional example 1708, in the interactive classroom, the teacher uses a traditional projector to display content on the screen. But for the young student, the excitement of learning is dampened by a challenge. The screen is just out of reach, making it difficult to point and answer questions about the planets. “Ugh, this frustrates me,” the child thinks.

[0264] In an exemplary embodiment 1710 of the present disclosure, recognizing the student's struggle, a smart projector equipped with “context-aware priority driven projection area selection in the multi-user environment” based on priority-based dynamic FOV calculation, adjusts its height, bringing the planets within easy reach, making it easier for a kid to reply and participate in the classroom.

[0265] As illustrated in Figure 17D, in a conventional example 1712, at a community center various events, workshops, and meetings are held regularly. The attendees include people with disabilities and older individuals who may have mobility issues or visual and hearing impairments. The content is displayed on a designated wall area without adjustment for audience needs or movements. Older attendees might find it hard to move different parts of the room to see the content better. Attendees on the left side struggle to see the content on the right side.

[0266] In an exemplary embodiment, as depicted by 1714, of the present disclosure, recognizing the older couple, attendees with mobility issues struggle to see the content clearly,as the projection content is static and cannot adapt to their needs. After identifying the optimal projection area based on real-time analysis of audience FOV, user movability, and importance, the robotic projector moves and shifts the projection. The system ensures that all participants regardless of their age or physical abilities have an optimal experience.

[0267] Figures 18 illustrate an embodiment of the electronic apparatus.

[0268] In an embodiment, an electronic apparatus includes at least one processor including processing circuitry, and memory storing instructions, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic apparatus to obtain sensing data related with one or more users with respect to a projection area, identify a field of view (FOV) for each of the one or more users based on the sensing data, obtain a target position and a size of content projection of the projection area based on the FOV, and obtain a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0269] The sensing data may refer to user-related environmental and behavioral information captured by one or more sensors around a projection area. The sensing data may be describe as sensing information, user data or user behavior information.

[0270] The FOV may refer to a spatial region visible to a user from the user’s position. The FOV may be described as a viewing region, a visual coverage area, a user-visible area, a gaze- visible region, or a viewing scope.

[0271] The target position may refer to a designated location within the proj ection area where content is intended to be displayed, determined according to the user’s FOV.

[0272] The target size may refer to a determined display dimension or scale of projected content within the projection area, determined according to the user’s FOV.

[0273] The position of the projecting device may refer to a spatial location of the projecting device. The position of the projecting device may be determined to place the projected image at the target position on the projection area. The position of the projecting device may be described as projecting position or projector position.

[0274] The orientation of the projecting device may refer to an angular alignment or pointing direction of the projecting device. The orientation of the projecting device may be adjusted to project content toward the projection area.

[0275] The projecting device may be described as projection unit or projector.

[0276] The at least one processor may output an image through the projecting device based on the position and the orientation of the projecting device.

[0277] The sensing data may include at least one of orientation information, location information, and viewing preference information corresponding to each of the one or more users with respect to the projection area.

[0278] The at least one processor may obtain a unique identification (ID) corresponding to each of the one or more users based on sensing data, identify at least one parameter associated with the one or more users based on the sensing data, the assigned unique ID, and the FOV, and obtain the target position and the size of content projection of the projection area based on the FOV and the at least one parameter.

[0279] The ID may be described as ID information, user ID information or user key data.

[0280] The at least one parameter may include at least one of state information, attention information or role information corresponding to each of the one or more users.

[0281] A parameter may refer to a user-related descriptive value derived from sensing data, representing conditions or behavioral attributes of each user. A parameter may refer to any user-associated indicator derived from sensing input, representing characteristics, conditions, or contextual factors relevant to determining how the projection area is controlled or adjusted.

[0282] The parameter may be described as a variable, an attribute, an indicator, a factor, or a characteristic.

[0283] The at least one processor may obtain a real-time engagement profile for each of the one or more users based on the at least one parameter, and obtain the target position and the size of content proj ection of the proj ection area based on the FO V and the real-time engagement profile.

[0284] The real-time engagement profile may refer to a user-related profile generated in real time. The real-time engagement profile may refer to a dynamically generated representation of a user’s current interest level, interaction tendency, or viewing involvement.

[0285] The real-time engagement profile may be described as a real-time user profile, a dynamic user profile, a real-time user metric, a dynamic user indicator, or a real-time interaction profile.

[0286] The at least one processor may obtain the real-time engagement profile by assigning weights to indicate user interest in viewing content on the projected area. The at least one processor may obtain the real-time engagement profile by assigning weighted values to user- related indicators to reflect the user’s level of interest in viewing content on the projected area.

[0287] The at least one processor may obtain the sensing data including a three-dimensional (3D) point cloud representation of a multi-user environment, estimating a user gaze direction based on the sensing data, assigning initial priorities during content projection based on the user gaze direction estimated for each of the one or more users, and identify the at least one parameter based on the initial priorities.

[0288] The at least one processor may identify a user of the one or more users from an input frame, wherein the input frame corresponds to an image of the user captured by an imaging device; segregate each user from the input frame based on an encoder-decoder-based image segmentation task; extract one or more discriminating features of each user, wherein the one or more discriminating features correspond to physical attributes associated with the user; and obtain the unique ID based on the extracted one or more discriminating features.

[0289] The at least one processor may align projection in a preferred direction in which the user is interested in watching content based on a user command comprising one of a voicebased command or a gesture-based command, wherein in case of the voice-based command amplifying and de-noising an audio signal received from the user; converting the amplified audio signal into text information; and feeding the text information to a large language model (LLM) framework to evaluate context and predict the preferred direction, wherein in case of the gesture-based command segmenting out body semantics from an input frame, wherein the body semantics correspond to a hand movement of the user; identifying key featuring points from the segmented body semantics; and generating a vector by performing global coordinate mapping on the identified key featuring points.

[0290] In an embodiment, a method of controlling an electronic apparatus, the method comprising: obtaining (SI 810) sensing data related with one or more users with respect to a projection area, identifying (SI 820) a field of view (FOV) for each of the one or more users based on the sensing data, obtaining (S1830) a target position and a size of content projection of the projection area based on the FOV, and obtaining (SI 840) a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

[0291] The method may include outputting an image through the projecting device based on the position and the orientation of the projecting device.

[0292] The sensing data may include at least one of orientation information, location information, and viewing preference information corresponding to each of the one or more users with respect to the projection area.

[0293] The method may include obtaining a unique identification (ID) corresponding to each of the one or more users based on sensing data, identifying at least one parameter associated with the one or more users based on the sensing data, the assigned unique ID, and the FOV, and obtaining the target position and the size of content projection of the projection area based on the FOV and the at least one parameter.

[0294] The at least one parameter may include at least one of state information, attention information or role information corresponding to each of the one or more users.

[0295] The present disclosure offers a system that may be implemented in any of the smart devices that may have displays like TV, Mobile, Family Hub, etc. The system may be useful for all age users and all types of devices and content. The present disclosure offers a platform as a service for official ecom-partners. The present disclosure offers a unique system to enhance the user experience by intelligently giving context-aware priority-driven projection area selection in a multi-user environment in diverse environments on the fly. The present disclosure considers a personalized field-of-view, an adaptive user behavior, and prioritizes user roles and preferences to enhance the overall viewing experience.

[0296] Although specific units / modules have been illustrated in the figure and described above, it should be understood that the system may include other hardware modules or software modules or combinations as may be required for performing various functions.

[0297] The various embodiments described above are provided by way of illustration only and should not be construed to limit the scope of the disclosure. Various modifications and changes may be made to the principles described herein without following the example embodiments and applications illustrated and described herein, and without departing from the spirit and scope of the disclosure.

[0298] Those skilled in the art will appreciate that the operations described herein in the present disclosure may be carried out in other specific ways than those set forth herein without departing from essential characteristics of the present invention. The above- described embodiments are therefore to be construed in all aspects as illustrative and not restrictive. The scope of the invention should be determined by the appended claims, not by the above description, and all changes coming within the meaning of the appended claims are intended to be embraced therein.

[0299] The drawings and the forgoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, orders of processes described herein may be changed and are not limited to the manner described herein.

[0300] Moreover, the actions of any flow diagram need not be implemented in the order shown; nor do all of the acts necessarily need to be performed. Also, those acts that are not dependent on other acts may be performed in parallel with the other acts. The scope of embodiments is by no means limited by these specific examples. Numerous variations, whether explicitly given in the specification or not, such as differences in structure, dimension, and use of material, are possible. The scope of embodiments is at least as broad as given by the following claims.

[0301] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any component(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or component of any or all the claims.

Claims

Claims

1. An electronic apparatus comprising: at least one processor including processing circuitry, and memory storing instructions, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic apparatus to: obtain sensing data related with one or more users with respect to a projection area, identify a field of view (FOV) for each of the one or more users based on the sensing data, obtain a target position and a size of content projection of the projection area based on the FOV, and obtain a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

2. The electronic apparatus as claimed in claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: output an image through the projecting device based on the position and the orientation of the projecting device.

3. The electronic apparatus as claimed in claim 1, wherein the sensing data includes at least one of orientation information, location information, and viewing preference information corresponding to each of the one or more users with respect to the projection area.

4. The electronic apparatus as claimed in claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: obtain a unique identification (ID) corresponding to each of the one or more users based on sensing data, identify at least one parameter associated with the one or more users based on the sensing data, the assigned unique ID, and the FOV, and obtain the target position and the size of content projection of the projection area based on the FOV and the at least one parameter.

5. The electronic apparatus as claimed in claim 4, wherein the at least one parameter includes at least one of state information, attention information or role information corresponding to each of the one or more users.

6. The electronic apparatus as claimed in claim 4, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: obtain a real-time engagement profile for each of the one or more users based on the at least one parameter,obtain the target position and the size of content projection of the projection area based on the FOV and the real-time engagement profile.

7. The electronic apparatus as claimed in claim 4, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: obtain the real-time engagement profile by assigning weights to indicate user interest in viewing content on the projected area.

8. The electronic apparatus as claimed in claim 4, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: obtain the sensing data including a three-dimensional (3D) point cloud representation of a multi-user environment, estimating a user gaze direction based on the sensing data, assigning initial priorities during content projection based on the user gaze direction estimated for each of the one or more users, and identify the at least one parameter based on the initial priorities.

9. The electronic apparatus as claimed in claim 4, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: identify a user of the one or more users from an input frame, wherein the input frame corresponds to an image of the user captured by an imaging device; segregate each user from the input frame based on an encoder-decoder-based image segmentation task; extract one or more discriminating features of each user, wherein the one or more discriminating features correspond to physical attributes associated with the user; and obtain the unique ID based on the extracted one or more discriminating features.

10. The electronic apparatus as claimed in claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: align projection in a preferred direction in which the user is interested in watching content based on a user command comprising one of a voice-based command or a gesturebased command, wherein in case of the voice-based command: amplifying and de-noising an audio signal received from the user; converting the amplified audio signal into text information; and feeding the text information to a large language model (LLM) framework to evaluate context and predict the preferred direction, wherein in case of the gesture-based command: segmenting out body semantics from an input frame, wherein the body semantics correspond to a hand movement of the user; identifying key featuring points from the segmented body semantics; and generating a vector by performing global coordinate mapping on the identified key featuring points.

11. A method of controlling an electronic apparatus, the method comprising: at least one processor including processing circuitry, and memory storing instructions, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic apparatus to: obtain sensing data related with one or more users with respect to a projection area, identify a field of view (FOV) for each of the one or more users based on the sensing data, obtain a target position and a size of content projection of the projection area based on the FOV, and obtain a position and an orientation of a projecting device to align the projection area with the target position and the size of the content projection.

12. The method as claimed in claim 11, further comprising: output an image through the projecting device based on the position and the orientation of the projecting device.

13. The method as claimed in claim 11, wherein the sensing data includes at least one of orientation information, location information, and viewing preference information corresponding to each of the one or more users with respect to the projection area.

14. The method as claimed in claim 11, further comprising: obtain a unique identification (ID) corresponding to each of the one or more users based on sensing data, identify at least one parameter associated with the one or more users based on the sensing data, the assigned unique ID, and the FOV, and obtain the target position and the size of content projection of the projection area based on the FOV and the at least one parameter.

15. The method as claimed in claim 14, wherein the at least one parameter includes at least one of state information, attention information or role information corresponding to each of the one or more users.

Images