Sign language cabin notification system

Abstract

Various embodiments of the present disclosure are directed to a computer server. The computer server includes at least one network interface, at least one processor connected to communicate through the at least one network interface with video display units (VDUs) and / or personal electronic devices (PEDs), and at least one memory. The one memory includes a sign language conversion module operative to translate digitized speech to a script of signs to be performed by a computer generated avatar. The one memory also includes a sign language animation module operative to generate video of an avatar performing signs forming a sign language responsive to the script of signs. The one memory also includes a distribution module operative to communicate the video of the avatar through the at least one network interface to VDUs and / or PEDs.

Description

SIGN LANGUAGE CABIN NOTIFICATION SYSTEMRELATED APPLICATION

[0001] This application claims priority under 35 U.S.C. 119(e) to U.S. Provisional Patent Application Serial No. 63 / 538,601 , filed September 15, 2023, the content of which is incorporated by reference herein in its entirety for all purposes.TECHNICAL FIELD

[0002] The present disclosure relates to providing notifications through aircraft and other vehicle environments.BACKGROUND

[0003] Passenger announcements within aircraft cabins are primarily audiobased, making them inaccessible to individuals with hearing impairments, particularly deaf passengers. Existing solutions, such as visual cabin signs, video announcements with closed captions, and text-based crew-to-passenger messages, fall short in effectively conveying information, especially for complex or lengthy announcements. Deaf passengers may not be proficient in written languages, and crew members often lack sign language skills, exacerbating the communication gap. As used herein, the terms "hearing impaired" and "deaf" are used interchangeably to refer to any person who is proficient at understanding one or more sign languages.SUMMARY

[0004] Various embodiments of the present disclosure are directed to a computer server. The computer server includes at least one network interface, at least one processor connected to communicate through the at least one network interface with video display units (VDUs) and / or personal electronic devices (PEDs), and at least one memory. The one memory includes a sign language conversion module operative to translate digitized speech to a script of signs to be performed by a computer generated avatar. The one memory also includes a sign language animation module operative to generate video of an avatar performing signs forming a sign language responsive to the script of signs; The one memory also includes adistribution module operative to communicate the video of the avatar through the at least one network interface to VDUs and / or PEDs.

[0005] Other aircraft communication terminals according to embodiments of the present disclosure will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. Moreover, it is intended that all embodiments disclosed herein can be implemented separately or combined in any way and / or combination.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiment(s) of the invention. In the drawings:

[0007] Figure 1 illustrates a simplified system block diagram of an In-Flight (IFE) system that provides IFE content to IFE displays and personal electronic devices and processes IFE content for delivery to components of an IFE system according to some embodiments;

[0008] Figure 2 illustrates a illustrative example of a picture-in-picture embodiments showing a video being played in a larger main box and a real-time digital signing avatars in a smaller box in the corner displaying a digital signing avatar signing the speech of the video being played according to some embodiments;

[0009] Figure 3 illustrates a simplified system block diagram of an IFE system that configured to provide computer-generated avatar signing according to some embodiments;

[0010] Figure 4 illustrates a simplified system block diagram of an IFE Server that processes text and speech inputs for translation to sign language animation according to some embodiments.DETAILED DESCRIPTION

[0011] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of aspects of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods,procedures, components and circuits have not been described in detail so as not to obscure the present invention.

[0012] Although various embodiments are explained herein in the context of generating and using a digital signing avatar with an In-Flight Entertainment (IFE) environment of an aircraft for display to passengers, other embodiments are not limited thereto and may be used with other types of vehicles, including, without limitation, ships (e.g., cruise ships), buses, and trains, and non-vehicle environments (e.g., broadcast television, streaming video services, audio announcement systems, etc.). Accordingly, embodiments are not limited for use with passengers but may instead be used for any persons who have hearing impairment or otherwise prefer to view sign language based announcements and notifications. The terms announcements and notifications are used interchangeably and can have the same meaning herein.

[0013] Some embodiments of the present disclosure are directed to addressing a longstanding issue related to providing passenger announcements within an aircraft cabin and how to make such announcements accessible and acceptable to deaf passengers. The technical challenge arises from the lack of effective communication and situational awareness for deaf passengers during flights due to the primarily audio-based nature of in-flight announcements.

[0014] Currently, the majority of passenger announcements in aircraft cabins are conveyed through audio messages played via the aircraft's loudspeakers and headphone interfaces. These announcements fall into various categories, including Passenger Announcements (PA), Pre-Recorded Announcements (PRAM), and Video Announcements (typically for safety videos). Deaf passengers, however, receive limited information from these announcements, typically relying on visual cues such as cabin signs or video announcements with closed captions in one or more written languages, if available. Crew-to-passenger messages can also be displayed on the in-flight entertainment (IFE) screens, but these are typically textbased again in one or more written languages, making them less effective for deaf passengers, especially those who may not be proficient in the corresponding written language.

[0015] The existing solutions to this problem are inadequate for several reasons. Deaf passengers often struggle to fully comprehend long or complex announcements through limited visual cues, and text-based crew-to-passengermessages may not be easily understood by passengers with limited proficiency in the displayed written language(s). Additionally, crew members typically do not possess sign language communication skills, further limiting effective communication with deaf passengers.

[0016] In light of these challenges, various embodiments are directed to enhancing communication and situational awareness for deaf passengers by generating a computer-generated avatar which is displayed on IFE seat displays, other cabin screens, and / or personal electronic devices (PEDs). A computergenerated avatar can provide real-time signing in a passenger-selectable or designated sign language for predefined or real-time announcements, as well as for various in-cabin messaging and entertainment purposes.

[0017] In summary, the technical domain of some embodiments of the present disclosure pertains to aviation technology and is directed to addressing the issue of inadequate communications of announcements to deaf passengers and other hearing impaired passengers. Existing approaches for communicating announcements, some of which provide limited visual cues and text-based messages, are inadequate for providing effective communication of announcements and related situational awareness to deaf passengers. This results in not only uninformed deaf passengers, but can also cause anxiety and stress due to lack of understanding of cabin events for the deaf and other hearing impaired passengers. There is therefore a need for improvements, which can be provided through various digital avatar-based solutions in accordance with embodiments disclosed herein.

[0018] Various embodiments of the present disclosure are directed to a computer server. The computer server includes at least one network interface, at least one processor connected to communicate through the at least one network interface with video display units (VDUs) and / or personal electronic devices (PEDs), and at least one memory. The one memory includes a sign language conversion module operative to translate digitized speech to a script of signs to be performed by a computer generated avatar. The one memory also includes a sign language animation module operative to generate video of an avatar performing signs forming a sign language responsive to the script of signs; The one memory also includes a distribution module operative to communicate the video of the avatar through the at least one network interface to VDUs and / or PEDs.

[0019] Signs in sign language are structured, systematic, and part of a linguistic culture, whereas gestures are informal, variable, and often used to complement spoken language. The distinction between the two helps with understanding the unique nature of sign languages and the informal nonverbal communication used by hearing individuals. However, in various embodiments of the present disclosure, the term “signs” embodies both signs in sign language and gestures.

[0020] Various embodiments of the present disclosure provide operations that address the critical challenge of communication and situational awareness for deaf passengers during various modes of transportation, primarily focusing on aircraft cabins. Embodiments can provide the ability to display a digital signing avatar in quasi-real time on In-Flight Entertainment (IFE) displays, PEDs, etc., serving as a comprehensive and effective means of resolving the existing problem.

[0021] By utilizing IFE displays and / or PEDs, these embodiments may provide a distinct advantage by providing a clear and unequivocal signed message that is readily understandable to deaf passengers. Unlike traditional text-based messages or limited visual cues, the digital signing avatar ensures that deaf passengers receive information in their preferred sign language, eliminating potential misunderstandings and reducing stress or frustration.

[0022] Furthermore, these embodiments align with accessibility requirements and regulations such as the Americans with Disabilities Act (ADA) in the USA, Directive 2019 / 882 in the European Union, and the World Federation of the Deaf 2020-2030 Strategic Direction. They contribute to the promotion of the legal recognition of national sign languages, thus fostering inclusivity and equal access for all passengers.

[0023] In addition to its primary benefit of enhancing communication for deaf passengers, these embodiments offer several other advantages discussed below.

[0024] First, digital signing avatar supports multiple sign languages. There exist many different sign languages, such as International Sign Language (ISL) , American Sign Language (ASL), British Sign Language (BSL), French Sign Language (LSF), Chinese Sign Language (CSL), and others. The digital signing avatar is designed to accommodate multiple sign languages, making it a versatile solution that aligns with airlines' and original equipment manufacturers' (OEMs) accessibility strategies. This goes beyond the limitations of legacy technologies like "closed captioning."

[0025] Second, a digital signing avatar is more cost-efficient than hiring a human translator for pre-determined notifications and videos. These embodiments are cost- effective as they eliminate the need for recording video messages with signing professionals for each announcement type. Instead, some embodiments use Machine Learning and Al technologies, enabling the quick generation and deployment of the digital signing avatar to meet airlines' evolving needs, including accommodating new information such as new notification types, new sign languages, new use-cases and variations associated to routes, seasons, health and safety topics.

[0026] Third, a digital signing avatar provides customizability. Using capabilities of computer technologies for speech and text translation, animation, etc., the digital signing avatar is highly customizable. Airlines can tailor the avatar's appearance, including factors like gender, demographics, styling, dress code, and color palette. This flexibility not only promotes inclusiveness but also aligns with airlines' branding strategies.

[0027] Fourth, a digital signing avatar may provide real-time translations. Another disadvantage of utilizing a human translator for pre-recording videos of the signs is that the IFE system is then limited to only the signs that were pre-recorded. A digital signing avatar may be used to translate text and speech in real-time to provide sign language to passengers in more situations than the predetermined situations with pre-recorded messages.

[0028] Fifth, a digital signing avatar may provide emotion, tones, and context for more accurate communication. Speech translated to text is inherently less accurate as it fails to provide important context, tones, and emotion in communicated in speech. Hand movements, facial expressions, and body postures of the digital signing avatar further define emotions, tones, and context of the communication to provide more accurate hand movements, facial expressions, and body postures for the animated avatar. The integration of avatar body movements and / avatar facial expressions into the animated signing conveys emotion, tone, and context effectively and accurately.

[0029] Various embodiments are applicable in vehicles that transport deaf passengers and rely predominantly or solely on audio messages for notifications and alerts. As explained above, these embodiments are applicable beyond the aviation domain, including other modes of transportation such as trains, tramways, and busesand non-vehicle environments (e.g., broadcast television, streaming video services, etc.). Furthermore, the technology can be adapted for personal electronic devices (PEDs) carried by deaf passengers, enabling real-time translation of announcements. These embodiments, therefore, have a broad potential domain, encompassing both vehicular and personal electronic applications.

[0030] The core elements of various embodiments include:

[0031] - Aircraft or Vehicle / Announcement System: This is the system responsible for initiating and coordinating announcements within the vehicle.

[0032] - Message Receiver, Interpreter, and Sender: This component interprets the announcements and triggers corresponding actions.

[0033] - Any Type of Monitor: This includes in-cabin monitors, IFE displays (e.g., seatback displays, seat armrest displays, bulkhead displays, etc.), and PEDs.

[0034] - Sign Language Content Database (Pre-defined): This comprises short video clips or files in multiple signed languages.

[0035] - Real-time Sign Language Translation System and Digital AvatarGenerator: This system generates real-time sign language translations and digital avatars for announcements.

[0036] Any In-Flight Entertainment System may trigger notification and / or announcements to passengers by interfacing with the aircraft notification system (using various messaging types and protocols) or via its crew control panel. When a notification or announcement is triggered by the aircraft notification system or the crew control panel, the IFE system interprets the action to be taken and broadcasts the corresponding audio, sometimes displaying an "Announcement In Progress" message. In the context of the present innovation, those audio messages would be shown as animated signing videos on IFE displays and PEDs for deaf passengers.

[0037] For example, the aircraft notification system and the crew control panel may be used to notify passengers of events and crew initiated messages such as: aircraft doors closing; aircraft doors opening; initiation of taxiing and takeoff procedure; initiation of approach and landing procedure; initiation of taxiing and takeoff procedure; initiation of approach and landing procedure; activation of a fasten seatbelt sign; deactivation of the fasten seatbelt sign; and / or depressurization of the aircraft cabin.

[0038] Other information which would be useful for speech / text to sign language conversion would be video IFE content. For example, video IFE content may includebroadcast television, streaming video services, and video content stored locally on the aircraft.

[0039] Other IFE content may be fed to the IFE system and displayable on the video display units and / or PEDs as text and may be converted to sign language with a digital signing avatar. For example, this text based IFE content may include news, literature, and live weather feeds. Contextual information for any type of application may also be converted to sign language with a digital signing avatar, such as apps related to maps, shopping, and meals and beverage.

[0040] In some embodiments, as discussed later in the context of the example illustration in Figure 2, a video display unit or PED may display a large box displaying IFE content with a smaller box overlaying the IFE content in a corner or a determined location displaying a digital signing avatar signing the speech of the IFE being displayed.

[0041] In another embodiment, the video display unit or PED may display the large box displaying IFE content with digital signing avatar signing the speech of the IFE being displayed overlaying the IFE content in a corner or a determined location without the smaller box blocking more of the IFE content.

[0042] The determined location might be based on defining a location in which the digital signing avatar would cause least occlusion of the IFE content while maximizing the efficiency and understandability of the signed message being delivered alongside the video IFE content.

[0043] The IFE system can include a database mapping announcement types with corresponding sign language animation signs or completed sign language videos, and other related information.

[0044] Identification of deaf passengers can be performed through passenger opt-in settings, crew selection, regional location of the passenger, an origin location of the passenger, or an airline database of opted-in passengers.

[0045] The opt-in feature to receive sign language announcements and other informative content may be made by the passenger through the IFE directly or through the passenger's PED device (e.g., after pairing with the IFE while onboard and / or through a ground-based server before boarding).

[0046] Various crew selections may be performed using a crew control panel.

[0047] Passenger selection to opt-in for sign language announcements and other informative content may be automated. An airline database containing a listing ofdeaf passengers who have opted-in may be used. Alternatively, a passenger manifest may be used to identify hearing-impaired, preferred sign language, and seating location. Selection of preferred sign language may also be based on a departure city, arrival city, and / or country of citizenship of the passenger.

[0048] Accordingly, operations to display sign language announcements may be configured to occur automatically for a particular passenger (e.g., when the passenger is declared as a deaf passenger in the airline database or airline application on a PED) or could be selectively enabled / disabled as desired by the passenger. In one embodiment, when the sign language announcement is performed, a button or other passenger selectable indicia can be displayed on a user interface (e.g., seat video display unit) that allows the passenger to enable or disable the sign language translation, and which enable the passenger to define a preferred sign language among a defined set of sign languages which the system is configured to operationally perform. In a further embodiment, the system may simultaneously display a plurality of different sign language translations of an announcement, e.g., in an array of sign language windows displayed on the passenger display device. The system may default to simultaneously displaying more than one sign language translation of an announcement (e.g., when a passenger preference is not known) or may perform such simultaneous display responsive to a related passenger defined preference or request. Moreover, sign language translation of an announcement may be performed on both a seat video display unit facing the passenger (e.g., seat back display) and a PED associated with the passenger.

[0049] In some embodiments, the sign language conversion module is enabled to generate of video of an avatar performing signs forming a sign language for distribution and playout to a user responsive to a request received from the user to receive sign language communications and / or responsive to information indicating the user has a hearing impairment.

[0050] In some of these embodiments, the sign language conversion module is further operative to access a database of sign languages to convert input text and / or digitized speech to a scripts of signs, and to select a first sign language for use in translation to the script of signs based on a preferred language indicated by a first user, and to select a second sign language for use in translation to the script of signs based on a preferred language indicated by a second user. The sign language animation module operative to generate a first video of an avatar performing signsforming the first sign language and to simultaneously generate a second video of an avatar performing signs forming the second sign language. The distribution module is further operative to stream the first video of the avatar through a first communication session established between the computer server and a first VDU or PED operated by a first user, and to stream the second video of the avatar through a second communication session established between the computer server and a second VDU or PED operated by a second user.

[0051] In some embodiments, the sign language animation module is further operative to generate a plurality of videos of avatars performing signs forming a plurality of different sign language. The distribution module is further operative to communicate the plurality of videos in parallel through the at least one network interface to a same VDU or PED for simultaneous display.

[0052] In some embodiments, the distribution module is further operative to stream the video of the avatar through a communication session established between the computer server and an individual VDU or PED operated by a user having a hearing impairment.

[0053] As discussed in further detail later in the context of Figure 4, technology for real-time sign language translation with corresponding digital avatar generation may leverage advancements in Al, machine learning, and 3D graphics animation. Various embodiments cover both static database mapping of announcements to sign language content files (e.g., where a database contains a set of announcements tied to corresponding sign language content files) and real-time creation of sign language videos (e.g., where a spoken announcement or textual announcement is translated and used to render a computer-generated avatar that performs corresponding sign language signing).

[0054] Other embodiments extend the use of digital signing avatars to any incabin entertainment, providing real-time sign language translations for various forms of entertainment.

[0055] Figure 1 illustrates a simplified system block diagram of an IFE system that provides IFE content to IFE displays and personal electronic devices and processes IFE content for delivery to components of an IFE system according to some embodiments.

[0056] Referring to Figure 1 , the system includes IFE displays (e.g., displays, seat armrest displays, etc.), in-cabin monitors, and crew control panels thatcommunicate through wired networks (e.g., ethernet) with an IFE server. Personal electronic devices (PEDs) communicate through wireless access points with the IFE server. PEDs can include, but are not limited to, smart phones, tablet computers, laptop computers, gaming computers, etc. The IFE server may be connected to a satellite communications (SatCom) terminal that is operative to communicate through satellites with ground-based servers, e.g., content servers, operations servers, etc.

[0057] The IFE server can be connected to sense defined events, such as cabin doors open / closed indications, takeoff / landing based on weight-on-wheels indications, turbulence indications, etc. An aircraft interface converter may be included to provide analog to digital conversion of signaling to be provided to the IFE server. Notifications (also "messages") can be triggered by such events, including pre-takeoff passenger briefings, return to seats and seatbelt indications, etc. Notifications can also be triggered by pilot and crew selections made through the cockpit and crew control panel. Pilot and crew notifications can be received as an audio stream from microphones in the cockpit, crew control panel, etc.

[0058] The term "notification" herein refers to any type of communication that may include spoken instructions played through speakers, written instructions displayed on the IFE displays, in-cabin monitor, personal electronic devices, etc. Moreover, notifications can include an audio track for media (e.g., movie, TV show, music, etc.). However, in accordance with various embodiments herein, notifications can be provided in the form of computer-generated avatar signing which is performed to convey the meaning of the spoken and / or written notifications.

[0059] Figure 2 illustrates an illustrative example of a picture-in-picture embodiment showing a video being played on a video display unit and / or PED in a larger main box and a real-time digital signing avatars in a smaller box overlaying the video in the corner displaying a digital signing avatar signing the speech of the video being played. However, the sign language avatar in the small box may also be displayed in situations other than when a video is playing such as an in flight announcement, a cabin notification change like a fasten seatbelt light being activated, or if text or speech is being displayed on the video display unit and / or PED.

[0060] Figure 3 illustrates a simplified block diagram of an IFE system 300 that is configured to provide computer-generated avatar signing according to some embodiments.

[0061] Referring to Figure 3, the IFE system 300 includes video display units (VDUs) 320 and PEDs 322 that communicate through communication distribution components 110 with an IFE server 350 and communication terminal 330. Some VDUs communicate through wired networks (e.g., ethernet) via seat electronics boxes 312, and other VDUs and PEDs 322 communicate through wireless router(s) 314, such as WiFi access point and / or cellular access point (e.g., a pico-cell radio base station) of distribution components 310. The VDUs can include, without limitation, seat displays which are mounted to individual seats, e.g., seatbacks or armrests. Example PEDs 322 transportable onto the aircraft can include, without limitation, smart phones, tablet computers, laptop computers, gaming computers, etc.

[0062] PEDs can be communicatively connected to the IFE server 350, for example, using WiFi communications routed through the wireless router 314 and / or using Bluetooth communications routed through Bluetooth pairings between PEDs and VDUs 320. The IFE server 350 can determine which PEDs are associated with a deaf passenger, e.g., by a passenger’s request or by accessing known passenger information related to being deaf (hearing impaired), and stream video of the generated signing avatar to those PEDs.

[0063] The IFE server 350 can provide broadcast and / or on-demand distribution of passenger-requested content to passengers. The IFE server may receive announcements from a crew control panel 360, cockpit equipment, and / or an aircraft notification interface 362 (e.g., connected to the aircraft interface converter) through network 340. The communication terminal 330 can communicate through satellite networks with ground-based radio access network(s) 372 (e.g., ground relays) and may through direct-air-to-ground communications (DA2GC) communicate with cellular 3GPP NR 5G base stations (e.g., eNodeBs, gNodeBs, etc.), etc. In some optional embodiments, the IFE system 300 communicates through the wireless networks with an IFE system operations platform 305 which controls access to content from an IFE content provider platform 304 and / or other networked content servers such as Internet content servers 374.

[0064] Figure 4 illustrates a simplified system block diagram of the IFE Server 350 of Figure 3 that is configured to process IFE content 400 and external text and external speech to perform translation to sign language animation provided to selected passengers according to some embodiments.

[0065] IFE content 400 can include digital media (e.g., songs, audio podcasts, movies, television programs, gaming, etc.), crew notifications, and automated aircraft notifications. The IFE content can have text components 402 and / or speech (audio) components 404 (digitized speech), which may be extracted from audio tracks and / or closed caption text provided with digital media.

[0066] External text and external speech may also be provided to the IFE server for translation to sign language. The IFE server may receive analog speech signals which are converted to digitized speech, or may receive digitized speech. Examples of external text and external speech include such as passenger audio and notifications delivered by the crew or pilot “in real time” during a flight.

[0067] The text components 402 can include textual descriptions, closed caption text, etc. The speech components 404 can be converted to text from the speech components 404 and / or the text components 402, by a speech-to-text conversion module 420 of the IFE Server 350. The converted text is then translated to sign language by a text-to-sign language conversion module 410 of the IFE Server 350. There exist many different sign languages, such as International Sign Language (ISL), American Sign Language (ASL), British Sign Language (BSL), French Sign Language (LSF), Chinese Sign Language (CSL), and others. A database of sign languages 412 may be stored in or accessible by the text-to-sign language conversion module 410 for use in converting the text to a defined one of the sign languages.

[0068] As discussed in further detail later, the text-to-sign language conversion module 410 may include a machine learning model 414 trained to associate text and / or audio speech to animated signing moments.

[0069] The sign language conversion module may be further operative to access a database of sign languages to convert input text and / or digitized speech to a scripts of signs.

[0070] The sign language conversion module may be further operative to select one of the sign languages for use in conversion based on a preferred language indicated by the user, based on an operator selection of a preferred language, basedon a regional location of the user, based on a departure location and / or a destination location of a user onboard a vehicle.

[0071] In some embodiments, the at least one memory further includes a speech- to-text conversion module configured to convert input digitized speech to a sequence of text words. The sign language conversion module is further operative to convert the sequence of text words to a scripts of signs.

[0072] In some embodiments, the sign language conversion module is further operative to determine contextual information as indicative of emotion or tone intended for communication by the digitized speech, and generate the script of signs to further convey the contextual information.

[0073] The sign language output of the text-to-sign language conversion module 410 is then provided to a sign language animation module 430. The sign language input to the sign language animation module 430 may correspond to commands or other indications of what signing is to be communicated to passengers. In one operational embodiment, the sign language animation module 430 generates a video of a digital signing avatar performing the commanded / indicated signing which is streamed to display devices associated with certain deaf passengers. In another operational embodiment, the sign language animation module 430 generates a script of signs or movements to be performed by the avatar generated through computer operations performed by the display devices associated with certain passengers. The translation and control of the digital signing avatar may be performed in realtime to correspond to spoken and / textual announcements broadcast to passengers and / or to correspond to media content being played through the IFE display associated with the deaf passenger. A distribution module 440 then provides the output of the sign language animation module 430 through the network 340 of Figure 3 for distribution to selected one(s) of the video display units 320 and PEDs 322. The digital signing avatar may be combined with video or text content in the IFE server 350 or at the video display units 320 and / or PEDs 322. For example, the digital signing avatar may be displayed as an inset picture-in-picture box overlaid on other content, e.g., as shown in Figure 2.

[0074] As discussed above, in some embodiments, the sign language animation module 430 may include a customization module 432 used to customize the digital signing avatar in many ways. Airlines may tailor the avatar's appearance, including factors like gender, demographics, styling, dress code, and color palette. Thisflexibility not only promotes inclusiveness but also aligns with airlines' branding strategies.

[0075] The sign language animation module 430 can generate the digital signing avatar to satisfy certain viewability requirements, and which can be adapted based on known or sensed conditions that can affect viewability by passengers in-general or by a particular passenger having known or determined sight disability or viewing preference. In some embodiments, the size of certain features involved in performing signing, e.g., the size of hands, facial movements such as mouth movement, etc., and / or the distance of movement of the features to perform signing can be increased relative to a default size and / or distance responsive to a passenger being known or determined have a visual impairment. The determination may be made based on passenger data such as medical information (e.g., declared as deaf passenger) or preferences recorded in an airline application or airline database, based on a passenger inputted request, and / or based on other defined preference indicative of a visual impairment or viewing preference.

[0076] In some additional or alternative embodiments, the size of certain features involved in performing signing and / or the distance of movement of the features to perform signing can be increased relative to a default size and / or distance responsive to a priority level of an announcement (e.g., an urgent announcement indicating seatbelt requirement due to severe turbulence) can be signed using larger features (e.g., hands) and with increased distance of movement to perform the signing compared to how signing is performed for a non-urgent announcement (e.g., duty free shopping offer).

[0077] The sign language animation module may be further operative when generating video of the avatar performing signs forming the sign language responsive to the script of signs, to increase the size of certain avatar features involved in performing signing and / or the distance of movement of the avatar features involved in performing signing responsive to determining a user viewing the video of the avatar has a visual impairment.

[0078] In some embodiments, the sign language animation module is further operative to determine a user viewing the video of the avatar has a visual impairment, based on user data including medical information, based on preferences recorded in an airline application or airline database, and / or based on a user inputted viewing related preference.

[0079] In still some additional or alternative embodiments, the size of certain features involved in performing signing and / or the distance of movement of the features to perform signing can be increased relative to a default size and / or distance responsive to detecting that a reduced visibility condition is occurring. The reduced visibility condition can be determined to occur when turbulence exceeds a threshold level, smoke is sensed in cabin, bright cabin lighting or sunlight is determined to be possibly shining on seat video displays along one side of the cabin (or a section of one side of the cabin), and which may be determined based on signaling from ambient light sensor(s) and / or determined based on the known orientation of seating in the aircraft cabin (e.g., aircraft heading and maybe also aircraft angle-of-attack) relative to the determined location of the sun.

[0080] In some embodiments, the sign language animation module is further operative when generating video of the avatar performing signs forming the sign language responsive to the script of signs, to increase the size of certain avatar features involved in performing signing and / or the distance of movement of the avatar features involved in performing signing responsive to determining an announcement, formed by the digitized speech, has a threshold priority level.

[0081] The text-to-sign language conversion module 410 may include a machine learning model 414 trained to associate text and / or audio (digitized) speech to a script of signs used to form animated signing movements. The machine learning model 414 may be used to preprocess the text inputs to extract key features corresponding to hand movements, facial expressions, and body postures. Audio and or video inputs may also be used to further define emotions and tones of the communication to provide more accurate hand movements, facial expressions, and body postures for the animated avatar. These features can be important for replicating through the computer-generated avatar movements the nuances and grammar of sign languages which provide a more lifelike rendering and resulting in more meaningful clearer communication.

[0082] In some embodiments, the sign language animation module is further operative when generating video of the avatar performing signs forming the sign language responsive to the script of signs, to increase the size of certain avatar features involved in performing signing and / or the distance of movement of the avatar features involved in performing signing responsive to determining occurrenceof a reduced visibility condition of the environment affecting viewing of the video of the avatar.

[0083] The type of machine learning model 414 which is implemented can be based on the complexity of the task. Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), or more advanced models like Transformerbased architectures may be suitable choices. Transfer learning techniques may also be employed, utilizing pre-trained models for tasks related to body movement recognition or facial expression analysis.

[0084] The sign language conversion module may include a machine learning model that is trained to associate input text and / or digitized speech to scripts of signs. The sign language conversion module is operative to input text and / or digitized speech through the machine learning model to generate the script of signs to be performed by a computer generated avatar.

[0085] In some embodiments, the machine learning model is trained to break down input text and / or digitized speech into smaller linguistic units of words and / or phrases, and associate individual linguistic units with an animated signing movement.

[0086] In some embodiments, the machine learning model is trained to break down input text and / or digitized speech into smaller linguistic units of words and / or phrases, and associate a sequence of linguistic units with a sequence of animated signing movements.

[0087] The machine learning model 414 may be trained to map text input and / or audio speech input into a sequence of signing movements. This operation can involve breaking down the input text and / or audio speech into smaller linguistic units, such as words or phrases, and associating each unit with appropriate sign language movements.

[0088] Grammatical rules and contextual information are used during this mapping process. For instance, understanding the order of words in a sentence or the use of sign language-specific grammatical structures is crucial for accurate conversion.

[0089] Once the model has been trained, it can generate a sequence of signing movements that correspond to the input text and / or audio speech. These movements can be generated frame by frame in a video format or can be generated as commands or indications of movements of avatar body parts to create signingmovements, e.g., movement of avatar body parts along pathways between coordinates, by a real time avatar animation rendering engine, taking into account the timing and coordination required for fluent signing. Facial expressions and avatar body movements are integrated into the animated signing to convey emotion, tone, and context effectively and accurately.

[0090] The sign language conversion module may be operative when generating the script of signs to apply grammatical rules relating to the order of words in a sentence to corresponding animated signing movements defined by sign languagespecific grammatical structures.

[0091] In some embodiments, the sign language conversion module is further operative when generating the script of signs to apply contextual information obtained for the input text and / or digitized speech.

[0092] In some embodiments, the sign language conversion module is further operative to determine the contextual information as indicative of emotion or tone intended for communication by the input text and / or digitized speech.

[0093] Further Definitions and Embodiments:

[0094] In the above description of various embodiments of the present disclosure, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0095] When an element is referred to as being "connected", "coupled", "responsive", or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected", "directly coupled", "directly responsive", or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, "coupled", "connected", "responsive", or variants thereof as usedherein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and / or clarity. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0096] As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes", "have", "has", "having", or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation "e.g.", which derives from the Latin phrase "exempli gratia," may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation "i.e.", which derives from the Latin phrase "id est," may be used to specify a particular item from a more general recitation.

[0097] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or devices) and / or computer program products. It is understood that a block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and / or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and / or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions / acts specified in the block diagrams and / or flowchart block or blocks, and thereby create means (functionality) and / or structure for implementing the functions / acts specified in the block diagrams and / or flowchart block(s).

[0098] These computer program instructions may also be stored in a non- transitory computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, suchthat the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the block diagrams and / or flowchart block or blocks.

[0099] A non-transitory computer-readable medium may include an electronic, magnetic, optical, electromagnetic, or semiconductor data storage system, apparatus, or device. More specific examples of the computer-readable medium would include the following: a portable computer diskette, a random-access memory (RAM) circuit, a read-only memory (ROM) circuit, an erasable programmable readonly memory (EPROM or Flash memory) circuit, etc.

[0100] The computer program instructions may also be loaded onto a computer and / or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer and / or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions / acts specified in the block diagrams and / or flowchart block or blocks. Accordingly, embodiments of the present disclosure may be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as "circuitry," "a module" or variants thereof.

[0101] It should also be noted that in some alternate implementations, the functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved. Moreover, the functionality of a given block of the flowcharts and / or block diagrams may be separated into multiple blocks and / or the functionality of two or more blocks of the flowcharts and / or block diagrams may be at least partially integrated. Finally, other blocks may be added / inserted between the blocks that are illustrated. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0102] Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate everycombination and subcombination of these embodiments. Accordingly, the present specification, including the drawings, shall be construed to constitute a complete written description of various example combinations and subcombinations of embodiments and of the manner and process of making and using them, and shall support claims to any such combination or subcombination.

[0103] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present invention. All such variations and modifications are intended to be included herein within the scope of the present invention.

Claims

Claims:

1. A computer server comprising: at least one network interface; at least one processor connected to communicate through the at least one network interface with video display units (VDUs) and / or personal electronic devices (PEDs); and at least one memory comprising: a sign language conversion module operative to translate digitized speech to a script of signs to be performed by a computer generated avatar; a sign language animation module operative to generate video of an avatar performing signs forming a sign language responsive to the script of signs; and a distribution module operative to communicate the video of the avatar through the at least one network interface to VDUs and / or PEDs.

2. The computer server of Claim 1 , wherein: the sign language conversion module comprises a machine learning model trained to associate input text and / or digitized speech to scripts of signs; the sign language conversion module is operative to input text and / or digitized speech through the machine learning model to generate the script of signs to be performed by a computer generated avatar.

3. The computer server of Claim 2, wherein the machine learning model trained to break down input text and / or digitized speech into smaller linguistic units of words and / or phrases, and associate individual linguistic units with an animated signing movement.

4. The computer server of Claim 2, wherein the machine learning model trained to break down input text and / or digitized speech into smaller linguistic units of words and / or phrases, and associate a sequence of linguistic units with a sequence of animated signing movements.

5. The computer server of Claim 2, wherein the sign language conversion module is operative when generating the script of signs to apply grammatical rules relating to the order of words in a sentence to corresponding animated signing movements defined by sign language-specific grammatical structures.

6. The computer server of Claim 5, wherein the sign language conversion module is further operative when generating the script of signs to apply contextual information obtained for the input text and / or digitized speech.

7. The computer server of Claim 6, wherein the sign language conversion module is further operative to determine the contextual information as indicative of emotion or tone intended for communication by the input text and / or digitized speech.

8. The computer server of Claim 1 , wherein the sign language conversion module is further operative to access a database of sign languages to convert input text and / or digitized speech to a scripts of signs.

9. The computer server of Claim 8, wherein the sign language conversion module is further operative to select one of the sign languages for use in conversion based on a preferred language indicated by the user, based on an operator selection of a preferred language, based on a regional location of the user, based on a departure location and / or a destination location of a user onboard a vehicle.

10. The computer server of Claim 1 , the at least one memory further comprises: a speech-to-text conversion module configured to convert input digitized speech to a sequence of text words, wherein the sign language conversion module is further operative to convert the sequence of text words to a scripts of signs.11 . The computer server of Claim 10, wherein the sign language conversion module is further operative to: determine contextual information as indicative of emotion or tone intended for communication by the digitized speech; andgenerate the script of signs to further convey the contextual information.

12. The computer server of Claim 1 , wherein the sign language animation module is further operative when generating video of the avatar performing signs forming the sign language responsive to the script of signs, to increase the size of certain avatar features involved in performing signing and / or the distance of movement of the avatar features involved in performing signing responsive to determining a user viewing the video of the avatar has a visual impairment.

13. The computer server of Claim 12, wherein the sign language animation module is further operative to determine a user viewing the video of the avatar has a visual impairment, based on user data including medical information, based on preferences recorded in an airline application or airline database, and / or based on a user inputted viewing related preference.

14. The computer server of Claim 1 , wherein the sign language animation module is further operative when generating video of the avatar performing signs forming the sign language responsive to the script of signs, to increase the size of certain avatar features involved in performing signing and / or the distance of movement of the avatar features involved in performing signing responsive to determining an announcement, formed by the digitized speech, has a threshold priority level.

15. The computer server of Claim 1 , wherein the sign language animation module is further operative when generating video of the avatar performing signs forming the sign language responsive to the script of signs, to increase the size of certain avatar features involved in performing signing and / or the distance of movement of the avatar features involved in performing signing responsive to determining occurrence of a reduced visibility condition of the environment affecting viewing of the video of the avatar.

16. The computer server of Claim 1 , wherein the sign language conversion module is enabled to generate of video of an avatar performing signs forming a sign language for distribution and playout to a user responsive to a request received from the user to receive sign language communications and / or responsive to information indicating the user has a hearing impairment.

17. The computer server of Claim 16, wherein: the sign language animation module is further operative to generate a plurality of videos of avatars performing signs forming a plurality of different sign language; and the distribution module is further operative to communicate the plurality of videos in parallel through the at least one network interface to a same VDU or PED for simultaneous display.

18. The computer server of Claim 16, wherein: the distribution module is further operative to stream the video of the avatar through a communication session established between the computer server and an individual VDU or PED operated by a user having a hearing impairment.

19. The computer server of Claim 16, wherein: the sign language conversion module is further operative to access a database of sign languages to convert input text and / or digitized speech to a scripts of signs, and to select a first sign language for use in translation to the script of signs based on a preferred language indicated by a first user, and to select a second sign language for use in translation to the script of signs based on a preferred language indicated by a second user; the sign language animation module operative to generate a first video of an avatar performing signs forming the first sign language and to simultaneously generate a second video of an avatar performing signs forming the second sign language; the distribution module is further operative to stream the first video of the avatar through a first communication session established between the computer server and a first VDU or PED operated by a first user, and tostream the second video of the avatar through a second communication session established between the computer server and a second VDU or PED operated by a second user.

Images