Method and system for the automatic translation of sign language into a different idiom
Patent Information
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- CLOUDIA RESEARCH SPA
- Filing Date
- 2024-06-28
- Publication Date
- 2026-05-06
Smart Images
Figure IB2024056339_02012025_PF_FP_ABST
Abstract
Description
METHOD AND SYSTEM FOR THE AUTOMATIC TRANSLATION OF SIGN LANGUAGE INTO A DIFFERENT IDIOMDESCRIPTIONField of application of the invention
[0001] The present invention concerns the technical field of rapid communication via electronic devices (e.g. smartphones, tablets, laptops, personal computers, etc.) and the subject of the invention is a method for automatic translation of sign language into a different idiom.
[0002] The invention also includes a system for the automatic translation of sign language into a different idiom based on this method.State of the art
[0003] As is known, in recent years there has been an increasing will to implement the use of sign language as an inclusive form of communication for hearing-impaired people.
[0004] Thanks to the advances in technology, electronic devices have opened up new lines of communication to facilitate interaction between people who use sign language and those who do not.
[0005] Sign language is a visual-gestural language that is used worldwide in communities of deaf or severely hearing-impaired people
[0006] Previously, communication between these people was 'local', i.e. carried out mainly in person, whereas the development of new technologies has made it possible to use this form of communication also at a distance, through the use of common electronic devices.
[0007] Information technology and new communication platforms have provided new opportunities for hearing-impaired people to connect more easily with the rest of the world, helping to break down language barriers and foster inclusion.
[0008] One of the main applications of the use of sign language in electronic devices is through video calls and real-time messaging platforms.
[0009] Thanks to smartphones, tablets, personal computers and other devices equipped with webcams and an Internet connection, people can communicate using sign language, allowing them to establish real-time conversations with friends, family, colleagues, etc.
[0010] In addition, applications and software have been developed for helping hearing-impaired people, to offer help in interpreting texts or voices in order to allow them to interact with electronic devices designed to act as virtual assistants or voice recognition systems.
[0011] These technologies are making digital services more accessible to all hearing- impaired people, and therefore their use is rapidly spreading among users of all age groups in order to promote the inclusion of a part of the population that cannot currently take full advantage of the resources and benefits offered by digital communications.
[0012] It is evident, therefore, how the diffusion of new digital communication systems capable of integrating the sign language allows a greater appreciation and integration of the community of hearing-impaired people by the rest of society.
[0013] Despite the progress achieved, the integration of sign language in digital devices still has some drawbacks.
[0014] Firstly, current electronic systems available to a hearing-impaired user only allow the automatic generation of subtitles, for the language spoken by an able-bodied user. These systems are an aid for hearing-impaired people that help them to interact with an able-bodied interlocutor.
[0015] However, such systems are slow and imprecise, thus preventing smooth, realtime communication between a hearing-impaired person and one or more able-bodied individuals.
[0016] Moreover, current electronic systems do not provide a translation from sign language to a different idiom (and vice versa).
[0017] Furthermore, current digital systems for converting sign language to other idioms have a low degree of integration and interoperability with the common messaging / video platforms used by the majority of the population.
[0018] This problem makes it rather difficult for a hearing-impaired person to fully exploit the potential of social media, with the consequence of relegating them to a more marginal role within society.Presentation of the invention
[0019] The present invention intends to overcome the above-mentioned technical drawbacks by providing a particularly effective and efficient method for the automatic translation of sign language into a different idiom.
[0020] In particular, the main purpose of the present invention is to provide a method capable of providing an automatic translation of sign language into a different idiom in asmooth, essentially seamless manner.
[0021] A further purpose of the present invention is to provide a method for the automatic real-time translation of sign language into a different idiom, i.e. without any particular delay from the moment the action is performed by the person to when the output provided to a listener is generated.
[0022] Another purpose of the present invention is to provide a method for the automatic translation of sign language into a different idiom that can be easily reproduced by all devices that are able to interface with all current software / platforms.
[0023] A further purpose of the present invention is to provide a method for the automatic translation of sign language into a different idiom able to provide high interoperability with all devices / apparatus / platforms currently in use.
[0024] Another purpose of the present invention is to provide a method for the automatic translation of sign language into a different idiom that is particularly easy to reproduce and does not require the use of dedicated or complex equipment.
[0025] Again, the last but not least purpose of the present invention is to make available a method for the automatic translation of sign language into a different idiom that can easily operate with all social media platforms currently in use, as well as being easily reproduced in new generation platforms (metaverse, etc.).
[0026] These purposes, together with others that will be better explained below, are achieved by a method for the automatic translation of sign language into a different idiom of the type according to claim 1.
[0027] Other purposes that will be better described below are achieved by a method for the automatic translation of sign language into a different idiom in accordance with the dependent claims.
[0028] According to a further aspect of the present invention, a system for the automatic translation of sign language into a different idiom is provided of the type in accordance with claim 10.Brief description of the drawings
[0029] The advantages and features of the present invention will become clear from the following detailed description of some preferred but not limiting configurations of a method for the automatic translation of sign language into a different idiom with particular reference to the following designs:Figure 1 shows a block diagram of a system implementing the method forthe automatic translation of sign language into a different idiom according to the invention.Detailed description of the invention
[0030] The present invention relates to a method for the automatic translation of sign language into a different idiom.
[0031] As will be further clarified in the following description, this method allows the realtime translation of sign language by a user (e.g. a hearing-impaired person) into a different idiom, e.g. the Italian language (or other languages such as English, German, French, etc.).
[0032] in order to promote the automatic translation of sign language into a different idiom, the present method comprises a number of specific steps.
[0033] Firstly, there is a step (a) of providing means 2 suited to allow the acquisition of images and / or videos.
[0034] In particular, the means provided during step a) may include any technical apparatus capable of generating a series of digital data (or numerical data) associated with a sequence of images or video frames.
[0035] in the present context, the expressions 'digital data' and 'numerical data' can be interpreted as synonyms and are intended to refer to discretely expressed data (by means of elementary elements defined as bit) that can be interpreted and processed by processor-based processing means, which are well-known and commonly used per se.
[0036] For example, means for images and / or video acquisition 2 may include one or more digital cameras arranged to both capture images and record video sequences.
[0037] In particular, such means 2 (digital cameras or other similar elements) may be integrated into a commonly used electronic device and known in itself such as, for example, a smartphone, tablet, laptop or personal computer, smartwatch, etc.
[0038] Conveniently, the method may comprise a step (b) of provision of first digital data Di and a step (c) of provision of second digital data Ds.
[0039] As is well known, sign language is characterized by the fact that it conveys meaning through precise gestures and movements made with the limbs or other parts of the body.
[0040] In particular, sign language differs from other languages in the use of the upper body, i.e. the gestures and expressions that the person defines with the face, arms, hands, head, eyes and mouth.
[0041] A sequence of information associated with a precise facial expression and the movements of the other parts of the body (head, hands, arms, etc.) allow to define a specific 'gesture' to which the sign language associates a precise meaning in sign language.
[0042] Like all idioms, therefore, sign language also has its own grammatical, syntactic, morphological and lexical rules.
[0043] Conveniently, certain characterizing parameters (or phonemes) are combined simultaneously in each sign:Configuration;Location;Orientation;Movement;Expression
[0044] The term 'Configuration' used in the present invention refers to the form adopted by the hand(s) of a user communicating via sign language.
[0045] The number and the particular shape adopted by the hand(s) in each configuration may vary depending on the type of sign language considered; for example, Italian sign language presents fifty-six possible configurations (source: Radutzky, 1992), even though other sign languages (e.g. British, American, etc.) present a different number of configurations, i.e. shapes that can be performed with the hands alone.
[0046] The term 'Location' indicates the position in space in which the sign (or gesture) is conveyed.
[0047] In general, the Location within which the sign is conveyed may present a three- dimensional space area generally included inside a square measuring 40cm x 40cm.
[0048] In practice, the Location is defined in relation to the user's position and, therefore, defines a relatively large area within which the gesture can be performed. The latter may be performed in front of the user's chest, or to the user's right, left, up or down.
[0049] The term 'Orientation', on the other hand, is defined as the position of the palm of the hands maintained by the user during the execution of the gesture.
[0050] More specifically, the orientation defines the existing relationship between the user’s hands and the location, and indicates which part of the hand is directed towards the location of articulation.
[0051] The relevant parts of the hand to define the Orientation are generally the palm,the back, the ulnar side (located near the little finger), the radial side (located near the thumb), and the wrist.
[0052] The term 'Movement' is intended to refer to the dynamic condition that signs present; this parameter is divided into two specific categories: primary movements, or secondary movements.
[0053] Primary movements present a trajectory that follows the hand as it changes Location. In fact, some signs, in order to be performed, require a movement along a precise path due to the fact that the hands have to progressively change their position in space starting from an initial position (e.g. close to the user's body) to a final position further away from the body than the initial position (e.g. in front of the user).
[0054] Secondary movements, on the other hand, involve changes in Configuration and / or Orientation. For example, a change (and therefore a secondary type of movement) is obtained when the hand undergoes a transition from an open 'U' to a closed 'U' (i.e. from an extended to a flat position).
[0055] Secondary movements also include changes in the Orientation of the palms; for example, changing the articulatory orientation of a user's hand from a prone position (hand palm pointing downwards) to a neutral position (hand standing vertically in its default position) falls into this category.
[0056] The Movement also presents its own particular characteristic in terms of time. In particular, signs may include non-repeated or repeated movements.
[0057] Primary movements and secondary movements are composed of both signs that can be repeated over time and individual signs, i.e. signs that do not have to be repeated in order to understand their meaning.
[0058] Finally, the term 'Expression' refers to the movements associated with the facial muscles designed to give the user a predetermined facial expression.
[0059] Through the facial expression (and thus the face), the user can give a particular emphasis to the sign he or she is executing with the rest of the body (e.g. to indicate a direct or indirect question; an imperative, etc.).
[0060] Each sign, therefore, can be defined by a precise sequence of the above parameters: Orientation, Location, Movement, Configuration and Expression.
[0061] Conveniently, the information regarding the Location, Configuration, Orientation and Expression constitute the static parameters associated with a sign.
[0062] The term "static" used in this context is intended to refer to those characteristicsof each sign that do not change during the execution of the sign itself, and thus represent a constant (i.e. the equivalent of a "still image").
[0063] In contrast, the information associated with the Movement constitutes the dynamic parameters of a sign.
[0064] The term dynamic used in this context is intended to refer to those features of the sign that change during the execution of the sign itself in the spatial sense of the term (i.e. a change in position assumed by a body part from a starting position at the beginning of the sign to a final position at the end).
[0065] Conveniently, the first digital data Di prepared during step b) are associated with the static parameters of the sign language.
[0066] The second digital data D2 prepared in step c), on the other hand, are associated with the dynamic parameters of the sign language.
[0067] In other words, the first digital data Di contain information about the Configuration, Location, Orientation and Expression parameters.
[0068] The second digital data D2 contain information about the movement parameter.
[0069] The source of the first digital data Di and the second digital data D2 can be represented by a database in which records associated with the static and dynamic parameters of the sign language are stored.
[0070] In particular, this database may be of the distributed type, i.e. located on one or more servers accessible via the Internet. In this way, the first data Di and the second data D2 can be obtained by accessing one or more platforms or Internet websites found in the network. Thanks to this access, therefore, it is possible to create a dedicated database containing a 'dictionary' (i.e. a collection) of data referring to static and dynamic parameters and parameters used in sign language.
[0071] Alternatively, the database containing the first data Di and the second data D2 can be created through the use of an Artificial Intelligence, the definition of which will be better described later in this description.
[0072] This artificial intelligence can be instructed to partially or fully interpret certain sign language features (e.g. the Configuration and / or Location and / or Orientation and / or Movement and / or Expression).
[0073] By applying said Artificial Intelligence to a sample source (e.g. a subject producing the fundamental signs of the language, as if they were basic elements of the alphabet), it will be possible to extract the static and dynamic characteristics for eachsign. These characteristics can then be used to create the respective static / dynamic parameters that can be stored on a database in the form of first data Di and second data Dz,
[0074] From the above, it is therefore possible to highlight how the source of the first data D-i and the second data Dscan be constituted by all the digital information that together identify the individual sings of the language (e.g. a static image available on the internet or acquired in real-time, digital documentation corresponding to dictionaries and reference texts, etc.).
[0075] Starting from these digital sources and through an ad hoc data processing (carried out by dedicated algorithms under human supervision, or directly by Artificial Intelligence), it is possible to derive the first data Di and second data Di as defined in steps b) and c) of this method.
[0076] This method includes a step (d) of provision of third digital data D3 associated with the metadata in an idiom other than sign language.
[0077] The term 'metadata' is intended to refer to all information related to grammar, idiomatic expressions, meaning of words, sound of the pronunciation of a particular idiom, etc., but transduced as digital data.
[0078] Metadata represent the digital transposition of all the information required to generate the translation of a written or oral text into the particular idiom to which that metadata refer.
[0079] For example, the third digital data Ds prepared during step d) could be associated with the metadata of the Italian language, or of the English language, or of any other official language of one or more countries.
[0080] Furthermore, the third digital data D3 prepared during the execution of step d) may be associated with the metadata of a plurality of official languages (Italian, English, German, etc.).
[0081] Conveniently, the method may present a step e) of provision of storage means 3.
[0082] In particular, storage means 3 can be configured to store digital data and in particular can be provided to store first digital data Di and second digital data D2 respectively associated with the static and the dynamic parameters of sign language.
[0083] Furthermore, the storage means 3 provided during step e) may also be suited to store the third digital data D3 associated with an idiom other than sign language.
[0084] Following step e), the method involves the execution of step f), during whichdigital processing means 4 are operatively connected to the images and video acquisition2 and the storage means 3.
[0085] For the purposes of the present invention and method, the storage means 3 and / or the processing means 4 may be of the local or remote type, i.e., both may be installed in the same electronic device accessible by a user (e.g., a smartphone, a tablet, a laptop, etc.), or may be of the remote or distributed type, i.e., may be contained, for example, in servers possibly located at a distance from the user and connected to each other via network connections, Internet network, etc.
[0086] The structure and architecture of storage means 3 and processing media 4 may be the most functional for the implementation of the present method, and since these aspects are known in themselves and not relevant to the inventive core of the present application, they will not be further described herein.
[0087] The method also includes a step g) of processing by the processing means 4 the first digital data Di and the second digital data D2previously stored in the storage means3 during the execution of step e).
[0088] The aim of step g) is to obtain digital reference data DREFassociated with the meaning of sign language.
[0089] The digital reference data DREFobtained in step g) contain information on the static and dynamic parameters of each sign defining the alphabet of the specific sign language to which the first digital data D1and the second digital data D2refer.
[0090] As will be better described below, the reference digital data DREFcontain the basic information with which it will be possible to digitally transduce the sign language used by the user.
[0091] For this reason, the method includes a step h) to save the reference data DREFin the storage means.
[0092] Steps a) to h) described above allow the definition of a digital data structure capable of allowing the automatic interpretation of any input sequence (i.e. input data) using sign language.
[0093] In fact, every sign made by the user can be correctly interpreted in digital format thanks to the prior preparation and storage of the digital reference data.
[0094] At this point, the method involves a phase i) of activation of the means for images and / or videos acquisition 2 so as to capture a sequence of images and / or videos associated with the user’s body movement reproducing sign language.
[0095] From a practical point of view, the user can record a video using a common electronic device including a digital camera suited to capture a sequence of images or video frames.
[0096] For example, when the means for images and video acquisition 2 are integrated into a user’s personal device, such as a smartphone, the user himself may activate these means through his mobile phone in order to promote the capturing of images or video.
[0097] In general, the acquisition of the video frame sequence can occur in real time by the user during the execution of step i).
[0098] For the purposes of this method, it is important to highlight that, during the execution of step i), the user acquires / records a video and / or image sequence in which he / she expresses him / herself using sign language.
[0099] This video sequence, therefore, will define the user as the main subject during a conversation developed through sign language.
[0100] It is, therefore, important that the video sequence contains all the signs performed by the user during his or her conversation, i.e. that ail the characteristics associated with the Configuration, Location, Orientation, Expression and Movement parameters of each gesture performed by the user can be found in this sequence.
[0101] Following the activation of the means for images and / or video acquisition 2 in step i), step j) for the provision of fourth digital data D« associated with the sequence of images or videos acquired during the activation of these means 2 is to be performed.
[0102] In this way, the fourth digital data D« will contain information associated with the signs actually performed by the user during the video recording.
[0103] The fourth data D4represent the transduction into digital format of the signs made by the user during video recording.
[0104] In this sense, therefore, the ‘arrangement’ of fourth data D4should be considered as the action performed by the means for image / video acquisition 2 in providing an output data stream corresponding to the video recorded by the user.
[0105] In other words, fourth datarepresent the result of the activation of acquisition means 2, i.e. all information associated with the video recorded by the user is contained in these fourth data D4.
[0106] Thus, the fourth data D4are not dependent on the first data Di and the second data D2(nor are they correlated with the latter), but nevertheless contain information associated with a plurality of signs performed by the user during the recording of thevideo.
[0107] in order to promote the identification of the signs actually performed by the user during the recording of the video, it is therefore necessary to compare the information contained in that video (expressed in digitai format with the fourth data D4) to the reference signs (expressed in digitai format with the reference data format DREF).
[0108] it is therefore possible to arrange a step k) to compare the fourth digitai data D« provided in step j) with the reference data DREFsaved in the storage means (step h) ) and obtained foliowing the execution of processing step g).
[0109] in this comparison, therefore, the information associated with the signs actually performed by the user during the recording of the video will be related with a library of information associated with the reference signs.
[0110] The outcome of this comparison allows to obtain information regarding the meaning, in sign language, of the body movement performed by the user in the image or video sequence acquired during the execution of step i).
[0111] The comparison is performed by processing means 4, and the outcome of this comparison may generate further digital data associated with the meaning of the signs performed by the user during the recording of the video occurred in step i).
[0112] Consequently, the method provides for a subsequent step I) during the execution of which it is possible to associate the meaning of the signs performed by the user during step j) and obtained following the comparison described above, with the meaning associated with the third data D3 relating to a different idiom and defined in step d).
[0113] Step I) is performed by processing means 4; the aim of this step is to determine the semantics of the conversation performed by the user during the recording of the video; this semantics is closely related to the preparation of a ‘base’ of such signs, i.e. a set of signs suited to define elementary units that are then progressively joined during the generation of a conversation so as to define a sequence having a precise meaning.
[0114] This structure is similar to that associated with the letters of the alphabet which, when joined together according to a precise criterion, form a precise word to which a certain meaning corresponds.
[0115] The aim of step I) is to determine the overall meaning of all signs performed by the user during the recording of the video in order to extend this meaning to the entire conversation and so determine the complete meaning of the latter.
[0116] At this point, the method provides for a step m) of generating fifth digital data Dsthat present information associated with the meaning found in the previous step 8), but nevertheless expressed through the meaning of the third digital data D3prepared in step d).
[0117] In other words, the meaning of the entire conversation recorded through sign language by the user is completely translated into the idiom with which the third digital data D3are associated.
[0118] It is therefore possible to determine how the fifth digital data D5generated during the execution of step m) are associated with the translation of the meaning of the sign language conversation recorded by the user into a different idiom (e.g. Italian, English, etc.).
[0119] Conveniently, the method may present a step n) of preparation of display means S.
[0120] The display means 5 prepared in step n) can also be operatively connected to the processing means 4.
[0121] In general, display means 5 may be defined by a screen (e.g. liquid crystal display or OLED) visible to the user or other people.
[0122] For example, the display means 5 could be the screen of a personal electronic device associated with the user (smartphone, tablet, laptop, etc.).
[0123] Alternatively, the display means 5 may be arranged in a remote location with respect to the user generating the sequence of video frames, in order to possibly be accessible also to people situated in a location relatively distant from that of the user.
[0124] Conveniently, processing means 4 can be configured to process the fifth digital data D5(associated with the meaning expressed by the user during the recording of the video in an idiom other than sign language) into a written text showed directly on the display means.
[0125] In this way, it will be possible to promote the displaying of the sequence of the fifth data D5on the display means 5; a possible application of this step n) could be the provision of subtitles (in a predetermined idiom) of the meaning expressed by the user in the video recorded with sign language.
[0126] Preferably, the method may include a step o) of provision of an Artificial Intelligence system 6 operationally connected to said processing means 4.
[0127] Artificial intelligence 6 used in the context of this method can be describedgenerically as a neural network capable of executing machine learning algorithms suitable for processing images according to a predefined pattern in order to extract language-related information from these images.
[0128] in particular, Artificial intelligence 6 prepared during step o) will be suited to undergo a specific training by connecting to extensive databases containing sign language information.
[0129] For this reason, the method includes a training step p) during the execution of which the Artificial Intelligence enables the provision of first digital data Di and second digital data D2 in a particularly accurate and precise manner.
[0130] In essence, the training step p) can be performed by first allowing the Artificial Intelligence system 6 to access a database particularly containing static images of gestures to define the alphabet of a sign language.
[0131] In addition, during the execution of the training step p), the Artificial Intelligence 6 will also be able to access databases containing video sequences associated with each individual gesture defining the sign language alphabet.
[0132] In this way, the Artificial Intelligence 6 will be able to refine the definition of the signs that form the alphabet of a specific sign language so as to determine in a particularly accurate manner the static and dynamic parameters associated with these signs.
[0133] The training step o) will therefore allow to accurately determine all parameters associated with the Configuration, Location, Orientation, Expression and Movement of each gesture defining the sign language alphabet
[0134] In addition to this, the Artificial Intelligence 6 may be trained on particularly large databases of users conversing through the use of sign language, in order to determine the gestures that form the sign language alphabet even when these are performed with a certain degree of approximation by users.
[0135] In this way, the use of an Artificial Intelligence 6 combined with processing means 4 will allow to define reference data associated with single gestures in sign language that are particularly precise and relevant to those actually performed by deaf persons.
[0136] In addition, the Artificial Intelligence 6 training may also be useful to improve the processing performed by processing means 4 on the fourth digital data D4associated with the images or video frames recorded by the user.
[0137] Consequently, the use of the Artificial Intelligence 6 may improve both the reference alphabet of the sign language represented by the digital reference data, and the definition in sign language of the gestures made by the user during the recording of the video.
[0138] Therefore, the Artificial Intelligence system 6 will allow to considerably improve the comparison that occurs during the execution of step k); in particular, starting from extremely precise data (fourth data D« and reference data DREF), it will be possible to accurately determine the entire meaning associated with the conversation recorded by the user during the activation of the image or video acquisition means 2 operated in step i).
[0139] The Artificial intelligence 6 prepared in step o) can also be used to define in a particularly precise and complete manner all metadata associated with the idiom (other than the sign language) to which the second digital data D2prepared during the execution of step c) correspond.
[0140] In this way, also the translation of sign language into text and / or audio associated with a different idiom may be particularly accurate and faithful to the language actually used by people in their daily lives.
[0141] Conveniently, the Artificial Intelligence system 6 can be trained (during step o) ) by accessing all information concerning the characterizing parameters of the sign, i.e. Configuration, Location, Orientation, Movement and Expression.
[0142] In this case, therefore, the training of the Artificial Intelligence system 6 can also be carried out with the help of the first D< and second Dz digital data referring to the static and dynamic parameters of each sign.
[0143] The present invention relates to a system 1 for the automatic translation of sign language into a different idiom.
[0144] This system 1 for the automatic translation of sign language comprises an electronic device provided with means for images and / or videos acquisition 2.
[0145] Conveniently, the means for image and / or video acquisition 2 are of the type previously described herein.
[0146] For example, the means image and / or video acquisition 2 may include one or more digital cameras arranged to both capture images and record video sequences.
[0147] In particular, these means 2 may be integrated into an electronic device in current use and known in itself (e.g. a smartphone, tablet, laptop or personal computer,smartwatch, etc.).
[0148] The system 1 for the automatic translation of sign language into a different idiom which is the object of the present invention also comprises storage means 3 and processing means (CPU) 4.
[0149] Storage means 3 and processing means 4 are also of the type described above and known in themselves.
[0150] Conveniently, the storage means 3 and the processing means 4 may be operatively connected to each other and integrated in the same electronic device in which the means for image and / or video acquisition 2 are installed.
[0151] In addition, system 1 comprises display means S suited to be seen by an able- bodied user in order to access the visualization of the automatic translation into a given idiom of signs performed by a hearing-impaired user.
[0152] As described above, the access to display means S is primarily aimed at able- bodied people who, through these means, can read the translation from sign language into an idiom known to the able-bodied person himself.
[0153] For this reason, the display means 5 can be remotely positioned with respect to the location of the hearing-impaired user.
[0154] Conveniently, the system 1 includes a computer program installed in the processing means 4.
[0155] In particular, these instructions may be stored in the storage means 3 so that they can be executed by processing means 4 by accessing said storage 3.
[0156] The instructions defining the computer program can be executed by processing means 4 in such a way as to implement the method for the automatic translation of the sign language into a different idiom of the type described above.
[0157] In this way, system 1 may provide to all users (whether hearing-impaired or able-bodied) a platform or computer application suited to manage certain hardware equipment (means for image / video acquisition, storage means, processing means and display means) in order to provide a real-time translation of sign language into a different idiom.
[0158] This real-time translation can be carried out when a hearing-impaired user develops a conversation through the use of an electronic device on which the corresponding computer program mentioned above has been installed.
[0159] The present invention can be implemented in other variants all falling within thescope of the inventive features claimed and described; these technical features can be replaced by different technically equivalent elements, and the materials, shapes and dimensions of the invention can be any as long as they are compatible with its use.
[0160] The numbers and reference marks inserted in the claims and description are only intended to improve the clarity of the text and must not be considered as limiting the technical interpretation of the objects or processes identified by them.
Claims
CLAIMS1. A method for the automatic translation of sign language into a different idiom, wherein the sign language involves the adoption of signs reproduced by a user and consisting of static parameters and dynamic parameters associated with Configuration, Location, Orientation, Expression, and Movement, wherein such method comprises the following steps: a) provision of means for images and / or video acquisition (2); b) provision of first digital data (Di) associated with the first statistic parameters of sign language; c) provision of second digital data (D2) associated with the dynamic parameters of sign language; d) provision of third digital data (D3) associated with metadata of a predetermined idiom and different from sign language; e) provision of storage means (3) suited to save said first digital parameters and said second digital parameters; f) provision of processing means (4) operatively connected to said means for images and / or video acquisition (2) and said storage means (3); g) processing by said processing means (4) of said first digital data (Di) and said second digital data (D2) to obtain digital reference data (DREF) associated with the meaning of sign language; h) saving of the reference data in said storage means (3); i) activation of said means for images and / or video acquisition (2) so as to capture a sequence of images and / or video associated with the user's body movement reproducing sign language; j) provision of fourth digital data (D4) associated with the sequence of images and / or video acquired in said activation step i); k) comparison between said fourth digital data (D4) provided in said step j) and the reference data (DREF) obtained in said processing step g) and saved in said storage means (3) during the execution of said step h), said comparison being suited to determine the meaning of the body movement performed by the user in the sequence of images and / or video acquired in sign language during step i); l) association of the meaning determined in said step j) with the thirddigital data (Da) defined in said step d); m) generation of fifth digital data (Ds) in the idiom associated with said third digital data (Da), the meaning of said fifth digital data (Ds) corresponding to the meaning determined during the execution of said step k).
2. Method according to claim 1 , characterized in that it comprises a step I) of provision of display means (5), said display means (5) being operatively connected to said processing means (3).
3. Method according to claim 2, characterized in that said display means (5) are suited to promote the display of the sequence of fifth digital data (Ds) generated during the execution of said step m).
4. Method according to one or more of the preceding claims, characterized in that said first digital data (Di) provided in said step b) comprise information relative to the Location, Configuration and Orientation in the manner defined in sign language.
5. Method according to one or more of the preceding claims, characterized in that said second digital data (D2) provided in said step c) comprise information related to the Movement in the manner defined by sign language.
6. Method according to one or more of the preceding claims, characterized in that it includes a step o) of providing and training an Artificial Intelligence system (6).
7. Method according to claim 6, characterized in that said step o) of training of said Artificial Intelligence system (6) is performed by using said first digital data (Di) and said second digital data (D2).
8. Method according to claim 6 or 7, characterized in that said first digital data (Di), said second digital data (D2) and said third digital data (D3) are suited to define respective vector-type parameters.
9. A system for the automatic translation of sign language into a different idiom, including:- an electronic device provided with:- means for images and / or video acquisition (2);- storage means (3);- processing means (4) operatively connected to said storage means (3);- display means (5);- a computer program; characterize in that said computer program comprises a plurality of instructions storable in said storage mean (3) and executable by said storage means (3), said plurality of Instructions being suited to implement the method for the automatic translation of sign language into a different idiom according to one or more of the preceding claims.