Laboratory system having a portable device with a microphone and method thereof - Patents.com
The laboratory system uses a portable device with voice input and correction software to convert technical language, addressing inefficiencies in manual control and contamination risks, enabling hands-free, accurate operation of laboratory equipment.
Patent Information
- Application Number
- JP2021556359
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-03-18
- Filing Date
- 2020-03-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-03-13
AI Technical Summary
Existing laboratory equipment control systems are inefficient and complex due to the need for manual input with gloves, which can lead to incorrect operations and contamination risks, and existing voice control systems are not designed for technical language used in laboratory settings.
A laboratory system with a portable device equipped with a microphone that uses voice input to control laboratory devices and software modules, incorporating a speech-to-text conversion system and correction software to accurately convert technical language, allowing hands-free operation and seamless interaction with laboratory equipment.
Enables efficient, contamination-free control of laboratory equipment and software modules by converting voice commands into accurate technical language, allowing users to move freely within the laboratory and interact with various systems without manual intervention.
Smart Images

Figure 0007586405000001 
Figure 0007586405000002 
Figure 0007586405000003
Abstract
Description
[Technical field]
[0001] The present invention relates to the control of laboratory equipment, and in particular to the control of complex laboratory systems. [Background technology]
[0002] In chemical laboratories, due to the various hazards from substances and equipment, various rules are applied to ensure safe work there. Thus, depending on the type of laboratory, the activities carried out there, and the substances used, safety regulations may include the following: the obligation to wear personal protective equipment, which in addition to a laboratory coat may also include protective glasses or a protective mask and protective gloves. As a rule, food and beverages must not be brought in or consumed, and also, to avoid contamination, the office area with desks, manuals, product documentation in paper form, computer workstations and Internet access, and the laboratory work area are physically separated from each other. The physical separation may be stipulated that it is only possible to transfer between the office area and the laboratory area through a security gate. It may also be mandatory to remove the safety clothing when leaving the laboratory area.
[0003] Safety measures make the workflow complicated, sometimes very complicated. Laboratory equipment, especially complex laboratory systems, comprise several laboratory devices and offer the user their own user interface, which often consists of a screen, a mouse and a keyboard, and / or a touch screen. These are components of standard equipment that are often difficult or impossible to operate with laboratory gloves. There is an increased occurrence of incorrect inputs, which slows down the workflow and can lead to incorrect control and operation of the laboratory equipment. Removing gloves before operating the laboratory system involves the risk of contamination, slows down the work, and is sometimes simply not possible for safety reasons.
[0004] In individual cases, some laboratory equipment has particularly large keyboards, for example in the form of large touch screens, to facilitate input with gloves. However, this special hardware is expensive and is not available for all laboratory equipment. In particular, standard computers and standard notebooks do not have such "glove-compatible" keyboards. However, the ability to also operate a standard computer with a conventional Internet browser, for example in a laboratory situation, to be able to study chemical databases available online, is becoming increasingly important.
[0005] Currently available options for controlling or interacting with laboratory equipment and systems are very limited and inefficient in the context of a chemistry or biology laboratory. Summary of the Invention
[0006] The object of the present invention is to provide an improved laboratory system and method for controlling at least one laboratory device or laboratory software module according to the independent claims, which allows for improved control of hardware and software functions in a laboratory context, in particular in a laboratory context. Embodiments of the invention are set out in the dependent claims. The embodiments of the invention can be freely combined with one another, if they are not mutually exclusive.
[0007] In one aspect, the invention relates to a laboratory system. The laboratory system includes at least one laboratory device designed to analyze and / or synthesize chemicals. The laboratory system further includes at least one laboratory software module designed to process data obtained from the at least one laboratory device. The laboratory system further includes a data processing device having control software that provides an interface for operating the at least one laboratory device and / or the at least one laboratory software module. The laboratory system further includes a portable device having a microphone. The device is interoperably connected to and designed for interoperation with the control software via a network to enable a user to operate the at least one laboratory device and / or the at least one laboratory software module by speaking into the microphone without manual intervention.
[0008] Embodiments of the present invention are advantageous because they allow for facilitated and improved control and management of laboratory equipment by one or more personnel, for example in biology and chemistry laboratories. Because control of laboratory equipment and associated laboratory software modules is via voice input, there is no need to remove gloves or take time-consuming measures to input control commands. Voice-based input allows information to be input anywhere within the acoustic range of the portable device, particularly within the laboratory area. Thus, personnel providing voice input do not need to leave their laboratory workstation or can move freely between different workstations within the laboratory area as long as they do not leave the microphone reception range of the portable device.
[0009] Control of target systems by voice input is known per se. The market offers low-priced terminal devices with microphones and powerful applications for voice-based input of commands, such as the Echo Dot from Alexa (Amazon®), smart speakers from various manufacturers with Cortana (Microsoft®), Google Home® Max / Mini from Google Assistant (Google®), and HomePod® from Siri (Apple®). However, these are designed to support the end user's daily activities, such as shopping, selecting radio programs, or booking hotels. The terminal devices and applications are therefore designed for everyday situations and to support only standard languages. Existing terminal devices are not designed or suitable for laboratory use or for the recognition of technical words. Moreover, currently available terminal devices are not designed to be subsequently modified by the user. Therefore, users have no possibility to use the currently available SmartSpeaker systems with voice recognition in a laboratory setting. According to an embodiment of the present invention, a control software has been developed that has the ability to correctly convert voice input, including technical language, into text, either alone or in interaction with other components, and a portable device with a microphone that can interoperate with the control software. In this way, a laboratory system that allows voice-based control of laboratory equipment and associated laboratory software modules is provided in a similarly simple manner and without manual intervention, as is possible today in the field of smart homes, for example on the basis of Amazon's Echo Dot®.
[0010] According to an embodiment, the control software is designed as a virtual laboratory assistant for the operation of at least one laboratory device and at least one laboratory software module.
[0011] This may be advantageous as it further facilitates the input of control commands by the user and the interaction between the user and the laboratory device or laboratory software module. For example, the virtual laboratory assistant may implement one or more voice generators, each of which generates a computer-generated, but natural-sounding voice, and reads out to the user in a natural voice the result of the performance of a particular function by the laboratory device or laboratory software module. It may also be possible for the virtual laboratory assistant to "listen" for a particular name, i.e. to look at the voice input or the text created from the voice input to see if that name is represented, and if so, to perform a particular function itself or have it performed by the target system. The laboratory researcher may thus interact with the control software and the virtual laboratory assistant implemented thereby in a manner similar to that which he would do with his human colleagues.
[0012] According to an embodiment, the handheld device is configured to receive audio signals from a user via a microphone and forward the received audio signals to the control software to enable conversion of the audio signals into text by the control software or by a speech-to-text system operably connected to the control software.
[0013] According to an embodiment of the invention, the portable device further comprises a speaker. The portable device is configured to receive from the control software a result of the execution of a function in response to transmission of an audio signal, the function being performed by the at least one laboratory device and / or by the at least one laboratory software module by text. The portable device is further configured to output the received result to a user via the speaker.
[0014] For example, the received result may be text that is converted to an audible signal via the device's text-to-speech software. According to an alternative embodiment, the control software includes text-to-speech functionality and the result is already sent to the portable device in the form of an electronic audible signal, so that in some embodiments the terminal device, which has only limited computing power, no longer needs to perform the text-to-speech conversion.
[0015] The output of results as acoustic signals via a speaker can be advantageous as it does not restrict the freedom of movement of the user, which is achieved by voice-based input of commands. The user can therefore move freely from any point in the laboratory within the detection range of the microphone and within the audibility of the acoustic output of the speaker, thereby both issuing control commands and research requests by voice input, and receiving the results of the execution of these commands and / or information retrieval. This is particularly useful in the context of laboratory work, since laboratory personnel frequently need to switch between different equipment and laboratory areas, for example to reload or reset machines, check the progress of process steps, open blocked components or transport boxes, and the like.
[0016] According to some embodiments, the results of the software or hardware functions implemented by the laboratory software modules or laboratory devices through voice input are displayed to the user via a screen. This can be in addition to or instead of output via a speaker. In particular in cases of complex and far-reaching results, such as documents determined in the course of a database or internet query, output on a screen can be advantageous, since with output only via a speaker, the user may not be able to accept and understand all the information of a long text.
[0017] According to an embodiment, the portable device is located in the same room as at least one laboratory device and / or is located in the same room as a main control computer of a laboratory facility that includes at least one laboratory device.
[0018] This can be advantageous because most types of microphones, and particularly those on portable devices, have the capability to record audio input in the room and / or the volume of most common speakers, and particularly those on portable devices, is sufficient to be heard and understood by personnel in the same room.
[0019] According to some embodiments, the laboratory system comprises a portable device having two or more microphones as described herein and an optional speaker. These devices are preferably evenly distributed and located in the laboratory. This may be advantageous as it allows good coverage for both receiving audio input and outputting results via the speaker.
[0020] According to an embodiment of the present invention, the portable device is a single board computer, in particular a Raspberry Pi computer.
[0021] This can be advantageous because such single-board computers are low-cost devices designed to be adapted and modified, for example by end users, by installing over a network client software, e.g., control software, that is interoperable with freely definable server-side software, to enable users to use the functionality of the control software via voice input.
[0022] For example, the "Google Assistant SDK" allows Google's voice recognition service to be integrated into end devices such as the Raspberry Pi, and numerous video tutorials about this can be found on Youtube®.
[0023] However, in accordance with an embodiment of the present invention, the portable device is not directly connected to the Google service or other speech-to-text services, but instead only client software is installed which enables the portable device to interoperate with the networked speech-to-text conversion system and controlling software which coordinates the exchange of data with the networked speech-to-text conversion system and corrections of text received from the conversion system.
[0024] For example, the software "Google Assistant on Raspberry Pi" can be installed on the Raspberry Pi and configured to transmit the voice signals received by the portable device to the control software, the address of which is thus predefined and stored in the portable device. This can be advantageous, since it provides a portable and very cheap terminal device for the purpose of facilitating the interaction and service of data processing devices in a laboratory, which terminal device can be configured to interoperate with any software application, for example a control software implementing a virtual laboratory assistant.
[0025] Since the Raspberry Pi computer has only low computing power, it may be advantageous to outsource this coordination function to the control software, since the coordination of the conversion of voice signals to text, as well as any subsequent text correction, can therefore be performed more quickly by other, more powerful devices.
[0026] According to an embodiment of the present invention, the control software is configured to receive a voice signal picked up by the microphone based on a voice input from the portable device. The voice signal includes standard language and technical language spoken by the user. The control software is further configured to input the received voice signal into a speech-to-text system, which only supports conversion of the voice signal into a target vocabulary that does not include technical language. The control software then receives from the speech-to-text system a text created by the speech-to-text system on the basis of the voice signal. The control software is further configured to create a corrected text from the received text, the corrected text being created by the control software alone or in interpretation with text correction software operably connected to the control software locally or via a network. The corrected text is created by automatically replacing words and phrases of the target vocabulary in the received text with technical language according to an assignment table of words in text form. The assignment table assigns at least one word from the target vocabulary to each of a plurality of technical languages. At least one word of the target vocabulary assigned to the technical language is a word or phrase that was incorrectly recognized by the speech-to-text system when the technical language was input in the form of a speech signal. The control software is configured to use the corrected text to cause the at least one laboratory device and / or the at least one laboratory software module to analyze, synthesize, and / or perform a software function as specified in the corrected text.
[0027] The use of a speech-to-text system that does not support technical words, combined with subsequent correction of the text based on an assignment table, can be beneficial for several reasons.
[0028] On the one hand, a number of common language speech-to-text systems already exist, e.g. Google's "Speech to Text", which have at least a high degree of accuracy for common language words and phrases, and which can be used partly free of charge and integrated into proprietary software programs via an open API. Voice input in laboratory situations usually contains a mixture of technical words and common language words, so that at least the correct conversion of common language words into text is guaranteed. Google and other cloud service provider speech-to-text systems are under continuous development, a service that manufacturers of laboratory equipment and laboratory management systems usually cannot offer with the same quality as the major cloud service providers such as Google, Amazon, and Apple.
[0029] The general language speech-to-text system is, however, not suitable for use in a laboratory context, since technical language ("technical words") are not supported by the speech-to-text system. In the biology and especially chemical industries, a large number of technical words are used in laboratory contexts. This does not occur in general language. These are not recognized by speech-to-text systems, so instead words in general language with similar acoustics are incorrectly recognized and converted to text. However, especially in chemical laboratory contexts, high accuracy of speech recognition is particularly important. In everyday language, small mistakes are often recognizable as such and can be easily corrected or compensated for by the user or receiver system (e.g., incorrect recognition of singular / plural does not result in corresponding entries returning significantly different results in different Internet search engines), but in the context of chemical synthesis, even minimal deviations (e.g., "bis" instead of "tris") can lead to a substance being "recognized" that is completely different from what the speaker actually meant, leading to the resulting product being unusable. This speech-to-text system for everyday use is therefore not suitable for use in biology and chemistry laboratories.
[0030] Some of the existing speech-to-text systems are specifically designed for the needs and vocabulary of a particular professional field. For example, Nuance offers software for lawyers called "Dragon Legal," which includes legal terminology in addition to the vocabulary of everyday language. However, the vocabulary required in a particular laboratory (e.g., in the field of paint and varnish manufacturing and analysis) is very specific, and even speech recognition software with chemical terminology that may be taken from a standard chemistry textbook, for example, has the drawback of being highly variable and, in many cases, often not suited to the practice of a particular company or a particular branch of the chemical industry, since laboratories work with trade names for substances. These trade names may change, or many new trade names are added each year for the products involved. In particular, many additional products and product variations that can be used for the manufacture of paints and varnishes appear on the market each year under new trade names. Even if there was a language-to-text conversion system that approached the accuracy of Google's or Apple's everyday language systems and included most important chemical terms (which is not the case in this case), this system would not be very suitable for practical use since, due to the dynamics and large number of names that play a role in chemical laboratories, especially in the manufacture of paints and varnishes, most words relevant in practice would not be supported, or at least the vocabulary would become completely outdated after a few years.
[0031] According to an embodiment of the invention, this problem is solved as mentioned above by using a speech-to-text conversion system that is known not to support the relevant technical terms. Hence, here we do not try from the outset to carry out expensive and complex specific developments that would only cater to a very small market and would therefore almost certainly not achieve the recognition accuracy of known leading conversion systems (from Amazon®, Google®, Microsoft® Azure® or Apple®) as far as the voice input involves general language terms, which in addition to technical chemical terms also need to be taken into account and correctly recognized. Instead, an embodiment of the invention exploits the already very good recognition accuracy of existing service providers for general language terms and performs corrections before outputting the recognized text. In the course of correction, incorrectly recognized words are replaced by technical words on the basis of the allocation table, so that the corrected text that is finally output is generated. The highly specific technical vocabulary, which has to be continuously updated due to the dynamics of the field and the large number of market participants, products and product names in order to keep the software suitable for practical use, is finally accommodated in the allocation table. This can be kept up to date with very little effort. Modification of the program source code of the statistical language model, recompilation, and / or retraining of the machine learning program is therefore not necessary according to embodiments of the present invention: modifying or adding tables is sufficient.
[0032] A new technical word can be added by simply adding it to the allocation table together with one or more words of the target vocabulary that were incorrectly recognized for that technical word in each case. From a technical point of view, the storage and update of technical words is therefore completely decoupled from the logic of the actual speech recognition. This also has the advantage of avoiding dependency on a specific provider of speech recognition services. The field of speech recognition is still young and it is not possible to predict which of the many parallel solutions will be the best choice in the long run in terms of recognition accuracy and / or price. According to an embodiment of the invention, the connection to a specific speech-to-text conversion system is only established upon the first transmission of the received speech signal to this conversion system and upon receipt of the (incorrect) text. In addition, the allocation table contains the incorrectly recognized words of the target vocabulary that were (incorrectly) returned by this specific conversion system for a specific technical term. However, both of these can be easily modified by using a different speech-to-text conversion system to create the (incorrect) text, and for this purpose also regenerate the allocation table using the other conversion system. For example, no complex modifications to the logic of the syntax parser and / or the neural network are necessary.
[0033] The combination of an everyday language speech-to-text system according to an embodiment of the present invention may also be advantageous for sales agents in the chemical industry or chemical production, since these people often use computers or at least smartphones in the course of their work anyway and are less likely to be distracted from customers or their own work by voice input into a correction software designed as an app or browser plug-in than by text input via a keyboard. These users may also be given access to or control over the functions of laboratory equipment and / or laboratory software modules of the laboratory system via client software on their terminal devices, which client software is interoperable with the control software, in order to send voice signals from these users to the speech-to-text system via the control software.
[0034] Another advantage is that the portable device according to the embodiment can only receive the voice signal, send it to the control software, and optionally receive and output the result of the execution of the function according to the specification of the voice input. The actual speech-to-text conversion of the voice signal to text, which is certainly the most computationally intensive step, is performed by a speech-to-text conversion system. The speech-to-text conversion system can be, for example, a server connected to the control software via a network (for example, the Internet). Therefore, portable devices with low processing power, for example, smartphones or single-board computers, can also be used for inputting long and complex voice inputs, and the resources of the computer on which the control software is installed are also saved.
[0035] According to an embodiment of the present invention, technical words are words from one of the following categories: - names of chemical substances, in particular paints and varnishes, in particular chemical names according to chemical naming conventions, e.g. IUPAC nomenclature; -The physical, chemical, mechanical, optical, or tactile properties of a chemical substance; - laboratory equipment and equipment from the chemical industry (e.g. trade names or unique names assigned by users to laboratory equipment in laboratories); - names of laboratory consumables and supplies, - Trade name in the paints and varnishes sector.
[0036] According to an embodiment of the present invention, the technical words are words from the field of chemistry, in particular from the chemical industry, in particular from paint and varnish chemistry.
[0037] According to an embodiment of the present invention, the target vocabulary consists of a set of words in a common language.
[0038] According to another embodiment of the invention, the target vocabulary consists not only of a set of general language words, but also of words derived from them. These derivatives can be, for example, dynamically created concatenations of two or more general language words. In the German language, for example, many words, especially nouns, are formed by combining several other nouns. For example, the German word "Schiffsschraube" (ship's screw) is very common and usually appears in most German general language dictionaries. In contrast, somewhat less commonly used German terms such as "Besfestigungsschraube" (fastening screw) are not found in most German general language dictionaries. However, some speech-to-text systems, by means of heuristics and / or neural networks, can also recognize words like "Besfestigungsschraube" provided that the individual word components "Besfestigung" and "Schraube" are part of the target vocabulary. In this sense, the word "Besfestigungsschraube" therefore also belongs to the target vocabulary of this type of speech-to-text system.
[0039] According to another embodiment of the invention, the target vocabulary consists of a set of words of a common language, supplemented by words formed by combining the syllables to be recognized. These speech-to-text systems are therefore more flexible in terms of the words they can recognize, since recognition can be performed not only at the level of individual words, but at the level of individual syllables (at least at the level of individual syllables as well). However, syllable-based recognition is also particularly prone to errors, since the risk of incorrectly recognizing words that are not in any known vocabulary is particularly high. Due to the finiteness of the set of supported or known syllables, and the limitations of the set of syllables that can be combined due to the length of typical words, the set of target words that can be created based on syllables is also finite. Therefore, even speech-to-text systems that support the creation of words based on syllables have a finite target vocabulary, despite their greater flexibility. Even in such systems, due to their flexibility, many chemical terms that in theory can be dynamically recognized are not in the pre-known dictionary, and the accuracy of recognition in practice is very low for practical use, and even these systems that ultimately have a target vocabulary do not include or support these chemical terms.
[0040] In some embodiments of the invention, the target vocabulary comprises a set of words of the general language, supplemented by words derived from them and by words formed by combining recognized syllables. These conversion systems are also based on a target vocabulary that does not accommodate technical words or cannot recognize them with sufficient accuracy for practical use, but instead incorrectly recognize other words, typically words of the general language, and convert them into text.
[0041] Therefore, the various different speech-to-text systems already available today can be used with embodiments of the present invention, even though these systems "support" substantially only everyday languages (i.e., can correctly recognize and convert them to text with sufficient accuracy).
[0042] According to an embodiment of the invention, a computer system performing text correction, e.g., a computer having control software or a correction computer, receives or calculates frequency information for at least some words in the text produced by the speech-to-text system from the audio signal, the frequency information indicating, for each word in the text, statistically how frequently the word is expected to occur in the text.
[0043] When producing the corrected text, only those words of the target vocabulary in the received text whose statistically expected frequency of occurrence according to the received frequency information is lower than a predefined threshold are selectively replaced by technical language in accordance with the assignment table.
[0044] This may be advantageous since the speech input from the user usually contains a mixture of general language words and technical words. Hence, the text received by the conversion system may contain words of the target vocabulary that are assigned to technical words in the assignment table and that would normally be replaced. For example, the returned text may contain the phrase "polymer innovation". Since the term "polymer innovation" is assigned to the technical word "polymerization" in the assignment table, this term would normally be replaced by "polymerization" during the text correction process. However, if the term "polymer innovation" is assigned a frequency that represents a high probability of occurrence, the correction software will assume that the term "polymer innovation" is correct based on the frequency of this occurrence, although it is assigned to the technical word in the assignment table, and will consequently leave the term "polymer innovation" unchanged in the text. For example, a contextual analysis of the words in a sentence, or in the entire speech input, may reveal that the word "innovation" occurs frequently alone in the text, for example because the text comes from a sales representative describing the benefits of a particular polymer product. In this context, the term "polymer innovation" may be a correctly recognized phrase. In contexts where neither "polymer" nor "innovation" is mentioned alone, the probability decreases. Words also have different probabilities of occurrence independent of context, which may be derived, for example, from a large text corpus.
[0045] Replacing words by an assignment table depending on the probability of occurrence of the word in the received text can be advantageous since it avoids situations where, in a few individual cases, a target language word which has a high probability of occurrence on its own or in the context of the text in question is mistakenly replaced by a technical word and an error is introduced instead of being corrected by the replacement.
[0046] According to one embodiment, the frequency of occurrence of words in the text is calculated by the speech-to-text system and returned to the control software together with the text from the speech-to-text system. For example, the speech-to-text system may use a hidden Markov model (HMM) to calculate the probability of occurrence of a particular word in the context of a sentence. Additionally or alternatively, the speech-to-text system can equate the frequency of occurrence of a word with the frequency of occurrence of the word in a large reference corpus. For example, the entirety of newspaper texts over several years, or some other large text data set, can serve as a reference corpus. The ratio of the number of counted words in the corpus to the total number of words in the corpus is the frequency of occurrence of this word observed in this reference corpus. The frequency data can be sent from the speech-to-text system to the control software together with the text.
[0047] According to a further embodiment, the frequency of occurrence of the words of the text is calculated by the control software or the correction software after receiving the text. As already described previously, the calculation of the probability of occurrence of individual words or phrases can be performed by HMM, taking into account the text context of the words or using the frequency of the words in a reference corpus. The reference corpus can be, for example, the entire text previously sent by the speech-to-text system to the control software or the correction software.
[0048] Therefore, according to an embodiment, the calculation of the frequency data (e.g., by the control software or the correction software) is performed using a hidden Markov model. For example, the expected frequency of occurrence, i.e. the probability of occurrence, can be calculated as the product of the occurrence probabilities of individual words in a sequence of words, as described, for example, in B. Cestnik, "Estimating probabilities: A crucial task in machine learning," In Proceedings of the Ninth European Conference on Artificial Intelligence, pages 147-150, Stockholm, Sweden, 1990.
[0049] According to an embodiment of the invention, for at least some of the words in the text generated from the speech signal by the speech-to-text system, in addition to the text, the program performing the text correction also receives part-of-speech tags (POS tags). The POS tags are received by the speech-to-text system directly from the control software or from the correction software via the control software, and include at least noun tags, adjective tags and verb tags. It is also possible that the POS tags include additional types of syntactic or semantic tags. The exact composition of the POS tags considered may also depend on the language. In the assignment table, technical language is stored in conjunction with the POS tags. When creating the corrected text, only those words in the received text whose POS tags match the target vocabulary are replaced with technical language by the assignment table.
[0050] This may be advantageous since it increases the accuracy of the text correction process. The correctness of the POS tags in the assignment table can be assumed, since the entries in the table are created semi-automatically by one or more speakers who input words or phrases of technical language into a microphone, the resulting audible signal is converted by the speech-to-text system into a (wrong) word or (wrong) phrase of the target vocabulary, and this wrong word or this wrong phrase is stored in the assignment table in association with the phrase of technical language. Since it is known what the term of technical language stands for, and whether it is, for example, a noun, a verb or an adjective, the term of technical language may also be stored in association with the correct POS tag at the same time that the table is created or updated. Thus, if the assignment table requires the replacement of certain words and phrases in the text by technical language, but the POS tag of the text to be replaced does not match the POS tag of the technical language, this is an indication that the corresponding word in the text may be correct after all. The recognition rate of POS tags is relatively high, so this measure can improve the quality of the correction process. For example, a technical word can be the trade name "Platilon®", which designates a thermoplastic polyurethane film manufactured by Covestro. In the table, the POS tag "noun" is assigned to this technical word. It is known that speech-to-text systems often incorrectly convert the spoken word "Platilon" to the target vocabulary word "platinum". The target vocabulary word "platinum" is therefore assigned to the technical word "Platilon" in the assignment table. However, in a recent voice input by a user, this word was used adjectivally: "The addition of a platinum-based catalyst or a zinc-based catalyst [...]". Based on the POS tag of "platinum" in the text returned by the conversion system, it may be possible to recognize that the word "platinum" is correct here and should not be replaced by "Platilon".
[0051] According to an embodiment, the control software is provided as a cloud service from a cloud computer system or a single server computer system, which may be, for example, a system operated by an operator in a laboratory (e.g., a university or a company).
[0052] According to one embodiment, the control software performs the text corrections either by itself or in interaction with a program installed locally on the same computer as the control software.
[0053] According to another embodiment, the correction of the text received from the speech-to-text system is performed by another computer connected to the computer of the control software via a network. The other computer may be, for example, a cloud system or a single server (for example, also operated by a laboratory operator). The control software transmits the received text to the software and computer by which the correction is performed on the basis of the allocation table, and receives the corrected text from this software or computer ("correction software" or "correction computer"). The network connecting the control software to the remote correction software may be, for example, the Internet or an organization's intranet. This embodiment may be advantageous, since it allows a better separation of the functions and data access rights of the control software on the one hand, and the correction software on the other hand. If the text correction is performed on a separate correction computer or correction cloud system, the user may grant selective access to the correction computer or correction cloud system for the purpose of updating the table, without having to grant the control software access to sensitive data (for example, which may control sensitive functions of laboratory equipment).
[0054] Therefore, according to embodiments of the present invention, the coordination of data exchange with the speech-to-text system, the text correction and the transmission of the corrected text to the execution system is fully performed or organized and coordinated by the control software. Therefore, according to some method embodiments, the portable device is essentially a device with a microphone and an optional output interface for the result of the execution of the corrected text.
[0055] According to an embodiment of the present invention, the speech-to-text system is implemented as a service provided over the Internet to multiple terminal devices. For example, the speech-to-text system can be a cloud computing system using Google's speech-to-text cloud service. This can be advantageous since functionally powerful API client libraries (e.g. for .NET) are available for this.
[0056] The portable device may include pre-configured client software for exchanging data with the control software, i.e., the client software on the portable device is configured to transmit voice signals over the network to the control software so that the control software can effect conversion of the voice signals to text, technical language correction of the text, and execution of a function with the corrected text. Results of the execution of the corrected text can be received by the portable device in response to the transmission of the voice signals and can be output through a speaker.
[0057] According to an embodiment of the present invention, the target language words and phrases stored in the assignment table represent erroneous text outputs of a speech-to-text system created based on speech input of technical words by a number of different individuals.
[0058] According to an embodiment of the invention, the method comprises a step of creating an allocation table. For each of a plurality of technical languages, at least one reference speech signal is recorded, which selectively represents the technical language. The reference speech signal is emitted by at least one speaker. For phrases of technical language, at least one reference speech signal selectively reproducing the phrase of technical language can also be spoken and recorded in each case by at least one speaker. The further steps are substantially identical for words and phrases, so that in the following, when technical language is mentioned, phrases of technical language are included. Each of the recorded reference speech signals is an input for a speech-to-text system. The input may in particular be made via a network (e.g. the Internet). For each input reference speech signal, a device for inputting the reference signal receives at least one word of the target vocabulary created by the speech-to-text system from the input reference speech signal. This device may for example be a microphone and a connection to any data processing system with speech-to-text conversion. Preferably, the reference speech signal is input via a device as similar as possible to the handheld device, in terms of its microphone type and / or its positioning relative to the noise source to ensure that the same errors occur reproducibly. Since the target vocabulary of the speech-to-text system does not support technical languages, at least one word (which may be a phrase) of the target vocabulary received for each of the technical languages represents an error conversion. Finally, an assignment table is created which assigns, for each of the technical languages for which at least one reference speech signal has been recorded, at least one word of the target vocabulary in text form returned by the speech-to-text system for the technical language.
[0059] This can be advantageous, since the table can be modified and supplemented very easily, without the need for source code modifications, program recompilation, or neural network retraining. Even if another speech-to-text system is used, only the appropriate client interface needs to be adapted, and the technical language phrases of the table are re-entered via microphone by one or more speakers and sent to the new speech-to-text system. The incorrect words and phrases in the target language are returned by this new system and form the basis for a new allocation table. It is therefore possible to functionally extend any everyday language speech-to-text system in such a way that spoken texts containing technical language words and phrases are also correctly converted to text, without extensive or complex modifications and without retraining the speech software.
[0060] The assignment table may be stored, for example, as a table in a relational database, or as tab-delimited text, or as another functionally comparable data structure.
[0061] According to an embodiment of the invention, for at least a portion of each of the technical words (or phrases of the technical language), a number of reference speech signals, each from a different speaker, are recorded. The reference speech signals reproduce the technical words (or phrases). An assignment table assigns each of the at least a portion of the technical words (or phrases) to a number of words (or phrases) of a target vocabulary in text form. The words (or phrases) of the target vocabulary represent miscommunications for the different speakers by the voices generated by the speech-to-text system.
[0062] For example, a specific technical word such as "1,2 methylenedioxybenzene" can be read aloud by 100 different people, each of whom can be recorded with a microphone as a reference speech signal. Preferably, these people are people who are familiar with the pronunciation of chemical phrases. Thus, there are 100 reference speech signals for this one substance name. Each of these 100 reference speech signals is sent to a speech-to-text system, which returns in response 100 words or phrases of the target vocabulary, not all of which represent the actual technical language name. Often, the 100 returned words are identical, but this is not always the case. Different people have different voices, i.e., the speech input differs in terms of emphasis, volume, pitch, and pronunciation. Thus, for a given technical language (or phrase), a given speech-to-text system may return several different, incorrect words or phrases, all of which will be included in the assignment table.
[0063] Considering the voice inputs of many different people for the creation of the allocation table can be advantageous since it can better take into account the variability of human voices and therefore achieve improved error correction rates.
[0064] According to an embodiment of the present invention, the at least one laboratory device is a plurality of laboratory devices, each selected from the group consisting of: - analytical stations designed to analyse chemical substances or mixtures of substances, and / or a synthesis station designed to carry out the synthesis of chemical substances or mixtures of substances, and / or - A pre- or post-processing station designed to purify, combine with other substances, dilute, concentrate, or otherwise modify a chemical substance before or after an analytical or synthetic step to allow for a subsequent processing or transport step.
[0065] According to an embodiment of the present invention, the at least one laboratory apparatus is a plurality of laboratory apparatuses, each of which serves as a processing station for chemical substances. The laboratory apparatus and optionally the at least one laboratory software module may be part of a laboratory facility that additionally also comprises a transport unit. The transport unit is designed to transport substances used for synthesis or analysis to the processing station, in order to enable the processing station to subject the substances to at least one processing step.
[0066] According to an embodiment of the present invention, the laboratory system further comprises a master control computer of a laboratory facility comprising at least one laboratory device, the master control computer including control software configured to coordinate process steps and chemical delivery by the laboratory devices of the laboratory facility based on instructions in the form of structured text.
[0067] According to an embodiment of the invention, the control software comprises an NLP processor (NLP = Natural Language Processing) or is interoperable with an NLP processor connected via a network. Here, for a software application that is interoperable with another software application, this means that the interfaces and routines of the two interoperable software applications are coordinated and adapted to each other, so that both can exchange requests, control commands and data in a coordinated manner, so that through this coordinated exchange, a specific software-based function is performed and the result is returned to the requesting software application. The control software is configured to use the NLP processor to transform the corrected text into a structured text that can be interpreted by a target system. The target system is at least one laboratory software module, and / or at least one laboratory device, and / or the control software of the main control computer of a laboratory facility that comprises at least one laboratory device. The control software is configured to receive from the NLP processor and input the structured text formed on the basis of the text corrected by the NLP processor into the target system. Inputting the structured text into the target system causes the target system to perform laboratory-related software and / or hardware functions.
[0068] For example, M. Hummel, D. Porcincula, and E. Sapper, in the article "NATURAL LANGUAGE PROCESSING. A semantic framework for coatings science-robots reading formulations" in the European Coatings Journal (01.02.2019), describe the transformation of natural language text input into structured text using syntactic parsers, POS tagging, and other techniques.
[0069] For example, the target system may be at least one laboratory apparatus and a transport unit that is prompted by the input of structured text to perform multiple process steps and transport of chemicals between the laboratory apparatuses according to the information contained within the structured text.
[0070] According to another embodiment, the target system is the control software of a laboratory facility, and the input of the structured text prompts it to perform software functions according to the information contained in the structured text, such as the calculation of certain concentrations or other parameters used in the course of performing certain analyses and / or syntheses defined in the text, for example to configure or calibrate laboratory equipment accordingly, or to adjust pH values and viscosities.
[0071] According to another embodiment, the target system is a chemical database of a laboratory facility, containing the results of analyses and syntheses already performed by the facility. The input of structured text causes it to perform a database search for the information and terms contained within the structured text. The search results can be returned directly to the user via the control software as a result, or may include input for further data processing steps, for example for simulation and prediction of advantageous formulations based on the searched substances.
[0072] According to an embodiment, the laboratory system comprises a chemical facility for the analysis and / or synthesis of chemical substances and / or a facility for the manufacture of substance mixtures, in particular paints and varnishes. The system also comprises a main control computer with a control software. At least one laboratory device and at least one laboratory software module are part of a high-throughput facility and may also be controlled and managed by and through the control software. The facility is designed to interpret the corrected text as a specification of a synthesis or as a specification of the constituents of a substance mixture or as a specification of an analysis to be performed.
[0073] The facility may be a high throughput facility (HTE facility), such as a high throughput facility for the analysis and manufacture of paints and varnishes. For example, the HTE facility may be a system for automated testing and manufacture of chemical products as described in WO 2017 / 072351 A2.
[0074] The corrected text is converted into a control software of the facility, where the output of the corrected or structured text causes the control software to control one or more laboratory devices and / or one or more laboratory software modules of the facility as specified in the corrected text. This may be particularly beneficial in the context of a biology or chemistry laboratory, since the voice input is processed in such a way that it can be directly transferred to and correctly interpreted by the technical system, for example, without the user having to remove gloves for this purpose. For example, the laboratory device may be a device for determining the oil value of a pigment, or a device for determining the viscosity of a varnish.
[0075] The facility may be designed to carry out one or more of the following processing steps fully automatically, for example in response to input of corrected text or structured text obtained therefrom via a machine-machine interface: - rheological analysis of substances and substance mixtures; - Determination of the storage stability of substances and substance mixtures, in particular on the basis of the inhomogeneity and the tendency to sedimentation of liquid substance mixtures, for example these analyses can be carried out using optical measurements in cuvettes after sampling, - Determination of the pH value of substances and substance mixtures, - Foam testing of substances and substance mixtures, in particular the determination of the defoaming effect and the determination of the foam breakup kinetics, - viscosity measurement of substances and substance mixtures, which can include an automatic dilution step, since the viscosity of a diluted solution is easier to determine, especially in the case of highly viscous substances or mixtures, and the viscosity of the original substance or substance mixture is calculated on the basis of the diluted solution; - determination of the scraping behavior of substances or substance mixtures, in particular of finished products (abrasion tests), - Determination of the color values of substances and substance mixtures using spectrophotometers working, for example, with light scattering (so-called LAB values), haze and gloss, -Layer thickness measurement of substances and substance mixtures applied to surfaces under various defined parameters (temperature, humidity, surface condition of the surface, etc.); - Image analysis methods of images of substances and substance mixtures, in particular for characterizing the substance surface, for example the number, size and distribution of air bubbles, scratches, in paints and varnishes.
[0076] The substances and mixtures of substances may in particular be substances and mixtures of substances used for the production of paints and varnishes. In addition, the substances and mixtures of substances may not only be final products, such as paints and varnishes in liquid or dry form, but also intermediate products and solvents used.
[0077] According to an embodiment of the present invention, the operation without manual intervention includes converting the voice input into text, in particular, the conversion into natural language text.
[0078] At least one laboratory software module may be designed, for example, as a chemical database designed to interpret the input text as a search input, determine information about the search input in a database, and return this as a result to the control software. The database, in particular a database management system (DBMS), may be, for example, MySQL or PostgreSQL.
[0079] Additionally or alternatively, the laboratory system may include a laboratory software module, which is a simulation software. The simulation software is designed to simulate the properties of chemical products, in particular varnishes and paints, based on a given substance mixture specification. The simulation software is designed to interpret input text as a specification of the product whose properties are to be simulated.
[0080] Additionally or alternatively, the laboratory system may include a laboratory software module configured as a control software of a laboratory facility including at least one laboratory device. The control software is installed on a main control computer. For example, it may be the control software already described herein for the embodiments. The control software is designed to prompt the laboratory devices and transport units of the laboratory facility to perform and coordinate a number of process steps and transport of chemicals between the laboratory devices according to the information contained in the input text. The laboratory facility may be designed as a HTE facility for controlling chemical analyses and / or synthesis and / or for producing substance mixtures, in particular paints and varnishes. The control software is designed to interpret the corrected text or the structured text extracted therefrom as a specification of the components of the analysis or synthesis or of the substance mixture.
[0081] Additionally or alternatively, the laboratory system may include laboratory software modules designed to perform simple data processing steps of data exchanged between components of the laboratory system in an analytical or synthetic context, which may be, for example, the translation, checking, and / or addition of control commands from the control software to the laboratory equipment, the storage, checking, or normalization of measurement data, or similar data processing steps.
[0082] The text input to the laboratory device or laboratory software module in each case may be corrected text or structured text created by an NLP processor from the corrected text.
[0083] According to an embodiment, the control software is configured to use at least some of the corrected text, e.g., text including a particular keyword (such as "Internet search"), as input to an Internet search engine. The control software receives the results of the search, e.g., by parsing the browser window, and forwards the results for output to a target system near the user, e.g., a portable device having a speaker.
[0084] According to an embodiment of the present invention, the control software includes a registry of multiple user accounts registered with the control software. The control software includes a voice recognition function for recognizing each of the registered users by the user's voice. The control software is designed to allow a user, by speaking into a microphone, to operate, without manual intervention, only those laboratory equipment and / or laboratory software modules that the user has been authorized to operate.
[0085] This can be advantageous because access control and management of laboratory equipment and laboratory software modules are seamlessly interwoven by making both dependent on the user's voice, without the need to remember or securely store passwords or special access tokens.
[0086] According to an embodiment of the invention, the control software allows a user to select a laboratory device from a plurality of laboratory devices and selectively control its functions through voice input by entering keywords, in particular the name of the laboratory device and / or the name of the room, as part of the voice input. For example, the control software is configured to analyze the corrected text for predefined keywords. If a keyword is recognized, the control software transfers the corrected text, either as natural language text or structured text, accurately to the recognized laboratory device or its control software. This may be advantageous, since the same language-based interface may be used to control a variety of different laboratory devices.
[0087] According to an embodiment of the present invention, the control software is configured to automatically convert the corrected text created based on the user's voice input into a structure that is correctly interpreted by the target system intended for this text input. For example, the control software dynamically recognizes that the user, by stating in the voice input the name of a particular laboratory device or laboratory facility, has selected it as the target system from various available target systems to perform a particular function. For each of the target systems, the control system has a stored format required by the particular target system as an input format, and automatically transfers the corrected or structured text obtained therefrom into the required format of the dynamically recognized target system. For example, a first laboratory device may require certain parameters in the form of an XML file as input. Another may have an NLP processor and directly use the corrected text as input. Another laboratory device requires certain parameter values or formulas in the form of a table. This may be advantageous, since the user can enter the voice input in the normal syntax of a sentence in a natural language. Thus, the user does not have to laboriously memorize the required syntax of each individual laboratory device and to speak "differently" to each individual device. A special adaptation of the user's language structure to the vocabulary or syntax of a particular laboratory device is therefore not necessary: the user can enter his input in his familiar speech syntax, which he also uses in conversations with colleagues. The adaptation to the special requirements of the interface of the individual laboratory device is carried out by the control software.
[0088] In another aspect, the present invention relates to a method for controlling laboratory equipment and / or laboratory software modules, the method comprising: - providing at least one laboratory device and at least one laboratory software module, the at least one laboratory device being designed for analyzing and / or synthesizing chemical substances, the at least one laboratory software module being designed for processing data obtained from the at least one laboratory device; providing a data processing device having control software that provides a user with an interface for operating at least one laboratory device and / or at least one laboratory software module; providing a portable device having a microphone, interoperably connected to control software via a network and designed to enable a user, in interoperation with the control software, to operate at least one laboratory device and / or at least one laboratory software module by speaking into the microphone without manual intervention.
[0089] According to an embodiment, providing the device includes placing the portable device in the same space as at least one laboratory device and / or placing the portable device in the same space as a master control computer of a laboratory facility including at least one laboratory device.
[0090] "Structured text" is understood herein to be text, especially ASCII text, that contains words and phrases of particular meaning in predefined positions. Structured text can take the form of, for example, a comma-separated file, a table, an XML document, or the like. Structured text can be more easily interpreted and processed by machines and computers than natural language text.
[0091] "Natural language text" is understood here to be text, especially ASCII text, written in a language that is spoken by humans and that typically consists of complete sentences.
[0092] An "NLP processor" is a software program or function that is capable of understanding "natural language" text, i.e., processing it in such a way that relevant information contained in the text can be extracted from the text, and optionally transforming it into other formats, e.g., structured text formats. NLP tools are therefore capable of processing text consisting of complete sentences, as well as extracting information from single sentences.
[0093] A "transport unit" is understood here to be an automated system consisting of one or more components capable of transporting chemicals in liquid and / or solid form, and optionally also other objects such as consumables or containers, from one processing station to the next. The transport unit can be, for example, a robotic arm, a collection of robotic arms, a conveyor belt, a collection of conveyor belts, or a combination of these. The transport unit can be part of a high-throughput system and can be controlled by control software. The processing unit can be a laboratory device for the analysis or synthesis of chemicals, or a device for pre- and post-treatment of these substances (dilution, concentration, staining, mixing, etc.).
[0094] A "speech-to-text system" is a data processing system designed to convert speech signals from a person into text. A data processing system may, for example, consist of a combination of hardware and software, and may, for example, be a cloud-based or server-based computer system that provides speech-to-text services over a network to one or more client devices.
[0095] "Operation without manual intervention" of a device is operation that does not involve any manual interaction of the user with the device or the human-machine interface of the device to the part. It is therefore a "non-contact", "non-touch" operation.
[0096] A "single-board computer" or "single-board processor" is a computer system in which all electronic components required for operation are combined on a single printed circuit board. Typically, the power supply is the only separately housed component.
[0097] A "voice signal" is an electronic signal recorded by a microphone when a person speaks into the microphone.
[0098] A "virtual laboratory assistant" is software or software routines that are operatively connected to one or more laboratory devices and / or software programs located in a laboratory in such a manner that they can receive information from the laboratory devices and software programs and can send commands to perform functions from the laboratory assistant to the laboratory devices and software programs. The laboratory assistant thus has an interface for exchanging data with and controlling the one or more laboratory devices and software programs. The laboratory assistant also has an interface to a user, which is configured to allow the user to facilitate use, monitoring, and / or control via the interface of the laboratory devices and software programs. For example, the interface to the user can be an audio interface or a natural language text interface.
[0099] Preferably, the "virtual laboratory assistant" includes a function for performing a dialogue between the virtual assistant and a person that is as similar as possible to a dialogue between a person and a person. For example, the virtual laboratory assistant may have a software function for reading aloud text in a substantially natural voice. The text to be read aloud may include the result of the execution of a function, for example, output through a speaker of a portable device. Additionally or alternatively, the virtual laboratory assistant may be configured to search for keywords in the received voice signal or text created therefrom, and to call a specific hardware or software function in response to the keyword. For example, one of the keywords may be a name, for example, a common person's undername, which is interpreted by the virtual laboratory assistant as its own name and understood as a command to address it to perform or call a specific function.
[0100] Here, the term "laboratory equipment" refers to electronic equipment used in a laboratory to carry out chemical processing steps, such as synthesis or analysis, or other processing steps, to prepare or post-process materials in view of a subsequent / previous processing step.
[0101] Here, a "laboratory facility" is understood to be a system consisting of several laboratory instruments, a transport unit and one or more software modules capable of controlling the laboratory instruments and laboratory software modules together in an integrated manner to perform a chemical workflow automatically or semi-automatically. The workflow can be, for example, a synthetic workflow or an analytical workflow or a combination of both workflows.
[0102] A "laboratory software module" is understood here to be a software application, a submodule of a software application or a software routine that is configured to process data provided by a laboratory device as measurement data or data used as input parameters. Preferably, a laboratory software module is a manual and / or functional component of a laboratory device or laboratory equipment.
[0103] "Vocabulary" is understood here to mean a linguistic space, i.e. a set of words that an entity, for example a speech-to-text system, can exploit.
[0104] A "word" is understood here as a coherent sequence of characters that occurs within a particular vocabulary and represents an independent linguistic unit. In natural languages, words have an independent meaning, in contrast to sounds or syllables.
[0105] A "phrase" is here understood as a linguistic unit consisting of two or more words.
[0106] "Technical language" or "technical words" are understood here as words from the technical vocabulary, which do not belong to the target vocabulary and which are typically not part of the general language vocabulary.
[0107] Phrases for which the speech-to-text system only supports the conversion of speech signals into the target vocabulary mean that words of another vocabulary cannot be converted into text at all or only into text with a very high error rate, which exceeds a limit value per converted word or phrase of the error rate that must be considered as the maximum acceptable value for a functional conversion from speech to text. For example, this limit value can be an error probability per word or phrase of more than 50%, preferably already more than 10%.
[0108] POS tags (i.e. part-of-speech tags) are understood here as special markings ("labels") assigned to each word in a text corpus to indicate the part of speech, and often other grammatical categories such as tense, number (plural / singular), capitalization, etc., that this word represents in this particular text context. The set of all POS tags used in a corpus is called a tagset. Tag sets for different languages are typically different. A basic tagset contains tags for the most common linguistic constituents (e.g. N for noun, V for verb, A for adjective, etc.).
[0109] A "portable device" is understood here to be a portable, preferably battery-powered, data processing device that can be freely positioned and can be connected, preferably wirelessly, to a network, such as the Internet or an organization's intranet. For example, a so-called "smart speaker" with a microphone and / or a single-board system such as a Raspberry Pi computer are used as a "portable device". However, for a user, it is also possible to use his or her own smartphone as a portable device, which has a microphone and client software that can interoperate with the control software.
[0110] In the following figures, embodiments of the invention are explained in greater detail in an exemplary manner. [Brief description of the drawings]
[0111] [Figure 1] FIG. 1 shows a flow diagram of a method for controlling laboratory equipment and laboratory software modules. [Diagram 2] FIG. 2 shows a block diagram of a distributed system for voice-based control of laboratory equipment and laboratory software modules. [Diagram 3] FIG. 3 shows a block diagram of an alternative distributed system for voice-based control of laboratory equipment and laboratory software modules. [Figure 4] FIG. 4 shows a block diagram of another distributed system for voice-based control of laboratory equipment and laboratory software modules. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0112] FIG. 1 shows a flow diagram of a computer-implemented method for voice-based control of laboratory equipment and laboratory software modules.
[0113] In a first step 102, one or more laboratory devices and one or more laboratory software modules are provided.
[0114] The laboratory apparatus may be individual devices or processing units within a complex laboratory facility. The laboratory apparatus may each be configured and equipped to carry out one or more processing steps in the context of the analysis and / or synthesis of substances or substance mixtures or for the preparation and work-up of substances and substance mixtures within a complex analytical or synthetic workflow.
[0115] Analogously, the laboratory software modules can be individually installable software applications, for example simulation software, which can issue opinions or make predictions regarding the chemical, physical, tactile, optical or other properties of the laboratory equipment based on the data obtained therefrom. The software modules can also be provided with a database in the form of a DBMS, which stores the results of previous analyses and syntheses of the laboratory equipment, the contents of which can be used, for example, to perform a database search for substances (for example paints and varnishes) that have already been manufactured and whose properties are known. The search query can, for example, be a search for all varnishes that have a viscosity below a limit value specified in the search query. The software modules can also be modules and functions of a larger software application, for example the control software of a complex laboratory facility.
[0116] In a further step 104, a data processing device with control software is provided. The control software provides one or more users with an interface for operating at least one laboratory device and / or at least one laboratory software module. The data processing device with control software can be located in the same room or laboratory area as the laboratory equipment or laboratory facility including the laboratory equipment. For example, the data processing device can be a desktop computer or a server. However, preferably, the data processing device is set up in another space, for example in the data center of the organization running the laboratory or in the data center of an external cloud storage provider. However, access to this data processing device and control software, for example for maintenance purposes, can be provided from a computer in the laboratory area, for example.
[0117] In step 106, a portable device with a microphone is provided and set up to be connected and interoperable with the control software via a network, for example the Internet or the organization's intranet. The device is designed to allow a user to operate at least one laboratory device and / or at least one laboratory software module by speaking into the microphone without having manual intervention in interaction with the control software. The portable device is preferably provided in such a way that it is placed near the laboratory researcher's work area from which he wishes to control the laboratory device or otherwise perform his laboratory work. It is also possible to position the device with the microphone in a central location in the laboratory area from which voice input from all areas of the room or laboratory area can be received more or less equally well and recorded by the microphone.
[0118] According to an embodiment, providing the device includes placing a portable device, e.g., a single board computer, having a microphone in the same room as at least one laboratory device and / or placing the portable device in the same room as a main control computer of a laboratory facility including at least one laboratory device.
[0119] The steps can be performed in any order. This allows for manual intervention-free operation of one or more laboratory devices and associated laboratory software modules by voice input from any area of the laboratory, without having to remove gloves for this purpose. By having the control software convert the voice signal recorded by the microphone by the speech-to-text system into a natural language text, which is then corrected in such a way that by a special text correction step, even technical language of special sub-fields, such as for example the chemical industry or paint and varnish manufacturing, is correctly reflected in the text of the voice input.
[0120] The steps performed by the control software, laboratory equipment, and laboratory software modules may vary slightly in different embodiments of the laboratory system, and the laboratory system may include other components, such as an external text correction system and / or an NLP processor, to transform the corrected natural language text into structured text. However, in some embodiments, the laboratory facility itself includes such an NLP processor that performs this transformation, so that the corrected natural language text can be input directly into the laboratory facility control software.
[0121] FIG. 2 shows a block diagram of a distributed system 200 for voice-based control of laboratory equipment 252, 254 and laboratory software modules 248, 250. The system 200 comprises laboratory equipment 252, 254 and laboratory software modules 248, 250, a control computer 213 having control software, and a handheld device 212 having a microphone. Optionally, the system can include other handheld devices 212 that are substantially identical in terms of construction and function. Optionally, the system can also include a main control computer having control software for controlling the laboratory equipment. Optionally, the system 200 can include correction software 225, which can be implemented as a function of the control software or as a standalone software application installed locally or remotely. Optionally, the system 200 can also include an NLP processor 258. Optionally, the system 200 can also include a speech-to-text system 226.
[0122] The laboratory equipment and laboratory software modules included in the system 200 are controlled by a user 202, e.g., a laboratory researcher physically located near a handheld device 212 having a microphone 214, via voice input 204. For example, the user can input the command "Synthesize a varnish called <name of varnish> and determine the rheological properties of the synthesized varnish!" into the microphone. A voice signal 206 recorded by the voice input microphone is routed by the handheld device 212 over a network 236, such as the Internet or an intranet, to a control software 222 installed on a control computer 213. This forwards the voice signal 206 over the network to a general language speech-to-text system 226 that converts the received voice signal 206 into text 208, the words and phrases of which are taken exclusively from a general language target vocabulary 234 that does not include technical language such as the name of the varnish or the word "rheological". For example, the speech-to-text system 226 can be configured as a cloud-based computer system that provides the conversion to multiple clients in the form of a software-based service, referred to herein as a "speech recognition processor" 232. The speech-to-text system 226 can be, for example, a Google cloud computer system that provides Google's speech-to-text cloud service over the Internet. Thus, in this example, the interface 224 is a Google cloud-based API. The speech-to-text system returns the recognized text 208 to the control software 222.
[0123] According to the embodiment shown in Figure 2, the control software includes a correction function 225. This has access to an assignment table 238 in which a plurality of technical words and phrases are each assigned to one or more words or phrases of the target vocabulary.
[0124] In the assignment table 238, the words of the text form are assigned to each other. More specifically, the assignment table assigns at least one word from the target vocabulary to each of a plurality of technical languages or technical language phrases. The at least one word of the target vocabulary associated with a technical language (or a phrase of technical language) is a word or phrase that the speech-to-text system will recognize incorrectly (and was previously recognized incorrectly when the table was created) when the technical language is input to the speech-to-text system in the form of an audible signal.
[0125] Optionally, POS tags can also be assigned to technical phrases and can also be optionally evaluated and taken into account by the correction function 225 during the course of text correction. The correction function 225 can also be formed as a separate correction software application 225 installed on the control computer 213 and with which the control software 222 can interoperate.
[0126] The control software 222 forwards the received text 208 to a correction function or software 225, which prompts the correction of the text 208 using an assignment table 238. During the correction process, target language words and phrases incorrectly recognized by the table are replaced by the technical language phrases assigned in each case. The result is a corrected text 210 that is either sent directly by the control software 222 to a specific target system, e.g., a chemical facility 244, or first forwarded to an NLP processor 258.
[0127] The NLP processor shown in Fig. 2 is installed on the control computer 213. However, according to an alternative embodiment, for example as shown in Fig. 3, the NLP processor can also be provided as a service on the network. The NLP processor analyzes the syntactic structure of the corrected text 210 in order to detect the semantic content of the corrected text and to extract relevant information from it, and converts it into a structured text 211. Since many laboratory systems and laboratory software modules currently on the market are still unable to correctly interpret natural language text, the transfer of the corrected text 210 to the corrected structured text 211 can be advantageous. However, many target systems have an input interface for structured text, for example a table or an XML file of a certain predefined structure. In another embodiment, the target system includes its own NLP processor, so that the corrected text 210 can be an input in natural language form to the target system. In this case, the control software sends the corrected text 210 directly to the target system.
[0128] For example, the target system can be a single laboratory instrument or a laboratory facility including multiple laboratory instruments. The laboratory facility 244 shown in FIG. 2 can be a high throughput environment (HTE) facility including multiple laboratory instruments 252, 254 and one or more laboratory software modules 250, 248.
[0129] For example, the laboratory devices 252, 254 can be analytical devices configured to perform various chemical analyses (e.g., viscosity, pH, color, surface structure, layer thickness). For example, the software module 250 can be a chemical database that interprets the input text 211 or 210 as a search query and performs a corresponding search in a chemical database. The laboratory software module 248 is control software installed on the main control computer 246 of the chemical facility 244 and is configured to coordinate the processing steps to be performed by the individual laboratory devices of the facility. For example, the control software 248 controls not only the analytical steps performed by the analytical devices, but also the operation of the transport unit 256 that transports materials and intermediate products from one processing station to the next within the chemical facility 244.
[0130] Control of chemical facilities by voice input can be particularly advantageous, since these facilities are now capable of producing specific chemical products, for example specific paints or specific varnishes, almost automatically and on the basis of specific recipes. It is therefore possible to enter a recipe for the production of a specific chemical substance or substance mixture by voice input, thereby prompting the chemical facility 244 to automatically produce the corresponding substance or substance mixture. Likewise, it is possible to start a corresponding analysis of a substance by voice input, or to control its execution, interrupting, modifying or stopping it if necessary.
[0131] The internal database 250 can be used not only for storing chemicals and substance mixtures and their properties, but also for storing the corresponding formulations. The database 250 can thus contain, for example, paint and varnish formulations as well as their raw materials and their respective physical, chemical, optical and other properties. In addition, other relevant data can be stored in the database, such as product data sheets of substance manufacturers, safety data sheets, parameters for configuring individual modules of the HTE system for the analysis or synthesis of a particular substance or product, and the like. The HTE system is designed to perform the analysis and synthesis based on formulations and instructions entered in text form.
[0132] In some embodiments, one or more laboratory devices of the chemical facility have their own interface through which they can directly receive the natural language or structured corrected text 210, 211 from the control software. In some other embodiments, the laboratory devices of the chemical facility can only indirectly receive the corrected text 210, 211 from the control software 222 via the control software 248.
[0133] According to an embodiment, the voice input includes keywords that the control software 222 uses to determine the number and identity of each required target system. The corrected text is then sent by the control software in natural language or structured form to each of the identified target systems. For example, the user can determine which target device the control software should send the corrected text to by entering the keyword "target system" followed by the name of a particular target system (laboratory device or software module). However, additionally or alternatively, the control software itself can search for a keyword and autonomously determine a suitable target system based on this keyword. For example, the voice input can simply specify a particular type of synthesis, and the control software dynamically determines for the requested synthesis which of the various laboratory devices available for synthesis to use. For example, the decision can be made based on the current workload, so that a laboratory device that is currently underutilized or not used at all is selected.
[0134] Preferably, the results 242 of the execution of a particular hardware or software function are returned from the target system to the control software 222. The control software can then return the results of the execution of the function to the user 202. Various paths are possible for this purpose. For example, the user can send an email, an SMS, or a message in other formats that includes the results of the execution of the function. According to the embodiment shown in FIG. 2, the results 242 are returned at least on the portable device 212 and also (or exclusively) on the speaker via its speaker 218. This is particularly advantageous, since it allows the laboratory researcher 202 maximum freedom of movement both with regard to inputting control commands, chemical preparations, and search queries, as well as with regard to receiving the results of the execution of the corresponding hardware or software functions. For example, the user can be in a particular working area of the laboratory and can start the analysis of a particular substance by voice command in parallel with his own manual activity. The results of the analysis are communicated acoustically via the speaker. Thus, the user does not have to interrupt his manual activity to request the results. Rather, the user can enter into a normal conversation with the control software and, for example, after receiving the result that a particular synthesis is complete, can also initiate a corresponding analytical process on the generated substances by voice input. The efficiency of laboratory work can thus be dramatically improved since the user is largely capable of working in parallel, for example, initiating and controlling the control of complex syntheses and analyses in parallel with other laboratory work.
[0135] Common activities within a laboratory that can be controlled by voice input according to embodiments of the present invention are, for example, given below as examples of activities and voice inputs. -A laboratory researcher started the analysis of a particular varnish for its rheological properties the previous day and now wants to query the results stored in the database of the HTE system. Possible voice input: "Control computer, please show me the results of the rheological analysis by the HTE facility in room 22 on February 24, 2019." -Laboratory researchers want to save money and are looking to reduce the use of certain solvents.<SLV EXP> >More economical<<SLV ECO> I am thinking of replacing it with >.<SLV ECO> The name > is the manufacturer's trade name. However, the laboratory researcher is not sure whether a more economical solvent would be suitable for the varnish being produced and would like to consult the product data sheet, which specifies further information regarding the chemical and physical properties of the more economical solvent. Possible voice input: "Control computer, <<SLV ECO> "Can I see the product data sheet for the <<SLV ECO> >Please show me the product data sheet." -Solvent<<SLV ECO> After studying the product data sheet of >, the laboratory researcher is of the opinion that for the production of a particular varnish, this solvent can possibly be used instead of a more expensive solvent. However, some parameters such as pH, rheological properties, polarity, and others will be different from those of the more expensive solvent, so some adjustments may be necessary for the formulation. These properties are interrelated with each other, so it is not possible for the laboratory researcher to identify the necessary adjustments to the formulation based on theoretical considerations. Running test series is laborious and time consuming. However, the laboratory has software that can predict (simulate) the properties of chemical products, e.g., paints and varnishes, based on a particular formulation. The simulation can be based, for example, on CNN (Convolutional Neural Networks). The laboratory researcher wants to use this simulation software to simulate, based on a known formulation, the predicted properties of a varnish, where the expensive solvent is replaced by an economical one. Possible voice input: "Please prompt the control computer to have the HTE simulation software calculate the properties of a varnish with the following formulation: 70.2 g naphthenic oil, 4 g methol n-amyl ketone, 1.5 g n-pentyl propionate, 1 g Ultrasorb, 50 g <<SLV ECO> >, [...]". -The simulation shows that for the production of varnishes the economical solvent is not suitable. The laboratory employee now wants to search the Internet for other solvents that can replace the expensive solvent without having an effect on the quality of the product, in order to reduce costs. Possible voice input: "Control computer, please search the Internet for: <<High viscosity solvents for paint production>>".
[0136] According to embodiments of the present invention, all of these inputs and commands to the various execution systems can be made without the user having to leave the lab and / or remove their gloves for this purpose.
[0137] Most of the speech inputs given as examples above contain words and phrases of both general and technical language. For example, the words or phrases "rheological", "naphthenic oil", "metol n-amyl ketone", and "n-pentyl propionate" are technical chemical terms and are also<SLV ECO> > is a trade name for a chemical product. These words or phrases are typically not included in the vocabulary supported by current common language speech-to-text systems (the "target vocabulary"). Subsequent correction by the correction software ensures that the speech input is correctly converted to text despite the use of a common language speech-to-text system 226. The control software performs various coordination and control activities for the management and processing of the speech signal and the text generated therefrom, which vary slightly depending on the system architecture.
[0138] Figure 3 shows a block diagram of an alternative distributed system 300 for voice-based control of laboratory equipment and laboratory software modules. The system 300 can include substantially the same components and functions as those shown in Figure 2. Unlike the system shown in Figure 2, the correction software 225 is not a component of the control software and is not installed locally on the control computer 213 as a separate software application, but is provided as a cloud service over the Internet from a cloud computer system 302. The control software 222 is configured to correct the text 208 received from the speech-to-text system, here using an assignment table stored on the cloud computer system 302, via a corresponding interface 304, and receives the corrected text 210.
[0139] 4 shows a block diagram of an alternative distributed system 400 for voice-based control of laboratory equipment and laboratory software modules. The data exchange for controlling the individual laboratory equipment 406-414 of the HTE system 244 is illustrated in more detail in FIG.
[0140] The laboratory includes a laboratory area 404 with various stand-alone laboratory equipment 416 (eg, centrifuges) and HTE facilities 418 .
[0141] The HTE facility includes a number of modules and hardware units 406-414 that are managed and controlled by control software on a main control computer 246. The control software in this embodiment serves as a central interface for externally monitoring and controlling the equipment contained within the HTE facility. The control software 222 on the control computer 213 has access to the control software and is configured to transmit the corrected text 210, 211 as input in natural language or in structured form to the control software and optionally also to other target systems such as laboratory equipment 416 and / or an Internet search engine 402. The creation of the corrected text 210, 211 from the voice input 204 of the user 202 is performed as previously described according to embodiments of the present invention.
[0142] The control software 222 may be implemented as a "virtual lab assistant" 223. For example, the virtual lab assistant may respond to names such as "control computer," "Luisa," or "Eva." The control software is configured to search the corrected text 210 for keywords and to invoke specific hardware or software functions of the target system in response to these keywords. For example, the control software searches the text 210 for keywords such as "control computer" or "Eva." If the corrected text contains this keyword, the virtual lab assistant 223 is prompted to further analyze the corrected text to determine whether the corrected text contains commands to perform hardware or software functions, and if so, by which hardware or software under the control of the control software or lab assistant these commands are to be performed. For example, the corrected text may contain equipment or laboratory names specifying which equipment and software the commands are to be forwarded to.
[0143] The phrase "control computer" or other names of virtual assistants in voice input can therefore be used analogously to the names "Siri" and "Alexa" of well-known virtual assistants for "everyday problems" in the field of chemical laboratories to allow more efficient and more reliable control of hardware and software functions.
[0144] In one possible implementation, evaluation of the corrected text 210 by the virtual laboratory assistant results in an Internet search engine 402 searching for the particular material specified in the corrected text 210 as technical language or phrases. The corrected text, or portions thereof specified, is input to the search engine via the Internet by the virtual assistant. Results 424 of the Internet search are returned to the controlling software or assistant, which forwards these results to a suitable output device near the user 202, such as a handheld device 212 having a microphone and speaker 214. The results are preferably forwarded such that the virtual laboratory assistant, i.e., the controlling software, reads the results 424 to the user via the speaker 214 in a natural sounding voice.
[0145] In another possible implementation, evaluation of the corrected text 210 by the virtual laboratory assistant reveals that the standalone laboratory device 416, the centrifuge, pellets a particular material at a particular speed. This is sufficient since the name of the centrifuge and the material are specified as technical words or phrases in the corrected text 210, and the centrifuge automatically retrieves the parameters of the centrifugation used, such as duration, number of revolutions, etc., from an internal database based on the material name. The corrected text, or a certain part of it, is sent by the virtual assistant via the Internet to the centrifuge 416. The centrifuge starts the centrifugation program associated with the material and returns a message about the success or failure of the centrifugation in the form of a text message 422. The result 422 is returned to the assistant 223, which outputs it via the speaker 214 and optionally also via other output interfaces, such as a screen, as described for the result 424.
[0146] In another possible implementation, evaluation 210 of the corrected text by the virtual laboratory assistant reveals that the HTE system 244 is for the synthesis of a specific varnish. The constituents of the varnish are also specified in the corrected text and consist of a mixture of trade names and IUPAC substance names of chemical products. The HTE facility receives the corrected text 210 and autonomously decides to carry out the synthesis in the synthesis unit 414. A message about a successful synthesis or an error message is returned as a result 426 from the synthesis unit 414 to the control software of the HTE unit, which in turn returns the result 426 to the control software or the virtual laboratory assistant, which transfers the results to a suitable output device, for example a terminal device (handheld terminal 212) which outputs these results 426 via a speaker 214 as described for the result 424, and optionally also via a further output interface, for example a screen.
[0147] The software and / or hardware from which the corrected text 210, 211 is output is preferably software and hardware located within the laboratory 404. However, it is additionally possible to transmit the text or parts thereof to a target system outside the laboratory, the result of which is designed or suitable for use for chemical analysis or synthesis.
[0148] The voice control of the associated laboratory equipment and laboratory software modules can be used to study and output analytical and synthetic results already performed in the laboratory, laboratory protocols, and product data sheets corresponding to laboratory databases, and also by voice control complementary searches can be performed in the Internet and public or proprietary databases accessible via the Internet. Voice commands including special trade names of chemicals or laboratory equipment or laboratory consumables, and / or names and adjectives of chemical terminology can also be correctly converted into text and therefore correctly interpreted by the target system. According to embodiments of the present invention, a highly integrated operation of a chemical or biological laboratory or laboratory HTE facility, largely voice controlled, is thus made possible.
[0149] Known speech-to-text cloud service provider hardware (smart speakers) pursues the objective of direct control and uses services developed by the cloud provider itself. Applications in the field of technical vocabulary have not been developed so far or have been developed only to a very limited extent. All the system architectures 200, 300, 400 shown here enable the use of existing speech-to-text APIs of various cloud providers with its own cloud provider independent hardware to enable speech recognition and control of subject specific laboratory equipment and electronic search services on this basis in the laboratory.
[0150] In the context of chemical synthesis of paints and varnishes, efficient retrieval of information on chemicals, as well as voice-based control of laboratory equipment and HTE facilities, is particularly advantageous, since paint and varnish production requires a wide variety of starting materials, the properties of which interact with each other in a complex manner and strongly influence the properties of the product. Hence, in the context of paint and varnish production, a large number of analysis, control steps, and test series are required. Paints and varnishes are very complex mixtures of up to 20 raw materials and many more, e.g., solvents, resins, hardeners, pigments, excipients, and a large number of additives (dispersants, wetting agents, adhesion promoters, defoamers, biocides, flame retardants, and others). Efficient access to information on the individual components and the corresponding analysis and control of the synthesis system can significantly accelerate the production process and the reliability of the product. [Explanation of symbols]
[0151] 102~106 process 200 Distributed Systems 202 Users 204 Voice Input 206 Audio Signals 208 Recognized Text 210 Corrected Text (Natural Language) 211 Corrected Text (Structured Text) 212 Portable Devices 213 Control Computer 214 Microphone 218 Speakers 222 Control Software 223 Virtual Lab Assistant 224 Speech-to-text interface (client side) 224' Speech-to-text interface (server side) 225 Correction Software 226 Speech-to-text conversion system / cloud system 232 Speech Recognition Processor 234 Target Vocabulary 236 Network 238 Allocation Table 242 Results of execution of corrected text 246 Main Control Computer 248 Control Software 250 Chemical Database (DBMS) 252 Analyzer 254 Analyzer 256 Transport Unit 300 Distributed Systems 302 Server with text correction service 304 Correction Service Interface (Client Side) 304' Correction Service Interface (Server Side) 308 NLP Interface 310 Laboratory Facility Control Interface 400 Distributed Systems 402 Internet Search Engine 404 Laboratory Area / Laboratory 406 Analyzer 408 Analyzer 410 Mixer 412 Composite Unit 414 Composite Unit 414 Control Computer 416 Standalone Laboratory Equipment 422 Results of the execution of the corrected text (text form) 424 Results of execution of corrected text (text form) 426 Results of execution of corrected text (text form)
Claims
1. 1. A laboratory system comprising: at least one laboratory device designed for analyzing and / or synthesizing chemical substances; at least one laboratory software module designed to process data obtained from said at least one laboratory device; a data processing device having control software providing an interface for operating said at least one laboratory device and / or said at least one laboratory software module; a portable device having a microphone, interoperably connected via a network to said control software and designed to enable, in interoperation with said control software, a user to operate said at least one laboratory device and / or said at least one laboratory software module by speaking into said microphone without manual intervention, The control software comprises: receiving from said portable device, on the basis of a voice input, a voice signal picked up by said microphone, said voice signal including general language and technical language spoken by said user; - inputting the received speech signal into a speech-to-text system, the speech-to-text system only supporting conversion of speech signals into a target vocabulary that does not include the technical language; - receiving from the speech-to-text conversion system text produced by the speech-to-text conversion system from the audio signal; - generating a corrected text from the received text, the corrected text being generated by the control software alone or by the control software operatively connected thereto and using text correction software in its interpretation, the corrected text being generated by automatically replacing words and phrases of the target vocabulary in the received text with a technical language according to an assignment table of words in text form, the assignment table assigning at least one word from the target vocabulary to each of a plurality of technical languages, the at least one word of the target vocabulary assigned to a technical language being a word or phrase that the speech-to-text conversion system incorrectly recognizes when that technical language is input in the form of a speech signal, - a laboratory system configured to use the corrected text to prompt the at least one laboratory apparatus and / or the at least one laboratory software module to perform the analysis, synthesis, and / or software functions according to specifications in the corrected text.
2. 10. The laboratory system of claim 1, wherein said control software is designed as a virtual laboratory assistant for said operation of said at least one laboratory device and said at least one laboratory software module.
3. The portable device, - receiving a voice signal of the user via the microphone, A laboratory system as claimed in claim 1 or 2, configured to forward the received audio signals to the control software for conversion of the audio signals into text by the control software or by a speech-to-text system operably connected thereto.
4. The portable device further comprises a speaker, the portable device further comprising: - receiving, in response to the transmission of said audio signal, a result of the execution of a function from said control software, said function being executed by said at least one laboratory device and / or by said at least one laboratory software module according to said text; and A laboratory system as claimed in claim 3, configured to output said results to said user via said speaker.
5. 5. The laboratory system of any one of claims 1 to 4, wherein the portable device is located in the same room as the at least one laboratory device and / or in the same room as a main control computer of a laboratory facility comprising the at least one laboratory device.
6. A laboratory system according to any one of claims 1 to 5, wherein the portable device is a single board computer.
7. The technical language falls into the following categories: - names of chemical substances, in particular paints and varnishes; - the physical, chemical, mechanical, optical or tactile properties of a chemical substance; - names of laboratory equipment and chemical industry equipment; - names of laboratory consumables and supplies; - trade names in the field of paints and varnishes.
8. The laboratory system of any one of claims 1 to 7, wherein the target language words and phrases stored in the assignment table represent incorrect text output of the speech-to-text system created on the basis of speech input of technical words by a number of different personnel.
9. The at least one laboratory device is a plurality of laboratory devices, the laboratory devices comprising: an analytical station designed to analyze chemical substances or substance mixtures, and / or a synthesis station designed to carry out the synthesis of chemical substances or substance mixtures, and / or - pre- or post-treatment stations designed to modify chemicals before or after an analytical or synthetic step to enable a subsequent processing or transport step.
10. a plurality of laboratory apparatus, at least one of the laboratory apparatuses each serving as a chemical processing station; a transport unit designed to transport the chemicals used for synthesis or analysis to said processing stations, in order to enable said processing stations to subject said chemicals in each case to at least one processing step; A laboratory system according to any one of claims 1 to 8, further comprising: a main control computer of a laboratory facility including a plurality of laboratory devices, the main control computer including control software configured to coordinate the processing steps and the transport of the chemicals on the basis of instructions in the form of structured text.
11. The control software comprises an NLP processor or is interoperable with an NLP processor, and the control software - using the NLP processor to transform the corrected text into a structured text that can be interpreted by a target system, the target system being the at least one laboratory software module, the at least one laboratory device, and / or the control software of a main control computer of a laboratory facility comprising the at least one laboratory device; and A laboratory system according to any one of claims 1 to 8, configured to input said structured text into a target system to prompt the target system to perform laboratory related software and / or hardware functions.
12. the operation without manual intervention includes converting the voice input into text; - said at least one laboratory software module is a chemical substance database designed to interpret an input text as a search input, to determine information about said search input in a database and to return this as a result to said control software; said at least one laboratory software module is a simulation software designed to simulate the properties of chemical products, in particular varnishes and paints, on the basis of a given substance mixture specification, said simulation software being designed to interpret said input text as a specification of said chemical products whose properties are to be simulated, The laboratory system of claim 10, wherein said at least one laboratory software module is control software of a main control computer of a laboratory system including said at least one laboratory apparatus, said control software being designed to prompt said at least one laboratory apparatus and a transport unit to perform a plurality of process steps and said transport of said chemicals between said laboratory apparatus according to said specifications contained within said input text.
13. 13. A laboratory system according to any one of claims 1 to 12, wherein the control software includes a registry of accounts of a plurality of users registered with the control software, the control software is equipped with a voice recognition facility for recognizing each of the registered users by their voice, and the control software is designed to enable the user to operate the laboratory equipment and / or the laboratory software modules that the user is authorized to operate without manual intervention, by simply speaking into the microphone.
14. 1. A method for controlling laboratory equipment and / or laboratory software modules, comprising: Providing at least one laboratory device and at least one laboratory software module, said at least one laboratory device being designed to analyze and / or synthesize chemical substances, said at least one laboratory software module being a laboratory software module for processing data obtained from said at least one laboratory device; - providing a data processing device having control software, said control software providing a user with an interface for operating said at least one laboratory device and / or said at least one laboratory software module; providing a portable device having a microphone, said portable device being interoperably connected to said control software via a network, said portable device being designed in such a way that in interoperation with said control software a user can operate said at least one laboratory device and / or said at least one laboratory software module by speaking into said microphone without manual intervention, The control software comprises: receiving from said portable device, on the basis of a voice input, a voice signal picked up by said microphone, said voice signal including general language and technical language spoken by said user; - inputting the received speech signal into a speech-to-text system, the speech-to-text system only supporting conversion of speech signals into a target vocabulary that does not include the technical language; - receiving from the speech-to-text conversion system text produced by the speech-to-text conversion system from the audio signal; - generating a corrected text from the received text, the corrected text being generated by the control software alone or by the control software operatively connected thereto and using text correction software in its interpretation, the corrected text being generated by automatically replacing words and phrases of the target vocabulary in the received text with a technical language according to an assignment table of words in text form, the assignment table assigning at least one word from the target vocabulary to each of a plurality of technical languages, the at least one word of the target vocabulary assigned to a technical language being a word or phrase that the speech-to-text conversion system incorrectly recognizes when that technical language is input in the form of a speech signal, - the at least one laboratory apparatus and / or the at least one laboratory software module are configured to use the corrected text to prompt the analysis, synthesis and / or software functions to be performed according to specifications in the corrected text.
15. 15. The method of claim 14, wherein providing the portable device comprises placing the portable device in the same room as the at least one laboratory device and / or in the same room as a main control computer of a laboratory facility including the at least one laboratory device.
Citation Information
Patent Citations
Experimental apparatus for processing liquids
JP2014500132A
Configuration control device for a laboratory instrument, a laboratory instrument with the configuration control device for instrument-controlled treatment of at least one laboratory sample and a method for configuring the laboratory instrument by means of the configuration control device
US20150105877A1
