Laboratory system with portable device having integrated microphone and related methods

Abstract

The invention relates to a laboratory system comprising: at least one laboratory device (252; 254; 406; 408; 412; 414; 416) designed for analyzing and / or synthesizing chemical substances; at least one laboratory software module (248) designed for processing data obtained by the at least one laboratory device; a data processing device (213) having control software (222), the control software providing an interface for operating the at least one laboratory device and / or the at least one laboratory software module; and a portable device (212) having a microphone (214), wherein the device is connected to the control software in an interactive manner via a network (236), wherein the device is designed to allow a user (202) to operate the at least one laboratory device and / or the at least one laboratory software module without intervention by means of voice input (204) into the microphone when interacting with the control software.

Description

Technical Field

[0001] The present invention relates to the control of laboratory equipment, in particular to the control of a complex laboratory system. Existing technology

[0002] In chemical laboratories, numerous regulations apply to ensure safe working, due to the various hazards arising from the substances and equipment. Depending on the type of laboratory, the activities carried out there, and the substances used, the following safety regulations may apply: Personal protective equipment must be worn, which, in addition to a lab coat, may also include protective glasses or a face shield and protective gloves. Food and drink are usually not allowed, and to prevent contamination, the office area, which contains desks, manuals, paper product documentation, computer workstations, and internet access, is physically separated from the laboratory work area. This physical separation may require that access between the office and laboratory areas is only possible via a safety door. Safety clothing may also be required upon leaving the laboratory area.

[0003] Safety regulations can sometimes make work quite difficult: Laboratory equipment, and especially complex laboratory systems with multiple pieces of equipment, provide users with a separate operating interface, typically consisting of a screen, mouse, keyboard, and / or touchscreen. These are often standard instrument components that are difficult or impossible to operate while wearing laboratory gloves. Consequently, incorrect inputs frequently occur, slowing down the workflow and potentially leading to incorrect control and operation of laboratory equipment. Removing gloves before operating a laboratory system carries a risk of contamination, slows down work, and is sometimes even impossible from a safety perspective.

[0004] In some cases, laboratory equipment has large keyboards, for example, in the form of large touchscreens, to facilitate gloved input. However, this specialized hardware is expensive and not all laboratory equipment is suitable. In particular, standard computers and laptops do not have such "glove-compatible" keyboards. However, the ability to operate a standard computer with a conventional internet browser is becoming increasingly important in laboratory settings, for example, to be able to search online chemical databases.

[0005] In the case of chemical or biological laboratories, the current possibilities for controlling or interacting with laboratory equipment and laboratory systems are limited and inefficient.

[0006] Overview

[0007] The object of the present invention is to provide an improved laboratory system and method for controlling at least one laboratory device or laboratory software module according to the independent claims, which allow improved control of laboratory-related hardware and software functions, in particular in a laboratory environment. Embodiments of the invention are described in the dependent claims. The embodiments of the invention can be freely combined with one another, unless they are mutually exclusive.

[0008] In one aspect, the present invention relates to a laboratory system. The laboratory system includes at least one laboratory device designed for analyzing and / or synthesizing chemical substances. The laboratory system also includes at least one laboratory software module designed for processing data obtained from the at least one laboratory device. The laboratory system also includes a data processing device having control software, wherein the control software provides an interface for operating the at least one laboratory device and / or the at least one laboratory software module. The laboratory system also includes a portable device with a microphone. The device is interoperably connected to the control software via a network and is designed to allow a user to non-invasively operate the at least one laboratory device and / or the at least one laboratory software module by voice input into the microphone while interoperating with the control software.

[0009] Embodiments of the present invention are advantageous because they allow for easier and better inspection and control of laboratory equipment, such as in a biochemistry laboratory, by one or more individuals. Because the laboratory equipment and the associated laboratory software modules are monitored via voice input, there is no need to remove gloves or take other time-consuming steps to enter control commands. Voice-based input allows information to be entered anywhere within the acoustic range of the portable device, particularly within the laboratory area. The person performing the voice input thus does not have to leave their laboratory workstation, or they can move freely between different workstations within the laboratory area, provided they remain within the range of the portable device's microphone.

[0010] Controlling target systems using voice input is already known. Consequently, inexpensive terminals with microphones and powerful applications for voice-based commands are available on the market, such as the Echo Dot with Alexa (Amazon), smart speakers from various manufacturers with Cortana (Microsoft), the Google Home Max / Min with Google Assistant (Google), and the HomePod with Siri (Apple). However, these terminals are designed to support everyday end-user activities, such as shopping, selecting radio programs, or booking hotels. These terminals and applications are therefore designed for everyday situations and only support common vocabulary. Existing terminals are neither designed nor suitable for use in laboratories or for term recognition. Currently available terminals are not designed for user modification, making it impossible to use currently available smart speaker systems with voice recognition in laboratories. According to embodiments of the present invention, control software has been developed that can accurately convert voice input containing terminology into text, either alone or in conjunction with other components, and a portable device with a microphone has been developed that interacts with the control software. A laboratory system is thus provided which enables voice-based control of laboratory equipment and associated laboratory software modules in a simple and hands-free manner, just as is currently achieved in the field of smart homes based on, for example, the Amazon Echo Dot.

[0011] According to an embodiment, the control software is designed as a virtual laboratory assistant for operating the at least one laboratory facility and the at least one laboratory software module.

[0012] This can be advantageous because it further simplifies user input of control commands and interaction between the user and the laboratory equipment or laboratory software modules. For example, the virtual laboratory assistant can implement one or more sound generators that each generate computer-generated but natural-sounding sounds and read out the results of specific functions performed by the laboratory equipment or laboratory software modules to the user in a natural voice. It is also possible for the virtual laboratory assistant to "recognize" certain names, that is, to check whether the voice input or the text generated thereby mentions the name and, if so, to perform or cause the target system to perform certain functions. Thus, laboratory staff can interact with the control software and the virtual laboratory assistant implemented thereby in a manner similar to how they interact with human colleagues.

[0013] According to an embodiment, the portable device is configured to receive a user's voice signal via a microphone and forward the received voice signal to the control software. The voice signal is forwarded to enable the control software or a speech-to-text conversion system operatively connected to the control software to convert the voice signal into text.

[0014] According to an embodiment of the present invention, the portable device further includes a speaker. The portable device is configured to receive a function execution result from the control software in response to the voice signal. The function is executed by at least one laboratory device and / or at least one laboratory software module based on text. The portable device is further configured to output the received result to the user via the speaker. For example, the received result may be text converted into an audio signal using the device's text-to-speech software. According to an alternative embodiment, the control software includes text-to-speech conversion functionality, and the result is already transmitted to the portable device in the form of an electronic audio signal, eliminating the need for text-to-speech conversion on a terminal with very low computing power in some embodiments.

[0015] Outputting the results as audio signals via a loudspeaker can be advantageous because this does not restrict the user's freedom of movement, which is achieved by inputting commands based on voice. Thus, the user can freely move from any point in the laboratory that is within the detection range of the microphone and within the audible range of the loudspeaker's sound output, while simultaneously inputting control commands and search requests via voice input, and receiving the results of executing these commands and / or information searches. This is particularly useful in a laboratory work environment, as laboratory personnel often need to move between different equipment and laboratory areas, for example, to reload or configure machinery, check the progress of work steps, or release blocked components or shipping boxes.

[0016] According to some embodiments, the results of software functions or hardware functions executed by a laboratory software module or laboratory device in response to voice input are displayed to the user via a screen. In addition to or as an alternative to this output, this can be performed via a loudspeaker. Output on a screen can be particularly advantageous in the case of complex and extensive results, such as documents identified during a database or internet search, since the user may not be able to receive and understand the entire message if long text is output solely via a loudspeaker.

[0017] According to an embodiment, the portable device is located in a common room with the at least one laboratory device and / or in a common room with a host computer of a laboratory facility containing the at least one laboratory device.

[0018] This may be advantageous because most microphone types, and particularly microphones on portable devices, are able to pick up speech input in the room, and / or because the volume of most common speakers, and particularly speakers on portable devices, is sufficient to be heard and understood by people in the same room.

[0019] According to some embodiments, a laboratory system includes two or more portable devices described herein with microphones and optional speakers. The devices are preferably evenly distributed throughout the laboratory. This can be advantageous because it allows for particularly good coverage both when receiving voice input and when outputting results via the speakers.

[0020] According to an embodiment of the present invention, the portable device is a single-board computer, in particular a Raspberry Pi computer.

[0021] This can be advantageous because such single board computers are inexpensive components designed to be adapted and modified by the end user, e.g. by installing client software that interacts with freely configurable server software, e.g. control software, via a network so that the user can access functions of the control software via voice input.

[0022] For example, the Google Assistant SDK allows you to integrate Google's speech recognition service into terminals such as the Raspberry Pi. You can find a large number of video tutorials on YouTube.

[0023] However, according to an embodiment of the present invention, the portable device is not directly connected to the Google service or other speech-to-text conversion service, but only has client software installed, which enables the portable device to interact with control software that coordinates the data exchange with the speech-to-text conversion system connected via the network and the correction of text received from the conversion system.

[0024] For example, the "Google Assistant for Raspberry Pi" software is installed on the Raspberry Pi and configured to transmit voice signals received by the portable device to the control software. The address of the control software is thus set and stored in the portable device. This can be advantageous because it provides a very inexpensive portable terminal that can be configured to interoperate with any software application, such as the control software that implements a virtual laboratory assistant, for simple interaction with in-laboratory data processing equipment and services.

[0025] It may be advantageous to outsource this coordination function to the control software, since the Raspberry Pi computer has only low computing power, so the conversion of the speech signal into text and perhaps the subsequent text correction can be coordinated more quickly by other more powerful devices.

[0026] According to an embodiment of the present invention, control software is configured to receive a voice signal from a portable device, captured by a microphone based on voice input. The voice signal includes common words and terminology words spoken by the user. The control software is further configured to input the received voice signal into a speech-to-text conversion system, wherein the speech-to-text conversion system only supports converting the voice signal into a target vocabulary that does not contain terminology words. The control software then receives text generated by the speech-to-text conversion system based on the voice signal. The control software is further configured to generate corrected text based on the received text, wherein the corrected text is generated by the control software alone or by the control software after being interpreted by text correction software, to which the control software is operatively connected locally or via a network. The corrected text is generated by automatically replacing words and phrases of the target vocabulary in the received text with terminology words according to a word assignment table in text form. The assignment table assigns at least one word from the target vocabulary to each of a plurality of terminology words. The at least one word of the target vocabulary assigned to the terminology word is a word or phrase that the speech-to-text conversion system misidentifies when the terminology word is input in the form of a voice signal. The control software is configured to use the correction text to cause the at least one laboratory device and / or at least one laboratory software module to perform an analysis, synthesis and / or software function according to the information in the correction text.

[0027] For a number of reasons it may be advantageous to use a speech-to-text system that does not support terminology, while subsequently correcting the text based on the assignment table.

[0028] On the one hand, there are already a range of general language speech-to-text conversion systems, such as Google's "Speech to Text" service, which have high accuracy for at least general language words and phrases, are sometimes free to use, and can be integrated into proprietary software programs through open APIs. Speech input in laboratory environments often includes a mixture of technical terms and general language words, so it is necessary to ensure that at least general language words are correctly converted to text. The speech-to-text conversion systems of Google and other cloud service providers are constantly evolving, and manufacturers of laboratory equipment and laboratory management systems generally cannot provide the same quality of service as large cloud service providers such as Google, Amazon, and Apple.

[0029] However, the speech-to-text conversion system of the described universal language itself is not suitable for use in a laboratory environment, because this language does not support terminology (" terminology") to the text conversion system. In biology and especially in the chemical industry, a large number of terms that do not appear in the universal language are used in a laboratory environment. They cannot be identified by the speech-to-text conversion system, and the universal language words with similar pronunciations are therefore mistakenly identified and converted to text. Especially in a chemical laboratory environment, the high precision of speech recognition is also particularly important. In everyday language, small mistakes are normally identifiable and can be easily corrected or compensated (such as the mistaken identification of single / plural forms can not cause the corresponding input in the Internet search engine to return significantly different results) by the user or receiving system, and under the environment of chemical synthesis, very small deviations (such as "double" instead of "three") may cause "identification" to go out a material that is completely different from the speaker's true meaning, and the product caused is unavailable. Therefore, the described speech-to-text conversion system designed for daily use is not suitable for use in a biochemical laboratory with corresponding danger.

[0030] Sometimes, speech-to-text conversion systems are designed specifically for the relevant objects and vocabulary of a particular profession. For example, Nuance offers "Dragon Legal" software for legal professionals, which includes legal terminology in addition to everyday language vocabulary. However, the disadvantage is that the vocabulary required in a laboratory, such as the production and analysis of paints and varnishes, is highly specialized and dynamic, so speech recognition software that uses chemical terminology, such as that found in a standard chemistry textbook, is generally not suitable for practical use in a specific company or a specific branch of the chemical industry. This is because laboratories often use substance trade names. These trade names may change, or a large number of new trade names for related products may be added annually. In particular, a large number of other products and product variants that can be used to manufacture paints and varnishes are launched on the market under new trade names every year. Even if a speech-to-text conversion system were as accurate as Google or Apple's everyday language systems and would include the most important chemical terms (which is not the case), the system would still be unsuitable for practical application due to the dynamic and large number of terms that are crucial in chemical laboratories, especially in the manufacture of paints and varnishes, because most practical terms would not be supported, or at least the vocabulary would be completely outdated after a few years.

[0031] According to an embodiment of the present invention, this problem is solved as described above by using a speech-to-text conversion system that is known to not support the relevant terminology. Therefore, no expensive and complex special development was attempted from the outset, which would only serve a very small market segment and would therefore hardly be able to achieve the recognition accuracy of well-known large-scale conversion systems such as Amazon, Google, or Apple, because this involves general language concepts, which, in addition to chemical terms, should generally be considered and correctly recognized in speech input. Instead, an embodiment of the present invention utilizes the already very good recognition accuracy of existing service providers for general language concepts and performs corrections before outputting the recognized text. During the correction process, incorrectly recognized words are replaced by terms according to the allocation table, thereby generating the corrected text that is ultimately output. Due to the dynamics of the field and the large number of market participants, products, and corresponding product names, highly specialized terminology vocabulary should be continuously updated to keep the software practical, so it is ultimately placed in the allocation table. This is a relatively simple way to keep it up to date. Therefore, according to an embodiment of the present invention, there is no need to modify the program source code and statistical language model, nor is there any need to recompile and / or retrain the machine learning program; modifying or supplementing the table is sufficient.

[0032] New terms can be added in such a simple way that the new terms are added to the allocation table together with one or more corresponding target vocabulary words that are incorrectly recognized for the terms. Thus, in terms of technology, the storage and updating of terms are completely separated from the actual speech recognition logic. This also has the advantage of avoiding dependence on a specific provider of speech recognition services. The field of speech recognition is still in its infancy, and it is not yet possible to foresee which of the numerous parallel solutions will be the best choice in the long term in terms of recognition accuracy and / or price. According to an embodiment of the present invention, the association with a specific speech-to-text conversion system is only carried out in this way, that is, first the received speech signal is sent to the conversion system and the (error) text is received. In addition, the allocation table contains the incorrectly recognized words of the target vocabulary, which have been returned by the special conversion system for a specific term (error). However, both can be easily changed by using different speech-to-text conversion systems to generate the (error) text and, for this purpose, recreating the allocation table with the help of the different conversion systems. There is no need, for example, to make complex changes to the logic of the grammar parser and / or the neural network.

[0033] According to embodiments of the present invention, the combination of a speech-to-text system for everyday language may also be advantageous for field service employees in the chemical industry or chemical production, as these employees often use computers or at least one smartphone in the course of their work activities. Voice input into calibration software, such as in the form of an application or browser plug-in, allows them to focus more on their clients or their activities than inputting text via a keyboard. These users can also access and control the functions of laboratory equipment and / or laboratory software modules of the laboratory system via client software on their terminals, which interacts with control software to forward the user's voice signals to the speech-to-text system via the control software.

[0034] Another advantage may be that, depending on the embodiment, the portable device only captures the voice signal, forwards it to the control software, and optionally receives and outputs a function execution result based on the information in the voice input. The actual speech-to-text conversion of the voice signal into text, a computationally intensive step, is performed by the speech-to-text conversion system. The speech-to-text conversion system can, for example, be a server connected to the control software via a network, such as the internet. This allows portable devices with low processor performance, such as smartphones or single-board computers, to input long and complex voice inputs, while conserving the computer resources on which the control software is installed.

[0035] According to an embodiment of the present invention, a term word is a word from one of the following categories:

[0036] - the names of chemical substances, in particular paints and varnishes, in particular also the chemical names according to chemical naming conventions, for example according to IUPAC nomenclature;

[0037] - the physical, chemical, mechanical, optical or tactile properties of the substance;

[0038] - the names of laboratory equipment and chemical industry equipment (e.g. trade names or proprietary names assigned by users to laboratory equipment);

[0039] -Names of laboratory consumables and laboratory supplies;

[0040] - A trade name in the paint and varnish sector.

[0041] According to an embodiment of the present invention, terminology words are words from the field of chemistry, in particular the chemical industry and in particular the field of paint and varnish chemistry.

[0042] According to an embodiment of the present invention, the target vocabulary consists of a set of common words.

[0043] According to other embodiments of the present invention, the target vocabulary is composed of a collection of general terms and their derivatives. For example, these derivatives can be dynamically created series combinations of two or more general terms. For example, in German, many words, especially nouns, are composed of a plurality of other nouns. For example, the word "ship propeller" (Schiffsschraube) is very common, so that it generally appears in most general language dictionaries. However, most general language dictionaries do not include relatively rarely used terms such as "fastening screw" (Befestigungsschraube). However, some speech-to-text conversion systems can also recognize words such as "fastening screw" (Befestigungsschraube) by means of heuristics and / or neural networks, provided that the separate word components "fastening" (Befestigung) and "screw" (Schraube) are components of the target vocabulary. Therefore, in this sense, the word "fastening screw" (Befestigungsschraube) also belongs to the target vocabulary of such speech-to-text conversion systems.

[0044] According to other embodiments of the present invention, the target vocabulary is composed of a collection of general words supplemented by words formed by combining recognized syllables. Therefore, in terms of which words can be recognized, the speech-to-text conversion system is more flexible because recognition can at least be carried out at the level of individual syllables rather than just individual words. But syllable-based recognition is also particularly prone to error because the risk of misidentifying words that do not exist in the known vocabulary is particularly large. Because the limited number of supported or known syllables and the restriction of the typical word length on the number of combinable syllables, the number of target words that can be generated based on syllables is also limited. Therefore, although the speech-to-text conversion system that supports word formation based on syllables has higher flexibility, it also has a limited target vocabulary. Even if such a system can dynamically recognize many chemical terms that are not included in previously known dictionaries due to its flexibility in theory, in practice the recognition accuracy is very low, so that in practice such a system ultimately has a target vocabulary that does not include or support these chemical terms.

[0045] In some embodiments of the present invention, the target vocabulary consists of a collection of common words supplemented by their derivatives and words formed by recognizing syllable combinations. These conversion systems are also based on target vocabularies that do not contain terms or that are not sufficiently accurate in actual use to recognize terms, but instead misidentify other words, often common terms, and convert them into text.

[0046] Thus, a wide variety of speech-to-text conversion systems available today can be used with embodiments of the present invention, even if these systems generally only "support" everyday speech (ie, are sufficiently accurate to correctly recognize and convert to text).

[0047] According to an embodiment of the present invention, a computer system performing text correction, e.g., a computer with control software or a correction computer, receives or calculates frequency information for at least some words in a text generated by a speech-to-text conversion system based on a speech signal. The frequency information indicates, for each word in the text, the statistically expected frequency of occurrence of the word.

[0048] When generating the correction text, only those words of the target vocabulary in the received text whose statistically expected occurrence frequency according to the received frequency information is below a specified threshold are selectively replaced by terminology words according to the allocation table.

[0049] This can be advantageous because user voice input typically contains a mix of common words and technical terms. Therefore, it's possible that the text received by the conversion system contains words from the target vocabulary that are assigned to corresponding terms in the assignment table and would normally be replaced. For example, the returned text might contain the phrase "Polymer Innovation." Because this phrase is assigned to the term "Polymerization" in the assignment table, it would normally be replaced by "Polymerization" during text correction. However, if the frequency information assigned to the phrase "Polymer Innovation" indicates a high probability of occurrence, the correction software might assume that the phrase "Polymer Innovation" is correct based on this frequency, despite being assigned to the corresponding term in the assignment table, and therefore retain the phrase "Polymer Innovation" unchanged in the text. For example, contextual analysis of words within a sentence or the entire voice input might indicate that the word "Innovation" frequently appears alone in the text, such as in a field employee describing the advantages of a polymer product. In this context, the phrase "Polymer Innovation" could also represent a correctly recognized phrase. This probability decreases if neither "polymer" nor "innovation" is mentioned alone. Regardless of the context, words themselves, which can be derived from a large text corpus, for example, also already have different probabilities of occurrence from each other.

[0050] It may be advantageous to replace words according to an assignment table based on the probability of their occurrence in the received text, since this avoids the situation in which, in a few individual cases, a word in the target language itself or in the context of the corresponding text with a high probability of occurrence is incorrectly replaced by a term, which would result in an error rather than a correction.

[0051] According to one embodiment, the frequency of occurrence of the words in the text is calculated by the speech-to-text conversion system and returned to the control software by the speech-to-text conversion system together with the text. For example, the speech-to-text conversion system can use a hidden Markov model (HMM) to calculate the probability of a certain word occurring in a sentence environment. As a supplement or alternative, the speech-to-text conversion system can equate the frequency of occurrence of a word with the frequency of occurrence of the word in a large reference corpus. For example, the entire text of a newspaper over several years or another large text data set can be used as a reference corpus. The ratio of the word count in the corpus to the total number of words in the corpus is the frequency of occurrence of the word observed in this reference corpus. The frequency information can be sent to the control software by the speech-to-text conversion system together with the text.

[0052] According to another embodiment, after receiving the text, the control software or the correction software calculates the word frequency of the text. As described above, the probability of occurrence of each word or phrase can be calculated using an HMM, taking into account the textual context of the word or based on the frequency of the word in a reference corpus. For example, the entire text previously transferred from the speech-to-text conversion system to the control software or the correction software can be used as the reference corpus.

[0053] Therefore, according to an embodiment, the frequency information is calculated using a hidden Markov model (e.g., by control software or calibration software). For example, the expected frequency of occurrence, i.e., the probability of occurrence, can be calculated as the product of the emission probabilities of the individual words of a single word sequence, as described, for example, in B. Cestnik, "Estimating Probability: A Key Task in Machine Learning" (Proceedings of the 9th European Conference on Artificial Intelligence, pp. 147-150, Stockholm, Sweden, 1990).

[0054] According to an embodiment of the present invention, a program for performing text correction receives not only the text but also the part-of-speech tags (POS tags) for at least several words in the text, which is generated by a speech-to-text conversion system from a speech signal. The part-of-speech tags are received from the speech-to-text conversion system directly by the control software or by the correction software while bypassing the control software, and include at least tags for nouns, adjectives, and verbs. It is also feasible that the part-of-speech tags include additional types of syntactic or semantic tags. The exact composition of the POS tags considered may also depend on the corresponding language. In an allocation table, terminology words are stored in an associated manner together with their POS tags. When generating the corrected text, according to the allocation table, only the words whose part-of-speech tags of the target vocabulary in the received text are replaced by terminology words.

[0055] This can be advantageous because the accuracy of the text correction step will be improved thereby. It can be assumed that the POS tags in the assignment table are correct because the entries in the table are created semi-automatically in that one or more speakers input a term word or a term phrase into a microphone, the resulting audio signal is converted by the speech-to-text conversion system into a (wrong) word or a (wrong) phrase of the target vocabulary, and the wrong word or the wrong phrase is stored in the assignment table in an associated manner together with the term phrase. Since it is known what the term represents and whether it is, for example, a noun, a verb or an adjective, the term phrase can also be stored in an associated manner together with the correct POS tag when creating or updating the table. Therefore, if a word or a phrase in the text should be replaced by a term word according to the assignment table, but the part-of-speech tag of the text to be replaced does not match the part-of-speech tag of the term word, this indicates that the corresponding word in the text may still be correct. The recognition rate of the POS tags is high, so that the quality of the correction step can be improved by this measure. For example, it is possible that the term word is, for example, a trade name. This term refers to a thermoplastic polyurethane film from Covestro. In the table, the part-of-speech tag "noun" is assigned to this term. The speech-to-text conversion system knows that it often incorrectly converts the spoken word "Platilon" into the target vocabulary word "Platin" (platinum), so it assigns the target vocabulary word "Platin" to the term "Platilon" in the assignment table. However, in the user's current voice input, the word is used as an adjective: "Add a platinum-based or zinc-based catalyst [...]." Based on the part-of-speech tag for "Platin" in the text returned by the conversion system, it can be recognized that the word "Platin" is correct here and should not be replaced by "Platilon."

[0056] According to an embodiment, the control software is provided by a cloud computer system or a separate server computer system in the form of a cloud service.The cloud computer system or the server system may be a system operated by a laboratory operator such as a university or a company, for example.

[0057] According to one embodiment, the control software performs text corrections either on its own or by interoperating with a program installed locally on a computer common to the control software.

[0058] According to another embodiment, the correction of the text that speech-to-text conversion system receives is performed by another computer, and it is connected to the computer of control software by a network.Another computer can be for example a cloud system or a separate server, which for example is also operated by a laboratory operator.Control software will receive text and send it to the software and computer that corrects according to the allocation table, and receives the correction text from this software or computer (" correction software " or " correction computer ").The network connecting control software and remote correction software can be for example the Internet or an intranet.This embodiment may be advantageous, because control software and correction software can be better separated to the access rights of function and data.If text correction is carried out on an independent correction computer or correction cloud system, then the user can be given the authority of selective access correction computer or correction cloud system to update form at this, and do not or need not also grant the access rights to the sensitive data of the control software that can check for example laboratory equipment key functions thus.

[0059] According to an embodiment of the present invention, the coordination of data exchange with the speech-to-text conversion system, text correction, and forwarding of the corrected text to the execution system is thus completely performed or organized and coordinated by the control software. According to some embodiments of the method, the portable device is thus essentially a device having a microphone and, optionally, an output interface for the text execution result of the correction.

[0060] According to an embodiment of the present invention, the speech-to-text conversion system is implemented as a service provided to multiple terminals via the Internet. For example, the speech-to-text conversion system can be a cloud computing system with Google's "Speech-to-Text" cloud service. This can be advantageous because a powerful API client database is available for this purpose, such as for .NET.

[0061] The portable device may include client software preconfigured for data exchange with the control software. This means that the client software on the portable device is configured to transmit voice signals to the control software via a network, so that the control software can facilitate the conversion of the voice signals into text, the correction of terminology in the text, and the execution of functions based on the corrected text. In response to the transmission of the voice signals, the portable device receives the results of the execution of the corrected text and outputs them through a speaker.

[0062] According to an embodiment of the present invention, the words and phrases in the target language stored in the assignment table represent erroneous text outputs of a speech-to-text conversion system, which are generated based on the terminology speech inputs of a large number of different people.

[0063] According to an embodiment of the present invention, the method includes a distribution table generation step. For each of a plurality of terminology words, at least one reference speech signal is recorded that selectively reproduces the terminology word. The reference speech signal originates from at least one speaker. Similarly, for terminology phrases, at least one reference speech signal that selectively reproduces the terminology phrase can be spoken by at least one speaker and recorded. The remaining steps are essentially the same for words and phrases, so when referring to terminology words, terminology phrases are also included below. Each recorded reference speech signal is input into a speech-to-text conversion system. Input can be performed, in particular, via a network, such as the Internet. For each input reference speech signal, a device to which the reference signal has been input receives at least one word of a target vocabulary generated by the speech-to-text conversion system from the input reference speech signal. The device can, for example, be any data processing system equipped with a microphone and connected to the speech-to-text conversion system. The reference speech signal is preferably input via a device that is as similar as possible to the portable device in terms of microphone type and / or positioning relative to noise sources to ensure that the same errors are reproducibly generated. Because the target vocabulary of the speech-to-text conversion system does not support terminology words, the at least one word (or phrase) of the target vocabulary received for each terminology word represents an erroneous conversion. Finally, an assignment table is generated as a table which assigns to each term word for which at least one reference speech signal has been acquired at least one word of the target vocabulary in text form which has been returned by the speech-to-text conversion system for the corresponding term word.

[0064] This can be advantageous because the table can be easily modified and supplemented without having to change the source code, recompile the program, or retrain the neural network. Even when using a different speech-to-text system, all that's required is to adapt the corresponding client interface and have one or more speakers re-enter the table's terminology and phrases using a microphone and transfer them to the new speech-to-text system. The incorrect words and phrases in the target language returned by the new system form the basis for the new assignment table. This makes it possible to functionally expand any everyday speech-to-text system to correctly convert spoken text containing terminology and phrases into text without requiring extensive or complex modifications or retraining the language software.

[0065] The allocation table may be stored, for example, as a table in a relational database or as a tab-delimited text file or other functionally similar data structure.

[0066] According to an embodiment of the present invention, for each of at least a plurality of terminology words (or terminology phrases), a plurality of reference speech signals corresponding to different speakers are recorded. The plurality of reference speech signals reproduce the terminology word (or terminology phrase). An assignment table assigns a plurality of words (or phrases) of a target vocabulary in text form to each of the at least a plurality of terminology words (or terminology phrases). The plurality of words (or phrases) of the target vocabulary represent error conversions generated by a speech-to-text conversion system for different speakers based on their voices.

[0067] For example, a specific term such as "1,2-methylenedioxybenzene" can be spoken by 100 different people and recorded accordingly using a microphone as reference voice signals. These people are preferably people who are familiar with the pronunciation of chemical phrases. Therefore, there are 100 reference voice signals for this substance name. Each of these 100 reference voice signals is sent to the speech-to-text conversion system, and in response, 100 words or phrases of the target vocabulary are returned, none of which correctly reproduce the actual term name. Usually, the 100 words returned are identical to each other, but not always. Different people have different voices, that is, the voice input varies in intonation, volume, pitch and clarity. Therefore, a speech-to-text conversion system may return multiple different, incorrect words or phrases for a certain term (or a certain term phrase), all of which are included in the allocation table.

[0068] It may be advantageous to consider the speech inputs of many different persons for the creation of the assignment table, since this better takes the diversity of human voices into account and thus a higher error correction rate can be achieved.

[0069] According to an embodiment of the present invention, the at least one laboratory device is a plurality of laboratory devices. The laboratory devices are all selected from the following group, the group comprising:

[0070] - analytical stations designed for the analysis of chemical substances or mixtures; and / or

[0071] - a synthesis station designed for the synthesis of chemical substances or mixtures; and / or

[0072] - Pre- or post-processing stations designed to purify chemical substances before or after analytical or synthetic steps, to combine them with other substances, to dilute them, to concentrate them or to modify them in another way, so as to enable subsequent work-up or transport steps.

[0073] According to an embodiment of the present invention, the at least one laboratory device is a plurality of laboratory devices, each of which functions as a processing station for a respective chemical substance. The laboratory devices and, optionally, the at least one laboratory software module can be components of a laboratory facility that additionally includes a transport unit. The transport unit is designed to transport substances for synthesis or analysis to the processing stations, thereby enabling the processing stations to perform at least one processing step on the substances.

[0074] According to an embodiment of the present invention, the laboratory system further comprises a master control computer of a laboratory facility comprising at least one laboratory device, wherein the master control computer comprises master control software configured to schedule processing steps and chemical substance transportation by the laboratory devices of the laboratory facility based on instructions in the form of structured text.

[0075] According to an embodiment of the present invention, the control software includes an NLP processor (NLP: Natural Language Processing) or is interoperable with an NLP processor connected via a network. "One software application is interoperable with another software application" refers to the mutual coordination and adaptation of the interfaces and routines of the two interoperable software applications, enabling them to exchange requests, control commands, and data in a coordinated manner, thereby executing a software-based function through the coordinated exchange and returning the results to the requesting software application. The control software is configured to use the NLP processor to transform the correction text into structured text that can be interpreted by a target system. The target system is the main control software of a main control computer of at least one laboratory software module and / or at least one laboratory device and / or a laboratory facility including at least one laboratory device. The main control software is configured to receive the structured text generated by the NLP processor based on the correction text from the NLP processor and input it into the target system. By inputting the structured text into the target system, it is prompted to execute laboratory-related software functions and / or hardware functions.

[0076] For example, M. Hummel, D. Porcincula, and E. Sapper describe in their article “Natural Language Processing: A Semantic Framework for Coatings Science – Recipes for Robotic Reading” in the European Coatings Journal (February 1, 2019) how to convert natural language text input into structured text using parsers, part-of-speech tags, and other techniques.

[0077] For example, the target system may be at least one laboratory device and a transport unit, and the input of the structured text causes them to execute a plurality of processing steps and transport chemical substances between the laboratory devices according to the information contained in the structured text.

[0078] According to another example, the target system is control software for a laboratory system, and the input of a structured document causes it to execute software functions based on the information contained in the structured document. The software functions may, for example, calculate certain concentrations or other parameters that are used, for example, to configure or calibrate laboratory equipment or to adjust pH and viscosity during the execution of certain analyses and / or syntheses defined in the document.

[0079] According to another example, the target system is a chemical database of a laboratory system, which contains the results of analyses and syntheses performed by the system. The input of structured text prompts the system to perform a database search based on the information and terminology contained in the structured text. The search results can be returned directly to the user as results via the control software, or can include favorable formulations based on the searched substances as input for further data processing steps (such as simulations and predictions).

[0080] According to an embodiment, a laboratory system includes chemical equipment for analyzing and / or synthesizing chemical substances and / or equipment for producing mixtures, particularly paints and varnishes. The system also includes a master computer with master control software. The at least one laboratory device and the at least one laboratory software module are components of a high-throughput system and can be checked and controlled by the control software via the control software. The system is designed to interpret a correction text as a detailed description of the synthesis, a detailed description of the components of the mixture, or a detailed description of the analysis to be performed.

[0081] The device may be a high throughput device (HTE device), for example a high throughput device for analyzing and producing paints and varnishes. For example, the HTE device may be a system for automatically testing and automatically producing chemicals as described in WO 2017 / 072351 A2.

[0082] By outputting the correction text or the structured text converted from the correction text to the control software of the device, the control software is prompted to control one or more laboratory devices and / or one or more laboratory software modules of the device as specified in the correction text. This can be particularly advantageous in biological or chemical laboratory environments, because the voice input is processed in such a way that it can be directly transmitted to the technical system and correctly interpreted by it, without the user having to remove gloves, for example. For example, the laboratory device can be a device for determining the oil absorption value of a pigment or a device for determining the viscosity of a paint.

[0083] The device can be designed, for example, to fully automatically execute one or more of the following work steps in response to a correction text input via a machine-to-machine interface or a structured text obtained therefrom:

[0084] - Rheological analysis of substances and mixtures;

[0085] - measuring the storage stability of substances and mixtures, in particular with regard to the inhomogeneity and precipitation tendency of liquid mixtures, for example, the analysis can be carried out by optical measurement in a cuvette after sampling;

[0086] - Determine the pH value of substances and mixtures;

[0087] - foam testing of substances and mixtures, in particular the measurement of the defoaming effect and the measurement of the foam collapse kinetics;

[0088] - Viscosity measurement of substances and mixtures. The viscosity measurement can include an automatic dilution step, especially in the case of highly viscous substances or mixtures, since the viscosity in the diluted solution can be determined more easily. The viscosity of the initial substance or mixture is calculated based on the viscosity of the diluted solution.

[0089] - measuring the rubbing behavior of substances or mixtures, and in particular finished products (abrasion test);

[0090] - measuring the color values ​​(so-called LAB values), haze and gloss of substances and mixtures by means of a spectrophotometer, for example, which works with light scattering;

[0091] - measurement of the layer thickness of substances and mixtures applied to a surface according to various specified parameters (temperature, humidity, surface characteristics of the surface, etc.);

[0092] - Image analysis processing of images of substances and mixtures, in particular for the characterization of surfaces, e.g. the number, size and distribution of bubbles or scratches in paints and varnishes.

[0093] The substances and mixtures can in particular be substances and mixtures for the production of paints and varnishes. In addition, the substances and mixtures can be end products such as paints and varnishes in liquid or dry form as well as intermediate products and used solvents.

[0094] According to an embodiment of the present invention, the non-intervention operation includes: converting the speech input into text, especially into natural language text.

[0095] The at least one laboratory software module can be designed, for example, as a chemical substance database, which is designed to interpret input text as search input, determine information related to the search input in the database, and return it as results to the control software. The database can, in particular, be a database management system (DBMS) such as MySQL or PostgreSQL.

[0096] Additionally or alternatively, the laboratory system may include a laboratory software module as simulation software. This simulation software is designed to simulate the properties of chemical products, in particular paints and varnishes, based on predefined mixture specifications. The simulation software is designed to interpret input text as a detailed description of the product whose properties are to be simulated.

[0097] In addition or as an alternative, the laboratory system may include a laboratory software module designed as control software for a laboratory system comprising at least one laboratory device. The control software is installed on a master computer. For example, it may be the master control software described with respect to the embodiments. The master control software is designed to cause the laboratory devices and transport units of the laboratory system to execute and orchestrate a large number of processing steps and the transport of chemical substances between the laboratory devices based on information contained in input text. The laboratory facility may be designed as HTE equipment for controlling chemical analysis and / or synthesis and / or the production of mixtures, particularly paints and varnishes. The control software is designed to interpret the correction text or structured text extracted therefrom into detailed instructions for the composition of the mixture or the analysis or synthesis.

[0098] Additionally or alternatively, the laboratory system may include laboratory software modules designed to perform simple data processing steps on the data, interacting between the components of the laboratory system during analysis or synthesis. This may include, for example, the conversion, checking, and / or supplementation of control commands from the control software to the laboratory equipment, the storage, checking, or standardization of measurement data, or similar data processing steps.

[0099] The corresponding text input into the laboratory device or laboratory software module may be a corrected text or a structured text generated by an NLP processor from the corrected text.

[0100] According to an embodiment, the control software is configured to use at least a portion of the correction text, for example, text containing a keyword such as "Internet search," as input to an Internet search engine. The control software receives the search results, for example by parsing a browser window, and transmits the output results to a target system near the user, such as a portable device with a speaker.

[0101] According to an embodiment of the present invention, the control software includes a register with a large number of user accounts registered with the control software. The control software also includes voice recognition functionality for identifying the corresponding registered user based on their voice. The control software is designed to allow the user to non-invasively operate only the laboratory equipment and / or laboratory software modules for which the user has permission, using voice input into a microphone.

[0102] This can be advantageous because access control and regulation of laboratory equipment and laboratory software modules is seamless, as both rely on the user's voice. No passwords or special access tokens need to be remembered or securely stored.

[0103] According to an embodiment of the present invention, control software allows a user to select a corresponding laboratory device from a large number of laboratory devices by entering a key term, in particular a laboratory device name and / or a room name, as part of a voice input, and selectively control its functions via voice input. For example, the control software is configured to analyze a correction text based on specified key terms. If a key term is recognized, the control software forwards the correction text in the form of natural language text or structured text to the corresponding laboratory device identified thereby or its control software. This can be advantageous because a large number of different laboratory devices can be controlled via a common voice interface.

[0104] According to an embodiment of the present invention, the control software is configured to automatically convert correction text created based on user voice input into a structure that can be correctly interpreted by the target system specified for the text input. For example, the control software dynamically identifies the target system selected by the user from a large number of available target systems by mentioning the name of a laboratory device or laboratory facility in the voice input to perform a specific function. The control system stores the format required for each target system, which is the input format required by the corresponding target system, and automatically converts the correction text or structured text obtained from it into the format required by the dynamically identified target system. For example, a first laboratory device may require certain parameters in the form of an XML file as input. Another device may have a natural language processor and may directly use the correction text as input. Another laboratory device may require certain parameter values ​​or recipes in a table format. This can be advantageous because the user can enter the voice input in a natural language sentence using normal syntax. Therefore, the user does not have to laboriously memorize the required grammar for each laboratory device or speak "differently" to each individual instrument. Therefore, it is not necessary to specifically match the user's language structure to the vocabulary or syntax of each laboratory device. The user can enter his input in the language syntax he is accustomed to, which he also uses when talking to his colleagues. The control software is responsible for meeting the special requirements of the interface of each laboratory device.

[0105] In another aspect, the present invention relates to a method for controlling a laboratory device and / or a laboratory software module. The method comprises:

[0106] - providing at least one laboratory device and at least one laboratory software module, wherein the at least one laboratory device is designed for analysis and / or synthesis of chemical substances, wherein the at least one laboratory software module is designed for processing data obtained from the at least one laboratory device;

[0107] - providing a data processing device with control software, which provides a user with an interface for operating at least one laboratory device and / or at least one laboratory software module; and

[0108] - providing a portable device with a microphone, wherein the device is interoperably connected to the control software via a network, wherein the device is designed to allow the user to operate the at least one laboratory device and / or the at least one laboratory software module non-invasively by voice input into the microphone when interoperating with the control software.

[0109] According to an embodiment, the providing of the device comprises placing the portable device in a common room with the at least one lab device and / or placing the portable device in a common room with a host computer of a lab facility containing the at least one lab device.

[0110] "Structured text" here refers to text, particularly ASCII text, that contains words and phrases with specific meanings in defined locations. For example, structured text can take the form of comma-delimited files, tables, XML documents, and so on. Structured text is easier for machines and computers to interpret and process than natural language text.

[0111] In this context, “natural language text” refers to text, in particular ASCII text, which is expressed in a language spoken by humans and usually consists of complete sentences.

[0112] An "NLP processor" is a software program or software function that can understand and process "natural language" text, i.e., extract relevant information from the text and optionally convert it into another format, such as structured text. Thus, NLP tools can process text consisting of entire sentences as well as extract information from single sentences.

[0113] "Transport unit" refers to an automatic system consisting of one or more components, which can transport chemical substances in liquid and / or solid form and optionally other objects such as consumables or containers from one processing station to the next processing station. The transport unit can be, for example, a robotic arm, a collection of robotic arms, a conveyor belt, a collection of conveyor belts, or a combination thereof. The transport unit can be an integral part of a high-throughput system and can be controlled by control software. The processing unit can be a laboratory device for analyzing or synthesizing chemical substances, or a device for pre-treatment and post-treatment (dilution, concentration, dyeing, mixing, etc.) of the substance.

[0114] A "speech-to-text conversion system" is a data processing system for converting human speech signals into text. The data processing system may be composed of a combination of hardware and software and may be designed as a cloud computer system or a server computer system, for example, that provides speech-to-text conversion services to one or more client devices via a network.

[0115] "Non-intervention operation" of a device means operation that does not involve any manual interaction between the user and the device or its human-machine interface. Therefore, it is a "non-touch" or "contactless" operation.

[0116] A "single-board computer" or "SBC" is a computer system in which all the electronic components required for operation are integrated onto a single circuit board. Typically, the power supply is installed separately as the only component.

[0117] A "voice signal" is an electronic signal picked up by a microphone when a person inputs voice content into the microphone.

[0118] A "virtual lab assistant" is software or a software routine that is effectively connected to one or more lab devices and / or software programs within a laboratory so that information can be received from these lab devices and lab software programs and commands for executing functions can be sent by the lab assistant to the lab devices and lab software programs. Thus, the lab assistant has an interface for exchanging data with and controlling one or more lab devices and lab software programs. The lab assistant also has a user interface and is configured to enable a user to more easily use, monitor, and / or control the lab devices and lab software programs through the interface. For example, the user interface can be designed as an acoustic interface or a natural language text interface.

[0119] The "virtual lab assistant" preferably includes a number of functions to facilitate interaction between the virtual assistant and a human in a manner that is as similar to human-to-human interaction as possible. For example, the virtual lab assistant may include a software function for reading text aloud in a voice that is as natural as possible. The read-out text may, for example, include the results of executing a function, which is output through the speaker of the portable device. In addition or alternatively, the virtual lab assistant may be configured to search the received voice signal or the text generated therefrom for keywords and to invoke certain hardware or software functions based on the keywords. For example, one of the keywords may be a name, such as an ordinary human name, which the virtual lab assistant interprets as its own name and understands as a command to execute or invoke a specific function.

[0120] "Laboratory equipment" refers here to electronic instruments used in laboratories, which are used to perform chemical processing steps such as synthesis or analysis or other processing steps, for pre- or post-processing of substances in relation to the next / previous processing step.

[0121] A "laboratory facility" is defined herein as a system consisting of multiple laboratory instruments, transport units, and one or more software modules that can be used to coordinate and control the laboratory instruments and software modules to automatically or semi-automatically execute a chemical workflow. For example, the workflow can be a synthetic workflow, an analytical workflow, or a combination of both.

[0122] A "laboratory software module" is defined herein as a software application, a submodule of a software application, or a software routine that is configured to process data provided by a laboratory device as measurement data or used as input parameters. A laboratory software module is preferably a physical and / or functional component of a laboratory device or laboratory equipment.

[0123] "Vocabulary" here refers to a language area, that is, the set of words available to entities such as speech-to-text conversion systems.

[0124] "Word" here refers to a coherent string of related characters that appears in a specific vocabulary and represents an independent linguistic unit. In natural language, unlike phonemes or syllables, words have inherent meaning.

[0125] A "phrase" here refers to a language unit consisting of two or more words.

[0126] A "term word" or "term" is a word of the term vocabulary. A term word does not belong to the target vocabulary and is usually not part of the common language vocabulary.

[0127] The statement "the speech-to-text system only supports the conversion of speech signals into the target vocabulary" means that words from another vocabulary are either not converted into text at all or only with a high error rate, where this error rate is higher than a limit error rate for each word or phrase to be converted. This limit value should be considered the maximum tolerance for the speech-to-text function. For example, this limit value could be more than 50%, preferably more than 10%, with respect to the error probability for each word or phrase.

[0128] A part-of-speech tag (POS tag) is a special marker ("label") assigned to each word in a text corpus to indicate the part of language it represents in its corresponding text context, and often also other grammatical categories such as tense, number (plural / singular), upper / lower case, etc. The set of all POS tags used in a corpus is called a tagset. Tagsets for different languages ​​are usually different from each other. A basic tagset contains tags for the most common language components (e.g., N for noun, V for verb, A for adjective, etc.).

[0129] A "portable device" is a portable, preferably battery-powered, data processing device that can be placed freely and connected to a network, such as the Internet or an organization's intranet, preferably wirelessly. For example, a "smart speaker" with a microphone and / or a single-board system (such as a Raspberry Pi computer) are used as a "portable device." However, it is also possible for a user to use their smartphone as a portable device, wherein the smartphone has a microphone and client software that can interact with the control software. Brief introduction of the attached figure

[0130] The embodiments of the present invention will be described in detail with examples in the following figures:

[0131] Figure 1 A flow chart illustrating a method for controlling laboratory equipment and laboratory software modules;

[0132] Figure 2 A block diagram illustrating a distributed system for voice control of laboratory equipment and laboratory software modules is shown;

[0133] Figure 3 A block diagram illustrating an alternative distributed system for voice control of laboratory equipment and laboratory software modules;

[0134] Figure 4 A block diagram of another alternative distributed system for voice control of laboratory equipment and laboratory software modules is shown.

[0135] Detailed description

[0136] Figure 1 A flow chart illustrating a computer-implemented method for voice control of laboratory equipment and laboratory software modules.

[0137] In a first step 102 , one or more lab-devices and one or more lab-software modules are provided.

[0138] A laboratory device can be a stand-alone device or a processing unit within a complex laboratory facility. A laboratory device can be configured and equipped to perform one or more work steps in a complex analytical or synthetic workflow, in the context of the analysis and / or synthesis of substances or mixtures, or in relation to the pre- and post-processing of substances and mixtures.

[0139] Similarly, a laboratory software module can be a separately installable software application, such as simulation software, that describes or predicts the chemical, physical, tactile, optical, or other properties of laboratory equipment based on data obtained from it. It can also be a DBMS with a database that already stores results from previous analyses and syntheses of laboratory equipment, the contents of which can be used, for example, to perform a database search for substances (such as paints and varnishes) that have been manufactured and whose properties are known. For example, a search request could be for all varnishes with a viscosity below a limit specified in the search request. Software modules can also be modules and functions of a larger software application, such as the control software for a complex laboratory facility.

[0140] In another step 104, a data processing device with control software is provided. The control software provides an interface for one or more users to operate at least one laboratory device and / or at least one laboratory software module. The data processing device with control software can be located in a room or laboratory area shared with the laboratory device or laboratory facilities containing the laboratory device. The data processing device can be, for example, a desktop computer or a server. However, the data processing device is preferably located in another room, such as in a data center of an organization operating the laboratory or in a data center of an external cloud storage provider. However, for example, for maintenance purposes, the data processing device and control software can be accessed by a computer in the laboratory area.

[0141] In step 106, provide and set up the portable device with microphone, make it be connected and interactive operation with control software by network such as the Internet or described mechanism intranet.This equipment is designed to allow the user to operate at least one laboratory device and / or at least one laboratory software module in a non-interventional mode by the voice input to microphone when interoperating with control software.Portable device is preferably provided like this, that is, this equipment is arranged near the work area of ​​laboratory staff, and laboratory staff wants to control laboratory device or laboratory staff from this work area and engages in its laboratory work there in other ways.Also feasible is that the device with microphone is placed in the center of the laboratory area, from here can basically always receive and gather the voice input from room or laboratory area whole range well.

[0142] According to an embodiment, providing the device comprises placing a portable device, such as a single-board computer with a microphone, in a room where at least one laboratory device is located and / or placing the portable device in a room where a main control computer of a laboratory facility including at least one laboratory device is located.

[0143] The steps described can be performed in any order. They allow one or more laboratory devices and their associated laboratory software modules to be operated in a hands-free manner using voice input from any area of ​​the laboratory, without having to remove gloves. Because the control software prompts the speech-to-text conversion system to convert the voice signal picked up by the microphone into natural language text, the text is then corrected using a special text correction step so that terminology from the voice input, for example, from the chemical industry or from specialized subfields such as paint and varnish production, is correctly reproduced in the text.

[0144] In different embodiments of the laboratory system, the steps performed by the control software, laboratory equipment, and laboratory software modules may vary slightly, and the laboratory system may include other components, such as an external text correction system and / or an NLP processor for converting natural language corrected text into structured text. However, in some embodiments, the laboratory facility itself includes such an NLP processor that performs this conversion, so that the natural language corrected text can be directly input into the control software of the laboratory facility.

[0145] Figure 2 A block diagram of a distributed system 200 for voice control of laboratory devices 252, 254 and laboratory software modules 248, 250 is shown. System 200 includes laboratory devices 252, 254 and laboratory software modules 248, 250, a control computer 213 with control software, and a portable device 212 with a microphone. Optionally, the system may include other portable devices 212 having substantially the same structure and function. Optionally, the system may also include a master computer with master control software for controlling the laboratory devices. Optionally, system 200 may also include calibration software 225, which may be implemented as a function of the control software or as a separate, locally or remotely installed software application. Optionally, system 200 may also include an NLP processor 258. Optionally, system 200 may also include a speech-to-text conversion system 226.

[0146] The laboratory equipment and laboratory software modules included in system 200 are controlled via voice input 204 by a user 202, such as a laboratory worker, who is physically close to a portable device 212 equipped with a microphone 214. For example, the user can enter the following command into the microphone: "Synthesize a paint named <varnish name> and determine the rheological properties of the paint after synthesis!" The voice signal 206 captured by the microphone in response to the voice input is forwarded by the portable device 212 via a network 236, such as the Internet or an intranet, to control software 222 installed on a control computer 213. The control software forwards the voice signal 206 via the network to a universal language speech-to-text conversion system 226, which converts the received voice signal 206 into text 208. The text consists solely of words and phrases drawn from a universal language target vocabulary 234 and does not include terminology such as paint names or the word "rheology." For example, the speech-to-text conversion system 226 can be designed as a cloud computing system that provides the conversion as a software-based service to multiple clients, referred to herein as a "speech recognition processor" 232. The speech-to-text conversion system 226 can be, for example, a Google Cloud computer system that provides Google's "speech-to-text" cloud service over the Internet. Thus, in this case, the interface 224 is a cloud-based API from Google. The speech-to-text conversion system returns the recognized text 208 to the control software 222.

[0147] according to Figure 2 In the embodiment shown, the control software includes a correction function 225. It has access to an assignment table 238 in which a plurality of terminology words and terminology phrases are correspondingly assigned one or more words or phrases of the target vocabulary.

[0148] In the assignment table 238, the words are associated with each other in text form. Specifically, the assignment table assigns at least one word from the target vocabulary to each of a plurality of term words or term phrases. The at least one word of the target vocabulary assigned to a term word (or term phrase) is a word or phrase that is incorrectly recognized by the speech-to-text conversion system when the term word is input into the speech-to-text conversion system in the form of an audio signal (and was previously incorrectly recognized when the table was created).

[0149] Optionally, POS tags can also be assigned to term phrases, which can optionally be evaluated and taken into account during text correction by the correction function 225. The correction function 225 can also be designed as a separate correction software application 225, which is installed on the control computer 213 and can interact with the control software 222.

[0150] Control software 222 transmits received text 208 to a correction function or correction software 225, prompting it to correct text 208 using a table 238. During the correction process, incorrectly recognized words and phrases in the target language are replaced with their corresponding terminology and phrases according to the table. The resulting corrected text 210 is transmitted directly from control software 222 to a target system, such as chemical plant 244, or is first forwarded to an NLP processor 258.

[0151] Figure 2 The NLP processor shown is installed on the control computer 213. However, according to e.g. Figure 3 In the alternative embodiment shown, the NLP processor also can be provided as a service through the network.The NLP processor analyzes the syntactic structure of proofreading text 210, to understand the semantic content of proofreading text and therefrom extract relevant information and change it into structured text 211. Proofreading text 210 is changed into the step of structured text 211 through proofreading and may be advantageous, because many laboratory systems and laboratory software modules available on the market today can not correctly interpret natural language text.But many target systems all possess the input interface that is used for structured text, promptly for example be used for form or XML file with specific regulation structure.In other embodiments, target system comprises the NLP processor of oneself, thereby can be in the proofreading text 210 input target system of natural language form.In this case, control software directly sends proofreading text 210 to target system.

[0152] The target system can be, for example, a single piece of lab equipment or a lab facility comprising a plurality of lab equipments. Figure 2 The illustrated lab facility 244 may be a high throughput system (high throughput environment / HTE device) comprising a plurality of lab devices 252 , 254 and one or more lab software modules 250 , 248 .

[0153] For example, laboratory equipment 252 and 254 may be analytical instruments configured to perform various chemical analyses (e.g., viscosity, pH, color, surface structure, layer thickness). For example, software module 250 may be a chemical database that interprets input text 211 or 210 as a search request and performs a corresponding search within the chemical database. Laboratory software module 248 is control software installed on main control computer 246 of chemical plant 244 and configured to coordinate the work steps performed by the various laboratory instruments within the plant. For example, control software 248 not only controls the analytical steps performed by the analytical instruments, but also controls the operation of transport unit 256, which transports substances and intermediate products from one processing station to the next within chemical plant 244.

[0154] Controlling chemical plants using voice input can be particularly advantageous, as these plants can now essentially automatically produce chemical products, such as paints or varnishes, based on a recipe. Thus, it is possible to input a recipe for producing a chemical substance or mixture using voice recognition, thereby causing chemical plant 244 to automatically produce the corresponding substance or mixture. Similarly, it is possible to initiate a corresponding substance analysis or control, or perhaps interrupt, modify, or stop its execution, using voice input.

[0155] Internal database 250 can be used not only to store chemical substances and mixtures and their properties, but also to store corresponding formulas. Thus, database 250 can contain, for example, the formulas for paints and varnishes, their raw materials, and their corresponding physical, chemical, optical, and other properties. Furthermore, other relevant data can be stored in the database, such as product data sheets from substance manufacturers, safety data sheets, and configuration parameters for various modules of the HTE equipment used to analyze or synthesize certain substances or products. HTE equipment is designed to perform analyses and synthesizes based on formulas and instructions entered in text form.

[0156] In some embodiments, one or more laboratory devices in a chemical plant have their own interface, through which they can directly receive natural language or structured correction texts 210, 211 from the control software. In some other embodiments, the laboratory equipment of the chemical plant can only receive correction texts 210, 211 indirectly from the control software 222 via the main control software 248.

[0157] Depending on the embodiment, the voice input includes key terms, which the control software 222 uses to determine the number and identity of the corresponding target systems. The control software then sends a correction text to each identified target system in natural language or structured form. For example, by entering the keyword "target system" along with the name of each target system (laboratory equipment or software module), the user can determine to which target device the control software sends the correction text. However, in addition or alternatively, the control software can also independently search for key terms and autonomously determine the appropriate target system based on the term. For example, the voice input can only determine a certain type of synthesis, and the control software dynamically determines which instrument from the large number of laboratory equipment available for synthesis will be used for the requested synthesis. For example, this determination can be based on the current load situation, thereby selecting laboratory equipment that is currently unloaded or has a low load rate.

[0158] The result 242 of executing a specific hardware or software function is preferably returned from the target system to the control software 222. The control software can now return the result of executing the function to the user 202. Various channels can be used for this purpose. For example, an email, SMS or a message in a different format can be sent to the user, containing the result of executing the function. Figure 2In the embodiment shown, the result 242 is at least also (or only) returned to the portable device 212 and output via its loudspeaker 218. This is particularly advantageous because, no matter in terms of inputting control commands, chemical formulas and search requests or in terms of receiving the execution results of corresponding hardware or software functions, the laboratory staff 202 has the maximum freedom of movement. For example, the user can be located in a specific work area of ​​the laboratory room and facilitate the analysis of a certain substance by voice command in a manner parallel to its manual activity. The analysis result is conveyed acoustically by the loudspeaker. Therefore, there is no need for the user to interrupt its current manual activity for the query result. On the contrary, the user can have a normal dialogue with the control software and, for example, after obtaining the result that a specific synthesis has ended, also start the corresponding analysis step for the generated substance by voice input. Therefore, laboratory work efficiency can be significantly improved because the user is enabled to work in parallel to a greater extent thereby, and, for example, facilitate and control complex synthesis and analysis while working in other laboratories.

[0159] Common activities in the laboratory that can be controlled by voice input according to embodiments of the present invention include, for example, the following activities and example related voice inputs:

[0160] The day before, a lab worker analyzed the rheological properties of a coating and now wants to query the results stored in the HTE equipment database. Possible voice input: "Control computer, show me the rheological analysis results for the HTE equipment in room 22 on February 24, 2019."

[0161] A laboratory worker needs to save costs and is considering replacing a solvent named "Solvent_Expensive" with a cheaper solvent named "Solvent_Cheap." The name "Solvent_Cheap" is the manufacturer's trade name. However, he is unsure whether the cheaper solvent is suitable for the varnish he wants to produce and would like to see the product data sheet, which details additional information about the chemical and physical properties of the cheaper solvent. Possible voice inputs include: "Control computer, show me the product data sheet for "Solvent_Cheap"" or "Control computer, show me the product data sheet for "Solvent_Cheap" stored in the HTE database in room 22."

[0162] After reviewing the product datasheet for the solvent "Solvent_Cheap," a lab worker concludes that it could potentially replace a more expensive solvent in the production of a certain clearcoat. However, they assume that the formulation requires some adjustments because several parameters, such as pH, rheological properties, and polarity, differ from those of the more expensive solvent. Because these properties influence each other, the lab worker cannot determine the necessary formulation adjustments based on theoretical assumptions. Performing a series of tests would be laborious and time-consuming. However, the lab has software that can predict (simulate) the properties of chemical products such as paints and clearcoats based on a given formulation. Such simulations can, for example, be based on a CNN (Convolutional Neural Network). The lab worker wants to use this simulation software to simulate the possible properties of a clearcoat based on a known formulation, in which the expensive solvent has been replaced by a cheaper one. Possible voice input: "Control the computer to have the HTE simulation software calculate the properties of a clearcoat based on the following formulation: 70.2 g naphthenic oil, 4 g methyl n-amyl ketone, 1.5 g n-amyl propionate, 1 g superabsorbent, 50 g <Solvent_Cheap>."

[0163] - Simulations show that cheap solvents are unsuitable for varnish production. Laboratory staff now want to search the internet for alternatives to expensive solvents without compromising product quality, thereby reducing costs. Possible voice input: "Control computer, search the internet for <<high-viscosity solvents for varnish production>>."

[0164] According to embodiments of the present invention, all of these inputs and commands to the corresponding execution systems can be achieved without the user having to leave the laboratory and / or remove gloves.

[0165] The speech inputs exemplified above mostly contain words and phrases in the form of general terms and terminology. Thus, for example, the words or phrases "rheology," "naphthenic oil," "methyl n-amyl ketone," and "n-amyl propionate" are chemical terms, and <<Solvent_cheap>> is a trade name for a chemical product. These words or phrases are typically not included in the vocabulary supported by common general-language speech-to-text conversion systems ("target vocabulary"). Post-correction by means of correction software ensures that the speech input is still correctly converted into text when using a general-language speech-to-text conversion system 226. This control software performs a number of coordination and control activities related to the management and processing of speech signals and the text generated therefrom, which vary slightly depending on the system architecture.

[0166] Figure 3 A block diagram of an alternative distributed system 300 for voice control of laboratory equipment and laboratory software modules is shown. The system 300 may include Figure 2 The same components and functions as shown. Figure 2Unlike the system shown, correction software 255 is not part of the control software and is not installed locally as a separate software application on control computer 213. Instead, it is provided as a cloud service via the Internet by cloud computer system 302. Control software 222 is configured to correct text 208 received from the speech-to-text conversion system via corresponding interface 304 using the assignment table now stored on cloud computer system 302 and to receive corrected text 210.

[0167] Figure 4 A block diagram of an alternative distributed system 400 for voice control of laboratory devices and laboratory software modules is shown. The data exchange between the individual laboratory devices 406-414 for controlling the HTE device 244 is described in detail in FIG. Figure 4 is shown in more detail in .

[0168] The laboratory includes a laboratory area 404 in which various individually located laboratory equipment 416 such as centrifuges and HTE equipment 418 are located.

[0169] The HTE device comprises a large number of modules and hardware units 406-414, which are managed and controlled by master control software on a master control computer 246. The master control software of this embodiment serves as a central external interface for monitoring and controlling the instruments included in the HTE device. The control software 222 on the control computer 213 can access the master control software and is configured to send correction texts 210, 211 as input to the master control software in natural language or structured form, and optionally to other target systems such as laboratory equipment 416 and / or an Internet search engine 402. The correction texts 210, 211 are generated based on the voice input 204 of the user 202, as described in accordance with an embodiment of the present invention.

[0170] The control software 222 can be implemented as a "virtual laboratory assistant" 223. For example, the virtual laboratory assistant can recognize names such as "control computer", "LUISA" or "EVA". The control software is configured to search the correction text 210 according to keywords and call the specific hardware function or software function of the target system according to these keywords. For example, the control software searches for keywords such as "control computer" or "EVA" in the text 210. If the correction text contains the keyword, the virtual laboratory assistant 223 is prompted to further analyze the correction text subsequently to determine whether the correction text includes instructions for executing hardware functions or software functions, and if the instruction is included, determine which hardware or software should be used to execute these commands under the control of the control software or laboratory assistant. For example, the correction text can include the name of an equipment or laboratory area, which names specify to which equipment and which software the command should be forwarded.

[0171] Therefore, the term "control computer" in voice input or other names of virtual assistants can be used in the field of chemical laboratories in a similar way to the names of virtual assistants "Siri" and "Alexa" known for "everyday problems" to achieve more efficient and secure control of hardware and software functions.

[0172] In one possible embodiment, the virtual lab assistant's evaluation of the corrected text 210 indicates that an internet search engine 402 should search for a substance specified as a term or term phrase in the corrected text 210. The corrected text, or portions thereof, is input into the search engine via the internet by the virtual assistant. The internet search results 424 are returned to the control software or assistant, which forwards them to a suitable output device near the user 202, such as a portable device 212 having a microphone and a speaker 214. The forwarding is preferably performed such that the virtual lab assistant, i.e., the control software, reads the results 424 to the user via the speaker 214 in a natural voice.

[0173] In another possible embodiment, the virtual laboratory assistant's evaluation of the correction text 210 indicates that an independent laboratory device 416 (i.e., a centrifuge) should pelletize a certain substance at a specific speed. The names of the centrifuge and the substance are specified in the correction text 210 as terminology or terminology phrases, which is sufficient because the centrifuge automatically reads the centrifugation parameters to be used, such as duration and speed, from an internal database based on the substance name. The correction text or some portion thereof is sent to the centrifuge 416 by the virtual assistant via the Internet. The centrifuge starts the centrifugation program for the substance and returns a message in the form of a text message 422 regarding whether the centrifugation was successful. The result 422 is returned to the assistant 223 and is output via the speaker 214 and optionally also via other output interfaces such as the screen as described for the result 424.

[0174] In another possible embodiment, the virtual laboratory assistant's evaluation of the correction text 210 indicates that the HTE device 244 should synthesize a specific varnish. The paint ingredients are also detailed in the correction text and are composed of a mixture of trade names and IUPAC substance names of chemicals. The HTE device receives the correction text 210 and independently decides to synthesize it in the synthesis unit 414. A message about the successful synthesis or an error notification is returned as result 426 from the synthesis unit 414 to the main control software of the HTE device. The control software then returns the result 426 to the control software or virtual laboratory assistant, which forwards it to a suitable output device, such as a terminal 312, which outputs the result 426 as described above for the result 424 via the speaker 214 and optionally also via other output interfaces such as a screen.

[0175] The target software and / or hardware when outputting the correction text 210, 211 is preferably the software and hardware within the laboratory 404. However, it is possible that the text or a part thereof is also sent to a target system outside the laboratory, the result of which is designed or suitable for chemical analysis or synthesis.

[0176] Voice control of laboratory equipment and associated laboratory software modules can be used to retrieve and output results of analyses and syntheses performed in the laboratory, laboratory protocols, and product data sheets in the laboratory's corresponding databases. Furthermore, voice-controlled searches can be performed on the internet and in publicly accessible or proprietary databases. Voice commands containing the names and qualifiers of chemical products, laboratory equipment, or laboratory consumables, and / or chemical terms, are also correctly converted to text and can therefore be correctly interpreted by the target system. According to embodiments of the present invention, highly integrated, essentially voice-controlled operation of a chemical or biological laboratory or laboratory HTE equipment can be achieved.

[0177] The goal of existing speech-to-text cloud service provider hardware (smart speakers) is to directly control and use services developed by the cloud service provider itself. Applications in specialized vocabulary areas are currently undeveloped or only developed to a very limited extent. All system architectures 200, 300, and 400 shown here allow the use of existing speech-to-text APIs from various cloud providers using independent hardware, independent of cloud providers, to implement specialized speech recognition and, based on this, control laboratory facilities and in-lab electronic search services.

[0178] The efficient collection of information about chemical substances and voice-based control of laboratory facilities and HTE equipment are particularly advantageous in the chemical synthesis of paints and varnishes, as the production of paints and varnishes requires a large number of raw materials, whose properties interact in complex ways and significantly influence the product properties. Consequently, paint and varnish production involves numerous analytical and control steps and test series. Paints and varnishes are highly complex mixtures of up to 20 or more raw materials, such as solvents, resins, hardeners, pigments, fillers, and numerous additives (dispersants, wetting agents, adhesion promoters, defoamers, biocides, flame retardants, etc.). The efficient collection of information about the individual components and the use of this information to control the corresponding analytical and synthesis equipment can significantly speed up the production process and improve product quality assurance.

[0179] Reference Signs List

[0180] Steps 102-106

[0181] 200 Distributed Systems

[0182] 202 users

[0183] 204 Voice Input

[0184] 206 Voice Signal

[0185] 208 Recognize Text

[0186] 210 Correction Text (Natural Language)

[0187] 211 Correction text (structured text)

[0188] 212 portable devices

[0189] 213 Control Computer

[0190] 214 Microphone

[0191] 218 speakers

[0192] 222 Control Software

[0193] 223 Virtual Laboratory Assistant

[0194] 224 Voice to Text Interface (Client Side)

[0195] 224' Speech to Text Interface (Server Side)

[0196] 225 Calibration Software

[0197] 226 Speech-to-text conversion system / cloud system

[0198] 232 Speech Recognition Processor

[0199] 234 target vocabulary

[0200] 236 Network

[0201] 238 Allocation Table

[0202] 242 Results of text correction

[0203] 246 Master Computer

[0204] 248 Master Control Software

[0205] 250 Chemical Databases (DBMS)

[0206] 252 Analytical Instruments

[0207] 254 Analytical Instruments

[0208] 256 transport units

[0209] 300 Distributed Systems

[0210] 302 Server with text correction service

[0211] 304 Correction service interface (client side)

[0212] 304' Correction service interface (server side)

[0213] 308 NLP interface

[0214] 310 Control interface for laboratory facilities

[0215] 400 Distributed Systems

[0216] 402 Internet Search Engine

[0217] 404 Laboratory Area / Laboratory Room

[0218] 406 Analytical Instruments

[0219] 408 Analytical Instruments

[0220] 410 Mixer

[0221] 412 Synthesis Unit

[0222] 414 Synthesis Unit

[0223] 414 Control Computer

[0224] 416 independent laboratory equipment

[0225] 422 Correction text execution result (text format)

[0226] 424 Correction text execution result (text format)

[0227] 426 Correction text execution result (text format)

Claims

1. A laboratory system comprising: - at least one laboratory device (252; 254; 406； 408；412； 414； 416), which are designed for the analysis and / or synthesis of chemical substances, at least one laboratory software module (248) designed to process data obtained by the at least one laboratory device; a data processing device (213) having control software (222) providing an interface for operating the at least one laboratory device and / or the at least one laboratory software module; as well as a portable device (212) having a microphone (214), wherein the device is interoperably connected to the control software via a network (236), wherein the device is designed to allow the user (202) to operate the at least one laboratory device and / or the at least one laboratory software module in an unobtrusive manner by means of voice input (204) into the microphone when interoperating with the control software, - The control software is configured to: - receiving a voice signal (206) collected by the microphone based on the voice input from the portable device, wherein the voice signal includes common words and terminology words spoken by the user; - inputting the received speech signal into a speech-to-text conversion system, wherein the speech-to-text conversion system only supports converting the speech signal into a target vocabulary (234) that does not contain the terminology; - receiving from the speech-to-text conversion system a text (208) generated by the speech-to-text conversion system according to the speech signal; - generating a correction text (210) based on the received text, wherein the correction text is generated by the control software alone or by the control software when interpreted by means of text correction software operatively connected to the control software, wherein the correction text is generated by automatically replacing words and phrases of a target vocabulary in the received text by terminology words according to an assignment table (238), wherein the assignment table assigns at least one word from the target vocabulary to each of a plurality of terminology words, wherein the at least one word of the target vocabulary assigned to the terminology word is a word or phrase that the speech-to-text conversion system erroneously recognizes when the terminology word is input in the form of a speech signal; and - using the correction text to cause the at least one laboratory device and / or the at least one laboratory software module to perform an analysis, synthesis and / or software function according to the information in the correction text.

2. The laboratory system according to claim 1, wherein: The control software is designed as a virtual laboratory assistant for operating the at least one laboratory device and the at least one laboratory software module.

3. The laboratory system according to claim 1 or 2, wherein: The portable device is configured to: - receiving the user's voice signal through the microphone (206); - forwarding the received speech signal to the control software for conversion of the speech signal into text (208) by the control software or by a speech-to-text conversion system (226) operatively connected to the control software.

4. The laboratory system according to claim 3, wherein: The portable device further comprises a speaker (218), wherein the portable device is configured to: - receiving a function execution result (242) from the control software in response to the voice signal transmission, wherein the function is executed by the at least one laboratory device and / or the at least one laboratory software module according to the text; and - outputting the result to the user via the speaker.

5. The laboratory system according to claim 1 or 2, wherein: The portable device is located in a room (404) shared with the at least one lab device and / or in a room shared with a host computer (246) of a lab facility including the at least one lab device.

6. The laboratory system according to claim 1 or 2, wherein: The portable device is a single board computer.

7. The laboratory system according to claim 1 or 2, wherein: - a computer system performing text correction receives or calculates frequency information of a plurality of words in the text generated by the speech-to-text conversion system according to the speech signal, wherein the frequency information indicates, for each word in the text, a statistically expected frequency of occurrence of the word; - when generating the corrected text, selectively replacing only those words of the target vocabulary in the received text with the terminology words according to the allocation table, the statistically expected occurrence frequency of which according to the received frequency information is below a specified threshold.

8. The laboratory system according to claim 1, wherein: The terminology word is a word from one of the following categories: - Name of the chemical substance; - the physical, chemical, mechanical, optical or tactile properties of the substance; - names of laboratory equipment and chemical industry equipment; -Names of laboratory consumables and laboratory supplies; - A trade name in the paint and varnish sector.

9. The laboratory system according to claim 1 or 8, wherein: The words and phrases in the target language stored in the assignment table represent erroneous text output of the speech-to-text conversion system, which text output is generated based on input terms from many different human voices.

10. The laboratory system according to claim 1 or 2, wherein: The at least one laboratory device is a plurality of laboratory devices, wherein the laboratory devices are respectively selected from the following group, the group comprising: - an analysis station (406; 408) designed for analyzing chemical substances or mixtures; and / or - a synthesis station (412; 414) designed to carry out the synthesis of chemical substances or mixtures; and / or A pre- or post-processing station (410) designed to modify the chemical substances before or after an analytical or synthetic step in order to enable subsequent work-up or transport steps.

11. The laboratory system according to claim 1 or 2, wherein: The at least one laboratory device is a plurality of laboratory devices, each of which is used as a corresponding chemical substance processing station, and further comprises: - a transport unit (256) designed to transport substances for synthesis or analysis to the processing station, in order to allow the processing station to perform at least one processing step on the corresponding substance, respectively; - a main control computer of a laboratory facility (244) comprising a plurality of laboratory devices, wherein the main control computer comprises main control software configured to arrange processing steps and transportation of the chemical substances based on instructions in the form of structured text.

12. The laboratory system according to claim 1, wherein: The control software includes or interacts with an NLP processor (258), wherein the control software is configured to: - using the NLP processor to convert the correction text into a structured text that can be interpreted by a target system (211), wherein the target system is the at least one laboratory software module, the at least one laboratory device and / or the main control software of a main control computer of a laboratory facility including the at least one laboratory device; and - inputting the structured text into the target system to cause the target system to execute software functions and / or hardware functions related to the laboratory.

13. The laboratory system according to claim 11, wherein: The non-invasive operation includes converting the speech input into text (210; 211), wherein - the at least one laboratory software module is a chemical substance database designed to interpret input text as a search input and to determine information in the database related to the search input and return it as a result to the control software; and / or the at least one laboratory software module is a simulation software designed to simulate properties of a chemical product based on a predefined mixture specification, wherein the simulation software is designed to interpret the input text as a detailed description of the product whose properties are to be simulated; - the at least one laboratory software module is a main control software (248) of a main control computer of a laboratory facility (244) containing the at least one laboratory device, wherein the main control software is designed to cause the at least one laboratory device and the transport unit to perform a plurality of the processing steps and the transport of chemical substances between the laboratory devices according to the information contained in the input text.

14. The laboratory system according to claim 1 or 2, wherein: The control software includes a register with multiple user accounts registered with the control software, wherein the control software includes a voice recognition function for identifying each registered user based on voice, wherein the control software is designed to allow the user to operate only the laboratory equipment and / or laboratory software modules that the user is authorized to operate without intervention through voice input to the microphone.

15. A method for controlling a laboratory device and / or a laboratory software module, comprising: - providing at least one laboratory device (252; 254; 406； 408；412； 414; 416) and at least one laboratory software module (248), wherein the at least one laboratory device is designed for analyzing and / or synthesizing chemical substances, wherein the at least one laboratory software module is a laboratory software module for processing data obtained by the at least one laboratory device (252; 254; 406; 408; 412; 414; 416); - providing a data processing device (213) with control software (222), wherein the control software provides a user with an interface for operating the at least one laboratory device and / or the at least one laboratory software module; and providing a portable device (212) with a microphone (214), wherein the device is interoperably connected to the control software via a network (236), wherein the device is designed to allow the user to operate the at least one laboratory device and / or the at least one laboratory software module non-invasively by voice input to the microphone when interoperating with the control software, - The control software is configured to: - receiving a voice signal (206) collected by the microphone based on the voice input from the portable device, wherein the voice signal includes common words and terminology words spoken by the user; - inputting the received speech signal into a speech-to-text conversion system, wherein the speech-to-text conversion system only supports converting the speech signal into a target vocabulary (234) that does not contain the terminology; - receiving from the speech-to-text conversion system a text (208) generated by the speech-to-text conversion system according to the speech signal; - generating a correction text (210) based on the received text, wherein the correction text is generated by the control software alone or by the control software when interpreted by means of text correction software operatively connected to the control software, wherein the correction text is generated by automatically replacing words and phrases of a target vocabulary in the received text by terminology words according to an assignment table (238), wherein the assignment table assigns at least one word from the target vocabulary to each of a plurality of terminology words, wherein the at least one word of the target vocabulary assigned to the terminology word is a word or phrase that the speech-to-text conversion system erroneously recognizes when the terminology word is input in the form of a speech signal; and - using the correction text to cause the at least one laboratory device and / or the at least one laboratory software module to perform an analysis, synthesis and / or software function according to the information in the correction text.

16. The method according to claim 15, wherein The provision of the device includes placing the device in a room (404) shared with the at least one lab device and / or in a room shared with a host computer of a lab facility including the at least one lab device.

Images