Communicating information about industrial machine to operator using operator context-specific communication assistant device

Abstract

A computer provides communication assistance to communicate information about an industrial machine to an operator. The computer detects (413) an operator context of the operator based on context data it receives from the industrial machine. The computer has a plurality of data modules adapted to process (443) queries to obtain responses. The computer enables (423-A, 423-B) at least one of the plurality of data modules in accordance with the detected operator context. When the computer receives (433-1) a question from the operator, it translates (433-2) the question as the query (340-Q) and directs (433-3) the query (340-Q) to one of the enabled data modules. The computer also translates (453-1) the response (340-R) into an answer (350-A) and forwards (453-2) the answer (350-A) to the operator.

Description

Technical Field

[0001] Generally speaking, this disclosure relates to communication between industrial machines and their operators. More specifically, this disclosure describes a computer-implemented communication assistant device that supports communication between a machine operator and the machine. Background Technology

[0002] Despite advancements in automation, almost every industrial machine still requires some interaction with a human operator. In short, the machine presents information about its internal state to the operator, who then reacts accordingly, for example, by controlling the machine.

[0003] Traditionally, machines interact with operators via a human-machine interface (HMI). The interface can present information visually (on a display, etc.). In addition, the interface can ultimately issue audible alarms.

[0004] Industry trends suggest making interactions more efficient, for example, by using alternative communication modalities or by augmenting data. For instance, machines could communicate with operators verbally (through text-to-speech translation). This method directly encodes information into a human-understandable form. Therefore, it allows machines to deliver information at a granularity best suited to the cognitive abilities of human operators. Simple alarm tones won't be eliminated, but machines can at least convey information through short, spoken phrases. For example, the interface will have to identify specific machine states (such as fault conditions) and will have to compose audio messages (e.g., "Temperature has risen above critical levels!") by combining predefined spoken language recordings.

[0005] The machine uses speech-to-speech technology to customize information for the operator. For example, the machine can choose natural language that the operator understands.

[0006] Another trend is the expansion of the time frame for interaction. For example, a machine can transmit information relevant to the present time, but it can also transmit information relevant to the future. The two trends are interconnected. A simplified example could be "the temperature will rise above a critical level within one hour from now."

[0007] However, there are technical limitations in at least the following aspects: ● Identifying the information to be conveyed to the operator is a time-consuming process, and this time depends on the nature of the information. For example, issuing alerts based on real-time data is relatively easier and less time-consuming than predicting potential (future) failures.

[0008] ● Transmitting information to the operator is also a time-consuming process. In many cases, signal propagation delay (i.e., data transmitted over a transmission network) is negligible, but spoken communication requires time to transmit messages to the operator. Spoken phrases have a minimum duration; otherwise, humans cannot understand them.

[0009] ●Enhancing information is also a time-consuming process, such as translating between formats (text to speech, etc.).

[0010] In some situations, restrictions can even amplify each other. For example, if a condition can be identified as posing a potential safety hazard to the operator, the corresponding information must be brought to the operator's attention as early as possible.

[0011] It is necessary to overcome limitations, and to speed up communication and make it more accurate.

[0012] US 2014 / 0365222 A1 discloses a voice-based mobile system with a non-voice interface for telematics service applications. The system processes natural language statements, determines commands to the vehicle, and thereby uses contextual information. Summary of the Invention

[0013] The computer implements a method for transmitting information about industrial machines to operators. The computer has the following modules: a context detector, a task manager, and a query interface. The computer can also access multiple data modules.

[0014] In the context detection step, the context detector detects the operator's operator context based on the context data received from the industrial machine. The operator context is defined by the relationship between the operator and the industrial machine and the operator's control functions over the industrial machine.

[0015] During the activation process, Task Manager enables at least one of a plurality of data modules based on the detected operator context. This approach is advantageous because inactive data modules do not consume computing resources (such as power, memory access, CPU sharing, etc.).

[0016] After receiving a question from the operator, the query interface translates the question into a query and directs the query to one of the enabled data modules. This approach can be advantageous because a data module does not necessarily have to be enabled; it may already be enabled. Therefore, the computer does not need to use resources to enable the data module. Directing the query to an enabled data module speeds up communication compared to directing the query to a data module that is not yet enabled. When the computer selects a data module to enable (and eventually disable) it, it uses the operator context; the likelihood of that data module being used is related to the operator context.

[0017] Once one of the enabled data modules receives a query, it processes the query to obtain a response. The query interface translates the response into an answer and forwards the answer to the operator. This reverse translation coordinates the communication format: questions from the operator have the same format as answers to the operator.

[0018] Enabling (at least one) data module may include any of the following: (i) The task manager may enable the data module to be loaded into the computer's main memory. The data module functions similarly to a memory-resident program, with the advantage that it eliminates the need to copy the module from storage to memory. (ii) The task manager may establish a data connection from the data module to the machine. The data module thus receives machine data representing the machine's current operating parameters, which can be processed immediately when a problem / query arrives. (iii) The task manager may associate model data (i.e., data to be used by the module) with the data module to be enabled. (iv) The task manager may instruct the module to begin operation, for example, to run a prediction. This prediction may already be available when a problem / query for the prediction results arrives.

[0019] Enabling (at least one) data module may include communicatively connecting a data source to at least one data module. Where the data module is adapted to process data from the past, it may be connected to a dataset of past operations of the industrial machine. Alternatively, where the data module is adapted to make predictions, it may be connected to a dataset of past operations of the industrial machine during a window interval.

[0020] In the context detection step, the context detector can receive updated context data as previously received context data. The context data originates from the industrial machine and is updated periodically. Examples are given in the following list: (i) Context detectors can determine that the operator of an industrial machine has changed to a different person. Operators typically log in to the machine (i.e., the computer associated with the machine) at the start of a work shift and typically log out at the end of the shift. Operators may wear ID tags or similar devices, allowing the computer to alternatively identify which person is the current operator.

[0021] (ii) Context detectors can determine that an operator of an industrial machine has performed a predefined activity that affects the machine. The context is the operator context and is therefore operator-specific. Different operators may make different decisions, and the state of the machine may change: components may be turned on (or off), the machine may stop due to emergency functions, etc.

[0022] (iii) Context detectors can determine that an operator has changed his or her physical position relative to an industrial machine. The basic principle is that operators in different locations may have different needs for information (about the machine) and may react in different ways. However, information about the machine's status may be important for the operator to make decisions. An example is a warning (and suggestion) to leave the machine as a measure to address potential safety hazards to the operator. However, such warnings are not necessary for operators in a safety control room.

[0023] (iv) Similarly, context detectors can determine the location of an operator who has left an industrial machine. This is again operator-specific: data modules that process data about potential safety hazards can be disabled, but other data modules (more specific to operator-free operation) can be enabled. Occupation data is typically obtained without the need for sensors. The data is usually used to report open emergency doors, making the source accessible to a computer.

[0024] (v) Operators typically wear protective clothing, and it is determined that the operator has changed clothing (putting on or taking off) or is wearing specific clothing relevant to the context. The computer can receive contextual data from the monitoring system and / or from sensors. Data modules for processing safety-related data can be enabled (or disabled) accordingly.

[0025] (vi) Last but not least, within this representative set of moments, the context detector can determine that the operator has changed the language used to interact with the operator interface of the industrial machine. The query interface may have (through training) learned that people switch to their native language, especially when they are afraid or in a real emergency. This is also processable contextual data.

[0026] Optionally, when translating a question into a query and a response into an answer, the computer saves data of the presented language statements. In this sense, the translation step can be bypassed. When the computer directs a query to one of the enabled data modules, the computer directs the query to the data module that processes the language statements. The query can be directed to the data module that processes the language statements, which are selected from: language statements in natural language; language statements in artificial language, wherein the statements are encoded using alphanumeric characters; and statements that are derived from the language statements.

[0027] Alternatively, when translating a question into a query (and a response into an answer), the computer can identify the domain of the question and can provide an answer within the same domain. Such domain-specific methods (along with enabling or disabling modules via operator context) can be advantageous in avoiding ambiguity (entering a question / answer loop).

[0028] Alternatively, when detecting operator context, the context detector can provide operator context by classifying the context data.

[0029] Optionally, after receiving a question from the operator via the query interface, and before translating the question into a query via the query interface, the computer executing the method checks the question for ambiguity. If there is no ambiguity, the computer continues to translate the question. If there is ambiguity, the computer (i) modifies the question according to the operator's context, (ii) translates the question according to the operator's context, or (iii) interacts with the operator to obtain further details in subsequent questions.

[0030] Optionally, the Task Manager (in the step of enabling at least one of multiple data modules) also handles the machine's status. This means that both the operator context and the machine status affect the enabling (or disabling) of a particular data module. For example, if the machine is in a "under maintenance" state, a data module that can perform predictions for an "operating" state can be disabled.

[0031] A computer program product, when loaded into the memory of a computer system and executed by at least one processor of the computer system, causes the computer system to perform the steps of a computer-implemented method.

[0032] A computer system includes multiple modules that perform the steps of a computer-implemented method. Alternatively, from a different perspective, a computer system is provided as an auxiliary system to transmit information about an industrial machine to an operator of the industrial machine, the auxiliary system including a computer, i.e., an assistant device, that performs the computer-implemented method.

[0033] A computer-implemented auxiliary system (“assistant device”) is provided to transmit information about industrial machines to operators of industrial machines.

[0034] A context detector is adapted to detect the operator's operator context based on context data. The context detector receives context data from the industrial machine. A data module is adapted to process queries to obtain responses. A task manager is adapted to enable at least one data module based on the detected operator context. A query interface is adapted to receive questions from the operator, translate the questions into queries, and direct the queries to one of the enabled data modules. The query interface is also adapted to translate responses into answers and forward the answers to the operator. Attached Figure Description

[0035] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, in which: [ Figure 1 This symbolically represents industrial machinery and its operators. [ Figure 2 A block diagram of a computer-implemented communication assistant device with modules is shown. [ Figure 3 This illustrates an overview of the sequence of steps for operating a communication assistant device in a method of transmitting information; [ Figure 4 The diagram shows the first type of machine, the second type of machine, and type-specific data as well as type-common data. [ Figure 5 A simplified flowchart showing the method for modifying transmitted information is shown; [ Figure 6 The flowchart illustrates the optional sub-steps of the operations used for the query interface; and [ Figure 7 The image shows a general-purpose computer. Detailed Implementation

[0036] Fictional example This manual explains how a computer-implemented communication assistant device enables so-called information communication between the machine and its operator. More simplified examples are occasionally used in this manual. The operator should perceive some atypical machine noise and should ask the assistant device, "When was the last time we heard this noise?"

[0037] The computer processes questions from the operator as queries to the data module and responses from the data module as answers. In the example, the computer would answer, "Last week, but don't worry."

[0038] Here, the computer provides an answer ("last week"), but also adds details that may be related to the machine's technical operation ("Don't worry").

[0039] Industrial machine and operator interface, context data Figure 1 The industrial machine 100 and the machine operator 190 are symbolically shown. (In short, this instruction manual will refer to them as "machine" and "operator".) Unless otherwise indicated, machine 100 is a specific machine located at a specific industrial site, and operator 190 is a specific natural person.

[0040] Operator 190 represents a functional role, and changes in the personnel of operator 190 do not interrupt the operation of machine 100.

[0041] Communication (between machine 100 and operator 190) is provided through the following interface: ● Operator Interface 180 (referred to as "Interface 180") is a traditional human-machine interface (HMI, two-way communication). ●Emergency Stop Item 101 (referred to as "Stop Item"), which is for one-way operation only and is usually implemented in the form of a red mushroom or rope, and ● Context-specific communication assistant device 200 (or simply "assistant device", bidirectional).

[0042] Figure 1 It has three parts to distinguish three scenarios (A), (B) and (C), but for simplicity, only the attached diagram for scenario (A) is given with reference numerals 100 / 101 / 200 / 180 / 190.

[0043] From a higher-level perspective, communication can be differentiated into: ● Control communication via interface 180 (and ultimately via stop element 101) enables operator 190 to control machine 100, and ● In many cases of the question-and-answer scheme, information communication is conducted via the assistant device 200 to provide the operator 190 with additional information (see the examples described above).

[0044] The operation of the assistant device 200 is context-specific, and this specification explains how the context can be identified. As used herein, the context is the operator context, and the data that allows identification of this operator context is called "context data" (see [link to documentation]). Figure 2 210-D in the middle).

[0045] Questions and the timing of answers Operator 190 can query assistant device 200 at almost any time, but it is expected that... ● Operator 190 only occasionally asks in scenarios (A) and (B), ● Operator 190 will inquire in situation (C) with a relatively high probability (due to some limitations in control communication). ● Operator 190 will ask in scenario (A), (B), or (C) if they believe that machine 100 (or a component) is malfunctioning. ● The assistant device 200 will contact the operator 190 after detecting an emergency stop event (the initial problem may come from the assistant device).

[0046] Control communication and information communication Double arrows indicate control communication, but Figure 1 It also indicates preferences: thick arrows (e.g., between operator 190 and interface 180, and between machine 100 and interface 180) represent common control communications, while thin arrows (e.g., between operator 190 and machine 100) represent less frequent control communications (or communications that transmit less information).

[0047] Control communication is related to the control machine 100 (its execution processes, etc.). The stop symbol (reference numeral 101) indicates a special control situation: the operator 190 can stop the machine (such as in an emergency).

[0048] In other words, machine 100 has interface 180, and machine 100 also includes an emergency stop device 101 if required by safety standards. The stop device 101 is an optional component, but almost every machine 100 has such a stop device 101, and this specification assumes that it exists.

[0049] The assistant device 200 can use information from interface 180 (and from stop element 101) but operates without interfering with the information. In other words, the assistant device 200 is not configured to control the machine. Control communication and information communication are separate topics. The operator 190 can use information communication to learn more about the machine (e.g., about its technical condition). The operator 190 can then decide how to proceed with control communication. In other words, the assistant device 200 (as indicated by its designation) assists the operator 190 in making decisions concerning the control of the machine.

[0050] Can also be used with Figure 1 Let's discuss the limitations mentioned above together, and at least in part, consider the context of the assistant device 200 as part of the solution.

[0051] Operator context Control communication is context-dependent, and (from a higher-order perspective) the operator context can be described as follows (as an example and not intended to be restrictive): ● In the communication mode between interface 180 and operator 190; ●The physical position of operator 190 relative to interface 180 and the physical position of operator 190 relative to machine 100; ● Specific circumstances of the operator, such as the operator's familiarity with the machine and the operator's measurable health condition (e.g., skin resistance, heart rate, which are still machine-related, as operators with health problems may tend to operate the machine incorrectly).

[0052] The operator context is defined by the relationship between the operator and the industrial machine and the functions the operator controls on the machine. Therefore, attributes of the operator that are not related to this control function are not defined in the operator context (e.g., his or her name).

[0053] The computer (which is associated with the communication assistant device 200) can collect context descriptors at virtually any time and can detect operator context (see [link]). Figure 2 Context data 210-C). This will target... Figures 2 to 3 Explain an example of a context detector (or context determiner). Depending on the context, machine 100 (i.e., interface 180) may interact with operator 190 in different ways (this interaction is context-sensitive), but (as will be described) the context is more relevant to the operation of assistant device 200.

[0054] Regarding modalities, control communication can often use different modalities in combination. Examples include the following: ● (Visual Communication) Operator 190 views interface 180 and / or views machine 100.

[0055] ● (Audio Communication) Operator 190 listens to the sound of machine 100, or hears alarm sounds emitted by interface 180, etc.

[0056] ● (Haptic) Operator 190 touches the keyboard or touchscreen of interface 180. Operator 190 may ultimately use stop element 101.

[0057] Modalities can be interdependent. In a simplified example, if operator 190 is in the control room, operator 190 may not be able to hear machine 100. The same applies if alarm tones can be recorded in only one location.

[0058] It should be noted that modalities can also be used as contextual data (see 210-D). In other words, modalities transmitted by the machine and the operator to control the machine can determine how the assistant device answers questions. For the example, operator 190 hears noise, and the communication assistant device 200 can therefore provide an answer not only via text-to-speech or similar audio messages, but also (considering the noisy environment) by repeating the answer on a display.

[0059] Regarding location, as shown by variations (A) and (B), operator 190 may be located near interface 180 (e.g., in a control room), or operator 190 may be located near machine 100. The availability of interface 180 at machine 100 may be limited (control rooms typically have more data), and in the simplified scenario, the only option for operator 190 may be to use stop piece 101.

[0060] Those skilled in the art can track the operator's position 190, making the position data usable by a context detector. In the example above, there can be a difference if the operator knows about the noise from the control room (A) or directly from the machine (B). In case (A), the noise may originate elsewhere, not from the machine.

[0061] In both variants, the operator can physically change the position: ●From scenario (A) to scenario (B): Operator 190 can leave the control room to inspect machine 100 (but will have fewer opportunities to use the interfaces located in the control room).

[0062] ●From Situation (B) to Situation (A): Operator 190 can return to the control room.

[0063] Similarly, applying the same principle, the context detector evaluates changes in entity location as changes in context. Therefore, if operator 190 is in the control room (A) or near the machine (B), assistant device 200 can communicate with operator 190.

[0064] Regarding the specific circumstances of the operator, the operator 190 can select a specific natural language for the interface 180 based on the specific circumstances of the operator.

[0065] Control was achieved through an emergency stop. Situations may arise where control communication becomes extremely simplified. Operator 190 may instruct machine 100 to stop. For a variety of reasons, operator 190 will avoid stopping whenever possible, as the machine may require time to resume normal operation. Of course, in some emergency situations, operator 190 may attempt to exit without interacting with the machine and potentially without stopping the machine.

[0066] A stop event (i.e., an action taken by operator 190 on stop device 101) is information that changes the context. In other words, a detected stop event can enable assistant device 200 to communicate with operator 190 in a specific manner.

[0067] In the example, the assistant device might add "Don't worry," because earlier, the noise had prompted different operators to stop the machine.

[0068] Control in complex situations Variant (C) illustrates a more complex scenario in terms of control interaction compared to (A) and (B).

[0069] Since operator 190 is located near machine 100 in (B), he / she may wear gloves and / or protective clothing. Figure 1 This situation is represented by a "rounded" rectangle surrounding the operator symbol.

[0070] The environmental characteristics of machine 100 can include sound, dust, darkness, humidity, and hot air. Figure 1 This situation is represented by the vertical line on the communication arrow.

[0071] Therefore, controlling communication may present some challenges.

[0072] ● Visual communication may be complex. Operator 190 can view machine 100, but visibility of displays, etc., may be poor.

[0073] ● Operator 190 listens to the sound of machine 100, but there are other sounds as well, and auditory communication via audio-enabled components of interface 180 (via headphones, etc., and telephones with other operators) is limited.

[0074] ● Gloves make it difficult for operators to use keyboards, etc. (tactile feedback). Even interface components that are suitable for wearing gloves provide fewer opportunities than normal interface components.

[0075] These (and other) challenges can be used as context descriptors (for assistant device 200). Sensors can represent visibility through data, measure ambient sound levels, measure vibrations, determine what kind of clothing the operator is wearing, etc.

[0076] Machine operation and the need for other information Despite the complexities inherent in the interaction, Figure 1 (A), (B), and (C) refer to the “normal” operating mode of machine 100: the machine executes the industrial process according to the predetermined plan, and there is no functional malfunction (of the machine or part of the machine).

[0077] However, some functions of machine 100 may malfunction, and operator 190 will become aware that something has changed. Operator 190 may perceive that machine 100 is operating abnormally. Operator 190 attempts to obtain additional information. In the example, the operator inquires about unknown noise.

[0078] Depending on the location, the perception (and reaction options) differ. Similarly, extreme reactions (emergency stops) should be avoided. Operators need time (and reliable information) to assess the situation. Likewise, other information, even information not yet available from Interface 180 (or available but difficult to capture), can be useful.

[0079] In scenario (A), interface 180 will indicate a large amount of data, and operator 190 can ultimately select software components (such as prediction modules) to perform the evaluation. In scenario (A), there should be no direct danger (or safety hazard) for operator 190. In scenario (A), operator 190 may have sufficient time and opportunity to manually identify the required information.

[0080] In scenarios (B) and (C), interface 180 can still indicate data for at least some elements that the operator can see at machine 100, but the operator 190 is far from the control room. Even with the most sophisticated interface (in a mobile device), only some data can be obtained. The operator may be in a hazardous area. The operator may perceive a "save and sound" situation as potentially dangerous.

[0081] Perception (and options for response) also vary over time. Regarding temporal aspects (which can also be considered part of the context), the following should be noted: In scenarios (B) and (C), operator 190 repositioning to (A) requires some time ΔT_ESCAPE, while in scenario (C), the path away from machine 100 may be blocked. Therefore, ΔT_ESCAPE in (C) may be longer than in (B). In other words, the computer can determine how long a potential disengagement will take.

[0082] Solutions The industrial machine 100 has already been explained with an emphasis on the operator context and the lack of information that the operator 190 may face has been highlighted. This specification now explains a solution.

[0083] As will be explained, the solution has several relevant aspects, at least in the following: ● Information and communication utilizes natural language as a modality for communication. (At a minimum, communication uses a language based on natural language. Other modalities may still be applicable.) ● Because the language modality requires the computer to (technically) process language (rather than data from buttons or keystrokes, such as those from machine 100 or interface 180), the computer requires additional processing time. To address this limitation, the data module that may provide information is enabled based on the likelihood of its use. It is even enabled before the operator asks a question.

[0084] ● Operator context (which can be collected at virtually any time) acts as a discriminator for selecting (or deselecting) specific data modules. This method can help save computation (memory and energy consumption of enabled modules). The likelihood of a data module being usable is relative to the operator context. A data module is more likely to provide a response if it is suitable for the operator context, or more likely to fail if it is not suitable.

[0085] ●Since data communication in language formats inherently involves some ambiguous restrictions, operator contexts are also used (in a cooperative manner) to remove ambiguity.

[0086] Re-examine the example The assistant device needs some time to break down the problem ("When was the last time we had this noise?") to identify subsequent activities, so that... ● Choose data from the past, meaning "the last time" (rather than the future). ● Identify whether it is necessary to measure or classify the current sound of the machine ("this noise", i.e. the noise the machine makes when the operation asks a question) and match that sound with the historical record (i.e., by representing the sound in the computer-recognizable data).

[0087] In the example, the operator context could be the following: (i) the operator's location at the machine's audible distance (where the operator will recognize the machine); (ii) the operator's experience (e.g., the person who recently started operating the machine); (iii) noise or even information that caused the machine to stop in the past, etc.

[0088] In addition to the context under point (iii), the operator context can be determined before the operator asks a question.

[0089] Therefore, the assistant device 200 can detect the recording to identify the absence of potential hazards (i.e., triggering of additional messages), and it also avoids stop commands.

[0090] Communication Assistant Device Figure 2 A block diagram of a communication assistant device 200 having modules 210, 220, 230 and 240 is shown.

[0091] Context detector 210 is communicatively connected to machine 100 and interface 180 (see...) Figure 1Optionally, the context detector 210 is coupled to a database (not shown). The context detector 210 is adapted to monitor the operator 190 and detect the operator context 210-C. Technically, the context detector 210 receives context data 210-D as the basis for such detection. Since the context data 210-D typically includes more data inputs than the operator context 210-C, the context detector 210 can be implemented as a classifier: context 210-C is the classification (which reduces the granularity of the context data).

[0092] Data module 220 is suitable for processing data (such as machine data, data representing machine operating parameters, etc.). From a high-level perspective, the data module can provide output information related to the following: ●In the past, ●Current (i.e., current operation), and ●The future.

[0093] Data module 220 may be, for example, a prediction module (for predicting the behavior of machine 100 at a future point in time), or a data retrieval module (for identifying past, present, or future data related to the operation of machine 100). An example of a data retrieval module with future output data could be a module for accessing maintenance plans.

[0094] Those skilled in the art can select a suitable data module, for example, by means of a database, by means of a database management program, by means of database middleware, by means of a computer program that interacts with a database (or an equivalent data source), or by means of other implementations of the data module.

[0095] Data module 220 can be implemented using machine learning tools, such as artificial neural networks (ANNs). For illustrative purposes only, referencing the following references concerning industrial machinery: methods involving the application of prediction to schedule machine maintenance. WO 2022 / 258835 A1, etc.

[0096] Data module 220 can be implemented by an application that acts as a simulator, for example, as cited in the following references.

[0097] ● WO 2022 / 069498 A1, etc.

[0098] Simulation can be used as an alternative method for predicting the behavior of machine 100 at a future point in time.

[0099] Data module 220 may have additional functions as a recommender. The data module can identify actions taken by previous operators in similar situations (based on data). Recommendation differs from prediction. Recommendation refers to operator action. For example, a predictor module can predict specific failures under certain operating conditions (e.g., a specific temperature for a pressure parameter), and the recommender can advise the operator to avoid such operating conditions. For example, a data module acting as a recommender can suggest touching (or not touching) the emergency stop switch, scheduling maintenance, or replacing specific spare parts.

[0100] Those skilled in the art can limit the data processing of the data module to specific time intervals. Modules that predict (and / or recommend) future events consider data from the past. However, past data is limited to a so-called "time context window," or simply "window," and predictions of the future are also time-bound.

[0101] Data module 220 can be implemented via a so-called chatbot function. This function can be domain-specific and context-specific (to be enabled or disabled).

[0102] Data module 220 is formed into multiple data modules, and individual modules in the multiple data modules are distinguished by the letters 220-A, 220-B, 220-C and 220-D.

[0103] Query interface 230 is suitable for: ● Convert the question 330-Q from operator 190, presented in a language format (“when we had ...”), into a computer-readable format (query 340-Q, which can be processed by modules 220-A, 220-B, 220-C, and 220-D), and ● Convert the response 340-R from the data module into a language format, and forward the answer 330-A to the operator 190.

[0104] The use of linguistically formatted questions and answers has limitations on ambiguity. In short, one word or the same word may have different meanings. In the example, the noise problem asked by operator 190 of machine 100 has a different meaning than the literally identical question asked by the operator at home who will hear music made by a neighbor.

[0105] Technically, query interface 230 can, for example, take into account contextual data (considering operator context, and also machine context, see also domain-specific details, see Figure 5 To resolve ambiguity.

[0106] The assistant device 200 also includes an internal task manager 240 that coordinates the operations of the context detector 210, the data module 220, and the query interface 230 to execute computer-implemented methods. Figure 2 The operation of Task Manager 240 is shown only symbolically: it receives operator context 210-C, selects data modules (see vertical arrows), and performs routing queries and responses (routes are not shown).

[0107] pass Figure 3 The method is summarized as method 403 for transmitting information about industrial machine 100 to operator 190 of industrial machine 100.

[0108] It is convenient to implement the context detector 210, data module 220, and query interface 230 using a neural network (to be) trained.

[0109] step Figure 3 An overview of the sequence of steps for operating the communication assistant device 200 is shown. Since the assistant device 200 is implemented via a computer, the sequence of steps can be considered as belonging to computer-implemented methods 402 and 403. This specification refers to the acting entity as a "computer." It should be noted that this is for simplification: the execution of the steps may be distributed across different physical computers.

[0110] There are two sequences at different stages: The training phase is marked with a last two digits (collectively referred to as Method 402), which has... ●Step 442 (Training data module 442-A, training data module 442-B, etc.) ●Step 412, train the context detector 210, and ●Step 432, train query interface 230.

[0111] Training is only required for modules that apply machine learning. The last two digits of the figure represent modules that are generally set up, such as by selecting specific data (e.g., specific variables in a multivariate time series), setting specific time intervals, etc.

[0112] The reference numeral ending in 3 marks the so-called operation phase (or method 403), which is a phase consistent with the operation (and control interaction) of machine 100.

[0113] In step 413, the computer monitors operator 190 (i.e., by receiving data from machine 100) and thereby detects operator context 210-C. More specifically, context detector 210 detects operator context 210 based on its data received from machine 100 (and, if applicable, from interface 180, see...). Figure 1 The received context data 210-D detects the operator context 210-C. Depending on the context, the computer enables multiple sub-data modules (here, modules A and B from multiple A, B, C, and D).

[0114] In step 423, the computer enables the data modules, for example, by loading the data modules (e.g., A and B) into main memory (and subsequently running the data modules). The diagram uses solid lines to represent enabled and running data modules and dashed lines to represent disabled (not yet enabled, or deactivated) data modules. More specifically, Task Manager 240 enables at least one of a plurality of data modules 220, which, in the example of the diagram, are data modules 220-A and 220-B.

[0115] In other words, the enabled data module is in "standby mode" and can start receiving queries immediately, while the disabled data is in "shutdown mode" and can start receiving queries after "warming up" (but it requires an activation time (TIME_TO_ACTIVATE)).

[0116] Steps 413 and 423 are repeated steps, and the sequence 413 / 423 is executed consecutively. When the computer detects a context change, it can enable (previously disabled) data modules and disable (previously enabled) data modules.

[0117] In step 433 (with sub-steps 433-1, 433-2, and 433-3), the computer receives a query (from operator 190, see below). Figure 1 (A dashed vertical arrow pointing upwards) and identifies the data module (among multiple data modules) that can provide a response. (Computers use a query interface; see [link]). Figure 2 ,but Figure 3 (Details omitted). In the example, the identified data module can be data module 220-B. Data module 220-B is already running (see step 423), and enabling it results in no time loss. More specifically, when query interface 230 receives (step 433-1) a question from operator 190, it translates question 330-Q (step 433-2) into query 340-Q and directs query 340-Q (step 433-3) to one of the enabled data modules. In the example, this is data module 220-B.

[0118] Step 433 may have other sub-steps, as described below. Figure 6 Explanation example.

[0119] Enabling a data module requires processing time TIME_TO_ACTIVATE, but depending on the context, enabling the data module is related to the likelihood that the question can be answered by the enabled module. In this case, TIME_TO_ACTIVATE is not part of the overall processing time. This technique can be advantageous, especially in situations where an operator needs an immediate response. (In the example above, noise could be an indicator of a safety hazard, and the response could be a recommendation for the operator (or other personnel) to disengage. Needless to say, such a "high-speed" response is highly desirable.)

[0120] The computer may also identify a data module that is not yet running (see step 423), and in such a case, the computer may begin its operation (i.e., during TIME_TO_ACTIVATE). Since the context (see step 413) is related to the selection of the data module to be initially run (step 423), the probability of running the data module may be higher than the probability of starting the data module.

[0121] Step 443 represents one of the enabled data modules used to process query 340-Q and obtain response 340-R.

[0122] In step 453, the computer interacts with the operator 190 by obtaining a response 340-R from the identified data module (e.g., from B) through a query interface, and provides the response to the operator 190 (e.g., by translating the response into a language format, such as answer 350-A).

[0123] Alternatively, the computer may also perform step 453 by taking the context into account.

[0124] Optionally, the computer implements a feedback loop (to the large arrow at the top) to training steps 412 and 432.

[0125] Feedback The question from operator 190 may not have been answered. There could be many reasons: the data module may not have access to enough data, the question may not be translatable into a query, or the response may not be translatable into an answer.

[0126] However, it is possible to retrain the module continuously. Figure 3 A large upward arrow is used as a symbol to indicate feedback. The step of forwarding the response 453-2 to the operator may include receiving feedback from the operator. Continuing the example, the response "four years ago" may not be helpful to the operator, but feedback may be helpful, for example, to enable Task Manager 240 to fine-tune the time interval at which the data module processes data. The technical effect of the feedback can be, for example, to provide annotation. In short, positive feedback can act as a positive annotation (i.e., the data and data processing led to the correct response) or as a negative annotation.

[0127] Translation Method 403 (and the training in method 402) for a specific machine and to assist a specific operator have been described; the description will now continue. ●Distinguish machine 100 and machine 2 according to machine type. ● Treat machines of the same type as belonging to the machine queue.

[0128] subdomain Figure 4 Machine 100-1 of type 1 and machine 100-2 of type 2 are shown, and type-specific (type 1 data and type 2 data) and type-common data are shown.

[0129] Machine 100-1 is a first-type machine (e.g., a blast furnace), and machine 100-2 is a second-type machine (e.g., an electric arc furnace). Both share some common attributes, such as their use in producing metal. However, differences exist. In other words, machine 100-1 and machine 100-2 belong to different subfields of the overall field of metal-making equipment.

[0130] Data can be obtained as a multivariate time series, regardless of whether the data is applicable to the past, present, or even future. Some variables are common to both domains. For example, common variables indicate: (i) whether the metal in the boiler is in a liquid or solid state; (ii) the temperature inside the boiler; (iii) the amount of material (e.g., iron and / or iron ore), etc. Some variables are subdomain specific, such as the current and voltage measured only for machine 100-2.

[0131] Therefore, assistant device 200-1 can be configured for machine 100-1, and assistant device 200-2 can be configured for machine 100-2. However, assistant device 200-1 and assistant device 200-2 will have some common functions. For example, the data module used to respond to emergency switch activity can be consistent (since each machine has a switch), the data module used to predict temperature can have the same structure (although it will be trained in different ways, especially in sub-domains), the question / answer translation can be similar (or even inconsistent, in which case a noise example would be suitable for both), and so on.

[0132] Context detector 210 (see Figure 2 Contexts can be established differently for the two subdomains 210-C. The same natural person can alternatively be an operator of a blast furnace or an electric arc furnace (possibly an expert in "blast furnace" but a beginner in "electric arc furnace," and vice versa). However, for both functions, some information for establishing the operator context is the same (e.g., the language must be French, not English).

[0133] Similarity (and shared data) allows assistant device 200-1 and assistant device 200-2 to be implemented using common modules. In other words, see [link to relevant documentation]. Figure 2 The cross-domain assistant device 200 can be implemented through the context detector 210, whose rules are partly machine-specific. The data module can be split into multiple sub-candidates (for the enablement in step 423, some are only for the first domain, some are only for the second domain, and some are for both domains). The query interface 230 may only find very minor modifications (e.g., operator questions about arc electricity from boiler operators can be ignored).

[0134] Software as a Service Figure 4 Two types of machines, either in multiples or "queues," are also shown, symbolized here by overlapping rectangles. A particular industrial site may have multiple blast furnaces or multiple electric arc furnaces. The assistant device 200 receives context data 210-D and interacts with its user (i.e., the boiler operator) through questions and answers (which are implied to be data). Those skilled in the art can configure data transmission to and from a centralized computer used to provide the assistant device 200 using a software-as-a-service (SaaS) configuration.

[0135] hierarchy Figure 5 Methods for modifying transmitted information (such as...) are shown. Figure 3 A simplified flowchart of method 403 in the document.

[0136] Figure 4 The relatively high number of machines (two or more machine types or subfields; and queue members) contrasts with the relatively low number of assistant devices (e.g., in...). Figure 4 (An assistant device 200-1 for a blast furnace, and an assistant device 200-2 for an electric arc furnace). As already mentioned, it is possible to reuse some of the modules.

[0137] Context-driven workflows can be established to select the computer module that provides the response (i.e., the translated answer). Configuring this module at the hierarchy level can save computational effort.

[0138] like Figure 5 In the middle, the problem of arrival (see Figure 3 The receiving step (433-1) can be handled by a so-called master assistant device (or a general assistant device, i.e., an assistant device with a data module that can respond). If such a general assistant device is available, it can be used (see Using a full-domain assistant device). The same applies if the question has been translated (step 433-2) into a query.

[0139] Also there Figure 5 In the event that a general-purpose assistant device (for a specific question or query) is unavailable, a domain (identified as a domain or subdomain) can be identified, and processing can be performed via either the first domain assistant device 200-1 or the second domain assistant device 200-2 (see [link to documentation]). Figure 4 ) handling.

[0140] Subdomain identification can be achieved by linking the source of the problem to a specific machine. The person raising the problem is the operator of that specific machine, as the assistant device is aware of the (sub)domain. Data combination techniques are familiar to those skilled in the art, so no further explanation is needed here.

[0141] Assistant devices 200-1 and 200-2 can share access to the enabled module. This is related to activation step 423 (see [link to activation step 423]). Figure 3 This is related to the possibility that there may be a context that allows some modules to remain enabled most of the time. Alternatively, the task managers in both assistant devices could eventually keep a module enabled before disabling a specific module used by assistant device 200-1, as the context in assistant device 200-2 could trigger its activation.

[0142] Cross-domain checks may also be implemented. For example, questions about the future (i.e., questions that can use the enabled prediction module) can be related to the (enabled) prediction module.

[0143] Figure 6 The query interface 230 is shown (see Figure 2 The flowchart of the optional sub-steps of the operation.

[0144] If already passed Figure 3 As shown and described, interface 230 performs sub-steps 433-1 (receiving the question), 433-2 (translating the question into a query), and 433-3 (directing the query to an enabled network).

[0145] One or more of the following may occur: ●Because for specific languages, query interface 230 may not have been trained for specific phrases (see step 432 for training). Figure 3 (in Chinese), therefore question 330-Q (see Figure 2 Unable to translate into query 340-Q.

[0146] ● Question 330-Q can be translated into multiple queries (not a single query 340-Q).

[0147] Such situations are considered as having internal ambiguity, but which the computer can examine. Detecting ambiguity from phrases (in natural language) is known in this technique. Consulting the preprint "Improving Language Understanding" by Alec Radford et al., "Generative Pre-Training," is helpful. It is available from the research lab OpenAI's website (openai.com) and from Semantic Scholar (corpus ID: 160025533).

[0148] In the event of a detected ambiguity, the computer executing the method (i.e., via its query interface 230 or via another module communicating with interface 230) may perform additional steps (between receiving 433-1 and booting 433-1).

[0149] This step is given here only as an example: ● In case (1), the computer can take into account the operator context (known from step 413, see below) Figure 3 To fix problem 433-5.

[0150] ● In case (2), the computer can keep the problem unchanged, but can express the query according to the operator context (translated according to operator context in translation 433-2').

[0151] ● In case (3), the computer may ask questions (or statements) relevant to the operator (i.e., in general, the computer may request clarification 433-6). In this case, it can be expected that the operator will ask updated questions so that the computer will execute step 433-1 again.

[0152] Other steps are possible, but it is suitable to briefly return to the example: "When was the last time we had this noise?". Depending on the training, interface 230 can detect ambiguity, for example, in the following: ● The word "we" proposed by a single person refers to multiple people. The operator context can indicate that a person can be an operator of the machine. In (1), the query can be expressed as (pseudocode here) "Noise near the machine, when?".

[0153] ● The term “this” identifies the noise currently arriving from the machine, but the term may have connotations implying that the operator making the inquiry may be dissatisfied with the noise (i.e., other meanings). Of course, the term “noise” already implies some discomfort. In other words, the terms “such noise” or “noise” may relate to a person’s subjective feeling, but not to the objective existence of the sound emission. The trained interface can (in case (3)) request clarification only by presenting the following: Vibrating sound? In step 433-1, the computer will then receive “Yes, vibration”, or receive something else. (Of course, the computer does not have to use precise grammar rules, and training can follow different methods to detect the same content).

[0154] Other implementation aspects Some questions / answers will appear more frequently than others. The translation functionality may be implemented differently depending on the expected usage frequency (query interface 230).

[0155] Converting data into language (and vice versa) can be viewed as a two-way mapping of numerical values ​​(e.g., values ​​in multivariate or univariate time series) to statements in a non-numerical format.

[0156] Non-numerical formats can be language statements, statements in man-made languages, etc.

[0157] For example, the time series {0, 1, 2, 2, 3, 4} can be transformed (or translated) into the statement: "The value increases over time". Because of the time intervals between known individual values ​​(e.g., 60 seconds between measurements), the total time from the first value to the last value can also be considered as: "The value increases over time at 5-minute intervals".

[0158] In the notation of artificial languages, statements can be given as "continuously increasing (CONTINUE INCREASE)" or "continuously increasing over a period of 5 minutes (CONTINUE INCREASE)".

[0159] Sentences can be created based on language components (such as words and phrases) understood by human operators. The technology can also incorporate numerical data into language statements.

[0160] Xue and Salim's article explains the use of templates for data-to-text transformation. (Hao Xue, Flora D. Salim: PROMPTCAST: A NEW PROMPT-BASED LEARNING PARADIGM FOR TIME SERIESFORECASTING), arXiv:2210.08964.

[0161] Data conversion (or translation) between different formats (used by the data module) into domain-specific formats (see [link to relevant documentation]). Figure 5 Furthermore, it is specific to the expected operator context. In other words, data modules out of context should not be candidates for activation.

[0162] Task Manager 240 will not only distinguish data modules by the letters A, B, C, and D (as shown in the figure), but will also apply more appropriate technologies for identification (such as ID, network address, service descriptor (when the data module is regarded as providing a service)).

[0163] Non-interference control information As already mentioned, control communications and information communications are separate topics.

[0164] As explained, the assistant device 200 can process data primarily related to controlling the machine. However, information communication (to / from the assistant device 200) is not intended to directly control the machine. Even if the assistant device 200 generates a response suggesting an emergency stop (see...), Figure 1 (In step 101), the assistant device 200 will not stop the machine. This is reserved for human operators. In other words, the assistant device 200 can identify the machine's technical status and transmit that status (in a language suitable for the operator) to the operator 190. However, it is up to the operator 190 to react accordingly. The decision is made by the operator 190, not by the assistant device 200.

[0165] train The description returns immediately. Figure 3 The top row of the diagram illustrates the training activity. Training is adapted to an implementation where modules are implemented using machine learning tools such as ANNs. Training may include: ● Step 442 is used to train modules 220-A, 220-B, 220-C, 220-D (and many other data modules, not shown). ●Step 412, used to train the context detector 210, and ●Step 432 is used to train query interface 230.

[0166] This training can be performed by someone skilled in the art. For example, the two references mentioned above by Schockaert et al. also explain the training.

[0167] Training can be based on historical data, which is machine data in the form of multivariate time series. Machine data includes machine parameters, data on materials and products, data on energy consumption, and data on specific events.

[0168] Historical data can be annotated for use as training data. In the example, it can be assumed that: (i) similar noise occurred previously (from machine 100, or from the same type of machine, in the same subdomain, see...). Figure 4 (ii) Vibration and sound have been captured as historical data (in time series, with the identification time points); and (iii) the operator experiencing the noise recorded the noise as “extremely unpleasant” in the log file, but the machine continued to operate normally.

[0169] Training steps can be interconnected to share data. For example, the bidirectional arrow between steps 412 and 432 indicates that training the context detector and training the query interface can use (or exchange) shared historical data, such as context data (see [link to documentation]). Figure 2 210-D in this training scenario (historical data).

[0170] The dashed lines illustrate the use of the training module: it participates in step 413 (its primary purpose) by training the context detector, and can also participate in providing the operator context to realize the functions of translation 433-2 and 453-1. In addition, it participates in steps 433-2 and 453-1 by training the query interface (participating in the main functions of translation).

[0171] Other fictional examples In a second example, operator 190 (who is the user of assistant device 200) may ask the following question: "Currently, due to a change in materials, I have noticed a sudden increase in vibration amplitude. Could you check if this phenomenon has occurred before?"

[0172] Assistant device 200 can decompose the problem (by translating 433-2 into a query) into the following: ● The variable {Xi} representing the oscillation (in the multivariate time series {{X}}) is increasing, where the derivative (its change over time) is above the threshold derivation, and during the minimum interval, ΔT NUMBER>THRESHOLD_DURATION.

[0173] ● The operator's observations must be used as annotations (for further use, see the training steps of the data module), for example, VIBRATION_INCREASE (or "sudden increase").

[0174] ● There is a correlation with the parameter "Material Input," which is also used as a comment. In computer languages, in an example of interpreted pseudocode, it might look like this: INPUT_CHANGE ---> VIBRATION_INCREASE.

[0175] ● There is a task (i.e., query) to be performed on a data module that has historical data, including vibration data (which includes material data). The historical data is not used as training data, but rather as information to inform the operator (in the simplest case, the answer is YES, indicating the past occurrence time). Consulting the references of Radford et al. mentioned above is also applicable.

[0176] Metaphorically speaking, the assistant device 200 receives data from the user via a computer, operating much like a "survey form" with different fields. Through training (see step 432), the query interface 230 has learned which "field" in the survey form must be relevant to the data.

[0177] In a third example, operator 190 (who is the user of assistant device 200) may have specific functions as a service technician. See also... Figure 3 In training step 412, the context detector 210 can learn that service personnel frequently request maintenance data ("When was the last maintenance?", "When is the next scheduled maintenance?"). Therefore, the corresponding data module will be activated.

[0178] In the fourth example, operator 190 may wear protective clothing. This example is particularly suitable for machines belonging to metallurgical equipment, such as those suitable for boilers. There is an opportunity to implement sensors to obtain health data (e.g., skin resistance, heart rate, body temperature, body vibration, hearing threshold). Such data also describes the operator context. Upon detecting certain anomalies (anomalies of the human operator, not the machine), the corresponding data module can be activated. Ultimately, if operator 190 faces a health problem, the data module may already be activated, for example, to show operator 190 a method to leave the machine's environment (but keep the machine's operation unchanged).

[0179] In the fifth example, the assistant device 200 acts as a monitoring tool to monitor the operator 190, for example, to detect drowsy conditions.

[0180] In the sixth example, operator 190 asks assistant device 200 if there is access to a specific location on the machine (e.g., the top of the boiler). Depending on the context, assistant device 200 may grant this access but adds a context-specific warning: "Yes, but please use flame-retardant clothing." This warning can be more specific and could even use a nickname for the clothing (via...). Figure 3 (Training in step 432 is known).

[0181] In the seventh example, operator 190 asks the assistant device a question that triggers conditional predictions, such as "What will happen if I press this particular switch?". The context detector can identify a specific switch (e.g., a switch for a specific motor) as context and can enable data modules that have a relationship to the switch from a data perspective, such as data about a specific motor. Through training, the computer learns that it must use the data modules for prediction, and that some of the input data can be the state transition of the switch (e.g., the motor to be turned on).

[0182] In the eighth example, operator 190 asks assistant device 200 the same question that triggers conditional prediction, but for predictions between past points in time. The question is, for example, "What would happen if I pressed this particular switch a week ago?". As in the seventh example, the context detector can identify a specific switch (e.g., a switch for a specific motor) as context and can enable data modules that have a relationship to the switch from a data perspective, such as data about a specific motor. Through training, the computer learns which data modules must be used for prediction, and some input data can be the state transition of the switch (e.g., a motor to be turned on). The prediction is shifted to the past only in terms of time.

[0183] As mentioned above, there is a difference between control communication (operator / machine) and information communication (operator / assistant device), and assistant device 200 supports decision-making (for control). In a negative example, operator 190 might ask assistant device 200 questions unrelated to control, such as inquiring about the price of spare parts. Since assistant device 200 (and its data modules) has been trained with technical data (in the broadest sense), assistant device 200 will refuse to answer.

[0184] Multivariate time series This specification occasionally refers to a multivariate time series representing multiple measurements. Generally, {{X}} represents a multivariate time series, where i = 1 to N variables {Xi}. Index i is the variable index, and N is the variable number. {{X}} includes multiple univariate time series. Variable {Xi} is given as a univariate time series. In alternative notation, a univariate time series can be given as a sequence of data samples.

[0185] M represents the number of samples in the observation interval WINDOW (i.e., the duration of the time series), and individual samples are identified by index m. The sampling interval has WINDOW = ΔT. The duration of M. ΔT represents the sampling interval. ΔT can be the same for all i. This is for ease of explanation, but is not actually necessary. Different univariate time series can use different sampling intervals. For example, if the temperature is measured once per minute, Δt = 60 seconds. At each time point tm, data representing the values ​​of each variable Xi can be obtained.

[0186] General-purpose computer Figure 7 An example of a general-purpose computer device that can be used with the techniques described herein is shown. Figure 7 This is a schematic diagram illustrating examples of a general-purpose computer device 900 and a general-purpose mobile computer device 950 that can be used with the techniques described herein. Computer device 900 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Computer device 950 is intended to represent various forms of mobile devices, such as personal digital assistants, mobile phones, smartphones, driver assistance systems, or in-vehicle computers, and other similar computing devices. For example, a user (e.g., a blast furnace operator) can use computer device 950 as a front end to interact with computer device 900. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the inventions described and / or claimed herein.

[0187] Computing device 900 includes a processor 902, a memory 904, a storage device 906, a high-speed interface 908 connecting the memory 904 and a high-speed expansion port 910, and a low-speed interface 912 connecting a low-speed bus 914 and the storage device 906. Each of components 902, 904, 906, 908, 910, and 912 is interconnected via multiple buses and can be mounted on a common motherboard or otherwise suitably mounted. Processor 902 can process instructions to be executed within computing device 900, including instructions stored in memory 904 or storage device 906, to display graphical information of a graphical user interface (GUI) on an external input / output device, such as a display 916 connected to high-speed interface 908. In other implementations, multiple processors and / or multiple buses, as well as multiple memories and various memory types, can be used as needed. Furthermore, multiple computing devices 900 can be connected, each providing some of the necessary operations (e.g., as a server group, a set of blade servers, or a multiprocessor system).

[0188] Memory 904 stores information within computing device 900. In one implementation, memory 904 is one or more volatile storage units. In another implementation, memory 904 is one or more non-volatile storage units. Memory 904 may also be other forms of computer-readable media, such as a magnetic disk or optical disk.

[0189] Storage device 906 provides ample storage space for computing device 900. In one implementation, storage device 906 may be or include computer-readable media, such as floppy disks, hard disks, optical disks, magnetic tapes, flash memory, or other similar solid-state storage devices; or it may include device arrays, including devices in a storage area network or other configuration. The computer program product may be tangibly embodied in an information carrier. The computer program product may also contain instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 904, storage device 906, or memory on processor 902.

[0190] High-speed controller 908 manages bandwidth-intensive operations of computing device 900, while low-speed controller 912 manages lower bandwidth-intensive operations. This functional allocation is merely illustrative. In one implementation, high-speed controller 908 is connected to memory 904, display 916 (e.g., via a graphics processor or accelerator), and high-speed expansion ports 910, which can accommodate various expansion cards (not shown). In this implementation, low-speed controller 912 is coupled to storage device 906 and low-speed expansion port 914. The low-speed expansion port can contain various communication ports (e.g., USB, Bluetooth, Ethernet, Wireless Ethernet) and can be coupled to one or more input / output devices, such as keyboards, pointing devices, scanners, or network devices (e.g., switches or routers), for example, via a network adapter.

[0191] As shown in the figure, the computing device 900 can be implemented in a variety of different forms. For example, the computing device 900 can be implemented as a standard server 920, or multiple implementations within a group of such servers. The computing device 900 can also be implemented as part of a rack server system 924. Furthermore, the computing device 900 can also be implemented as a personal computer, such as a laptop computer 922. Alternatively, components of the computing device 900 can be combined with other components (e.g., device 950) in a mobile device (not shown). Each of such devices can include one or more computing devices 900 and 950, and the entire system can consist of multiple interconnected computing devices 900 and 950.

[0192] The computing device 950 includes components such as a processor 952, a memory 964, input / output devices (e.g., a display 954), a communication interface 966, and a transceiver 968. The device 950 may also be provided with a storage device (e.g., a micro hard disk or other device) to provide additional storage space. Each of the components 950, 952, 964, 954, 966, and 968 is interconnected via multiple buses, and the multiple components may be mounted on a common motherboard or otherwise suitably mounted.

[0193] Processor 952 can execute instructions within computing device 950, including instructions stored in memory 964. The processor can be implemented as a chipset containing multiple independent analog and digital processors. For example, the processor can provide for coordinating other components of device 950, such as controlling the user interface, applications running on device 950, and wireless communications of device 950.

[0194] Processor 952 can communicate with the user via control interface 958 and display interface 956 coupled to display 954. Display 954 can be, for example, a thin-film transistor liquid crystal display (TFTLCD) or an organic light-emitting diode (OLED) display, or other suitable display technologies. Display interface 956 can include appropriate circuitry for driving display 954 to present graphics and other information to the user. Control interface 958 can receive commands from the user and translate them for submission to processor 952. Furthermore, an external interface 962 can be provided to communicate with processor 952, enabling device 950 to perform near-field communication with other devices. For example, external interface 962 can provide wired communication in some implementations, wireless communication in others, and multiple interfaces can also be used.

[0195] Memory 964 stores information within computing device 950. Memory 964 can be implemented as one or more of the following: computer-readable media, volatile storage units, or non-volatile storage units. Extended memory 984 may also be provided and can be connected to device 950 via an extended interface 982, for example, which may include a Single In-line Memory Module (SIMM) card interface. Such extended memory 984 can provide additional storage space for device 950 and can also store applications or other information specific to device 950. Specifically, extended memory 984 may include instructions for performing or supplementing the above processes and may also contain security information. Thus, for example, extended memory 984 can serve as a security module for device 950 and can be programmed with instructions allowing secure use of device 950. Furthermore, secure applications and other information can be provided via a SIMM card, for example, by placing identification information on the SIMM card in a manner resistant to hacking.

[0196] As discussed below, memory may include flash memory and / or non-volatile random access memory (NVRAM). In one implementation, a computer program product is tangibly embodied in an information carrier. This computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer-readable or machine-readable medium that can be received, for example, by a transceiver 968 or an external interface 962, such as memory 964, extended memory 984, or memory on the processor 952.

[0197] Device 950 can conduct wireless communication via communication interface 966, which may include digital signal processing circuitry if necessary. Communication interface 966 can provide communication under various modes or protocols, such as GSM voice calls, SMS, EMS or MMS message transmission, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS. Such communication can occur, for example, via radio frequency transceiver 968. Furthermore, short-range communication can occur, for example, using Bluetooth, WiFi, or other such transceivers (not shown). Additionally, a Global Positioning System (GPS) receiver module 980 can provide device 950 with additional navigation and location-related wireless data, which can be used as needed by applications running on device 950.

[0198] Device 950 can also use audio codec 960 for voice communication, which can receive the user's voice information and convert it into usable digital information. Audio codec 960 can also generate audible sounds for the user, such as through a speaker, for example, in the earpiece of device 950. Such sounds can include sounds from voice phone calls, recorded sounds (such as voice messages, music files, etc.), and sounds generated by applications running on device 950.

[0199] As shown in the figure, the computing device 950 can be implemented in many different forms. For example, it can be implemented as a cellular phone 980. It can also be implemented as part of a smartphone 982, a personal digital assistant, or other similar mobile devices.

[0200] Various implementations of the systems and technologies described herein can be implemented in digital electronic circuits, integrated circuits, specially designed application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These different implementations may include implementation in one or more computer programs executable and / or translatable on a programmable system, which at least includes a programmable processor (which may be dedicated or general-purpose) coupled to receive and send data and instructions from and to a storage system, at least one input device, and at least one output device.

[0201] These computer programs (also referred to as programs, software, software applications, or code) contain machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, “machine-readable medium” and “computer-readable medium” mean any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. “Machine-readable signal” means any signal used to provide machine instructions and / or data to a programmable processor.

[0202] To enable interaction with the user, the systems and techniques described herein can be implemented on a computer having a display device for showing information to the user (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor), a keyboard, and a pointing device (e.g., a mouse or trackball) that the user can use to provide input to the computer. Other types of devices can also be used to enable interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input from the user can also be any form, including sound, speech, or tactile input.

[0203] The systems and technologies described herein can be implemented in one of the following ways: a computing device including back-end components (e.g., a data server); a computing device including middleware components (e.g., an application server); or a computing device including front-end components (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with the implementation of the systems and technologies described herein); or a computing device including any combination of the aforementioned back-end components, middleware components, or front-end components. The components of the system can be interconnected via any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0204] Computing devices can include clients and servers. Clients and servers are typically geographically separated and usually interact via communication networks. The relationship between clients and servers arises from computer programs running on the respective computers that have a client-server relationship with each other.

[0205] Several implementation methods have been described herein. However, it should be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0206] Furthermore, the logical flow shown in the figures does not necessarily require a specific order or sequence to achieve the desired result. Other components may be added to or removed from the system. Therefore, other embodiments are within the scope of the appended claims.

Claims

1. A computer-implemented method (403) for transmitting information about an industrial machine (100) to an operator (190) of the industrial machine (100), the computer-implemented method (403) being performed by a computer (200), the computer-implemented method comprising: The operator context (210-C) of the operator (190) is detected (413) by the context detector (210) based on the context data (210-D) received from the industrial machine (100) by the context detector (210), wherein the operator context (210-C) is defined by the relationship between the operator (190) and the industrial machine (100) and the function of the operator in controlling the industrial machine (100); The computer (200) enables (423) at least one of a plurality of data modules (220-A, 220-B) based on the detected operator context (210-C) by the task manager (240), thereby enabling (423) the data module (423) whereby the probability of the data module being used is related to the operator context; After the query interface (230) receives (433-1) a question from the operator (190), the question is translated (433-2) into a query (340-Q) and the query (340-Q) is directed (433-3) to one of the previously enabled data modules (220-B). The query (340-Q) is processed (443) by one of the enabled data modules (220-B) to obtain a response (340-R); and The response (340-R) is translated (453-1) into an answer (350-A) and the answer (350-A) is forwarded (453-2) to the operator (190).

2. The method according to claim 1, wherein, The step of activating (423) the at least one data module (220-A, 220-B) includes any of the following: The data modules (220-A, 220-B) are loaded into the main memory of the computer (200); Establish a data connection with the industrial machine (100) to receive machine data representing the current operating parameters of the industrial machine (100); Associate the model data with the data modules (220-A, 220-B) to be enabled; as well as Instruct the data module to begin operation.

3. The method according to any one of claims 1 and 2, wherein, Activating (423) the at least one data module (220-A, 220-B) includes communicatively connecting a data source to the at least one data module (220-A, 220-B), the step including any of the following: When the data modules (220-A, 220-B) are adapted to process data from the past, the data modules are connected to a data set of past operations of the industrial machine (100); as well as When the data modules (220-A, 220-B) are adapted to make predictions, the data modules are connected to a set of data on the past operations of the industrial machine (100) during a window interval.

4. The method according to any one of claims 1 to 3, wherein, The step of detecting (413) the operator context (210-C) of the operator (190) by the context detector (210) includes receiving updated context data (210-D) from the industrial machine (100) at times selected from: It was determined that the operator (190) of the industrial machine had been changed to a different person; It is determined that the operator (190) of the industrial machine has performed a predefined activity that affects the industrial machine (100); It is determined that the operator (190) has changed his or her physical position relative to the industrial machine (100) ((A), (B)); It is determined that the operator (190) has left the position of the industrial machine (100); Determine that the operator (190) is wearing specific clothing or has changed clothing; as well as It is determined that the operator (190) has changed the language used to interact with the operator interface (180) of the industrial machine (100).

5. The method according to any one of claims 1 to 4, wherein, The steps of translating the question (330-Q) (433-2) into the query (340-Q) and translating the response (340-R) (453-1) into the answer (350-A) are performed by saving data of the presented language statement, wherein the step of directing the query (340-Q) (433-3) to one of the enabled data modules (220-B) includes directing the query to the data module for processing the language statement.

6. The method according to claim 5, wherein, The guidance (433-3) of the query (340-Q) includes directing the query to the data module for processing the language statement, the language statement being selected from: language statements in natural languages; language statements in artificial languages, wherein the statements are encoded using alphanumeric characters; and statements that are derived from the language statement.

7. The method according to any one of claims 1 to 6, wherein, In the steps of translating the question (330-Q) (433-1) into the query (340-Q) and translating the response (340-R) (453-1) into the answer (350-A), the computer identifies the domain of the question (330-Q) and provides the answer (350-A) in the same domain.

8. The method according to any one of claims 1 to 7, wherein, In the step of detecting (413) the operator context (210-C), the context detector (210) provides the operator context (210-C) by classifying the context data (210-D).

9. The method according to any one of claims 1 to 8, wherein, After receiving (433-1) the question from the operator (190) through the query interface (230), and before translating (433-2) the question into the query (340-Q) through the query interface (230), the computer executing the method checks the question (433-4) for ambiguity, wherein, if there is no ambiguity, the computer continues to translate (433-2) the question, and if there is ambiguity, the computer performs other steps selected from the following: (i) Modify the problem described in (433-5) according to the operator context. (ii) Translate the question (433-2') according to the operator context, and (iii) Interact with the operator (433-6) to obtain further details in subsequent questions.

10. The method according to any one of claims 1 to 9, wherein, In the step of activating (423) at least one of the multiple data modules (220-A, 220-B), the task manager (240) also processes the state of the machine (100).

11. An application of the method (403) according to claims 1 to 10 in an auxiliary system for transmitting information about the industrial machine to an operator of the industrial machine.

12. A computer program product, which, when loaded into the memory of a computer system and executed by at least one processor of the computer system, causes the computer system to perform the steps of the computer-implemented method according to any one of claims 1 to 10.

13. A computer system comprising a plurality of modules, the plurality of modules performing the steps of the computer-implemented method according to any one of claims 1 to 10.

14. A computer-implemented auxiliary system (200), hereinafter referred to as an assistant device, for transmitting information about an industrial machine (100) to an operator (190) of the industrial machine (100), the assistant device (200) comprising: A context detector (210) is adapted to detect (413) the operator context (210-C) of the operator (190) based on context data (210-D), wherein the context detector (210) receives the context data (210-D) from the industrial machine (100). Multiple data modules (220-B) are adapted to process (443) queries (340-Q) to obtain responses (340-R). Task manager (240) is adapted to enable (423) at least one of the plurality of data modules (220-A, 220-B) based on the detected operator context (210-C). A query interface (230) is adapted to receive (433-1) a question from the operator (190), to translate (433-2) the question into the query (340-Q) and to direct (433-3) the query (340-Q) to one of the enabled data modules (220-B), wherein the query interface (230) is also adapted to translate (340-R) the response (340-R) into an answer (350-A) and to forward (453-2) the answer (350-A) to the operator (190).

Images