Virtual reality-supported diagnostic procedure for a system with computer-based auralization

Abstract

The invention relates to a method (100) for detecting an unintended operating state of a system (10) comprising a plurality of components (12, 13, 14, 17), each configured as a sound source (56, 58), as an object that is at least partially sound-reflecting, and / or as an object that is at least partially sound-absorbing. In the method (100), a first and a second sound image (36, 37) in the system (10) are detected by means of a first and second sound sensor (18.1), respectively. Furthermore, a receiver position (26) within a virtual space (20) in which the system (10) is represented is specified by means of a virtual reality system (30). A virtual sound image (31) is then determined and output to a user (33) and / or an artificial intelligence (50). Based on this, an unintended operating state is detected.The invention also relates to a computer program product (45) with which such a method (100) is implemented and to an evaluation unit (40) equipped with such a computer program product (45). Furthermore, the invention relates to the use of a virtual reality system (30) for inspecting a plant (10).

Description

The invention relates to a method for detecting an incorrect-determination operating state in a system and to a computer program product embodied for this purpose. The invention also relates to an evaluation unit for implementing the method and to a system equipped with such an evaluation unit. The invention further relates to the use of a virtual reality system for inspecting a plant.From the patent application DE 10 2016 125 886 A1 a method for calculating auralization is known, in which a first unit is used for auralization in a time domain and a second unit for auralization in a frequency domain. Depending on a test criterion, switching is performed between the first and second units.From the promotion document "Physically Base Real-Time Auralization of Interactive Virtual Environments" by Dirk Schroder, RWTH Aachen, a real-time room acoustic simulation is known which is suitable for modelling residential rooms, lecture rooms, concert rooms or subway stations.From the patent U.S. Pat. No. 11,170,139 B1 a real-time capable acoustic ray tracing method is known, in which a first receiver position is defined in a space and a beam group emitted from a second position.The document DE 10 2016 125 886 A1 discloses an apparatus for the efficient calculation of auralization with a signal processing device which has a first unit for auralization in the time domain and a second unit for auralization in the frequency domain. Switching can be performed between the first and second units.Patent application DE 10 2017 103 385 A1 shows a method for evaluating structure-borne noise and a device suitable for this purpose. Therein, a neural network is constructed and trained, so that the structure-borne sound can be evaluated by deep learning. A system state of the machine is identifiable by means of the method.US 2017 / 0 286 572 A1 discloses a physical-digital system with a digital twin. The physical-digital system has at least one sensor which is linked to the digital twin. The digital twin is designed to simulate the operating behavior of an aircraft engine, for example, on the basis of the sensor signals.International patent application WO 2011 / 038 838 A1 discloses an acoustic representation of states in an industrial installation. Therein, an audio profile is selected with computer assistance based on a current state of the industrial installation and is converted into an acoustic signal. In this case, signals from a plurality of sources are filtered, weighted, aggregated and / or abstracted using a machine classifier.The publication EP 3 968 105 A1 shows a computer-implemented method for simulating a plant, in which input data about the plant are predefined for a simulation model. Acoustic data are also predefined for the simulation model. Based on the input data and the acoustic data, a simulation of the installation is effected by means of acoustic signals.To ensure an economical and low-failure operation of installations, in particular of automation systems, experts have been used to date who walk on the installation and inspect it on the basis of their own sensory impressions, for example acoustic impressions. Such attempts require personal presence. There is a need to make inspection of facilities more efficient and have it performed from the removal. At the same time, there is a need for more meaningful diagnostic possibilities in order to identify present determination-defective states. The invention is based on the object of providing a possibility suitable for this purpose.The object is achieved by a method according to the invention which is designed to identify an operation state of a system, in particular an automation system, which is not subject to determination. The plant can be, for example, a chemical plant, a refinery, a manufacturing plant, a power plant or an electrolyser. The installation comprises a plurality of components which cooperate and thereby implement an installation process. The components are each designed as a sound source, as an at least partially sound-reflecting object, as an at least partially sound-absorbing object, or as a combination thereof.The components are thus configured to emit, absorb, interact with, or a combination of sound. During operation of the installation, different sound images are thus produced, depending on a position within the installation, so-called location-dependent sound images. The method comprises a first step, in which the installation is provided in an active operating state. An active operating state is understood here to mean a state in which the plant process runs at least partially, starts up or runs out, so that sound is emitted. Furthermore, in the first step, a virtual space is provided, for example in a virtual reality system, in which the installation is at least partially reset. The virtual space is viewable and virtually walkable by a user.Furthermore, the method according to the invention has a second step, in which a first sound image is detected by a first sound sensor and a second sound image is detected by a second sound sensor. The first and second sound sensors are arranged in the system and connected to a suitable evaluation unit. In the second step, a sound emitted by at least one first sound source is detected at its respective positions by means of the first and second sound sensors. Furthermore, a third step belongs to the method according to the invention, in which a receiver position is specified within the virtual space which is provided in the first step. The receiver position can be specified by a user who uses a virtual reality system on which the method according to the invention is carried out. In the third step, a virtual sound image is also determined which is present at the receiver position. The determination of the virtual sound image can be effected on the basis of the sounds detected by the first and second sound sensors. The determination of the virtual sound image can be carried out in this case taking into account at least partially sound-reflecting and / or at least partially sound-absorbing properties of objects in the installation which are adjusted in the virtual space. In particular, on the basis of the sounds detected by the first and second sound sensors, the virtual sound image at the receiver position in the virtual space can be determined by calculating sound propagation paths within the system. The virtual sound image corresponds to a sound image that occurs when a user is located in the installation at the corresponding position.Furthermore, the method according to the invention comprises a fourth step, in which the virtual sound image determined in the third step is output to the user, in particular the user of the virtual reality system, and / or an artificial intelligence. On the basis of the output virtual sound image which is present at the receiver position, a determination-different operating state of the installation is detected in the fourth step. The determination and output of the virtual sound image represents a computer-implemented realization of an acoustic impression, i.e. of a sound image, which is present in the installation at the position corresponding to the predefined receiver position. The virtual sound image can be output as a binaural sound image. The detection of the non-determination operating state can be carried out by the user. Alternatively or additionally, the artificial intelligence can be designed as a trained neural network which determines deviations from a setpoint sound image at the receiver position in the virtual space or at the corresponding position in the installation, for example on the basis of a Fourier analysis. As a further alternative or in addition, the artificial intelligence can be coupled to an experience database in which sound images of disturbances are stored, with which pattern recognition in the virtual sound image can be carried out. In the case of a detected non-determination operating state of the installation, a warning can also be output in the fourth step.The method according to the invention makes it possible to evaluate acoustic sensation impressions, i.e. sound images, as virtual sound images by transmission into a virtual reality system. Complex sound images, and thus also correspondingly complex virtual sound images, can also be displayed with sufficient precision. The invention is based, inter alia, on the surprising finding that virtual sound images can be displayed with sufficient realism in virtual spaces. In the method according to the invention, a sufficiently low latency between the individual acoustic sensors is relevant. On the other hand, a total latency in the output of the virtual sound image is of secondary importance. The invention is also based, inter alia, on the surprising finding that already knowledge of latency differences between individual acoustic sensors is sufficient to determine and output a sufficiently realistic virtual sound image. Overall, latency due to data transmission between the plant and the output of the virtual sound image is not quality-limiting in puncto reality litter. As a result, the method according to the invention is suitable for carrying out an acoustically assisted inspection, and thus acoustically assisted recognition, of determination-defective operating states remotely, for example over hundreds or thousands of kilometers of distance. Overall, this realizes location-independent detection of determination-external operating states of the installation, which is carried out by the user and / or the artificial intelligence.In one embodiment of the claimed method, the first sound source can be mapped as a first virtual sound source in virtual space. For this purpose, at least the first and second sound images are used to determine by bearing, triangulation or trilateration which frequencies of the respective sound images come from which direction, based on the position of the sound sensors. Accordingly, a sound emission spectrum can be determined for individual components which are sound sources in the system. The respective acoustic emission spectrum of an acoustic source can thereby be output as a corresponding virtual acoustic source in a component-specific manner in the virtual space. As a result, the first virtual sound source can also be separately inspected as a virtual counterpart to a corresponding component of the installation in the virtual space. By means of the claimed method, targeted detection of conditions that are not subject to determination is thereby possible. In addition, in the method claimed, it is also possible to emit support signals which are transmitted in puptoFurthermore, at least the third and fourth steps can also be carried out for a second sound source. The claimed method can be applied overall to a plurality of sound sources, which allows a further targeted detection of non-determination conditions. In the fourth step, the virtual sound image at the receiver position may be determined based on a combination of the first virtual sound source and the second virtual sound source. This can be effected, for example, by means of so-called acoustic ray tracing. The virtual sound image thus corresponds to a superimposed sound image which results from the corresponding first and second sound source in the system taking into account the receiver position. The claimed method can be scaled in a surprisingly simple manner, in a manner saving computing capacity, to complex installations having a plurality of sound sources, as a result of which complex installations in virtual space can also be inspected.Likewise, in the method claimed, the virtual sound image can be output on a virtual reality headset, wherein the installation is shown virtually walkable thereon. Virtual reality headsets that belong to a virtual reality system provide an increased level of realism and are readily available. Furthermore, virtual reality headsets are used in other fields of application, for example in the entertainment industry. The claimed method uses an output means with the virtual reality headset, which is developed further quickly, so that advances in virtual reality headsets can be used quickly by the claimed method.Furthermore, a warning can be output in the fourth step of the method via the virtual reality headset.In a further embodiment of the claimed method, an operating data record of the corresponding sound source can be output for a virtual sound source in a representation on the virtual reality headset after a selection by the user. Therein, operating data of the component that is a sound source in the installation and is adjusted as a virtual sound source can be displayed. For example, the component can be an electric motor and its current rotational speed can be displayed as operating data. The user can thus plausibilize the output virtual sound image with respect to the operating data. Detection of an incorrect determination state is thus further supported. The display of the operating data record can also be designed as a graphical display which is superimposed on a visual reproduction of the installation reset in the virtual space. This can be, for example, a superimposed false color representation, which is also referred to as a heat map. Any available quantity describing the operation of the system, for example acoustic quantities representing a frequency band of a virtual sound source or a present volume level at the receiver position, can be displayed by means of the graphic display. The detection of an incorrect determination state of the installation is thus further facilitated and accelerated.In addition, at least one component of the plant can be a motor, a pump, a stirrer, a fan or another mechanical application. Such components have movable parts and generate operating noises which are characteristic depending on the present operating state, in particular a rotational speed. Accordingly, deviations therefrom are meaningful indicators of an incorrect operating state. Alternatively or additionally, at least one component of the system can also be lines through which a process medium can flow, which lines have an acoustic characteristic when there is a corresponding flow. Further alternatively or additionally, at least one component of the installation can be a pneumatic module in which a leak can be acoustically detected by means of the method claimed. Likewise, the at least one component may be a high-voltage application in which switching operations are acoustically identifiable. The claimed method is particularly suitable for identifying non-determination operating states in a wide range of components. The recognition can be performed by the user or by the artificial intelligence.Furthermore, the claimed method may comprise a fifth step, in which a virtual sound source is selected by the user, which corresponds to a component in the installation. This may be done, for example, within the virtual space provided by the virtual reality system. A sixth step follows, in which a sound-source-specific sound image is determined. The sound source-specific sound image is determined here at least on the basis of the first and second sound images which are acquired in the second step. The sound source-specific sound image includes an acoustic impression that is based on the sound emissions that are exclusively produced by the component corresponding to the selected virtual sound source. The sound source-specific sound image applies specifically to a position that corresponds to the predefined receiver position in the virtual space. The sound source-specific sound image can be determined corresponding to the first virtual sound source. In a further seventh step, the sound source-specific sound image in the virtual space is output as a virtual sound source-specific sound image. In this case, a sound gate is reduced, that is to say its volume in the output is reduced. The sound gate is understood to mean acoustic impressions which are not caused by the selected virtual sound source. This allows disturbing acoustic impressions in the virtual space to be suppressed and individual components to be inspected acoustically in a targeted manner. This allows more precise diagnosis if a determination-different operating state is present on the associated component. This further increases the significance of the claimed method. The fifth, sixth, and seventh steps may also be performed for a plurality of virtual sound sources. The more sound sensors are used, the more exactly the sound gate can be separated from the sound source-specific sound image.In one embodiment of the claimed method, the sound source-specific sound image of at least one virtual sound source of the artificial intelligence can be provided for identifying an incorrect-determination operating state of the corresponding sound source in the installation, that is to say one of its components. In this case, an evaluation mode can be predefined by the user, for example by selection in virtual space. By selecting the evaluation mode, an algorithm can be specified by which the virtual sound source is to be evaluated. As a result, in the method claimed, computing-intensive algorithms can also be used, which are too computing-intensive to run them continuously in the accompanying operation. Alternatively or additionally, the evaluation mode can also be proposed or selected by the artificial intelligence. Overall, the diagnostic possibilities for the claimed method are thus extended in a manner saving computing capacity.Furthermore, in the method claimed, a comparison data set can be selected which comprises a historical operation of the installation. The comparison data record is provided to the installation represented in the virtual space, so that the historical operation of the installation stored therein is at least acoustically adjusted and is output analogously to the third and fourth steps of the method. Accordingly, a virtual sound image is output at the predetermined receiver position on the basis of the comparison dataset. When outputting the comparison data set in virtual space, the installation can still be walked on therein. During the method, it is possible to switch in a predeterminable manner between the virtual sound image, which is based on the comparison data set, and the virtual sound image, which is based on the sound images acquired in the second step, during the output. The comparison data set can depict a good state of the installation. The user and / or the artificial intelligence are thus supported by a direct comparison of the virtual sound images at the same receiver position when identifying an incorrect-determination operating state. As a result of the virtual walking on, meaningful receiver positions can thus be specified and can be acoustically examined there. In order to recognize an incorrect determination state, a difference can be formed between the virtual sound image based on the comparison data set and the virtual sound image based on the detection in the second step, and a virtual difference sound image is thus present. The non-determination operating state can be recognized by the user and / or the artificial intelligence by evaluating the virtual difference sound image. Furthermore, the first and / or second sound sensor can be designed as a microphone, directional microphone or as a sound camera. Microphones, directional microphones and sound cameras offer increased measurement accuracy and can be coupled to the evaluation unit of the installation via powerful communicative data connections. Furthermore, this also allows sound emissions outside the human-audible frequency spectrum to be detected. As a result, the artificial intelligence is further supported in its evaluation and recognition of an operation state that is not dependent on the determination. Damage that causes characteristic sound images in the ultrasonic spectrum or infrasonic spectrum can thus also be used. The diagnostic capabilities of the claimed method are thereby further enhanced.In a further embodiment of the claimed method, a detected ultrasonic emission or a detected infrasonic emission is shifted in its frequency position. The infrasonic or ultrasonic emission is shifted in the frequency position to the extent that it lies in the human-audible spectrum. The ultrasonic or infrasonic emission thus frequency-shifted is output in the method as claimed as a virtual sound image in the fourth step. For this purpose, the corresponding component, i.e. its representation in virtual space, is selected by the user. Likewise, a frequency range in which the ultrasonic or infrasonic emission is to be output and / or evaluated can be specified by the user. The frequency-shifted output of infrasonic or ultrasonic emissions represents an acoustic counterpart to a visual representation with false colors, for example in infrared images. The claimed method is therefore suitable for providing audible frequency ranges in an evaluable manner even for humans. This further increases the diagnostic possibilities, in particular with respect to a request in the presence.The object described at the beginning is likewise achieved by a computer program product according to the invention. The computer program product is designed to receive and process measurement data from a plurality of sound sensors. Furthermore, the computer program product is designed to calculate a virtual space with virtual objects which map components of a system, in particular of an automation system, and to simulate them in cooperation with a suitable device, for example a virtual reality headset. The readjustment of the virtual space with the virtual objects comprises a visual and acoustic readjustment. According to the invention, the computer program product is designed to carry out at least one embodiment of the methods outlined above. The features of the underlying method can therefore also be easily transferred to the computer program product according to the invention. The underlying method can be carried out in a manner which protects the computing capacity. The computer program product is suitable in particular for receiving the measurement data from sound sensors which are remote from the device on which the virtual space is displayed. The computer program product is suitable overall for location-independent inspection, in particular acoustic inspection, of the associated installation. Furthermore, the computer program product can be monolithically embodied, i.e. executable on a single hardware platform. Alternatively, the computer program product can be modular and comprise a plurality of subprograms that can be executed on separate hardware platforms. The subprograms are connected to one another via at least one communicative data connection and cooperate via said data connection to provide the functionality of the claimed computer program product. The computer program product can be stored in a storage unit, for example a USB memory, a hard disk or an optical data carrier in remanent form.In one embodiment of the claimed computer program product, the latter can be coupled to a digital twin of the installation to be simulated. Alternatively, the computer program product may comprise a digital twin of the installation. The digital twin is connected via communicative data connections to components of the installation, for example a control unit and / or sound sensors. Consequently, the digital twin is suitable for detecting a present operating state of the installation and tracking it in a digital image of the installation. The digital image in the digital twin is designed to simulate an interaction of components of the installation with one another and with its environment. The digital twin can be designed, for example, as a digital twin according to the publication US 2017 / 286572 A1. The disclosure content of US 2017 / 286572 A1 is included in the present application. Digital twins typically have precise simulation routines whose performance is usable by the underlying method. The virtual sound image can be determined on the basis of a plurality of separate simulations of sound propagations, which cooperate as co-simulations in the sense of European patent EP 4 016 216 B1.The underlying object is also achieved by an evaluation unit according to the invention. The evaluation unit is configured to receive a plurality of measurement signals from a plurality of acoustic sensors. Furthermore, the evaluation unit has a data interface, via which operating data of a system to be reset can be received. The monitoring unit is suitable for storing and executing a computer program product which is suitable for evaluating the received measurement data and the operating data. According to the invention, the computer program product is designed according to one of the embodiments outlined above. The evaluation unit can be designed as a master computer, as a programmable logic controller, or as a computer cloud. Alternatively or additionally, the evaluation unit can be coupled to an operator station or can belong to the operator station of the installation.Furthermore, the object is achieved by a system according to the invention, which can be designed in particular as an automation system. The plant comprises a plurality of components which cooperate to execute a plant process. The components are each designed as a sound source, as an at least partially sound-reflecting object, as an at least partially sound-absorbing object, or a combination thereof. The installation also comprises a monitoring unit which is configured to detect an operation state of the installation which is not subject to determination. According to the invention, the monitoring unit of the installation is designed according to one of the embodiments described above.The object outlined at the beginning is likewise achieved by a system according to the invention having a virtual reality system and an evaluation unit. The virtual reality system has at least one virtual reality headset and is designed for computer-based auralizing of a sound image at a predefinable receiver position within a virtual space. According to the invention, the virtual reality system is used to identify an out-of-destination operating state of a system, in particular an automation system, that is being reset with it. The detection of the determination-disabled state can be carried out independently of location in this case. Furthermore, a warning can also be output to the user on the virtual reality system. According to the invention, the virtual reality system is connected to an evaluation unit according to one of the embodiments presented above. The features of the evaluation unit, and thus also the features of the underlying method, can thus be transferred to the virtual reality system. In particular, the virtual reality system can be configured to at least partially perform at least one of the methods described above.The invention is explained in more detail below with reference to individual embodiments in the figures. The figures are to be read in a complementary manner in so far that the same reference numerals have the same technical meaning in different figures. The features of the individual embodiments can also be combined with one another. Furthermore, the features of the embodiments shown in the figures can be combined with the features outlined above. They show in detail: FIG. 1 shows a schematic structure of an automation system on which a first embodiment of the claimed method is carried out; FIG. 2 shows a schematic structure of an automation system on which a second embodiment of the claimed method is carried out; FIG. 3 shows a schematic structure of an automation system on which a third embodiment of the claimed method is carried out.A structure of a plant 10 on which a first embodiment of the claimed method 100 is carried out is schematically illustrated in FIG. 1. The plant 10 comprises a plurality of components 12 which cooperate in an intended operation in order to carry out a plant process 15. One of the components 12 is connected as a motor 14, another component 12 as a fan 13 and yet another component 12 as a stirrer 17. The components 12 can be controlled by the control unit 60 by means of a control program 65 for carrying out the installation process 15. The system 10 also comprises a plurality of sound sensors 18, which are each designed as microphones or sound cameras. Among the components 12, the motor 14 and the agitator 17 are formed as sound sources and emit sound emissions that result in sound 19. The further components 12 are each designed as at least partially sound-reflecting objects and / or at least partially sound-absorbing objects. The components 12 thus have an effect on sound propagation paths 24, which result for the sound emissions or sounds 19. Depending on a position 11 in the installation 10, a location-dependent sound image 38 is thus obtained. The installation 10 is provided in an active operating state in a first step 110 of the method 100. An active operating state is understood here to mean an operation in which the system process 15 runs at least partially and sound emission, i.e. sound 19, occurs on at least one component 12.In the first step 110 of the method 100, a virtual space 20 is also provided in which the installation 10 is adjusted. In the virtual space 20 there is a digital twin 25 of the installation 10, which comprises a plurality of virtual objects 22, each of which simulates a component 12 of the automation system 10. Virtual space 20 is shown walkable to a user 33 via a virtual reality headset 32 associated with a virtual reality system 30. The user 33 can use this to specify a receiver position 26 within the virtual space 20, which corresponds to the position 11 in the installation 10 shown in FIG. 1. The walkability is shown in FIG. 1 by the arrow 27. The virtual space 20 is provided by a computer program product 45 which is stored in an executable manner on an evaluation unit 40 of the installation 10.The method 100 also includes a second step 120, in which a first sound image 36 is captured by a first sound sensor 18.1 and a second sound image 37 is captured by a second sound sensor 18.2. the first and second sound images 36, 37 are each an acoustic impression that arises at the position of the respective sound sensor 18.1, 18.2 as a function of the sounds 19 and the sound propagation paths 24 in the installation 10. In the second step 120, a component 12, namely the motor 14, is determined as the first sound source 56. For this purpose, triangulation, trilateration or bearing is carried out by means of the computer program product 45 which is executed on the evaluation unit 40 of the system 10. Overall, it is determined on the basis of the first and second sound images 37, 39 which sound 19 is emitted separately from the first sound source 56. Likewise, in the second step 120, a component 12, namely the stirrer 17, is determined as the second sound source 58. The determination of the second sound source 58 proceeds analogously to the first sound source 56. An emitted sound 19 is also determined with respect to the second sound source 58.Furthermore, the method 100 comprises a third step, in which the receiver position 26 in the virtual space 20 is predefined by the user 33. The recipient position 26 is specified by walking 27 on the virtual space 20 according to a representation by the virtual reality headset. In the third step 130, a virtual sound image 31 is determined, which corresponds to a location-dependent sound image 38 that is perceptible at the corresponding position 11 in the installation 10. To ascertain the virtual sound image 31, virtual objects 22 are adjusted as a first virtual sound source 28 and as a second virtual sound source 29 in the virtual space 20. Sound propagation paths 24 are calculated in the virtual space 20 and the virtual sound image 31 is thus determined at the receiver position 26. The virtual objects 22 are also virtual objects 22 which correspond to the components 12 and are at least partially sound-reflecting and / or at least partially sound-absorbing. In the virtual space 20, the corresponding at least partially sound-reflecting and / or at least partially sound-absorbing behavior is simulated therein in order to determine the sound propagation paths 24. The ascertainment of the virtual sound image 31 is likewise carried out by the computer program product 40, which is at least coupled to the digital twin 25.The method 100 further comprises a fourth step 140, in which the virtual sound image 31 is output to the user 33. The output to the user 33 is effected via the virtual reality headset 32. Accordingly, the virtual sound image 31 corresponds to the location-dependent sound image 38 that is perceptible at the position 11 in the installation 10 that corresponds to the predefined receiver position 26. The output virtual sound image 31 recognizes a determination-different state of the installation 10 by the user 33. Similarly, in the fourth step 140, the virtual sound image 31 is output 43 to an artificial intelligence 50, which at least interacts with the computer program product 45. The artificial intelligence 50 is designed as a neural network and is suitable for comparing the virtual sound image 31 with sound images from a database 46 which is connected to the evaluation unit 40 via a communicative data connection 42. In order to recognize the non-determination operating state, a target sound image from the database 46 that is associated with a region of the predefined receiver position 26 is compared with the virtual sound image 31 by the artificial intelligence 50. If an adverse operating state of the installation 10 is detected, a warning 48 is output. The warning 48 is transmitted via a communicative data connection 42 between the evaluation unit 40 and the control unit 60 and thus enables a reaction by the control program 65. Furthermore, in method 100, one of virtual objects 22 is selected by user 33. The selection is represented by the hand symbol 23. Selection 23 takes place within virtual space 20. The operating data 62 comprise measured values, present control commands, setpoint values and / or other parameters, by means of which the operation of the respective component 12 is characterized. The operational data 62 is presented to the user 33 in the virtual space 20 as a virtual representative 49, for example as a window floating in the virtual space 20. The user 33 is thus provided with the operating data 62 associated with the selected component 22 for plausibility checking of the output warning 48 and / or for further diagnosis in the virtual space. The method 100 allows the virtual space 20 in which the installation 10 is adjusted to be displayed remotely therefrom. The installation 10 can thus be identified by the method 100 according to FIG. 1 independently of location for the user 33.A structure of a plant 10 on which a second embodiment of the claimed method 100 is carried out is schematically illustrated in FIG. 2. The plant 10 comprises a plurality of components 12 which cooperate in an intended operation in order to carry out a plant process 15. One of the components 12 is connected as a motor 14, another component 12 as a fan 13 and yet another component 12 as a stirrer 17. The components 12 can be controlled by the control unit 60 by means of a control program 65 for carrying out the installation process 15. The system 10 also comprises a plurality of sound sensors 18, which are each designed as microphones or sound cameras. Among the components 12, the motor 14 and the agitator 17 are formed as sound sources and emit sound emissions that result in sound 19. The further components 12 are each designed as at least partially sound-reflecting objects and / or at least partially sound-absorbing objects. The components 12 thus have an effect on sound propagation paths 24, which result for the sound emissions or sounds 19. Depending on a position 11 in the installation 10, a location-dependent sound image 38 is thus obtained. The installation 10 is provided in an active operating state in a first step 110 of the method 100. An active operating state is understood here to mean an operation in which the system process 15 runs at least partially and sound emission, i.e. sound 19, occurs on at least one component 12.In the first step 110 of the method 100, a virtual space 20 is also provided in which the installation 10 is adjusted. In the virtual space 20 there is a digital twin 25 of the installation 10, which comprises a plurality of virtual objects 22, each of which simulates a component 12 of the installation 10. Virtual space 20 is shown walkable to a user 33 via a virtual reality headset 32 associated with a virtual reality system 30. The user 33 can use this to specify a receiver position 26 within the virtual space 20, which corresponds to the position 11 in the installation 10 shown in FIG. 2. The walkability is shown in FIG. 2 by the arrow 27. The virtual space 20 is provided by a computer program product 45 which is stored in an executable manner on an evaluation unit 40 of the installation 10.The method 100 also includes a second step 120, in which a first sound image 36 is captured by a first sound sensor 18.1 and a second sound image 37 is captured by a second sound sensor 18.2. the first and second sound images 36, 37 are each an acoustic impression that arises at the position of the respective sound sensor 18.1, 18.2 as a function of the sounds 19 and the sound propagation paths 24 in the installation 10. In the second step 120, a component 12, namely the motor 14, is determined as the first sound source 56. For this purpose, triangulation, trilateration or bearing is carried out by means of the computer program product 45 which is executed on the evaluation unit 40 of the system 10. Overall, it is determined on the basis of the first and second sound images 37, 39 which sound 19 is emitted separately from the first sound source 56. Likewise, in the second step 120, a component 12, namely the stirrer 17, is determined as the second sound source 58. The determination of the second sound source 58 proceeds analogously to the first sound source 56. An emitted sound 19 is also determined with respect to the second sound source 58.Furthermore, the method 100 comprises a third step, in which the receiver position 26 in the virtual space 20 is predefined by the user 33. The recipient position 26 is specified by walking 27 on the virtual space 20 according to a representation by the virtual reality headset. In the third step 130, a virtual sound image 31 is determined, which corresponds to a location-dependent sound image 38 that is perceptible at the corresponding position 11 in the installation 10. To ascertain the virtual sound image 31, virtual objects 22 are adjusted as a first virtual sound source 28 and as a second virtual sound source 29 in the virtual space 20. Sound propagation paths 24 are calculated in the virtual space 20 and the virtual sound image 31 is thus determined at the receiver position 26. The virtual objects 22 are also virtual objects 22 which correspond to the components 12 and are at least partially sound-reflecting and / or at least partially sound-absorbing. In the virtual space 20, the corresponding at least partially sound-reflecting and / or at least partially sound-absorbing behavior is simulated therein in order to determine the sound propagation paths 24. The ascertainment of the virtual sound image 31 is likewise carried out by the computer program product 40, which is at least coupled to the digital twin 25.The method 100 further comprises a fourth step 140, in which the virtual sound image 31 is output to the user 33. The output to the user 33 is effected via the virtual reality headset 32. Accordingly, the virtual sound image 31 corresponds to the location-dependent sound image 38 that is perceptible at the position 11 in the installation 10 that corresponds to the predefined receiver position 26. The output virtual sound image 31 recognizes a determination-different state of the installation 10 by the user 33. Similarly, in the fourth step 140, the virtual sound image 31 is output 43 to an artificial intelligence 50, which at least interacts with the computer program product 45. The artificial intelligence 50 is designed as a neural network and is suitable for comparing the virtual sound image 31 with sound images from a database 46 which is connected to the evaluation unit 40 via a communicative data connection 42. In order to recognize the non-determination operating state, a target sound image from the database 46 that is associated with a region of the predefined receiver position 26 is compared with the virtual sound image 31 by the artificial intelligence 50. If an adverse operating state of the installation 10 is detected, a warning 48 is output. The warning 48 is transmitted via a communicative data connection 42 between the evaluation unit 40 and the control unit 60 and thus enables a reaction by the control program 65.Furthermore, in method 100, in a fifth step 150, one of virtual objects 22 is selected by user 33. The selection is represented by the hand symbol 23. In a further sixth step 160, a sound-source-specific sound image 34 is determined, which is present at the predefined receiver position 26. The sound source-specific sound image 34 is based on the virtual sound image 31 at the receiver position 26. Accordingly, sounds that are classified as sound gate 53 by virtual objects 22 other than the first virtual sound source 28. The portion of the sound gate 53 is output in the output of the sound source-specific sound image 34 with an adjustable acoustic damper 44. The sound gate 53 is output with reduced volume by the adjustable acoustic damper 44 in a seventh step 170 of the method 100. The sound gate 53 can also be completely hidden in the seventh step 170 by the adjustable acoustic damper 44. In this case, the sound source-specific sound image 34 represents a theoretical acoustic impression in the system 10, which is obtained if the components 12 not considered were still. The sound source-specific sound image 34 can also be transmitted to the artificial intelligence 50 for identifying an operation state that is not eligible for determination. Overall, the method 100 according to FIG. 2 makes possible a diagnosis based on acoustic impressions, which is not possible when the installation 10 is walked on in the presence. At the same time, the spectrum of diagnostic possibilities is expanded with the method 100 according to FIG. 2 during location-independent inspection for the user 33.A structure of a plant 10 on which a first embodiment of the claimed method 100 is carried out is schematically illustrated in FIG. 3. The plant 10 comprises a plurality of components 12 which cooperate in an intended operation in order to carry out a plant process 15. One of the components 12 is connected as a motor 14, another component 12 as a fan 13 and yet another component 12 as a stirrer 17. The components 12 can be controlled by the control unit 60 by means of a control program 65 for carrying out the installation process 15. The system 10 also comprises a plurality of sound sensors 18, which are each designed as microphones or sound cameras. Among the components 12, the motor 14 and the agitator 17 are formed as sound sources and emit sound emissions that result in sound 19. The further components 12 are each designed as at least partially sound-reflecting objects and / or at least partially sound-absorbing objects. The components 12 thus have an effect on sound propagation paths 24, which result for the sound emissions or sounds 19. Depending on a position 11 in the installation 10, a location-dependent sound image 38 is thus obtained. The installation 10 is provided in an active operating state in a first step 110 of the method 100. An active operating state is understood here to mean an operation in which the system process 15 runs at least partially and sound emission, i.e. sound 19, occurs on at least one component 12.In the first step 110 of the method 100, a virtual space 20 is also provided in which the installation 10 is adjusted. In the virtual space 20 there is a digital twin 25 of the installation 10, which comprises a plurality of virtual objects 22, each of which simulates a component 12 of the installation 10. Virtual space 20 is shown walkable to a user 33 via a virtual reality headset 32 associated with a virtual reality system 30. The user 33 can use this to specify a receiver position 26 within the virtual space 20, which corresponds to the position 11 in the installation 10 shown in FIG. 3. The walkability is shown in FIG. 3 by the arrow 27. The virtual space 20 is provided by a computer program product 45 which is stored in an executable manner on an evaluation unit 40 of the installation 10.The method 100 also includes a second step 120, in which a first sound image 36 is captured by a first sound sensor 18.1 and a second sound image 37 is captured by a second sound sensor 18.2. the first and second sound images 36, 37 are each an acoustic impression that arises at the position of the respective sound sensor 18.1, 18.2 as a function of the sounds 19 and the sound propagation paths 24 in the installation 10. In the second step 120, a component 12, namely the motor 14, is determined as the first sound source 56. For this purpose, triangulation, trilateration or bearing is carried out by means of the computer program product 45 which is executed on the evaluation unit 40 of the system 10. Overall, it is determined on the basis of the first and second sound images 37, 39 which sound 19 is emitted separately from the first sound source 56. Likewise, in the second step 120, a component 12, namely the stirrer 17, is determined as the second sound source 58. The determination of the second sound source 58 proceeds analogously to the first sound source 56. An emitted sound 19 is also determined with respect to the second sound source 58.Furthermore, the method 100 comprises a third step, in which the receiver position 26 in the virtual space 20 is predefined by the user 33. The recipient position 26 is specified by walking 27 on the virtual space 20 according to a representation by the virtual reality headset. In the third step 130, a virtual sound image 31 is determined, which corresponds to a location-dependent sound image 38 that is perceptible at the corresponding position 11 in the installation 10. To ascertain the virtual sound image 31, virtual objects 22 are adjusted as a first virtual sound source 28 and as a second virtual sound source 29 in the virtual space 20. Sound propagation paths 24 are calculated in the virtual space 20 and the virtual sound image 31 is thus determined at the receiver position 26. The virtual objects 22 are also virtual objects 22 which correspond to the components 12 and are at least partially sound-reflecting and / or at least partially sound-absorbing. In the virtual space 20, the corresponding at least partially sound-reflecting and / or at least partially sound-absorbing behavior is simulated therein in order to determine the sound propagation paths 24. The ascertainment of the virtual sound image 31 is likewise carried out by the computer program product 40, which is at least coupled to the digital twin 25.The method 100 further comprises a fourth step 140, in which the virtual sound image 31 is output to the user 33. In addition, an acoustic mode of the method 100 is changed by the user 33 via a selection 23 in the virtual space 20. The acoustic mode change 41 adjusts the output of the virtual sound image 31. A frequency range of the virtual sound image 31 is selected which lies in a frequency spectrum 52 that is unaudable for humans. As a result of the acoustic mode change 41, the portion of the virtual sound image 31 is shifted via a frequency shift 54 into a frequency spectrum 51 audible to the human. Based on the selected portion in the human-unaudable frequency spectrum 52 of the frequency shift 54 a frequency-shifted sound image 39 is determined and output to the user 33. The frequency shifted sound image 39 is an acoustic counterpart to a false color representation, such as in infrared imaging. As a result, the frequency spectra 52, which are not audible to humans and are present in the location-dependent sound image 38, can be analyzed directly by the user when the virtual space 20 is being walked on. The human-unaudable frequency spectra 52 are ultrasonic emissions and / or infrasonic emissions.The output of the virtual sound image 31 and the frequency-shifted sound image 39 to the user 33 is effected via the virtual reality headset 32. Accordingly, the virtual sound image 31 corresponds to the location-dependent sound image 38 that is perceptible at the position 11 in the installation 10 that corresponds to the predefined receiver position 26. This applies analogously to the frequency-shifted sound image 39 taking into account the frequency shift 54; the output virtual sound image 31 and / or the frequency-shifted sound image 39 recognizes a determination-different state of the installation 10 by the user 33. Similarly, in the fourth step 140, the virtual sound image 31 and / or the frequency-shifted sound image 39 are output 43 to an artificial intelligence 50, which at least interacts with the computer program product 45. The artificial intelligence 50 is designed as a neural network and is suitable for comparing the virtual sound image 31 and / or the frequency-shifted sound image 39 with sound images from a database 46, which is connected to the evaluation unit 40 via a communicative data connection 42. In order to identify the non-determination operating state, a target sound image from the database 46 that is associated with a region of the predefined receiver position 26 is compared with the virtual sound image 31 and / or the frequency-shifted sound image 39 by the artificial intelligence 50. If an adverse operating state of the installation 10 is detected, a warning 48 is output. The warning 48 is transmitted via a communicative data connection 42 between the evaluation unit 40 and the control unit 60 and thus enables a reaction by the control program 65. The method 100 allows the virtual space 20 in which the installation 10 is adjusted to be displayed remotely therefrom. The installation 10 can thus be identified by the method 100 according to FIG. 3 independently of location for the user 33. In addition, the analysis of the frequency-shifted sound image 39 provides an additional diagnostic possibility which cannot be carried out in a practicable manner when the installation 10 is being walked on in the presence.

Claims

Method (100) for detecting an out-of-determination operating state of a plant (10) comprising a plurality of components (12, 13, 14, 17), each configured as a sound source (56, 58) as an at least partially sound-reflecting object and / or as an at least partially sound-absorbing object, comprising the steps: a) providing the plant (10) in an active operating state and providing a virtual space (20) in which the plant (10) is at least partially adjusted; b) detecting a first and a second sound image (36, 37) by means of a first and a second sound sensor (18.1), respectively, and determining a sound (19) emitted by a first sound source (56); c) presetting a receiver position (26) within the virtual space (20) and determining a virtual sound image (31) at the receiver position (26); d) outputting the virtual sound image ( 31) to a user ( 33) and / or an artificial intelligence ( 50) and recognizing an incorrect-determination operating state on the basis of the virtual sound image ( 31) present at the receiver position ( 26).Method (100) according to Claim 1, characterized in that, in step b), the first sound source (56) is mapped as a first virtual sound source (28) in the virtual space (20).Method (100) according to Claim 1 or 2, characterized in that steps a) to c) are carried out for a second sound source (58), and in step d) the virtual sound image (31) is carried out on the basis of a combination of the first and second virtual sound sources (56, 58).Method (100) according to one of Claims 1 to 3, characterized in that the virtual sound image (31) is output on a virtual reality headset (32) on which the installation (10) is shown to be virtually walkable.Method (100) according to one of claims 1 to 4, characterised in that for a virtual sound source (28, 29) in a representation on the virtual reality headset (32), after selection by a user (33), an operating data set (62) of the corresponding sound source (56, 58) is visually output.Method (100) according to one of Claims 1 to 5, characterized in that the component (12, 13, 14, 17) of the plant (10) is a motor (14), a pump, a stirrer (17), a fan (13) or a mechanical application.Method (100) according to one of Claims 1 to 6, characterized in that the method (100) further comprises the steps: e) selecting a virtual sound source (28, 29) by a user (33); f) determining a sound-source-specific sound image (34) on the basis of at least the first and second sound images (36, 37); g) outputting the sound-source-specific sound image (34) with reduced sound gate.Method (100) according to one of Claims 1 to 7, characterized in that the sound-source-specific sound image (34) is provided to at least one virtual sound source (28, 29) of the artificial intelligence (50) for identifying an operation state of the corresponding sound source (56, 58) which is not subject to determination in the installation (10).Method (100) according to one of Claims 1 to 8, characterized in that the first and / or second sound sensor (18, 18.1, 18.2) is designed as a microphone, directional microphone or as a sound camera.Method (100) according to one of Claims 1 to 9, characterized in that an ultrasonic emission or an infrasonic emission with a shifted frequency position is output in the human-audible spectrum (51).Computer program product (45) configured to receive and process measurement data from a plurality of acoustic sensors (18, 18.1, 18.2) and to visually and acoustically readjust a virtual space (20), characterized in that the computer program product (45) is configured to perform a method (100) according to one of claims 1 to 10.Computer program product (45) according to claim 11, characterised in that the computer program product (45) is coupled to a digital twin (25) of a system (10) to be simulated or comprises the digital twin (25).Evaluation unit (40) which is designed to receive measurement signals from a plurality of acoustic sensors (18, 18.1, 18.2) and is designed via a data interface to receive operating data (62) of a plant (10) to be simulated, a computer program product (45) being stored on the evaluation unit (40) in an executable manner, characterized in that the computer program product (45) is designed according to Claim 11 or 12.Installation (10) which comprises a plurality of components (12, 13, 14, 17) for carrying out an installation process (15), which components are each designed as a sound source (56, 58), as an at least partially sound-reflecting object and / or as an at least partially sound-absorbing object, characterized in that the installation (10) comprises an evaluation unit (40) according to Claim 13.A system comprising a virtual reality system (30) and an evaluation unit (40), wherein the virtual reality system (30) has a virtual reality headset (32) and which is designed for computer-based auralizing of a sound image at a predeterminable receiver position (26) within a virtual space and which is designed for detecting an adverse determination state of a system (10) reset therewith, characterized in that the evaluation unit (40) is designed according to claim 13.

Images