Measurement by signal-to-signal translation

Abstract

System for operating a pump, comprising: a balance bike a motor that is coupled to the impeller and designed to generate a torque acting on the impeller, a housing with an outlet opening, designed to accommodate the impeller and a fluid, and an acoustic sensor coupled to the housing and designed to output a torque-indicating signal of the impeller based on sound and a forward mapping function, wherein the forward mapping function is learned during a system training phase in which torque data from a torque sensor are mapped pairwise forward of acoustic data from the acoustic sensor.

Description

Technical area

[0001] This disclosure relates generally to a system and a method for measuring using sound. background

[0002] Estimating various physical parameters (e.g., temperature, pressure, torque) of a dynamic system involves installing a physical sensor that transforms input physical signals into an output signal (often electrical) suitable for downstream tasks. Physics-based models can be used to represent the relationship between the physical parameter and the observed sensor signal. These physics-based models can be complex, are often partially treatable, and involve many mathematical parameters.

[0003] For the prior art, reference is made to CN 1 02 589 770 A, which discloses a system and a method for estimating the torque and speed of a motor. The system comprises a sound receiver for receiving a sound signal from the motor, a device for capturing a sound characteristic value of the sound signal, and a neural network module for receiving training data. Based on the training data, a motor model is created, which determines the motor torque and speed according to the sound signal. Summary

[0004] A pump system comprises an impeller, a motor coupled to the impeller and designed to generate a torque acting on the impeller, a housing with an outlet opening designed to accommodate the impeller and a fluid, and an acoustic sensor coupled to the housing and designed to output a torque-indicating signal based on sound and a forward mapping function, wherein the forward mapping function is learned during a system training phase in which torque data from a torque sensor are pairwise mapped forward of acoustic data from the acoustic sensor.

[0005] A method for measuring a physical parameter of a mechanical device comprises: obtaining paired measurements of target data from a target sensor and source data from a source sensor during a training period; generating a forward mapping function from the source data to the target data based on the paired measurements; and interacting with the mechanical device to monitor the physical parameter by means of feedback from the source sensor, which is mapped forward onto the target sensor by the forward mapping function, wherein the source sensor is an acoustic sensor designed to output a signal indicating the torque of an electrical machine of the mechanical device using a forward mapping function and an output signal indicating the acoustic sensor using a backward mapping function.where the reverse mapping function is learned during a system training phase in which torque data from a torque sensor are mapped pairwise in reverse to acoustic data from the acoustic sensor.

[0006] A system for measuring a first physical parameter comprises a mechanical device designed to operate on the basis of the first physical parameter and a source sensor coupled to the mechanical device and designed to output a signal indicating the first physical parameter, based on a second physical parameter and a forward mapping function, wherein the forward mapping function is learned during a system training phase in which first physical parameter data from a first physical parameter sensor are pairwise mapped forward to second physical parameter data from the source sensor, wherein the source sensor is an acoustic sensor designed to provide a signal indicating the torque of an electrical machine of the mechanical device using a forward mapping function and an output signal.which indicates the acoustic sensor, to output using a backward mapping function, wherein the backward mapping function is learned during a system training phase in which torque data from a torque sensor are mapped pairwise in reverse to acoustic data from the acoustic sensor. Brief description of the drawings

[0007] They show: Fig. 1. A flowchart of a machine monitoring system using dynamic spatiotemporal beamforming, Fig. 2 a flowchart for obtaining spatially dynamically beam-shaped energy images, Fig. 3 a block diagram of a system for obtaining spatially dynamic beam-shaped energy images, Fig. 4 a graphical representation of the movement of a robot platform designed to obtain spatially dynamic beam-shaped energy images over an area, Fig. 5A a graphical representation of an acoustic image of an area obtained by spatial-dynamic beam shaping of data acquired by a sensor system on a mobile robot platform, Fig. 5B a graphical representation of an acoustic image of an area obtained by spatial-dynamic beam shaping of data acquired by a sensor system on a mobile robot platform, Fig. 6. A flowchart of a self-calibration and self-diagnostic system using spatial dynamic beamforming to generate an environmental image. Fig. 7. A block diagram of a system for a virtual measurement system. Fig. 8 a graphical representation of the mathematical foundations of a virtual measurement system in relation to the physical system, Fig. 9 a graphical representation of signal-to-signal translations by mapping target sensor data to source sensor data, Fig. 10 Another graphical representation of signal-to-signal translations by mapping target sensor data to source sensor data, Fig. 11 a schematic diagram of a control system designed to control a vehicle, Fig. 12 a schematic diagram of a control system designed to control a manufacturing machine, Fig. 13 a schematic diagram of a control system designed to control an electric tool, Fig. 14 a schematic diagram of a control system designed to control an automated personal assistant, Fig. 15 a schematic diagram of a control system designed to control a monitoring system, and Fig. 16 a schematic diagram of a control system designed to control a medical imaging system. Detailed description

[0008] Detailed embodiments of the present invention are disclosed herein as required; however, it should be understood that the disclosed embodiments merely illustrate the invention, which can be implemented in various and alternative forms. The figures are not necessarily to scale, and some features may be exaggerated or minimized to show details of certain components. Therefore, specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis to teach those skilled in the art how to use the present invention in various ways.

[0009] The term "essentially" can be used here to describe disclosed or claimed embodiments. The term "essentially" can modify a value or relative property that is disclosed or claimed in the present disclosure. In such cases, "essentially" can mean that the value or relative property thereby modified is within ± 0%, 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, or 10% of the value or relative property.

[0010] The term sensor refers to a device that detects or measures a physical property and records, indicates, or otherwise reacts to it. The term sensor includes an optical, light, imaging, or photon sensor (for example, a charge-coupled device (CCD), a CMOS active pixel sensor (APS), an infrared (IR) sensor, or a CMOS sensor); an acoustic, sound, or vibration sensor (for example, a microphone, a geophone, or a hydrophone); a motor vehicle sensor (for example, a wheel speed, parking, radar, oxygen, blind spot, or torque sensor); a chemical sensor (for example, an ion-sensitive field-effect transistor (ISFET), an oxygen sensor, a carbon dioxide sensor, a chemical resistance sensor, or a holographic sensor); an electric current, electric potential, magnetic, or radio frequency sensor (for example, a Hall effect sensor, magnetometer, etc.).magnetoresistance sensor, Faraday bowl, galvanometer), an environmental, weather, humidity or moisture sensor (e.g., weather radar, actinometer), a flow or fluid velocity sensor (e.g., mass airflow sensor or anemometer), a sensor for ionizing radiation or subatomic particles (e.g., ionization chamber, Geiger counter or neutron detector), a navigation sensor (e.g., a global positioning system (GPS) sensor or a magnetohydrodynamic (MHD) sensor), a position, angle, displacement, distance, velocity or acceleration sensor (e.g., LIDAR, accelerometer, ultra-wideband radar or piezoelectric sensor), a force, density or level sensor (e.g., strain gauge or nuclear density measuring device), a thermal, heat or temperature sensor (e.g., infrared thermometer, pyrometer, thermocouple,Thermistor, microwave radiation meter) or any other device, module, machine or subsystem whose purpose is to detect or measure a physical property and to record, indicate or otherwise respond to it.

[0011] The term image refers to a representation or artifact that depicts a perception (for example, a visual perception from a viewpoint), such as a photograph or other two-dimensional image representation that resembles an object (for example, a physical object, a scene, or a property) and accordingly provides a representation of it. An image can be multidimensional in that it may have temporal, spatial, intensity, or concentration components, or components of some other feature. For example, an image can comprise a time-series image.

[0012] Systems and methods for generating high-resolution energy maps of a given space using a beam shaper mounted on a mobile platform are disclosed. The energy maps can specify the signal intensity, for example, the power as a function of spatial dimensions. These methods can be used to obtain information from a variety of energy sources, such as acoustic or electromagnetic energy sources, but they are mainly discussed in the acoustic domain. The system can include a receiver array mounted on a mobile platform coupled with a variety of beam shaping algorithms, thereby recording spatiotemporal information as it moves through space.The methods disclosed herein utilize the spatiotemporal information coupled with the beam-shaped information obtained at each location to generate a high-resolution image of space. This image can be superimposed on visual information or used to monitor changes in acoustic or electromagnetic energy over time. Systems and methods for performing self-calibration tests to separate self-generated background noise during spatial-dynamic beam-shaping operations are also disclosed. These methods can be used to identify intrinsic noise associated with the recording platform, such as sensor hardware, moving hardware, such as robotic platforms, or other background noise.By using these methods, the output of spatial dynamic beamforming algorithms can be improved by separating background noise from foreground signals, which are signals associated with objects or regions of interest. The background intrinsic noise can then be used to perform intrinsic functionality diagnostics by monitoring changes in intrinsic noise over time.

[0013] The ability to accurately monitor various energy sources within a given environment is becoming increasingly important due to the growing focus on human health and safety in complex industrial settings. In particular, monitoring acoustic energy is crucial from both human health and industrial perspectives. For humans, monitoring acoustics is essential to ensure a safe working environment and prevent hearing loss. From an industrial perspective, acoustics can provide valuable insights into the functionality of machinery or equipment that visual monitoring cannot. However, acoustic monitoring of environments, spaces, and processes can be challenging for several reasons.First, sound emitted by a given source tends to be reflected by the surfaces in its vicinity, which can cause echoed versions of that signal to arrive at a receiver, such as a microphone, antenna, etc. Second, when multiple sources are involved, the signals from these sources overlap, making it difficult for a receiver to determine which source emitted a particular sound.

[0014] To address these problems, a variety of beamshaping algorithms have been developed. Beamshaping allows a receiver array to improve signal quality by estimating the direction of arrival (DOA) of energy sources by comparing the signals recorded by each receiver. Broadly speaking, beamshaping algorithms are often divided into three categories. The simplest category consists of maximizing the controlled response power (SRP) of the received signals, and examples of these beamshapers include delay-and-sum, filter-and-sum, and maximum-probability estimation beamshapers. The second category comprises approaches using time-of-arrival-time-difference (TDOA) estimators, which consider the arrival time of a signal at each receiver.Essentially, if a signal arrived at receiver "A" earlier than at receiver "B," it can be estimated that the signal originating closer to receiver "A" will eliminate echoes or other forms of interference. Given multiple receivers, TDOA estimators can both point in a particular direction from which a signal is assumed to have originated and filter out background noise to reconstruct the original signal of interest. The third category includes localizers based on spectral estimation, including the multi-signal classification (MUSIC) algorithm used in many state-of-the-art devices. However, regardless of the chosen beamforming algorithm, the performance of the selected beamformer is highly dependent on the geometry of the receiver array.

[0015] For any given receiver array, the limitations of its beamshaping capabilities are determined by the spacing between the receivers. As prescribed by the Nyquist-Shannon sampling theorem, the minimum spacing d min between receivers for beam shaping of a signal with a wavelength λ by d min = λ / 2 is given. For a given wave speed c in the propagation medium (for example, the speed of sound), the maximum frequency that can be beam-shaped by a receiver array at a receiver spacing d is therefore given by f. maxGiven a frequency of c / 2d, aliasing occurs above this frequency, making it impossible to determine the direction from which a signal was emitted. Therefore, beamforming high-frequency signals requires arrays with closely spaced receivers. However, if low-frequency signals are also important, a closely spaced array designed for high-frequency applications appears small, and the phase difference between receivers is minimal, resulting in a large beam. This essentially prevents beamforming from improving signal quality and accurately identifying the DOA (Dead End Location). Furthermore, in most beamforming applications, signals cannot be assumed to be narrowband. Because signals are often broadband, evenly spaced receivers result in a spectral shift of the beamformed signal.These facts have led to significant academic and industrial research efforts regarding optimal receiver array geometries and beamforming algorithms for the uniform acquisition of information over a broadband frequency range. A primary limitation of these approaches, and a major reason for such efforts, is that receiver array hardware is typically spatially static, such as an antenna array mounted at the top of a mast. For sources located far from an array, this can result in the appearance of two separate sources originating from the same location. For sources located near an array, the far-field assumption breaks down (incident waves cannot be modeled as plane waves), and source localization becomes challenging.

[0016] Synthetic aperture beamforming is a technique that typically uses a 1D receiver array to generate high-resolution 2D images by stacking many high-resolution 1D measurements acquired at different locations. In practice, this is typically done by installing a 1D array on, for example, an aircraft or a boat moving along a known path and reconstructing images of, for example, islands or the ocean floor using sonar or radar. These techniques are practically unsuitable for applications such as monitoring a factory or other enclosed environments, frequently monitoring changes in an environment over time, or providing information localized in three dimensions.

[0017] Spatial dynamic beamforming (SB), also known as spatial dynamic beamforming, described here, involves observing energy sources from multiple perspectives and using beamforming algorithms to combine this information to obtain a complete picture of the energy mapping of a given space. SB methods can be used to obtain information about a variety of energies, such as electromagnetic energies; however, they are discussed in this disclosure from an acoustic perspective. It should by no means be inferred, however, that these methods are only applicable to acoustic SB.

[0018] The field of foreground-background separation, often simply called "background subtraction," involves separating the foreground and background of a signal and is highly relevant to background analysis. Most of the work in this area has come from computer vision and video surveillance applications. The foreground contains objects of interest, such as people or vehicles moving around a frame. The background contains static or pseudo-static objects, such as trees, buildings, or roads. By estimating the signals that make up the background, the foreground can be extracted from the overall signal, allowing subsequent tasks to be performed with higher quality, such as tracking the movement of a person in a video.At a high level, the fundamental assumption behind background subtraction is that the background is relatively stationary, meaning that in a given temporal frame sequence of a scene, the background objects are the same in all frames. By observing many frames and identifying the objects that do not change or change very little, the background can be identified and thus subtracted from the overall signal to reveal the foreground objects. Several methods have been developed to solve this problem, such as kernel density estimation (KDE), Gaussian mixing models (GMMs), hidden Markov models (HMMs), various subspace approximation and learning techniques, and various other machine learning techniques in supervised, semi-supervised, and unsupervised learning, such as support vector machines (SVMs) and deep learning, for example, convolutional neural networks (CNNs).Background subtraction techniques have also been used for acoustic background subtraction. However, applications of this technology have been limited to stationary acoustics, where the recording platform does not change its spatial location. Furthermore, work in this area has primarily focused on identifying various background noises related to the environment, such as vehicle traffic, wind, HVAC systems, or other signals that can affect technologies like speech recognition devices. Little work has been done to isolate noises related to the recording platform, particularly in the emerging field of acoustic devices mounted on mobile robot platforms.

[0019] Because spatial dynamic beamforming typically involves some kind of moving platform, such as a drone, robotic platform, robotic arm, etc., some kind of intrinsic noise is typically associated with the signals captured by beamforming. Various forms of these intrinsic noise sources can include motors, servos, wheels, mechanical belts, fans, propellers, vents, beams, joints, gears, electronic devices, and other mechanical devices. Often, these processes generate structured noise, or noise containing known information about the signal. This knowledge may include factors such as the start and end times of the noise, but it can also include more complex factors such as frequency content, statistics, or other mathematical quantities.In many cases, this structured noise is consistent across frames, so it can be considered background. When performing spatial dynamic beamforming, this background noise is undesirable because it can alter measurements that describe the total energy of different parts of a given space. Given a method for separating this background from the rest of the signal, spatial dynamic beamforming techniques could be improved by including only noise from areas or objects of interest in the calculations. Furthermore, by isolating the background signals that are exclusively associated with the motion platform, the background signal provides insight into the platform's operations themselves.By monitoring changes in this background signal over time and comparing them with the platform's various activities, such as different movement types, loads, runtimes, etc., an overall picture of the platform's "functionality" can be understood.

[0020] Herein, systems and methods for generating high-resolution spatiotemporal images of energy for a region or space of interest are described by the novel combination, referred to here as spatiotemporal beamforming and also as spatiotemporal beamforming, of space-sensitive, mobile beamforming receivers. A receiver or receiver array, in conjunction with beamforming algorithms, is installed on a mobile platform that records information at various locations and uses this spatially distributed information to reconstruct a coherent model of the measured space. It should be noted that these methods can be used to obtain information about a variety of energy sources, including acoustic and electromagnetic emitters. However, for the sake of simplicity, this disclosure may be discussed primarily from an acoustic perspective.Systems and methods for isolating noise associated with a robotic or mobile platform and recording devices used in spatial dynamic beamforming to improve the output of spatial dynamic beamforming algorithms and also to provide insights into the operation of the platforms themselves are also disclosed. It should be noted that these methods can be used to obtain information about a variety of energy sources, including acoustic and electromagnetic emitters. However, for the sake of simplicity, this disclosure can be discussed primarily from an acoustic perspective.

[0021] According to a general aspect, a mobile receiver or a mobile receiver array with spatial sensitivity is disclosed. The system has some form of locomotion unit, including, but not limited to, a wheeled robotic platform, a rail, a beam, a propeller, air, a drone, or a robotic arm, by which the receiver array is moved. The mobile part of this system has some form of measurement and recording system telemetry. The telemetry part of this system may have a device attached to the mobile platform that provides this information, for example, optical imaging, radar, LiDAR, etc., or it may have a system not attached to the mobile platform, such as a motion detection or spatial localization and imaging (SLAM) system.The receiver array can consist of a single receiver or a set of many receivers organized in a variety of geometric 1D, 2D, or 3D configurations. The system can also include onboard computer hardware and software that operates independently of, or in conjunction with, separate computing hardware and software. Furthermore, the overall system can consist of multiple individual mobile platforms, each with its own set of recording devices and localization capabilities, all of which communicate with each other and / or with a master system.

[0022] According to another general aspect, a method for obtaining acoustic images of a given space is disclosed by combining beamformed information of the space from several different perspectives into a coherent image. This method involves applying a beamforming algorithm to data received by a receiver or receiver array at a specific location, pointing in a particular direction, to determine the acoustic output (AO) of areas of interest (ROI) or objects of interest (OOI) within the field of view (FOV) of the one or more receivers. The AO may consist of sound pressure level (SPL), frequency spectra, time-series signals, or other recording methods. The AO of each ROI / OOI is logged, and the acoustic array is then repositioned at a new location.This new location can be achieved by shifting the receiver array itself, for example by yaw, pitch, or roll, or by completely rearranging the array in three-dimensional space. At the new location, the same AO recordings are made for all ROI / OOI within the FOV, including all previously recorded ROI / OOI within the FOV or new ROI / OOI that were not present in any previous FOV. For ROI / OOI that were measured previously, the new AO is stored in a database along with previous measurements obtained from another location. This process of recording and repositioning is repeated indefinitely until a complete mapping of the mapped space is obtained using the algorithms described here.

[0023] According to another general aspect, systems and methods for performing self-calibration, wherein noises associated with the recording instrument or recording platform are isolated from the noises of interest.

[0024] According to another general aspect, noises associated with the recording instrument or recording platform are used to perform self-operation monitoring.

[0025] Certain aspects are now described in detail to provide a complete understanding of the class of devices, principles of use, design, manufacture, and associated methods, algorithms, and results disclosed herein. One or more examples of these aspects are illustrated in various non-exhaustive embodiments in the accompanying drawings. Persons skilled in the art will understand that the methods and devices described in this disclosure and the accompanying drawings are non-limiting examples and that the scope of protection of this disclosure is defined solely by the claims.

[0026] In many industrial, commercial, or consumer applications, monitoring environmental energy is crucial for human health, machine operation, the functionality and integrity of infrastructure, and the preservation of many other assets. This energy can include, but is not limited to, acoustic energy (including audible sound, ultrasound, and infrasound), visible light and the entire electromagnetic spectrum, nuclear energy, chemical energy, thermal energy, mechanical energy, and even gravitational energy. A wide variety of sensors and methods have been developed for monitoring these different forms of energy. These sensors are mostly used statically, meaning they are placed in a specific location and monitor their surroundings from that point of view, for example, in the manner of a security camera, microphone, or thermal imaging camera.However, these locations contain noise sources that interfere with the recordings of various ROI / OOI within the sensor's FOV. Several algorithms have been developed to compensate for these disturbances, such as beamforming, but this cannot solve two problems: (1) Not all noise sources can be isolated and removed from the ground truth signal of interest, and (2) not all signals of interest can be captured from a given location. Herein, methods and systems are described that aim to solve these problems through the novel combination of a mobile, location-sensitive platform with sensor technology. These methods and systems are primarily described from an acoustic perspective. However, operation in the acoustic domain should not be considered a limiting embodiment of this disclosure.

[0027] The Fig. Figures 11-16 show a set of exemplary embodiments of space-sensitive robot platforms that can be used to spatially modulate sensor hardware used for data acquisition. According to one embodiment, the robot platform is a wheeled platform that has a sensor array system mounted directly on the chassis. The platform can also be used to hold other loads. According to another embodiment, the same wheeled robot platform is used, but a specialized mounting system is employed to position the sensor array statically or dynamically at a specific location or set of locations. This robot platform can also induce locomotion by other means such as rails, ball bearings, rollers, or other means.According to another embodiment, the robotic platform is in the form of a drone that uses air propellers to induce locomotion. The sensor array system can be placed directly on the drone chassis or on a specialized arm that positions the sensors as needed. According to another embodiment, the robotic platform is in the form of a drone or flying object that uses another form of locomotion, such as jet propulsion or antigravity. Again, the sensor array system can be positioned as needed. According to yet another embodiment, the robotic platform is in the form of a robotic arm that provides multiple degrees of freedom for sensor positioning. This robotic arm can potentially be integrated with another robotic platform to enhance locomotion capabilities.According to another embodiment, the robot platform can take the form of a droid-like robot that can move around space similarly to a person. The sensor array system could be mounted on the body of the droid robot or in some other way. According to yet another embodiment in the... Fig. In the embodiment not shown in Figures 11-16, the robot platform may not be of a robotic nature, but rather a positioning device that is positioned and repositioned by hand by a person.

[0028] Depending on various aspects, the robot platform can be designed to localize itself in space relative to other objects around it or to an initialized location or set of locations in order to provide telemetry information. This telemetry information could include distances to the ROI or OOI, precise spatial coordinates for a given set of reference coordinates, travel distances, pitch, roll, and roll angles, or other localization means. The telemetry component of this system may involve a device mounted on the mobile platform that provides this information, such as optical imaging, radar, LiDAR, RF, etc., or it may involve a system not mounted on the mobile platform, such as a motion detection or spatial localization and mapping (SLAM) system.

[0029] Fig. Figure 1 is a flowchart of a machine monitoring system using dynamic spatiotemporal beamforming 100. In block 102, a control unit performs dynamic spatiotemporal beamforming, also known as spatiotemporal beamforming. Next, in step 104, the control unit performs a self-calibration. This self-calibration can be performed at regular or irregular intervals, for example, each time the system is powered on, at a predetermined time interval, after detecting a change in a physical measurement (e.g., temperature, pressure, detection of airborne particles such as oil droplets or smoke), or after detecting an anomaly in step 102. In step 106, the control unit performs a self-operational monitoring check in step 104.Similar to step 104, the self-calibration can be performed at regular or irregular intervals, for example, each time the system is powered on, at a predetermined time interval, after the detection of a change in a physical measurement (e.g., temperature, pressure, detection of airborne particles such as oil droplets or smoke), or after the detection of an anomaly in step 102. In step 108, the control unit performs a region of interest (ROI) in step 104. This self-calibration can be performed at regular or irregular intervals, for example, each time the system is powered on, at a predetermined time interval, after the detection of a change in a physical measurement (e.g., temperature, pressure, detection of airborne particles such as oil droplets or smoke), or after the detection of an anomaly in step 102.Because the system monitors an area, special attention can be paid to ROIs, and steps 102, 104, 108, and 106 can be used or combined to perform machine performance monitoring. This includes virtual measurement 110, production line monitoring 112, vehicle monitoring 114, or other machine performance monitoring 116. This information can be used in a database where a control unit (either separate or the control unit from step 102) performs knowledge-based decision analysis.

[0030] Fig. Figure 3 is a block diagram describing the hardware 300 in the system according to at least one aspect of the present disclosure. A processor 302 is powered by a power source 304, which may be, but is not limited to, a variety of power sources such as batteries, wired power supplies (for example, an AC power outlet), or solar panels. The processor 302 may be a single central processing unit (CPU) consisting of one or more cores, a set of CPUs, a graphics processing unit (GPU), a set of GPUs, a combination of CPUs and GPUs, or a number of other computer hardware platforms and devices. The processor 302 interacts with communication hardware 306, which communicates with external devices and systems 318. Some examples of communication protocols include Bluetooth, WiFi, RFID, and others.The external devices 318 can be other robot platforms, infrastructure, centralized control computers, and others. The processor 302 also interacts with sensors 308, which can include sensors used for recording environmental data for spatial dynamic beamforming, such as microphones or electromagnetic sensors. The sensors 308 can also include other sensor types used for localization, telemetry, and motion, such as cameras, ultrasonic detectors, LiDAR, radar, and other sensor modalities. The sensors 308 and the processor 302 interact with the telemetry system 310 and the motion system 312, which provide variations of position-sensitive locomotion for the robot platform.The processor 302 and the motion system 312 store information in memory 314, which can be in the form of, for example, random access memory (RAM), hard disk drive memory, semiconductor memory, or other memory types. A variety of data can be stored in memory 314. Embodiments of the hardware described by at least one aspect of the present disclosure can also include visualization hardware 316 and accompanying software to enable the robot platform to provide users or other robots with visual information indicating a variety of information, such as robot status, status of other hardware, recording mode, location information, or reconstructed spatial-dynamic images, and other information.Exemplary embodiments of visualization hardware 316 include, but are not limited to, LCD / LED screens, LEDs, digital displays and analog devices.

[0031] Fig. Figure 2 is a block diagram of a high-level method 200 for obtaining spatially dynamic beam-shaped energy images according to at least one aspect of the present disclosure. During the initialization step 202, certain parameters can be selected for a specific environment of interest. In particular, aspects such as ROIs, OOIs, spatial resolution, recording parameters (e.g., sampling frequency and duration), recording locations, etc., are selected, which can vary depending on the environment of interest.An exemplary embodiment incorporating several of these aspects can be selected for a specific type of environment, such as a production line comprising a variety of robots, machines, and conveyor belts. Other mechanical devices can have ROI / OOI that is specific to each mechanical device, certain parts of each mechanical device, parts or all parts of manufacturing processes, areas occupied by people, products of such a production line, and other areas of interest. According to this embodiment, the sampling frequency and duration selected for sensor recording can be chosen to monitor specific signals (i.e., narrowband signals) or a broad signal range (broadband) with varying spatial frequencies (e.g., continuous versus transient). Other embodiments of these parameters can vary drastically depending on the environment of interest.It is important that the recording locations can be selected during or even before this initialization period. These locations can be chosen to represent a specific pattern or randomly. Furthermore, according to one embodiment, these locations can be selected ad infinitum throughout the entire duration of the recording, which can be practically infinite, and the system can choose these locations either randomly or according to a principle or set of principles. After initialization, in step 204, the robot platform positions itself (or is positioned) at an initial location m. At this location, in step 206, data from the various ROIs / OOIs selected during the initialization stage q are recorded from location m.According to another embodiment, these various ROIs / OOIs are not selected during initialization, but rather during the recording process as new ROIs / OOIs are discovered. In step 208, the control unit checks whether the entire recording in step 206 is complete. If not all ROIs of interest have been covered, the control unit branches to step 210 and continues until data from all ROIs / OOIs has been recorded. Upon completion, the robot platform is repositioned to a new location in step 214. This pattern continues in steps 204–214 until all ROIs / OOIs have been mapped from all locations. At this point, the mathematical spatial-dynamic beamforming procedures are applied in step 216, and the environment mapping is returned in step 218.According to another embodiment briefly described above, the pattern of movement as in step 204 and the recording as in step 206 is continuously repeated over different locations, and the algorithm is performed in . Fig. 2 continued without many advantages. According to this embodiment, the environment image from step 218 is returned by spatial-dynamic beam shaping in step 218, whereby it can be returned at any time and continuously updated.

[0032] The following is a mathematical formulation of a non-restrictive embodiment of the spatial-dynamic beamforming algorithm used in conjunction with other methods and systems in this disclosure for reconstructing the acoustic images of the environment of interest. For a given microphone array with n ∈ [0,...,N] microphones, the frequency-domain beamformed output of the array at the spatial recording location m ∈ M for the spatial location of interest (ROI) q ∈ Q is given by Ym(ω,q)≡∑n=0NGn,m(ω)Xn,m(ω)ejωΔn defined where X n,m (ω) and G n,m (ω) are the frequency-domain signals of each microphone at each location or at the filter associated with each microphone. M and Q represent the sets of spatial locations in the three-dimensional space of the recording locations and the locations of interest, respectively. Here, it is initially assumed that the filters can change with location; however, it is possible that Gn,m (ω) = G n,k (ω) ∀ m,k ∈ M holds, for example in filter-and-sum beam shaping. Element-wise phasing is given by e jωΔn applied and is specific for a given q. The Y m The equation defining (ω, q) therefore represents a spatial beamforming operation for a microphone array. Then Ym(ω)≡Ym(ω,q)∀q∈Q It is defined as a matrix representation of the acoustic mapping of all spatial locations, recorded based on the locations in M. At a given spatial recording location, the signal captured for each spatial ROI does not perfectly represent the pure signal emitted by each spatial ROI via an acoustic mapping, even with the very best beamforming operation; that is, each signal can be associated with a certain amount of noise and distortion. This noise K m (ω) and this distortion A m(ω) can be due to a combination of signals from other sources, reflections, distortions, sensor noise, etc. Therefore, the acoustic imaging is also affected by Ym(ω)=Am(ω)⋅Y^(ω)+Km(ω) given, where Ŷ(ω) is the acoustic ground truth -

[0033] The figure represents all ROIs regardless of the recording location. Our goal is to find Ŷ(ω) \ because it contains the true signals emitted by each source / ROI unaltered by other sources, reflections, absorption due to attenuation and scattering, etc. It is important to assume that and . Km(ω)≠Kj(ω)∀m,j∈M,m≠j Am(ω)≠Aj(ω)∀m,j∈M,m≠j

[0034] It is also assumed that through A m(ω) Distortions caused primarily by sensor problems, such as lens scratches, damaged microphones, etc., are due to these factors and do not irreparably distort the overall signal. Under this assumption, A m (ω) ≈ I, where I is the identity matrix.

[0035] Therefore, the following apply ∵lim|M|→∞1|M|∑m∈MKm(ω)=0 Y^(ω)=lim|M|→∞1|M|∑m∈M(Am(ω)⋅Y^(ω)+Km(ω)) Y^(ω)=lim|M|→∞1|M|∑m∈MYm(ω)

[0036] Because the acoustic imaging noise K m (ω) depends on location; the average acoustic mapping from these locations approaches Ŷ(ω) if the acoustic mapping Y m (ω) is recorded from a sufficient number of locations.

[0037] Fig. Figure 4 is a representation of an exemplary embodiment of the movement of a robot platform through an area, a space, or an environment according to at least one aspect of the present disclosure. The robot system 400 in Fig. 4 most closely resembles a flying drone-like robot platform; however, it is understood that any embodiment of the robot platform, including those in the Fig. 11-16 or other embodiments, including those that have a movement 416 in any direction and all directions or a subset of all possible directions, and can, for example, move with three degrees of freedom on the floor of such an environment or enclosure. The robot 402 moves around in the space or environment 404 and collects data 408. The area, space, or environment 404 can be an enclosed space of any shape and size or an open or partially open environment, also of any shape and size. Data 408 can be acquired by a beamforming algorithm 406 that acquires data over a strip 414 which may comprise a particular segment or portion of the space 404, as in Fig. 4 is represented, or, for example, segments of a cone or a sphere can be recorded. Data 408 can be processed by spatial-dynamic beamforming to create an energy mapping of space 404. Furthermore, various algorithms in the form of classical signal processing techniques or modern data-driven approaches in the field of machine learning and / or deep learning, or others, can be applied to data 408 to compute various features 410 and output data 412 that indicate certain states of the environment 404. This data 412 can indicate conditions such as machine functionality, the health of the space, pedestrian traffic, and other features. Space 404 can therefore contain many different components or processes of interest that can be analyzed by means of features 410 and output data 412.

[0038] Fig. 5A is an acoustic imaging system 500 of an environment obtained by spatial-dynamic beamforming of data acquired by a sensor system on a mobile robot platform, according to at least one aspect of the present disclosure. According to the present embodiment, a 3D version of an area 502 is in the manner of a room; however, the room 502 can be represented by any number of other dimensions, for example, 1, 2, 3, 4, or more dimensions. The data 504 and 506 represent the energy levels defined by the scale. According to the present embodiment, the scale ranges from 0 to 1 and represents the intensity. However, this scale could also indicate other features such as the frequency content or object types. The data 506 and 508 are recorded together with 502, so that this data represents what has happened at that particular location in space and time.In this example, data 506 indicates that a region has low energy, and data 504 indicates that a region has high energy. According to this embodiment, space 502 is an empty space. However, space 502 in this figure can also contain other objects of interest.

[0039] Fig. 5B is an acoustic imaging system 550 of an environment obtained by spatial-dynamic beamforming of data acquired by a sensor system on a mobile robot platform, according to at least one aspect of the present disclosure. According to the present embodiment, a 3D version of space 552 is represented; however, space 552 can be represented by any number of other dimensions, for example, 1, 2, 3, 4, or more. Data 554 and 556 represent the energy levels defined by the scale. According to the present embodiment, the scale ranges from 0 to 1 and represents intensity. However, this scale could also indicate other features such as frequency content or object types. Data 556 and 558 are recorded together with 552, so that these data represent what has happened at that particular location in space and time.In this example, data 556 indicates that a region has low energy, and data 554 indicates that a region has high energy. According to this embodiment, space 552 is an empty space. However, space 552 in this figure can also contain other objects of interest.

[0040] An important aspect of the processes typically associated with sound leveling (SB) is a mobile platform of some kind that moves recording devices around the area of ​​interest. The movement associated with these platforms is typically accompanied by noise that can be recorded by the SB recording devices, thus altering the recorded measurements. This disclosure discloses methods for removing this noise from the SB measurements. When this noise has been correctly isolated and removed, it represents exclusively noise associated with the mobile platform. Methods for then monitoring the condition and functionality of the mobile platform with this given noise isolation are also disclosed here.

[0041] Fig. Figure 6 shows a high-level block diagram of various embodiments of the implementation of self-calibration and self-diagnostic algorithms in an implementation of a spatial-dynamic beamforming algorithm 600 according to at least one aspect of the present disclosure. Without self-calibration and self-diagnostic algorithms, the SB system would initialize its environment in step 602, move to location m (which may be its starting location) in step 604, record data in step 606, check in step 608 whether all regions of interest (ROI) have been recorded, and increment a counter in step 610. Then, in step 612, it would check whether all locations have been recorded and move to the next location 614 until all locations have been recorded. After completion of the recording, the SB is typically applied in step 616 to return the environment map in step 618.According to one possible embodiment of the methods in this disclosure, self-calibration and self-diagnostic algorithms (the “calibration”) can be applied at algorithm injection point 1, which is step 620, so that data is analyzed after each region of interest (ROI) or object of interest (OOI) q has been acquired. According to another possible embodiment, the calibration can be applied at algorithm injection point 2 in step 622 after all ROIs for a given recording location have been acquired. According to yet another possible embodiment, the calibration can be applied at algorithm injection point 3 in step 624 after all ROIs at all locations have been acquired. These three algorithm injection points offer several advantages. At algorithm injection point 1, the calibration is applied only to a small amount of data.Although this may lead to some loss of information or overfitting, it is likely the fastest algorithm injection point. According to one possible SB implementation, various beamforming techniques, such as delay-and-summing beamforming, can be applied in step 606 to acquire an aggregate signal from a ROI / OOI. It should be noted that, according to this embodiment, the calibration can be performed before or after the beamforming techniques used to acquire data q at a given ROI. At algorithm injection point 2, the calibration is applied to all ROIs from a particular recording location. This possible injection point has the advantage of providing more information about the noise associated with that particular location than injection point 1; however, it is also likely to be computationally more complex than injection point 2.The advantage of injection point 3 is that the calibration can be applied to all data across the entire recording area, i.e., all ROI / OOI from all recording locations. This injection point is likely the most computationally complex due to the data volume overhead, but it can also provide the most accurate calibration. According to yet another possible embodiment, the calibration can be applied over hours, days, weeks, years, etc. In this embodiment, the calibration can be applied periodically over time to understand different states of change in the system implementing the calibration and its environment.

[0042] The following is a mathematical formulation of several non-restrictive embodiments of the self-calibration algorithms used in connection with spatial-dynamic beamforming for obtaining accurate acoustic images of different environments and for performing a self-functionality diagnosis for the hardware used to obtain this information. Given a recording p→∈ℝk×1, so that k represents the number of samples in the recording, a function f" is defined such that fm:p→→P,P∈ℝm×n a mapping from the vector form p→ into the matrix P, where k = mn. The selection of m depends on the sampling frequency of the recording and the feature size of the foreground and background noise of interest. The value of m can be chosen as any value, but according to the preferred embodiment, it is chosen such that m ≅ n. The singular value decomposition (SVD) is then applied to P such that P=U∑VT applies, whereby ∑=[S000], S=diag(σ1,…,σn)∈ℝr×r,σ1≥…≥σr>0,U∈ℝm×m R m×m is and V ∈ ℝ n×nThe basic assumption of this algorithm is that the inherent noise associated with the recording platform, or "background noise," is relatively consistent at any given time. Then, the function shown in Figure ff is applied, and a sample of this background noise is fed into each column of P, such that most of the information stored in P is low-rank or even rank -1. A low-rank version of the signal contains background noise, while a high-rank version contains foreground data, a foreground signal, or a signal of interest. When used for self-diagnosis, the high-rank version of the signal contains foreground noise, while the low-rank version contains background data.Then the background noise is reconstructed based on the original signal, and the foreground parts of the signal are processed by a rank-r approximation of P. PBG=Ur∑r∑rVrT PFG=P−PBG isolated, where P BG and P FG The background and foreground parts of the signal are represented. The value of r is typically chosen to be small and, according to its preferred embodiment, is typically between 1 and 5. However, the optimal value of r can be determined by ropt=argminr‖P^−PFG‖F calculated where P is the ground truth foreground signal and || · ||F is the Frobenius norm. r opt = argmin ||P̂ - P̂ FG || F Of course, it cannot be calculated a priori without knowledge of the foreground signal, but it can be used, for example, for calibration and experimental testing in different environments.

[0043] In cases where the background noise is unstructured, it may be desirable to convert the data into a spectral range before performing SVD separation. PFG=IFFT[S−SBG]

[0044] An alternative implementation of spectral range-based separation involves converting the data into a spectrogram and using a series of spectrogram "images" as the different frames. These frames are then transformed into f by a similar mapping function. M vectorized, and then SVD separation is applied.

[0045] Yet another embodiment, which works particularly well when the background noise is highly structured, involves using the cross-correlation function to align the vectors in P, then truncating the ends of P to prevent zero stuffing, and then performing SVD background separation. Finally, using this known background, cross-correlation is again employed to align the estimated background, thereby extracting the foreground.

[0046] In yet another embodiment, the so-called "displacement matrix" is estimated, which essentially defines the phase variation between vectorized signals in the columns of P. Essentially, this displacement matrix is ​​estimated, a displacement cancellation operation is applied, and then our SVD separation algorithm is applied. Depending on how the displacement matrix is ​​calculated, this can ultimately be mathematically identical to the cross-correlation algorithm.

[0047] The next part we want to describe is how we incorporate the system's state knowledge into this calculation. We want to ensure that we only compare similar states, as the robot, for example, will likely sound different when moving in different ways or performing different tasks. We might also want to consider the case where we only need to worry about isolating the sounds when the robot is simply standing still rather than moving, which is probably the simplest scenario anyway.

[0048] Finally, given a good estimate of P, we can BGWe use this as a metric for understanding the state of the mobile system used with SB. We define n ∈ [0, N], which represents the total number of measurements obtained, i.e., the number of times the intrinsic noise of the mobile robot platform is calculated. The value of N can vary for a number of reasons, such as how frequently acoustic mappings of a given space are calculated, how the robot platform operates, or how the intrinsic noise compares to that of other robot platforms operating in the same general area. At a very high level, we can essentially use a distance measure between what we measured previously and what we are measuring now, and a predefined or possibly dynamic threshold to determine whether there is a problem.The distance measure we use could resemble the Frobenius norm or the KL divergence, but it depends heavily on how we mathematically represent this information. The predefined or dynamic distance threshold would likely need to depend on the robot system and its environment. We also want to discuss the addition of various pieces of information, such as environmental conditions or a load on the platform.

[0049] Details regarding the determination of the area of ​​interest (ROI) are set out here, as described in Block 108 of Fig. Figure 1 illustrates this. Below are ROI algorithms that provide an example of what this area of ​​interest (ROI) mapping might look like. It essentially appears as a set of square areas of varying sizes, each exhibiting different levels of signal output. There are numerous ways this could be displayed: total SPL, for example, in dBA levels, which could represent some metadata for a data structure containing all records for that area, or other information.

[0050] The set of algorithms (for example, algorithms 1-4 shown below) defines a procedure for dynamically assigning resolution levels over a given space for spatiotemporal ray shaping. The purpose of these algorithms is to use higher-resolution imaging for areas of interest and lower-resolution imaging for quiet areas. The high-level algorithm that implements the dynamically generated mapping of spatial resolution over multiple recording locations by spatiotemporal ray shaping is shown in algorithm 1. In this algorithm, the input is a list of recording locations, and the output is the resulting information obtained from these locations.

[0051] In Algorithm 1, the initial spatial resolution used to define the size S of an individual area of ​​interest (ROI) is set to an arbitrary unit. According to this embodiment, S ← 1. However, S should be set to a suitable value for a given space, given size constraints, the acoustic profile of the object of interest, and the physical capabilities of the recording device. The spatial index L is set to 0, and the data structure used for indexing is initialized as an empty, arbitrary data structure that may depend on both hardware and software requirements.

[0052] The aforementioned MOVE algorithm represents the command and subsequent actions undertaken by any robot platform to move to a new location. The MOVE algorithm is not detailed here. For a given recording location L, the basic investigation algorithm is presented in Algorithm 2, employing a recursive approach to examine progressively smaller areas until a sufficient resolution level is achieved. This is related to the well-known binary search algorithm.

[0053] Algorithm 3 determines a sufficient resolution level, which can be based on many factors. These factors could, for example, be based on the acoustic content of a given area, as well as the physical limitations of the implementation of these algorithms for resolving a smaller area. These factors are all contained in the "if interested then" statement. Algorithm 4 is used to define the spatial information of each ROI. The inputs to this algorithm are the current area r and the current segmentation factor s. The simplest implementation of 4 is simply to segment the area r in half by the factor s. However, other, more complex methods could segment it with more information. Furthermore, this entire procedure should in no way be restricted to a Cartesian system.The different areas could be of varying shape and size and do not necessarily have to be convex.

[0054] Next, the machine functionality monitoring will be carried out as described in Block 110 of Fig. Figure 1 illustrates this. Virtual measurement / signal-to-signal translation systems are disclosed here. Systems and methods for obtaining estimates of data from one or more modalities (e.g., source modalities) by analyzing data from another modality or set of modalities (e.g., target modalities) are disclosed. These methods can be used for the "virtual" measurement or acquisition of data from a variety of systems or processes by means of fundamental linear or nonlinear mappings between the virtually measured target data and the acquired source data, which are learned by observing paired data between all modalities. This mapping is typically associated with a fundamental physical process, but it can also be derived from a more abstract process.Virtual measurement can enable the replacement of expensive and / or difficult-to-install sensors with a more cost-effective and / or easier-to-install sensor. Furthermore, virtual measurement can facilitate predictive machine or process performance monitoring and diagnostics by obtaining estimates of informative data.

[0055] Sensors enable people to observe and record the world around them and to estimate predictions of future global conditions. For example, by using sensors to observe both local and global environmental metrics such as temperature, pressure, and humidity, and by tracking weather fronts, future weather conditions can be predicted with a high degree of accuracy over a relatively short time horizon. Other sensors allow the observation of more microscopic processes, such as the condition of an engine, the stability of infrastructure, or a person's health. Often, these processes and the sensors used to observe them are related to each other through a fundamental physical process.This could involve a combination of mechanical, electrical, or chemical processes, such as variations in current draw by a computer processor or hormonal signals within a living organism. In other cases, these processes and their associated signals and sensors are more abstract, for example, when using stock prices as sensors to estimate and predict the health of a market. Regardless of the system, however, the sensors used to elucidate the state of a system all relate to the fundamental processes associated with that system and could therefore be related to each other via a frequently nonlinear mapping or transfer function.

[0056] A common linear example of such a transfer function is the ideal gas law PV = nRT, where P, V, and T represent the pressure, volume, and temperature of a gas, respectively; n is a value representing the amount of gas in question; and R is the constant of an ideal gas. If the amount of gas in question is known, a temperature sensor could therefore be used to measure the pressure of the gas without needing a pressure sensor. Because a temperature sensor, given a suitable mapping equation, can also output the data of a pressure sensor, it can be said that a temperature sensor can "virtually" measure pressure. However, many other processes of interest exhibit sensor transfer functions that are less linear, such as the wear in various mechanical devices like a vehicle engine.A typical engine has many moving components, such as pistons, belts, fuel pumps, and many others. If the goal, for example, is to analyze fuel pump wear, several possible sensors could be used, including sensors for the temperature inside the pump, the torque or speed of rotating components, structural vibrations, airborne noise emitted by such vibrations, the flow rate induced by the pump itself, and many others. All these sensors essentially relate in some way to the condition of the fuel pump.For example, if the rotating components that induce pump pressure begin to wear and shed particles, friction within the pump can increase, thereby increasing torque and temperature. This alters the noise profile in both airborne and structural tones and likely reduces the flow rate due to decreased pressure within the pump or pump failure. Pump failure is defined as pump malfunction, and impending pump failure is defined as a prediction that a pump will fail within 24 hours, although the pump may actually fail sooner. While all these sensors can observe a fuel pump through different means and communicate different modes of information, they are all linked to the same fundamental physical processes associated with the pump itself.Therefore, a probably non-linear transfer function should exist that relates the values ​​of each sensor to each other in the same way that the ideal gas law relates pressure to temperature.

[0057] Measuring physical processes under real-world scenarios (for example, measuring combustion pressure within an engine or torque or flow rate within a fuel pump while the vehicle is operating on the road) can often be difficult or expensive. However, being able to do so can be crucial for the success of predictive diagnostics, monitoring machine functionality, and controlling processes. At the same time, it can be less difficult or costly to implement these systems and processes in laboratory environments to acquire sensor data that is practically impossible or extremely expensive to obtain in real-world, scaled-up installations.Given these challenges, the use of laboratory data (training data) from expensive sensors, which are difficult to install in the field, in conjunction with more cost-effective sensors that can be deployed on a large scale in the field, in a controlled environment for the "virtual" measurement of phenomena and / or physical processes that are difficult to measure in the field, is advantageous. Accordingly, enabling signal-to-signal translation from the signals acquired by more cost-effective measurement solutions to the signals of other sensors, whose extensive deployment in the field would otherwise be precluded for a variety of reasons, is beneficial.

[0058] In particular, systems and methods for estimating mappings, i.e., transfer functions, between sensors are described here to enable virtual measurement between sensor modalities. A variety of data-driven algorithms are trained using observations from all sensors of interest to predict the output of one or more sensors. These algorithms are then deployed in a variety of embodiments to extend the capabilities of existing sensors by enabling them to acquire data virtually in the manner of alternative sensors.

[0059] According to a general principle, various embodiments of a system for acquiring data and implementing a virtual measurement are disclosed in a variety of physical forms. The system includes some form of data processing hardware and software, through which both the training and installation components of the virtual measurement are implemented. A high-level system evaluates the output of the data processing system to inform an operator or the observed system itself about the state of the observed system. It also includes a training-observable process that enables real-world measurements of virtually implemented sensors at installation time.

[0060] According to another general aspect, complete methods for virtual measurement are disclosed, in which measurements of a process from one or more modalities are estimated from measurements of one or more other modalities by observing the same process via a learned mapping function. The method includes procedures for preprocessing data using both classical methods and modern approaches to prepare the data for virtual measurement. The method also includes a procedure for obtaining such a physical-to-virtual sensor mapping function by training algorithms using sensors that can only be used in a training environment, for example, in a laboratory, but not at scale or in a real-world environment.The method also includes specifications for estimating real sensors based on the virtual domain and methods for both joint and non-joint learning of these mappings.

[0061] According to another general aspect, specific methods for computing physical-to-virtual sensor mapping functions using data-driven models are disclosed. These methods exhibit various non-restrictive possible and preferred embodiments, including generative methods where different types of algorithms directly generate virtual sensor data from physical sensor data.

[0062] According to another general aspect, methods for monitoring a process and implementing predictive maintenance or diagnostics of various systems through virtual measurement are disclosed. These methods involve observing the output of virtual measurement systems and using virtually measured data, with or without physically measured data, to indicate various states of the process of interest. These states may include operating state, operating conditions, fault modes, detection of specific events, and others. Methods for implementing this monitoring include classical signal processing and statistics, machine learning, deep learning, and methods involving human-machine interaction.

[0063] Monitoring the status of various processes is critical in many industrial, commercial, consumer, and healthcare applications. A variety of measurement modalities are often used for this purpose. These sensors can include cameras, lasers, LiDAR, radar, SLAM systems, microphones, hydrophones, ultrasonic sensors, sonar, vibration sensors, accelerometers, torque sensors, pressure sensors, temperature sensors, fluid volume and flow rate sensors, altimeters, velocity sensors, gravity sensors, gas sensors, humidity sensors, heart rate monitors, blood pressure sensors, pulse oximeters, EEG systems, ECG systems, medical imaging devices, and others. Sometimes these sensors are inexpensive and easy to install on a large scale, such as low-cost microphones. Other sensors are more expensive and less easy to install, such as high-precision lasers.In some cases, the cost of the sensors or the procedures required for sensor implementation prevent the sensors from being installed in any practical environment. For example, while relatively inexpensive torque sensors exist and can be used to evaluate various machines in a controlled laboratory setting, they may be too bulky or difficult to install in every vehicle engine on the road. However, there are occasions when the sensors that cannot be installed are most critical, when it becomes necessary to understand the condition or functionality of a machine or process.This section describes methods and systems that aim to solve this problem by learning functions that map data from one sensor or set of sensors (e.g., "source" or "physical" sensors) to data from another sensor or set of sensors (e.g., "target" or "virtual" sensors).

[0064] Fig. Figure 7 provides an overview of a general physical embodiment of a virtual measurement system. A sensor suite 702 consists of all sensors available for the virtual measurement system. Exemplary embodiments of the sensors in the sensor suite 702 include optical, acoustic, vibration, environmental, electromagnetic, and other sensors. The sensors in the sensor suite 702 are those that can observe the process in the training state 708, for example, in a laboratory. The sensor suite 702 therefore also includes sensors that cannot be easily installed on a large scale; these are the sensors that are measured virtually. A data processor or control unit 704 interacts with and controls the sensor suite 702. The data processor 704 generally consists of computing technology.Exemplary embodiments of data processing systems include desktop computers, mobile computers, edge computers, mobile communication devices, mobility platforms such as vehicles or aircraft, security systems, and others. In addition to interacting with sensors, the data processor 704 also analyzes data and implements core algorithms for virtual measurement. A subset of the sensor suite 702 is the set of installable sensors 706. The installable sensors 706 are those that can observe the process of interest in a scaled installation 710 and are used as source sensors, enabling virtual measurement of the target, or as virtual sensors. In addition to the data processor 704, an evaluation and indication system 712 interacts with both the data processor 704 and the installation-observable process 710 (and therefore also has access to data from the sensors 702 and the sensors 706).The evaluation system 712 is primarily used to observe the output of the virtual measurement occurring at the data processor 704 in order to provide information regarding the state of the observed process. Examples of non-restrictive embodiments of the evaluation system 712 could be a computation-based system that uses data processing techniques to understand the virtually measured data, possibly based on prior knowledge or what has been previously observed; a machine learning algorithm that aims to classify the state of the process, such as a support vector machine, decision tree, random forest, k-nearest neighbors, k-means, neural network, or other machine learning algorithm.The output of this classification can trigger artificial intelligence within the 712 assessment system, the 704 data processor, or the 710 process itself to inform and justify subsequent decisions, such as a change of direction, the shutdown of a device, a modification of operating conditions, etc. The 712 assessment system can also involve human intervention, with a person monitoring the output from the 704 data processor or working with an automated assessment system to interpret the 704's output.

[0065] There are many examples of real-world embodiments of the concept in Fig. The system shown in Figure 7 is described, and several examples are discussed here. However, those skilled in the art will understand that the virtual measurement systems and methods disclosed here can be applied generally to practically any situation where the objective is to acquire data from a sensor that is difficult to install via an easy-to-install sensor substitute, and the following examples show several non-limiting examples of such situations, as described in the Fig. 11 - 16 shown.

[0066] Fig. Figure 8 shows the mathematical foundations 800 of the virtual measurement problem and how they relate to the physical embodiments described here. The system 802 has a process 804, which is the process of interest to be observed by physical and / or virtual measurement ranges. Sensors 806 represent all available sensors. The output of the sensors 806 includes all data X 808, including the subset of target sensor data X. Ω , Ω ⊂ {1,...,K} and source sensor data X Λ, Λ ⊂ {1,...,K}. An exemplary embodiment of the virtual measurement problem is set out below. For example, suppose that sensor data X1 represents a source sensor that can be installed, while sensor X2 represents a target sensor that cannot be installed but can be tested in a test environment, so that data pairs {X1(t), X2(t)} can be acquired for time t. The goal of the virtual measurement is to find a mapping function f θ to find a mapping function that maps X1 to X2 so that X2 can be estimated even if the sensor typically used to acquire this data cannot be installed. More generally, there is a desire to find a mapping function f θ 810 to find the X Λ on X Ω maps, i.e., X Λ = f θ (X Ω) for each Λ, Ω ⊂ {1,...,K}, where K represents the number of sensors. According to this formulation, the task can go from one source sensor to one target sensor, from one source sensor to several target sensors, from several source sensors to several target sensors, or from several source sensors to one target sensor.

[0067] Fig. Figure 9 illustrates the key concept behind virtual measurement through signal-to-signal translation. The measurement setup aims to estimate the hidden physical state of a linear or nonlinear dynamic system S, where S ∈ {S1, S2, S3...}. The sensor observations y1 and y2 can be viewed as the output of two observational models (transfer functions) H1(s) and H2(s), respectively. Both y1 and y2 encode information about the hidden state S in different ways, depending on the properties of the observational models (i.e., H1(s) and H2(s)). The key hypothesis of virtual measurement is that if y1 and y2 are detected by the system underlying the physical state S / phenomenon, then a mathematical relationship (linear or nonlinear) must exist between the two measurement modalities y1 and y2.The mathematical relationship represents the reciprocal information between the two measurement modalities y1 and y2 conditioned by S. This can be used to represent the mathematical relationship through a neural network model T. 12(s) to represent / approximate. Next, the neural network is trained by collecting a large number of y1-y2 pairs by traversing the physical system through various states S. Under a practical installation scenario, we can now estimate the transfer function model of y2 with y1 as input and y2 as output (i.e., virtually measuring y2 without a corresponding physical sensor being available, rather "virtually" measuring / reconstructing y2 based on an available physical sensor / data y1). The selection of y1 and y2, i.e., which sensor data is measured virtually and which is physically installed, depends on several factors / compromises.For example, y2 might be expensive and difficult to install in a scalable way, while y1 is inexpensive and easy to install, and it might be simpler to estimate the physical state S from y2 (the ultimate goal of the measurement for downstream tasks), meaning H2(s) is less complicated and better deployed. Under these scenarios, a system would install y1 and measure y2 virtually, then use y2 data to obtain / estimate S in order to perform downstream tasks, such as machine health monitoring / predictive diagnostics.

[0068] In addition to mapping source sensor data to target sensor data, it can also be important to be able to map target sensor data back to source sensor data. If the effectiveness of the mapping function f θIf the mapping function is tested in a test environment where target sensor data can be acquired, its accuracy can be measured directly. However, if it is installed in an environment where target sensors cannot be installed, the accuracy of the mapping function cannot be directly measured. Fig. Figure 10 shows a method 1000 for validating whether the virtually measured data map the virtual target data back into the observable source domain. Observable sensors 1002 are mapped to virtual sensors 1004 by a forward learning model 1006. The virtual sensors 1004 are mapped to virtual sensors 1004 by an inverse learning model 1008, which is defined by a function g. θ' is represented, mapped back to observable sensors 1002. It is important that the inverse learning model can be developed simultaneously with the forward learning model using real data from both the observable and the virtual domain. It is important and similar to the forward mapping function f. θ, that the inverse mapping function g θ' can map from all virtual sensor data to all observable sensor data or a combination of subsets of both virtual and observable data.

[0069] The following presents a mathematical formulation of a non-restrictive embodiment of the virtual measurement algorithm and the training process in conjunction with the other methods and systems in this disclosure for estimating data from virtual sensors of interest. First, consider a physical process monitored by a set of K sensors and denote X i The data captured by the i-th sensor. The data X iThese could correspond to unprocessed data from the sensor, processed data (for example, through filtering, normalization, calculation of a spectrum or spectrogram, etc.), a composition of different data processing methods, or a combination of these data. Next, consider the problem of estimating the measurements of the target data based on the measurements of the source data. To solve the problem of estimating X Ω based on X Λ Can a mapping function f be parameterized by parameter θ θ to be learned so that the minimization of a reconstruction loss L by argmin θL(XΩ,fθ(XΛ)) is achieved.

[0070] Next, designate X̂ n = f θ (X Λ )

[0071] Virtual sensor measurements. Instead of measuring the phenomena with sensors Ω to obtain non-virtual measurements XΩ, virtual measurements are obtained by measuring the phenomena with sensors Λ and applying a mapping function f θ to such measurements in X. Λ obtained. An exemplary non-restrictive embodiment of the reconstruction loss L is a l p -Standard, which defines the difference between the actual sensor measurements XΩ The value of p can be calculated based on problem requirements and / or specifications, or using expert knowledge. Furthermore, the reconstruction loss L can be adjusted to account for and prioritize certain sensors in situations where the reconstruction of either a specific sensor or a set of specific sensors is more important than all target sensors combined. For example, this is done using a weighted norm of difference, where the relative importance of sensor reconstruction is mediated by the weights of the norm.

[0072] To validate the accuracy of virtual measurement using the mapping function f θIt can be useful to be able to map back from the virtual measurement domain to the non-virtual domain. During the process of learning parameters θ, which parameterize the function f that maps from source sensor data to target sensor data, parameters θ' of an inverse mapping function g, which maps from the target sensor data to the source sensor data, can also be learned. This joint learning process of θ and θ' can be described as argminθ,θ' L(XΩ,fθ(XΛ))+τL(XΛ,gθ,(XΩ)) be formulated.

[0073] The composition of the two mapping functions f and g allows an estimation of the reconstruction of the virtual measurement X̂ Ω without explicit knowledge of the measurements of the target sensor XΩ.

[0074] Next, let γ(θ,X) λ )∞ L'(g' θ (f' θ (X Λ )),X λ), where L' denotes a reconstruction loss. The quality of the reconstruction estimate γ can also be part of the joint learning process of θ and θ', where the parameters θ and θ' are optimized in addition to learning the mappings such that γ is an accurate indicator of the reconstruction error, given by the following: argminθ,θ' L(XΩ,fθ(XΛ))+τL(XΛ,gθ,(XΩ))+L'(gθ'(fθ'(XΛ)),Xλ)

[0075] This additional consideration of the process of learning the parameters for the mappings (from source to target and from target to source) is an important concept with regard to the robustness of the mappings. This is useful in situations where the virtual sensors of interest can never be deployed as non-virtual sensors or cannot be deployed at scale. A particular mapping can be considered robust if imperceptible changes δ applied to the input of the mapping functions do not result in perceptible changes in the output of the mapping functions. More formally, this corresponds to a bound imposed by maxδ∈Δ‖f(x+δ)−f(x)‖22 / ‖δ‖22 maxδ∈Δ‖g(x+δ)−g(x)‖22 / ‖δ‖22 is given, where Δ denotes a space of permissible perturbations (for example, l p-sphere with radius ε. In addition to approximating the virtual sensor measurements to the real sensor measurements (minimizing the loss function L), the parameters of the mapping functions f and g can also take into account the minimization of these limitations. They denote Rθ(XΛ)=Ex∼XΛ[maxδ∈Δ||fθ(x+δ)−fθ(x)||22 / ||δ||22] Rθ'(XΩ)=Ex∼XΩ[maxδ∈Δ||gθ'(x+δ)−gθ'(x)||22 / ||δ||22] The measurements of the robustness of the mapping functions. Incorporating these robustness measurements as regularizers into the present optimization problem enables the training of a model that is robust, at least for perturbations in the training data. The problem of learning θ and θ' can therefore be considered as argminθ,θ' L(XΩ,fθ(XΛ))+τ1L(XΛ,gθ,(XΩ))+τ2γ+τ3Rθ(XΛ)+τ4Rθ'(XΩ) be formulated.

[0076] Many embodiments of the mapping functions f and g exist, and they can be parameterized by a variety of algorithms and methods. Exemplary embodiments of methods for obtaining these functions and their parameters include regression, principal component analysis, singular value decomposition, canonical correlation analysis, sequence-to-sequence modeling methods and multimodal representation methods, artificial neural networks, deep neural networks, convolutional neural networks, recurrent neural networks, U-networks, combinations and compositions of these methods, and many others. An exemplary set of preferred embodiments of the mapping functions f and g are generative models and, in particular, variational autoencoders. In this case, each mapping function consists of an encoder and a decoder. The encoder component of f receives X as input. Λ and gives the parameters of a divided by µ fθ , Σ fθA parameterized Gaussian distribution is used. Based on this distribution, a latent vector z is calculated. f sampled and provided as input to the decoder section of f, thereby X Ω is output. Conversely, the encoder component of g X receives Ω as input and returns parameters of a value determined by µ gθ , Σ g The parameterized Gaussian distribution is selected. Based on this distribution, a latent vector z is determined. g sampled and can then be provided as input to the decoder component of g, thereby enabling X Λ is issued.

[0077] Learning the parameters of the generative models, i.e., the encoder and the decoder, can be achieved by minimizing the loss function L, which measures the difference between the real target sensor measurements and the virtual target sensor measurements and provides a divergence measure between the distributions obtained by the encoder and a previous distribution (for example, a Gaussian distribution with a mean of zero and identity covariance).

[0078] Learning the mappings f and g using variational autoencoders can be a separate or a joint process. In the separate learning scenario, the parameters of the encoder-decoder pair for f and g are learned independently. In the joint learning scenario, the parameters of the encoder-decoder pair for f and g are learned together, where the encoder for f is the inverse of the decoder for g and the decoder for f is the inverse of the encoder for g. One method for implementing this process involves prescribing a similarity of the distributions µ. fθ , Σ fθ and µ gθ , Σ gθ (for example, by minimizing the Divergenz between these two distributions).

[0079] Fig. Figure 11 is a schematic diagram of a control system 1102 designed to control a vehicle, which may be at least partially autonomous or at least partially autonomous. The vehicle includes a sensor 1104 and an actuator 1106. The sensor 1104 may comprise one or more visible light sensors (for example, a charge-coupled CCD device or a video device), radar, LiDAR, microphone array, ultrasonic, infrared, thermal imaging, acoustic imaging, or other technologies (for example, GPS-type positioning sensors). One or more of these specific sensors may be integrated into the vehicle.Alternatively or in addition to one or more of the specific sensors identified above, the control module 1102 may include a software module designed to determine a state of the actuator 1104 after execution. A non-limiting example of a software module includes a weather information software module designed to determine a present or future state of the weather near the vehicle or at another location.

[0080] In embodiments where the vehicle is at least partially autonomous, the actuator 1106 can be implemented in the vehicle's braking system, drive system, engine, traction system, or steering system. Actuator control commands can be defined so that the actuator 1106 is controlled to prevent the vehicle from colliding with detected objects. Detected objects can also be classified according to what the classifier considers most likely to be, such as pedestrians or trees. The actuator control commands can be determined based on the classification. For example, the control system 1102 can classify an image (e.g., visual, acoustic, thermal) or other input from the sensor 1104 into one or more background classes and one or more object classes (e.g., pedestrians, bicycles, vehicles, trees, traffic signs, traffic lights, road debris, or construction cylinders / cones, etc.).The system can segment an image and send control commands to actuator 1106, which in this case is implemented in a braking or drive system, to avoid collisions with objects. In another example, control system 1102 can segment an image into one or more background classes and one or more marking classes (e.g., lane markings, guardrails, road edges, vehicle lanes, etc.) and send control commands to actuator 1106, which here is implemented in a steering system, to cause the vehicle to avoid crossing markings and to remain in its lane. In a scenario where an adversary attack may occur, the system described above can also be trained to better detect objects or to identify changes in lighting conditions or angles for a sensor or camera on the vehicle.

[0081] According to other embodiments, in which the vehicle 1100 is an at least partially autonomous robot, the vehicle 1100 can be a mobile robot designed to perform one or more functions such as flying, swimming, diving, and walking. The mobile robot can be an at least partially autonomous lawnmower or a at least partially autonomous cleaning robot. According to such embodiments, the actuator control command 1106 can be configured to control a drive unit, a steering unit, and / or a braking unit of the mobile robot so that the mobile robot can avoid collisions with identified objects.

[0082] According to another embodiment, the vehicle 1100 is an at least partially autonomous robot in the form of a gardening robot. According to this embodiment, the vehicle 1100 can use an optical sensor, designated as sensor 1104, to determine the condition of plants in an environment near the vehicle 1100. The actuator 1106 can be a nozzle designed for spraying chemicals. Depending on an identified species and / or condition of the plants, an actuator control command 1102 can be determined to cause the actuator 1106 to spray the plants with an appropriate quantity of suitable chemicals.

[0083] The vehicle 1100 can be a partially autonomous robot in the form of a household appliance. Non-restrictive examples of household appliances include a washing machine, a stove, an oven, a microwave oven, or a dishwasher. In such a vehicle 1100, the sensor 1104 can be an optical or acoustic sensor designed to detect the state of an object to be processed by the household appliance. For example, if the household appliance is a washing machine, the sensor 1104 can detect the state of the laundry inside the washing machine. An actuator control command can then be determined based on the detected state of the laundry.

[0084] According to this embodiment, the control system 1102 would receive image (optical or acoustic) and annotation information from the sensor 1104. By using this information and a prescribed number of classes k and a similarity measure K stored in the system, the control system 1102 can determine the in Fig. The procedures described in section 10 are used to classify each pixel of the image received by sensor 1104. Based on this classification, signals can be sent to actuator 1106, for example, to brake or change direction to avoid collisions with pedestrians or trees, to steer to stay between detected lane markings, or to execute any of the actions performed by actuator 1106 as described above. Signals can also be sent to sensor 1104 based on this classification, for example, to focus or move a camera lens.

[0085] Fig. Figure 12 shows a schematic diagram of a control system 1202 designed to control a system 1200 (for example, a manufacturing machine) in the form of a punch, cutter, or deep-hole drill of the manufacturing system 102 in the form of a part of a production line. The control system 1202 can be designed to control the actuator 14, which is designed to control the system 100 (for example, a manufacturing machine).

[0086] The sensor 1204 of the system 1200 (for example, a manufacturing machine) can be an optical or acoustic sensor or an optical or acoustic sensor array designed to detect one or more properties of a manufactured product. The control system 1202 can be designed to determine the state of a manufactured product based on one or more of the detected properties. The actuator 1206 can be designed to control the system 1202 (for example, a manufacturing machine) for a subsequent step in the manufacturing process, depending on the determined state of the manufactured product 104. The actuator 1206 can be designed to perform functions from Fig. 11 (for example, manufacturing machine) to control subsequently manufactured products of the system (for example, manufacturing machine) depending on the specific state of the previously manufactured product.

[0087] According to this embodiment, the control system 1202 would receive an image (for example, optical or acoustic) and annotation information from the sensor 1204. Using this information and a prescribed number of classes k and a similarity measure K stored in the system, the control system 1202 can perform the following actions in Fig. The 10 described methods for classifying each pixel of the image received by sensor 1204 are used, for example, to segment an image of a manufactured object into two or more classes, to detect anomalies in the manufactured product, or to ensure the presence of objects on the manufactured product, such as barcodes. Based on this classification, signals can be sent to actuator 1206. For example, if control system 1202 detects anomalies in a product, actuator 1206 can mark anomalous or defective products or remove them from the production line. In another example, if control system 1202 detects the presence of barcodes or other objects to be placed on the product, actuator 1206 can apply or remove these objects.Based on this classification, signals can also be sent to sensor 1204, for example to focus or move a camera lens.

[0088] Fig. Figure 13 shows a schematic diagram of a control system 1302 designed to control a power tool 1300, such as a drill or electric screwdriver, which has at least a partially autonomous mode. The control system 1302 can be designed to control an actuator 1306, which is designed to control the power tool 1300.

[0089] A sensor 1304 of the power tool 1300 can be an optical or acoustic sensor designed to detect one or more properties of a work surface and / or a fastener driven into the work surface. The control system 1302 can be designed to determine the state of a work surface and / or a fastener relative to the work surface based on one or more of the detected properties.

[0090] According to this embodiment, the control system 1302 would receive an image (for example, optical or acoustic) and annotation information from the sensor 1304. By using this information and a prescribed number of classes k and a similarity measure K stored in the system, the control system 1302 can perform the following functions: Fig. The methods described in Section 10 for classifying each pixel of the image received by Sensor 1304 are used to segment an image of a work surface or fastener into two or more classes or to detect anomalies in the work surface or fastener. Based on this classification, signals can be sent to Actuator 1306, for example, to increase pressure or adjust the speed of the tool, or to execute any of the actions performed by Actuator 1306 as described in the preceding sections. Signals can also be sent to Sensor 1304 based on this classification, for example, to focus or move a camera lens. In another example, the image can be a time-series image of signals from Power Tool 1300, for example, relating to pressure, torque, revolutions per minute, temperature, current, etc.the power tool shall be a hammer drill, a drill, a hammer (rotary hammer or demolition hammer), an impact wrench, a pendulum saw, an oscillating multi-tool and either cordless or corded.

[0091] Fig. Figure 14 shows a schematic diagram of a control system 1402 designed to control an automated personal assistant 1401. The control system 1402 can be designed to control the actuator 1406, which is designed to control the automated personal assistant 1401. The automated personal assistant 1401 can be designed to control a household appliance such as a washing machine, stove, oven, microwave oven, or dishwasher.

[0092] According to this embodiment, the control system 1402 would receive an image (for example, optical or acoustic) and annotation information from the sensor 1404. By using this information and a prescribed number of classes k and a similarity measure K stored in the system, the control system 1402 can perform the following functions: Fig. The methods described in section 10 are used to classify each pixel of the image received by sensor 1404, for example, to segment an image of a device or other object to be manipulated or operated. Based on this classification, signals can be sent to actuator 1406, for example, to control moving parts of the automated personal assistant 1401, to interact with household appliances, or to execute any of the actions performed by actuator 1406, as described in the preceding sections. Signals can also be sent to sensor 1404 based on this classification, for example, to focus or move a camera lens.

[0093] Fig. Figure 15 shows a schematic diagram of a control system 1502 designed to control a monitoring system 1500. The monitoring system 1500 can be designed to physically control access through a door 252. A sensor 1504 can be designed to detect a scene relevant to the decision of whether to grant access. The sensor 1504 can be an optical or acoustic sensor, or an optical or acoustic sensor array, designed to generate and transmit image and / or video data. This data can be used by the control system 1502 to detect a person's face.

[0094] The monitoring system 1500 can also be a surveillance system. According to such an embodiment, the sensor 1504 can be an optical sensor designed to detect a monitored scene, and the control system 1502 is designed to control a display 1508. The control system 1502 is designed to determine a scene classification, for example, whether the scene detected by the sensor 1504 is suspicious. A distracting object can be used to detect certain object types, enabling the system to identify such objects under less than optimal conditions (for example, night, fog, rain, distracting background noise, etc.). The control system 1502 is designed to send an actuator control command to the display 1508 in response to the classification. The display 1508 can be designed to adjust the displayed content in response to the actuator control command.For example, display 1508 can highlight an object that is considered suspicious by control unit 1502.

[0095] According to this embodiment, the control system 1502 would receive image (optical or acoustic) and annotation information from the sensor 1504. By using this information and a prescribed number of classes k and a similarity measure K stored in the system, the control system 1502 can determine the in Fig. The procedures described in section 10 are used to classify each pixel of the image received by sensor 1504, for example, to detect the presence of suspicious or unwanted objects in the scene, to identify types of lighting or visibility conditions, or to detect motion. Based on this classification, signals can be sent to actuator 1506, for example, to lock or unlock doors or other entry points, to activate an alarm or other signal, or to execute any of the actions performed by actuator 1506 as described in the preceding sections. Signals can also be sent to sensor 1504 based on this classification, for example, to focus or move a camera lens.

[0096] Fig. Figure 16 shows a schematic diagram of a control system 1602 designed to control an imaging system 1600, such as an MRI device, an X-ray imaging device, or an ultrasound device. The sensor 1604 can be, for example, an imaging sensor or an acoustic sensor array. The control system 1602 can be designed to determine a classification of an entire acquired image or a portion thereof. The control system 1602 can be designed to determine or select an actuator control command in response to the classification obtained by the trained neural network. For example, the control system 1602 can interpret an area of ​​an acquired image (optical or acoustic) as potentially anomalous. In this case, the actuator control command can be determined or selected to cause the display 1606 to display the image and highlight the potentially anomalous area.

[0097] According to this embodiment, the control system 1602 would receive image and annotation information from the sensor 1604. By using this information and a prescribed number of classes k and a similarity measure K stored in the system, the control system 1602 can perform the function described in Fig. The 10 described methods are used to classify each pixel of the image received by sensor 1604. Based on this classification, signals can be sent to actuator 1606, for example, to detect anomalous areas of the image or to execute any of the actions performed by actuator 1606, as described in the preceding sections.

[0098] The program code implementing the algorithms and / or methodologies described herein can be distributed individually or collectively as a program product in a variety of different forms. The program code can be distributed using a computer-readable storage medium containing computer-readable program instructions to instruct a processor to execute aspects of one or more embodiments. A computer-readable storage medium that is inherently non-transient may be a volatile or non-volatile, removable or non-removable physical medium implemented by a method or technology for storing information in the form of computer-readable instructions, data structures, program modules, or other data.Computer-readable storage media may also include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other semiconductor memory technology, portable compact disc read-only storage (CD-ROM) or other optical storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be read by a computer. Computer-readable program instructions can be downloaded over a network from a computer-readable storage medium, or to an external computer or storage device, to a computer, another type of programmable data processing device, or other device.

[0099] Computer-readable program instructions stored on a computer-readable medium can be used to instruct a computer, other types of programmable data processing equipment, or other devices to operate in a specific manner, such that the instructions stored on the computer-readable medium produce a manufactured item that includes instructions implementing the functions, steps, and / or operations specified in the flowcharts or diagrams. According to certain alternative embodiments, the functions, steps, and / or operations specified in the flowcharts and diagrams can be rearranged, processed serially, and / or processed concurrently according to one or more embodiments. Furthermore, any of the flowcharts and / or diagrams can have more or fewer nodes or blocks than those described according to one or more embodiments.

[0100] Although the entire disclosure has been explained by describing various embodiments, and although these embodiments have been described in considerable detail, the applicant does not intend to limit the scope of protection of the attached claims in any way to these details. Additional advantages and modifications will readily occur to those skilled in the art. This disclosure is therefore not limited in its broader aspects to the specific details, the representative apparatus and method, and the illustrative examples presented and described. Accordingly, deviations from these details are permissible without altering the concept or scope of protection of the general invention.

Claims

[1] System for operating a pump, comprising: a balance bike a motor that is coupled to the impeller and designed to generate a torque acting on the impeller, a housing with an outlet opening, designed to accommodate the impeller and a fluid, and an acoustic sensor coupled to the housing and designed to output a torque-indicating signal of the impeller based on sound and a forward mapping function, wherein the forward mapping function is learned during a system training phase in which torque data from a torque sensor are mapped pairwise forward of acoustic data from the acoustic sensor. [2] System according to claim 1, wherein the sound is airborne and the acoustic sensor consists of one or more microphones. [3] System according to claim 1, wherein the sound is structure-borne and the acoustic sensor consists of one or more contact microphones. [4] System according to claim 1, wherein the sound is fluid-borne and the acoustic sensor consists of one or more hydrophones. [5] System according to claim 1, wherein the acoustic sensor is further configured to output a signal indicating the torque of the wheel using a forward mapping function and an output signal indicating the acoustic sensor using a backward mapping function, wherein the backward mapping function is learned during a system training phase in which torque data from a torque sensor are mapped pairwise in reverse to acoustic data from the acoustic sensor. [6] System according to claim 1, wherein the power supply to the motor is interrupted in response to the torque-indicating signal indicating a pump failure. [7] System according to claim 1, wherein an error signal is output in response to the torque-indicating signal indicating an impending pump failure. [8] System according to claim 1, wherein a gradient signal of remaining service life is output in response to the torque-indicating signal indicating an estimated pump service life. [9] Method for measuring a physical parameter of a mechanical device, the method comprising: Obtaining paired measurements of target data from a target sensor and source data from a source sensor during a training period, Generating a forward mapping function from the source data to the target data based on the paired measurements and Interacting with the mechanical device for monitoring the physical parameter by feedback from the source sensor, which is mapped in the forward direction onto the target sensor by the forward mapping function, wherein The source sensor is an acoustic sensor designed to output a signal indicating the torque of an electrical machine of the mechanical device using a forward mapping function, and an output signal indicating the acoustic sensor using a backward mapping function, wherein the backward mapping function is learned during a system training phase in which torque data from a torque sensor is mapped pairwise in reverse to acoustic data from the acoustic sensor. [10] Method according to claim 9, wherein the mechanical device is further operated by feedback from the source sensor, which is imaged in the forward direction by the forward imaging function in order to adjust the physical parameter. [11] Method according to claim 9, wherein the target sensor consists of a first and a second target sensor and the first target sensor is different from the second target sensor. [12] Method according to claim 9, wherein the target sensor consists of several target sensors and each target sensor measures a physical parameter different from all the other target sensors. [13] Method according to claim 9, wherein the source sensor consists of a first and a second source sensor and the first source sensor is different from the second source sensor. [14] Method according to claim 9, wherein the source sensor consists of several source sensors and each source sensor is designed to measure one physical parameter different from all the other several source sensors. [15] Method according to claim 9, wherein the target sensor is a pressure sensor. [16] Method according to claim 9, wherein the physical parameter includes rotational speed, torque, electric current, RF intensity, RF frequency or EMI in connection with the mechanical device. [17] System for measuring a first physical parameter, comprising: a mechanical device designed to operate on the basis of the first physical parameter, and a source sensor coupled to the mechanical device and designed to output a signal indicating the first physical parameter, based on a second physical parameter and a forward mapping function, wherein the forward mapping function is learned during a system training phase in which first physical parameter data from a first physical parameter sensor are mapped pairwise in the forward direction of second physical parameter data from the source sensor, wherein The source sensor is an acoustic sensor designed to output a signal indicating the torque of an electrical machine of the mechanical device using a forward mapping function, and an output signal indicating the acoustic sensor using a backward mapping function, wherein the backward mapping function is learned during a system training phase in which torque data from a torque sensor is mapped pairwise in reverse to acoustic data from the acoustic sensor. [18] System according to claim 17, wherein the forward mapping function is obtained by paired measurements of target data from a target sensor and source data from the source sensor during a training period. [19] System according to claim 18, wherein the target sensor is a pressure sensor.

Images