Method and device for monitoring a work process

Abstract

The invention relates to a method for monitoring a work process, comprising the steps of: acquiring audio data by means of a recording unit and forwarding the audio data to a signal-processing unit; transforming the acquired audio data into the frequency range within a specified bandwidth B by means of the signal-processing unit; creating a spectrogram by spectral analysis, for example by applying a Fourier transform; forwarding the spectrogram to a data-processing unit; identifying at least one type of noise in the spectrogram by means of the data-processing unit by comparing the spectrogram with multiple predefined reference noises stored in a memory; marking, by means of the data-processing unit, the point in time at which the noise type was identified in the spectrogram; allocating the identified noise type to the marked point in time by means of the data-processing unit and creating an electronic allocation table; comparing, by means of the data-processing unit, the created allocation table with at least one reference table stored in a memory; and outputting a data signal by means of the data-processing unit if, when comparing the allocation table with the reference table, a deviation is detected which exceeds a threshold value.

Description

[0001] Method and device for monitoring a work process

[0002] The invention relates to a method and a device for monitoring a work process.

[0003] Methods and devices for monitoring work processes are generally known from the state of the art, primarily in the field of industrial process engineering and process technology. Monitoring the work process is generally based on measuring physical parameters such as pressure, temperature, or flow of the media and materials being processed. For this purpose, industrial plants are equipped with sensors to measure the corresponding parameters.

[0004] In contrast, the monitoring of non-industrial work processes, especially when they largely take place in unstructured, public spaces, is regularly based on manual data capture. Examples of such non-industrial work processes include the distribution of mail, security checks within a geographical area, or garbage collection. Here, the monitoring of the work process is regularly carried out by capturing and analyzing visual data, by manually entering process steps to be processed into a list, or by manually operating timers. However, such methods for monitoring work processes are prone to errors and misuse. The object of the invention is to resolve these and other problems and to provide an objective and reliable method and a corresponding device for monitoring a work process.

[0005] According to the invention, this object is achieved by a method according to claim 1.

[0006] A method according to the invention for monitoring a work process comprises the following steps.

[0007] In a first step, at least one recording unit captures audio data and forwards it to an electronic signal processing unit. The recording unit can be a microphone. The signal processing unit can be implemented as a digital signal processor.

[0008] In a second step, the signal processing unit transforms the acquired audio data into the frequency domain within a predetermined bandwidth B and performs a spectral analysis to create a spectrogram, for example, by applying a Fourier transform. The signal processing unit forwards the created spectrogram to a data processing unit. The bandwidth B can be in a range from approximately 5 Hz to 50 kHz, preferably 10 Hz to 30 kHz, particularly preferably 15 Hz to 20 kHz.

[0009] The data processing unit can be embodied as a microcontroller or microcomputer in a smartphone or the like and can include a central processing unit (CPU), a volatile semiconductor memory (RAM), a non-volatile semiconductor memory (ROM, SSD hard drive), a magnetic storage (hard drive) and / or an optical storage (CD-ROM), as well as interface units (Ethernet, USB), and the like. The components of such data processing units are generally known to those skilled in the art.

[0010] The aforementioned signal processing unit can also be provided as a software module in the RAM or ROM of the data processing unit. In a third step, the data processing unit identifies at least one noise type in the spectrogram by comparing the spectrogram with a plurality of predefined spectrograms of reference noises stored in a database. The database can be provided as a software module in the data processing unit or in an external memory. Instead of a single noise type, a plurality of identical or different noise types can also be identified in the spectrogram.

[0011] In a fourth step, the data processing unit marks the time(s) at which individual noise types were identified in the spectrogram. The data processing unit then assigns the identified noise types to the marked time(s) and creates an electronic assignment table. The assignment table shows at which time each of the predefined noise types was identified in the spectrogram. In the assignment table, a separate time point can be assigned to each of the different or identical noise types identified in the spectrogram.

[0012] The data processing unit then compares the created assignment table with at least one reference table stored in memory. The reference table can contain predefined times and predefined noise types and represent a reference process that should be adhered to wherever possible.

[0013] Finally, the data processing unit outputs a data signal if, when comparing the assignment table with the at least one reference table, a deviation exceeding a predetermined threshold is detected. The data signal can be used to control an image acquisition unit or sent as an email or SMS message.

[0014] To identify the at least one noise type in the spectrogram, the data processing unit can apply a matching algorithm or a classification method from the field of machine learning, in particular a neural network trained with spectrograms of predefined reference noises. The neural network can be part of the data processing unit. In particular, preliminary investigations can be conducted to identify typical noises of individual work steps in the work process—for example, during garbage collection, lifting the garbage bin, emptying the garbage bin into the truck, and placing the empty garbage bin can be defined as individual work steps, each with different reference noises. The corresponding spectrograms can be used to train a correspondingly trained neural network.

[0015] According to the invention, the audio data can be recorded first, followed by the creation and analysis of the spectrograms through spectral analysis and the identification of predefined noise types. For this purpose, the audio data can be acquired within a predetermined time window T. The time window T can range from 1 second to 120 seconds, preferably from 2 seconds to 90 seconds, and particularly preferably from 5 seconds to 6 seconds.

[0016] However, it can also be provided that the recording of the audio data and creation of the spectrograms takes place continuously and essentially in real time.

[0017] According to the invention, an image recording unit connected to the data processing unit can be activated as soon as a predefined noise type is detected in the spectrogram. This can be used to photograph or film the detected work step and thus also be able to visually verify it. In particular, the image recording unit can be activated when a data signal is received.

[0018] The invention further relates to a computer-readable storage medium comprising instructions that cause a data processing unit to execute a method according to the invention. The invention further relates to the use of a method according to the invention for analyzing workflows in a non-industrial work process, in particular in the field of municipal waste collection, wherein the audio data used are the sounds generated during the work process, in particular pouring sounds, and pre-recorded sounds during the work process, in particular pouring sounds of different types and combinations of waste, or pouring sounds when using broken waste bins, are used as reference sounds.

[0019] This makes it possible to identify during the work process which types of waste are being disposed of (residual waste, paper, glass, organic waste, etc.). If necessary, the audio data can also be used to determine whether a broken trash can is being used or whether the trash can has been damaged during handling.

[0020] The invention further relates to a device for monitoring a work process, comprising at least one recording unit, a signal processing unit, and a data processing unit, wherein the recording unit is designed to capture audio data and forward it to the signal processing unit, the signal processing unit is designed to transform the captured audio data into the frequency range within a predetermined bandwidth B and to create a spectrogram, for example by applying a Fourier transformation, and to forward the spectrogram to the data processing unit, the data processing unit is designed to identify at least one noise type in the spectrogram by comparing the spectrogram with a plurality of predefined reference noises stored in a memory, to mark the time at which the noise type was identified in the spectrogram,Assign the detected noise type to the marked time, create an electronic assignment table, compare the created assignment table with at least one reference table stored in a memory, and output a data signal if, upon comparing the assignment table with the at least one reference table, a deviation is detected that exceeds a predetermined threshold. The data processing unit can be configured to identify a plurality of identical or different noise types in the spectrogram and to assign a time in the assignment table to each identified noise type.

[0021] The data processing unit can be designed to apply a matching algorithm or a classification method from the field of machine learning, in particular a neural network trained with predefined reference sounds, to identify the at least one noise type in the spectrogram.

[0022] The recording unit can be configured to capture the audio data within a predetermined time window T. The time window T can be in a range from 1 second to 120 seconds, preferably 2 seconds to 90 seconds, particularly preferably 5 seconds to 6 seconds. However, the recording unit can also be configured to capture the audio data continuously.

[0023] The bandwidth B can be in a range from 5 Hz to 50 kHz, preferably 10 Hz to 30 kHz, particularly preferably 15 Hz to 20 kHz.

[0024] The data processing unit may be configured to spectrally analyze the audio data continuously and substantially in real time and to identify the predefined noise types continuously and substantially in real time.

[0025] An image acquisition unit connected to the data processing unit may be provided, wherein the data processing unit is configured to activate the image acquisition unit as soon as a predefined noise type is detected in the spectrogram. For this purpose, the image acquisition unit may be configured to receive a data signal from the data processing unit.

[0026] Further features of the invention emerge from the patent claims and the following description of an embodiment.

[0027] In an exemplary, non-limiting embodiment, a method according to the invention is used to monitor individual, predefined work steps in municipal waste collection. In the specific example, these are the lifting of the full garbage bin, the emptying of the garbage bin, and the setting down of the empty garbage bin. A neural network in a data processing unit was previously trained using typical spectrograms of these three work steps to recognize these work steps. Furthermore, typical times and durations of these work steps were stored in a reference table—for example, 5 seconds for lifting the garbage bin, 10 seconds for emptying the garbage bin, and 5 seconds for setting down the garbage bin.

[0028] In the embodiment, the method is used to monitor the work processes during waste disposal.

[0029] In a first step, a microphone captures audio data within a specified time window T of 60 seconds and forwards the audio data to a signal processing unit. This unit transforms the captured audio data into the frequency domain within a specified bandwidth of 20 kHz and creates a spectrogram through spectral analysis, namely by applying a Fourier transform. The signal processing unit forwards the spectrogram to a data processing unit. This unit identifies the noise types in the spectrogram by passing the spectrogram to the trained neural network, which was trained with the reference noises.

[0030] The neural network detects the noise types associated with the lifting, emptying, and setting down of the trash can in the spectrogram. The data processing unit then marks the time at which the noise type was identified in the spectrogram and assigns the detected noise type to the marked time.

[0031] The data processing unit then creates an electronic mapping table containing the times of the detected noise types and the detected noise types, and compares it with the stored reference table. If, when comparing the created mapping table with the reference table, a deviation in timing, duration, or other aspects exceeding a threshold is detected, the data processing unit outputs a data signal.

[0032] For example, a data signal can also be output if a typical noise is detected that should not occur in the work process in question, such as the noise of a shattering garbage bin or, in the case of the collection of organic waste, the noise of falling glass or residual waste.

[0033] However, the invention is not limited to the present embodiment, but includes all features within the scope of the following patent claims.

Claims

Patent claims 1. Method for monitoring a work process, comprising the steps of a. capturing audio data by at least one recording unit and forwarding the audio data to a signal processing unit, b. transforming the captured audio data into the frequency range within a predetermined bandwidth B by the signal processing unit and creating a spectrogram by spectral analysis, for example by applying a Fourier transform, and forwarding the spectrogram to a data processing unit, c. identifying, by the data processing unit, at least one noise type in the spectrogram by comparing the spectrogram with a plurality of predefined spectrograms of reference noises stored in a database, d. marking, by the data processing unit, the time at which the noise type was identified in the spectrogram, e.Assigning, by the data processing unit, the detected noise type to the marked point in time and creating an electronic assignment table, f. Comparing, by the data processing unit, the created assignment table with at least one reference table stored in a memory, g. Outputting, by the data processing unit, a data signal if, when comparing the assignment table with the at least one reference table, a deviation is detected that exceeds a predetermined threshold value.

2. Method according to claim 1, characterized in that a plurality of identical or different noise types are identified in the spectrogram, and a time point is assigned to each identified noise type in the assignment table.

3. The method according to claim 1 or 2, characterized in that the data processing unit uses a matching algorithm or a classification method from the field of machine learning to identify the at least one noise type in the spectrogram, in particular a neural network trained with spectrograms of predefined reference noises.

4. Method according to one of claims 1 to 3, characterized in that the detection takes place within a time window T, wherein the time window T is in a range from 1 second to 120 seconds, preferably 2 seconds to 90 seconds, particularly preferably 5 seconds to 6 seconds.

5. Method according to one of claims 1 to 4, characterized in that the bandwidth B is in a range from 5 Hz to 50 kHz, preferably 10 Hz to 30 kHz, particularly preferably 15 Hz to 20 kHz.

6. Method according to one of claims 1 to 5, characterized in that the spectral analysis and the identification of predefined noise types are carried out continuously and essentially in real time.

7. The method according to claim 6, characterized in that an image recording unit connected to the data processing unit is activated as soon as a predefined noise type is detected in the spectrogram or a data signal is received by the data processing unit.

8. A computer-readable storage medium comprising instructions causing a data processing unit to execute a method according to any one of claims 1 to 7.

9. Use of a method according to one of claims 1 to 8 for analyzing work processes in a non-industrial work process, in particular in the field of municipal waste collection, wherein the audio data used are the sounds generated during the work process, in particular pouring sounds, and the reference sounds used are previously recorded sounds during the work process, in particular pouring sounds of different types and combinations of waste or pouring sounds when using broken collection bins.

10. Device for monitoring a work process, comprising at least one recording unit, a signal processing unit, and a data processing unit, wherein a. the recording unit is designed to record audio data and forward it to the signal processing unit, b. the signal processing unit is designed to transform the recorded audio data into the frequency range within a predetermined bandwidth B and to create a spectrogram, for example by applying a Fourier transformation, and to forward the spectrogram to the data processing unit, c. the data processing unit is designed to i. identify at least one noise type in the spectrogram by comparing the spectrogram with a plurality of predefined reference noises stored in a memory, ii. mark the time at which the noise type was identified in the spectrogram, iii.to assign the detected noise type to the marked time and to create an electronic assignment table, iv. to compare the created assignment table with at least one reference table stored in a memory, and v. to output a data signal if, when comparing the assignment table with the at least one reference table, a deviation is detected that exceeds a predetermined threshold value.

11. Device according to claim 10, characterized in that the data processing unit is designed to identify a plurality of identical or different noise types in the spectrogram and to assign a time point to each identified noise type in the assignment table.

12. Device according to claim 10 or 11, characterized in that the data processing unit is designed to apply a matching algorithm or a classification method from the field of machine learning, in particular a neural network trained with predefined reference sounds, to identify the at least one noise type in the spectrogram.

13. Device according to one of claims 10 to 12, characterized in that the time window T is in a range from 1 second to 120 seconds, preferably 2 seconds to 90 seconds, particularly preferably 5 seconds to 6 seconds.

14. Device according to one of claims 10 to 13, characterized in that the bandwidth B is in a range from 5 Hz to 50 kHz, preferably 10 Hz to 30 kHz, particularly preferably 15 Hz to 20 kHz.

15. Device according to one of claims 10 to 14, characterized in that the data processing unit is designed to spectrally analyze the audio data continuously and substantially in real time and to identify the predefined noise types continuously and substantially in real time.

16. Device according to claim 15, characterized in that an image recording unit connected to the data processing unit is provided, wherein the data processing unit is designed to activate the image recording unit as soon as a predefined noise type is detected in the spectrogram or a data signal is received by the data processing unit.