Abnormal station determination method and device and electronic equipment

Abstract

The invention discloses a method and device for determining an abnormal station and electronic equipment. The method comprises the steps that a sound sampler is adopted to collect audio data of a sound source sampling object, a vibration sampler is adopted to collect vibration data of the sound source sampling object, and the sound source sampling object is a device with specific spectrum characteristic audio on each station of an assembly line; determining action point audio data of the audio data according to the vibration data, and determining a reference spectrum characteristic index and a reference interval duration according to target action point audio data of the sound source sampling object in a normal state; and comparing the sampling interval duration and the spectrum characteristic index of the action point audio data with the corresponding reference interval duration and the corresponding reference spectrum characteristic index, and determining an abnormal station in the assembly line according to a comparison result. According to the invention, the technical problem of high cost due to the fact that whether the station where the equipment is located has a fault or not is monitored through manual inspection in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular, to a method, device and electronic equipment for determining an abnormal workstation. Background Art

[0002] Under normal working conditions, the automated assembly line will complete each process from feeding to semi-finished products and finished products according to the set procedures. The time intervals of each process link are relatively fixed and the rhythm is stable. Once the equipment in a certain link fails, the rhythm from that process link onwards will be disrupted and will continue until the equipment is repaired.

[0003] Most of the related technologies use manual inspections to monitor whether equipment fails, which is relatively costly, and the audio-based fault diagnosis algorithm in the enterprise inspections of the related technologies is relatively complex.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device and electronic device for determining an abnormal workstation, so as to at least solve the technical problem in the related art of using manual inspection to monitor whether a workstation where the equipment is located has a fault, which is costly.

[0006] According to one aspect of an embodiment of the present application, a method for determining an abnormal workstation is provided, comprising: using a sound sampler to collect audio data of a sound source sampling object, and using a vibration sampler to collect vibration data of a sound source sampling object, wherein the sound source sampling object is a device having specific spectral characteristic audio at each workstation of the assembly line; determining action point audio data of the audio data based on the vibration data, and determining a reference spectral characteristic index and a reference interval duration based on the target action point audio data of the sound source sampling object in a normal state; comparing the sampling interval duration and spectral characteristic index of the action point audio data with the corresponding reference interval duration and reference spectral characteristic index, respectively, and determining the abnormal workstation in the assembly line based on the comparison result.

[0007] Optionally, the sound source sampling object is determined in the following manner: obtaining operating audio data of all sound-emitting devices at each workstation on the assembly line; extracting the frequency and amplitude of the operating audio data as operating spectrum feature indicators of the operating audio data; determining the operating audio data corresponding to the spectrum feature indicators that meet preset conditions in the operating spectrum feature indicators as target audio data, wherein each workstation corresponds to one target audio data; determining the sound-emitting device corresponding to the target audio data as the sound source sampling object, wherein adjacent workstations correspond to different sound source sampling objects.

[0008] Optionally, determining the action point audio data of the audio data based on the vibration data includes: performing spectrum analysis on normal vibration data of the sound source sampling object in a normal state to determine an amplitude threshold; determining vibration data with an amplitude exceeding the amplitude threshold as action point data from a spectrum time series diagram corresponding to the vibration data; determining a time point corresponding to the action point data, and determining a time period associated with the time point from the audio data; and intercepting audio data corresponding to the time period from the audio data to obtain the action point audio data.

[0009] Optionally, after determining vibration data with an amplitude exceeding an amplitude threshold as action point data, the method further includes: performing a fast Fourier transform on the audio data to obtain an amplitude-frequency spectrum curve; determining a frequency band associated with the action point data from the amplitude-frequency spectrum curve, and determining an average value and a standard deviation of the amplitude within the frequency band; determining a spectrum domain threshold based on the average value and the standard deviation; and determining data in the audio data that is higher than the spectrum domain threshold as action point audio data.

[0010] Optionally, a reference spectrum characteristic index is determined based on the target action point audio data of the sound source sampling object in a normal state, including: performing a fast Fourier transform on the target action point audio data to obtain the amplitude distribution of the audio signal of the target action point audio data at different frequencies; determining a target amplitude value within a preset frequency range from the amplitude distribution; matching the target amplitude value within the preset frequency range with the corresponding frequency value to obtain a feature vector; and determining the reference spectrum characteristic index based on the mean and standard deviation of the feature vector.

[0011] Optionally, the reference interval duration is determined based on the target action point audio data of the sound source sampling object in a normal state, including: determining the time interval between two adjacent action point audio data of a preset number of groups from the target action point audio data; determining the average duration of all time intervals, and determining the average duration as the reference interval duration.

[0012] Optionally, determining an abnormal workstation in the assembly line based on the comparison result includes: determining a device that meets a first condition in the assembly line as a first device, wherein the first condition is that the sampling interval duration is not within a preset variation range of the reference interval duration, and the degree of matching between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach a preset value; determining a device that meets a second condition in the assembly line as a second device, wherein the second condition is that the sampling interval duration is within a preset variation range of the reference interval duration, and the degree of matching between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach a preset value; determining a device that meets a third condition in the assembly line as a third device, wherein the third condition is that the sampling interval duration is not within a preset variation range of the reference interval duration, and the degree of matching between the spectrum characteristic index and the corresponding reference spectrum characteristic index reaches a preset value; and determining the workstations corresponding to the first device, the second device, and the third device as abnormal workstations.

[0013] According to another aspect of an embodiment of the present application, a device for determining an abnormal workstation is also provided, including: an acquisition module, used to use a sound sampler to collect audio data of a sound source sampling object, and use a vibration sampler to collect vibration data of a sound source sampling object, wherein the sound source sampling object is a device with specific spectral characteristic audio at each workstation of the assembly line; a determination module, used to determine the action point audio data of the audio data based on the vibration data, and determine the reference spectral characteristic index and reference interval duration based on the target action point audio data of the sound source sampling object in a normal state; a comparison module, used to compare the sampling interval duration and spectral characteristic index of the action point audio data with the corresponding reference interval duration and reference spectral characteristic index, and determine the abnormal workstation in the assembly line based on the comparison result.

[0014] According to another aspect of the embodiments of the present application, there is also provided an electronic device, comprising: a memory for storing program instructions; a processor, connected to the memory, for executing program instructions to implement the following functions: using a sound sampler to collect audio data of a sound source sampling object, and using a vibration sampler to collect vibration data of a sound source sampling object, wherein the sound source sampling object is a device with specific spectral characteristic audio at each workstation of an assembly line; determining action point audio data of the audio data based on the vibration data, and determining a reference spectral characteristic index and a reference interval duration based on the target action point audio data of the sound source sampling object in a normal state; comparing the sampling interval duration and spectral characteristic index of the action point audio data with the corresponding reference interval duration and reference spectral characteristic index, and determining an abnormal workstation in the assembly line based on the comparison result.

[0015] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned method for determining the abnormal workstation by running the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned method for determining abnormal workstations when executed by a processor.

[0017] In an embodiment of the present application, a sound sampler is used to collect audio data of a sound source sampling object, and a vibration sampler is used to collect vibration data of a sound source sampling object, wherein the sound source sampling object is a device having specific spectral characteristic audio at each workstation of the assembly line; the action point audio data of the audio data is determined based on the vibration data, and the reference spectral characteristic index and the reference interval duration are determined based on the target action point audio data of the sound source sampling object in a normal state; the sampling interval duration and spectral characteristic index of the action point audio data are compared with the corresponding reference interval duration and reference spectral characteristic index respectively, and the abnormal workstation in the assembly line is determined based on the comparison result, thereby achieving the purpose of real-time and automatic monitoring of the operating status of equipment in an unmanned assembly line workshop of industrial manufacturing, thereby achieving the technical effect of improving inspection efficiency and reducing labor costs, and further solving the technical problem of high cost in the related technology of using manual inspection to monitor whether the workstation where the equipment is located has a fault. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a method for determining an abnormal workstation according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of a method for determining an abnormal workstation according to an embodiment of the present application;

[0021] Figure 3 It is a structural diagram of a device for determining an abnormal workstation according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] In order to solve the problems existing in the related art, the embodiment of the present application provides a method for determining an abnormal workstation, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0025] The method for determining an abnormal workstation provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for determining an abnormal workstation. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0026] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0027] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for determining the abnormal workstation in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned method for determining the abnormal workstation is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0030] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0031] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a method for determining an abnormal workstation. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 2 is a flow chart of a method for determining an abnormal workstation according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0033] Step S202, using a sound sampler to collect audio data of a sound source sampling object, and using a vibration sampler to collect vibration data of a sound source sampling object, wherein the sound source sampling object is a device with specific frequency spectrum characteristic audio at each workstation of the assembly line.

[0034] In the above step S202, the sound source sampling object refers to a specific device on the assembly line, which will emit audio with unique spectral characteristics when running. In an industrial environment, each device will generate a specific audio spectrum when operating normally, that is, the frequency composition and amplitude distribution of the sound when the device is running. These spectral characteristics are the fingerprint of the device status and can be used to identify whether the device is working normally or whether there are potential abnormalities. Among them, each workstation can correspond to a most representative sound source sampling object. A sound sampler is a sensor or device for capturing audio signals. The sound data captured by the sound sampler will be used for subsequent sound spectrum analysis to identify the operating status of the device. A vibration sampler is a sensor that can record and measure vibrations, and is used to collect vibration data when the device is running. Vibration data provides physical vibration characteristics that cannot be captured by sound samplers. When used in combination with sound data, the health status of the equipment can be more comprehensively evaluated.

[0035] It should be noted that before collecting audio data and vibration data, it is necessary to install a sound sampler and a vibration sampler, an image acquisition device, and an alarm connected to the image acquisition device at each workstation of the workshop assembly line. For example, a sound sampler and a vibration sampler are installed on the vibration device of the sound-generating device at each workstation. The sound sampler is used to collect the operating audio data of the sound-generating device, and the vibration sampler is used to collect the vibration data of the sound-generating device. The image acquisition device is installed at a position where a clear picture of the entire workstation can be taken, and is used to record and store the operating image data of each workstation. The alarm is installed on the main equipment of each workstation, and the working status of the equipment at the corresponding workstation is fed back in the form of indicator lights and data transmission. At the same time, a schematic map with working status display of each workstation of the workshop assembly line is drawn and uploaded to the monitoring display terminal. By collecting the operating audio data of all sound-generating devices at each workstation on the assembly line within a process time period, the spectral feature indicators of each audio data are extracted, and the sound source sampling objects of each workstation are configured according to the spectral feature indicators.

[0036] Step S204 , determining the action point audio data of the audio data according to the vibration data, and determining the reference spectrum feature index and the reference interval duration according to the target action point audio data of the sound source sampling object in a normal state.

[0037] In the above step S204, the device will generate sound and vibration during operation, and the vibration data, especially the change in vibration amplitude, can be used as a direct indication of the device action. When the device performs a specific operation, its vibration amplitude will increase significantly. When the amplitude increases significantly (for example, an amplitude exceeding 5mm), it usually corresponds to an important working moment or action point of the device. The vibration data of the device is collected in real time by a vibration sampler and analyzed. The point in the vibration data that exceeds a preset amplitude threshold is regarded as the moment when the action occurs, that is, the above-mentioned action point audio data.

[0038] When the sound source sampling object corresponding to each workstation is in a normal state, a target action point audio data with unique spectral characteristics is selected as a representative, and its key frequency and amplitude characteristics are extracted. These characteristics together constitute the reference spectral characteristic index of the workstation. This index reflects the typical audio characteristics of the equipment under normal working conditions and is the benchmark for subsequent abnormality detection. The reference interval duration reflects the average action frequency or rhythm of the equipment under normal working conditions. By combining the dual monitoring of sound and vibration, the errors of manual sampling and interception of action point audio are reduced, and the accuracy and efficiency of fault detection are improved, which is especially suitable for abnormal detection of equipment with no obvious characteristic changes in appearance.

[0039] Step S206, comparing the sampling interval duration and spectrum characteristic index of the action point audio data with the corresponding reference interval duration and reference spectrum characteristic index respectively, and determining the abnormal workstation in the assembly line according to the comparison result.

[0040] In the above step S206, for the action point audio data intercepted by each workstation, a spectrum analysis is first performed to extract its frequency and amplitude characteristics to form a spectrum feature index of the audio data. Subsequently, these real-time acquired spectrum feature indicators are compared with the previously determined reference spectrum feature indicators. The reference spectrum feature indicators are determined based on a large amount of action point audio data when the device (such as the sound source sampling object) is in normal operation, and include the expected values ​​of the device-specific audio features and their range of variation. If the degree of match between the real-time spectrum feature indicator and the reference indicator is lower than the preset threshold, it means that the audio characteristics of the workstation have deviated from the normal state and there may be an abnormality.

[0041] At the same time, the real-time monitored sampling interval duration (i.e., the time difference between two consecutive action point audio data) is also compared with the reference interval duration. The reference interval duration is a statistical result based on the frequency of action points under normal operating conditions. If the sampling interval duration exceeds the preset variation range of the reference interval duration, it means that the action frequency of the equipment has changed abnormally, which may be that the equipment speed has increased or decreased, which also indicates a potential fault. By comprehensively evaluating the comparison results of the spectrum characteristic indicators and the sampling interval duration, it can be determined whether a certain workstation has an abnormality. If any one or both of the two indicators deviate from the preset normal range, the workstation is marked as an abnormal workstation and may require further inspection and maintenance. The determination of abnormal workstations is not just a simple alarm, but different levels of early warning signals are issued according to the degree of deviation. For example, based on the consideration of equipment operation safety and maintenance priority, mild abnormalities may only require visual monitoring and recording, while severe abnormalities may require immediate shutdown and inspection.

[0042] In the above-mentioned method for determining abnormal workstations, the sound source sampling object is determined in the following manner: obtaining the operating audio data of all sound-emitting devices at each workstation on the assembly line; extracting the frequency and amplitude of the operating audio data as operating spectrum feature indicators of the operating audio data; determining the operating audio data corresponding to the spectrum feature indicators that meet preset conditions in the operating spectrum feature indicators as target audio data, wherein each workstation corresponds to one target audio data; determining the sound-emitting device corresponding to the target audio data as the sound source sampling object, wherein the sound source sampling objects corresponding to adjacent workstations are different.

[0043] In an embodiment of the present application, at each station of an industrial manufacturing assembly line, multiple devices will generate sound during operation. The sound sampler will collect the operating audio data of all devices at these stations to comprehensively cover the various sound outputs of the stations. For the acquired operating audio data, spectrum analysis technology, such as fast Fourier transform (FFT), is applied to convert the sound signal in the time domain into a signal in the frequency domain, and the frequency and amplitude characteristics of the audio are extracted. These features are combined to form the operating spectrum characteristic index of the operating audio data. According to the preset conditions, unique characteristic indicators that can represent the normal operating state of the station are screened out from the operating spectrum characteristic indicators of each station. The preset conditions may include the significance and stability of the characteristics, etc., in order to ensure that the selected audio data can have clear spectrum characteristics when the equipment is operating normally. The screened operating audio data is defined as the target audio data, and the target audio data should be clearly distinguishable from the sounds generated by other devices, so as to ensure the accuracy and effectiveness of the monitoring. The sound-generating device corresponding to the target audio data, that is, the device regarded as having unique and stable spectrum characteristics after spectrum analysis, is determined as the sound source sampling object. Each workstation corresponds to a sound source sampling object, which serves as the main sound source for monitoring and is used to determine whether the workstation is in normal operation. In order to prevent the sound of adjacent workstation equipment from interfering with the monitoring results, the sound source sampling objects are different between different workstations.

[0044] In step S204 of the above-mentioned method for determining the abnormal workstation, the action point audio data of the audio data is determined based on the vibration data, including: performing spectrum analysis on the normal vibration data of the sound source sampling object in a normal state to determine the amplitude threshold; from the spectrum time series diagram corresponding to the vibration data, determining the vibration data with an amplitude exceeding the amplitude threshold as the action point data; determining the time point corresponding to the action point data, and determining the time period associated with the time point from the audio data; and intercepting the audio data corresponding to the time period from the audio data to obtain the action point audio data.

[0045] In an embodiment of the present application, the vibration data collected by the vibration sampler first needs to be subjected to spectrum analysis. For example, the vibration signal in the time domain can be sampled and converted to the frequency domain, and the spectrum of the vibration signal, i.e., the graph of the amplitude distribution with frequency, can be obtained by techniques such as fast Fourier transform (FFT). In the spectrum of the vibration data, when the device performs a specific action, its vibration amplitude will increase significantly within a certain frequency band. Therefore, an amplitude threshold can be determined by statistical analysis of normal vibration data under normal conditions. This threshold is a standard for distinguishing normal vibration from significant vibration, and is usually set to the average amplitude of normal vibration plus a multiple of a standard deviation to ensure that a clear action point vibration can be captured. In the spectrum time series diagram of the vibration data, any vibration data whose amplitude exceeds the above amplitude threshold will be marked as action point data. For each vibration data determined as an action point (i.e., action point data, or action point vibration data), the corresponding time point is recorded. These time points are the moments when the device performs a specific action and will be used for subsequent time synchronization with the audio data. Since the action point audio data and the action point vibration data correspond in time, a corresponding time period can be determined through the time point of the action point vibration data. This time period usually revolves around the action point time point of the vibration data and has a certain width to ensure that the audio characteristics of the device in action can be fully captured. Using the determined time period associated with the action point vibration data, the corresponding audio clips are intercepted from the audio data. These audio clips contain the audio characteristics when the device performs a specific action and are defined as action point audio data. By analyzing the action point audio data, it is possible to further determine whether the operating status of the device is normal, whether there is a fault, etc.

[0046] In the above steps, after the vibration data with an amplitude exceeding the amplitude threshold is determined as the action point data, the method further includes: performing a fast Fourier transform on the audio data to obtain an amplitude-frequency spectrum curve; determining a frequency band related to the action point data from the amplitude-frequency spectrum curve, and determining the average value and standard deviation of the amplitude within the frequency band; determining a spectrum domain threshold based on the average value and the standard deviation; and determining the data in the audio data that is higher than the spectrum domain threshold as the action point audio data.

[0047] In the embodiment of the present application, after the action point in the vibration data is determined, the audio data is then subjected to a fast Fourier transform. This is the process of converting the audio signal from the time domain to the frequency domain. Through FFT, an amplitude-frequency spectrum curve of the audio data can be obtained, that is, a graph of the amplitude changing with the frequency. In the amplitude-frequency spectrum curve, by analyzing the frequency band characteristics corresponding to the action point in the vibration data, the frequency range related to the device action, that is, the frequency band, can be identified. Usually, when the device performs a specific action, characteristic amplitude changes will occur at one or some frequencies, and these frequency bands can help locate the audio characteristics of the action point more accurately. The determined frequency band is statistically analyzed to calculate the amplitude mean and standard deviation of all frequency components in the frequency band. The mean reflects the general level of the amplitude in the frequency band, and the standard deviation indicates the discrete degree of the amplitude change. Based on the mean and standard deviation of the amplitude in the frequency band, a spectrum domain threshold can be set. This threshold is usually set to the mean plus a certain multiple of the standard deviation to distinguish normal amplitude changes from significant amplitude changes. By comparing the amplitude in the frequency band of the audio data with the spectral domain threshold, the audio data above the threshold can be identified, and these data can be regarded as action point audio data.

[0048] By analyzing in the frequency domain, we can better understand the dynamic characteristics of the equipment in different frequency bands. For the intelligent inspection system, this means that we can monitor the operation of the equipment at a deeper level, detect and respond to abnormal situations in a timely manner, and ensure the smooth progress of the production process. At the same time, this method reduces the sensitivity to background noise and non-critical audio information, and improves the robustness of the monitoring system.

[0049] In step S204 of the above-mentioned method for determining the abnormal workstation, a reference spectrum characteristic index is determined based on the target action point audio data of the sound source sampling object in a normal state, including: performing a fast Fourier transform on the target action point audio data to obtain the amplitude distribution of the audio signal of the target action point audio data at different frequencies; determining a target amplitude value within a preset frequency range from the amplitude distribution; matching the target amplitude value within the preset frequency range with the corresponding frequency value to obtain a feature vector; and determining the reference spectrum characteristic index based on the mean and standard deviation of the feature vector.

[0050] In an embodiment of the present application, a fast Fourier transform is performed on the audio data of the target action point intercepted from the sound source sampling object. This transform converts the audio signal in the time domain into a signal in the frequency domain, and obtains an amplitude distribution diagram of the audio signal at different frequencies, that is, an amplitude-frequency spectrum curve. In the amplitude-frequency spectrum curve, a frequency range related to the normal action of the device is obtained to obtain a preset frequency range. These frequency ranges are preset according to the physical characteristics and historical data of the device, for example, the typical frequency generated by the device when performing a certain operation. In another optional embodiment, the preset frequency range can be a frequency range of interest selected according to specific needs, for example, a low range can be selected to capture low-frequency sounds, or a high range can be selected to capture high-frequency sounds, and these frequency ranges can be adjusted according to actual application needs. Within the preset frequency range, the amplitude value of the frequency component closely related to the normal action of the device is selected from the amplitude distribution, that is, the target amplitude value. The target amplitude value reflects the intensity of a specific frequency component when the device performs an action in a normal state. The target amplitude value and the corresponding frequency value within the preset frequency range are combined into a feature vector. This vector contains multiple frequency components and their corresponding amplitude values, which can fully reflect the audio characteristics of the device in a specific frequency band. Based on the mean and standard deviation of the feature vector, a reference spectrum characteristic index is determined. The mean represents the central trend of the amplitude within the preset frequency range under normal conditions, while the standard deviation reflects the fluctuation range of the amplitude. The reference spectrum characteristic index is usually set to the mean plus or minus one or more standard deviations, forming a range of normal amplitude values.

[0051] In step S204 of the above-mentioned method for determining the abnormal workstation, the reference interval duration is determined based on the target action point audio data of the sound source sampling object in a normal state, including: determining the time interval between two adjacent action point audio data of a preset number of groups from the target action point audio data; determining the average duration of all time intervals, and determining the average duration as the reference interval duration.

[0052] In an embodiment of the present application, multiple groups (preset number of groups, for example, 10 groups) of time points at which two adjacent action point audio data appear are selected from the target action point audio data of the sound source sampling object under normal operating conditions. The two adjacent action points here may refer to actions that are continuously performed by the device in the normal operating process, for example, two consecutive cutting, stamping or welding, etc. By recording the time points of these continuous actions, the interval duration between the actions can be calculated. All time intervals obtained in the above steps are statistically analyzed, that is, the average value of these interval durations is calculated, which reflects the average interval time for the device to perform continuous actions under normal operating conditions, thereby obtaining a reference interval duration.

[0053] In step S206 of the above-mentioned method for determining abnormal workstations, the abnormal workstations in the assembly line are determined based on the comparison results, including: determining the equipment that meets the first condition in the assembly line as the first equipment, wherein the first condition is that the sampling interval duration is not within the preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach the preset value; determining the equipment that meets the second condition in the assembly line as the second equipment, wherein the second condition is that the sampling interval duration is within the preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach the preset value; determining the equipment that meets the third condition in the assembly line as the third equipment, wherein the third condition is that the sampling interval duration is not within the preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index reaches the preset value; determining the workstations corresponding to the first equipment, the second equipment, and the third equipment as abnormal workstations.

[0054] In the embodiment of the present application, during the monitoring process, if it is found that the sampling interval duration of a certain device is not within the preset variation range of the reference interval duration (i.e., the rhythm is abnormal), and the matching degree of the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach the preset value (i.e., the sound is abnormal), that is, both the sound and the rhythm deviate from the normal state, then the device will be marked as the first device. If the sampling interval duration of a certain device is within the preset variation range of the reference interval duration (i.e., the rhythm is normal), that is, the operation rhythm of the device does not obviously deviate from the normal state, but the matching degree of the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach the preset value (i.e., the sound is abnormal), that is, the sound characteristics of the device are significantly different from those during normal operation, but the operating frequency or time interval is still within an acceptable range. In this case, the device will be marked as the second device. For those sampling interval durations that are not within the preset variation range of the reference interval duration (i.e., the rhythm is abnormal), but the matching degree of the spectrum characteristic index and the corresponding reference spectrum characteristic index reaches the preset value (i.e., the sound is abnormal), that is, the operation interval of the device is abnormal, but the sound characteristics are still consistent with normal operation, that is, the rhythm is abnormal and the sound is normal. Such equipment will be marked as the third equipment, wherein the rhythm abnormality includes two states of speed increase and speed decrease. The workstations corresponding to the first equipment, the second equipment and the third equipment will be uniformly determined as abnormal workstations.

[0055] It should be noted that if the matching degree between the frequency spectrum characteristic index of the sampled action point audio data and the reference frequency spectrum characteristic index of the corresponding workstation reaches 95%, the sound is normal; otherwise, the sound is abnormal; if the sampling interval is within ±20% of the reference interval of the corresponding workstation, the rhythm is normal. For example, if the reference interval is 1S, the allowable fluctuation range is 0.8S-1.2S. The action points within this range are considered to have normal rhythm; otherwise, the rhythm is abnormal. The abnormal rhythm includes two states: faster and slower speed. Since a workstation may have multiple actions, there are time differences in the connection between actions, which causes the overall processing time of a certain workstation to speed up or slow down. The range of the interval time should be appropriately relaxed to reduce the misjudgment rate.

[0056] The image data of abnormal workstations is sent to the monitoring display terminal, and the corresponding level of warning signals are issued according to the spectrum characteristic indicators and interval duration of the sampled audio data. The working status of each workstation is updated on the schematic map of the monitoring display terminal in the form of status indicators of different colors, and combined with the received image data, the on-duty personnel can view and deal with it at any time. Different levels of warnings are issued according to the following situations:

[0057] If the sound and rhythm are normal, the on-site alarm will display a green light and no alarm will sound; if the sound is normal but the rhythm is abnormal, the on-site alarm will display a blue light and sound a third-level alarm, and at the same time send the image data of the abnormal workstation to the monitoring display end; if the sound is abnormal and the rhythm is normal, the on-site alarm will display a yellow light and sound a second-level alarm, and at the same time send the image data of the abnormal workstation to the monitoring display end; if the sound and rhythm are abnormal, the on-site alarm will display a red light and sound a first-level alarm, and at the same time send the image data of the abnormal workstation to the monitoring display end.

[0058] Furthermore, it also includes installing an operation controller connected to the alarm on the equipment of each workstation, and setting a function that when an alarm of a preset level is issued, the operation controller connected to the alarm executes an action to stop the equipment at the corresponding abnormal workstation or perform other operations.

[0059] Through this series of comparisons and classifications, the monitoring system can quickly identify equipment and workstations with abnormal operating conditions in the assembly line, providing key information for rapid response and troubleshooting. This intelligent analysis method based on sound and vibration data not only improves monitoring efficiency, but also reduces reliance on manual inspections, which is of great significance for improving the intelligence level and production efficiency of industrial manufacturing assembly line workshops. At the same time, by setting different abnormal conditions, the system can carefully distinguish different types of equipment problems, providing targeted guidance for subsequent maintenance and optimization.

[0060] Figure 3 is a structural diagram of a device for determining an abnormal workstation according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0061] The acquisition module 30 is used to acquire audio data of a sound source sampling object by using a sound sampler, and to acquire vibration data of a sound source sampling object by using a vibration sampler, wherein the sound source sampling object is a device having a specific frequency spectrum characteristic audio at each station of the assembly line;

[0062] A determination module 32, configured to determine the action point audio data of the audio data according to the vibration data, and to determine the reference spectrum characteristic index and the reference interval duration according to the target action point audio data of the sound source sampling object in a normal state;

[0063] The comparison module 34 is used to compare the sampling interval duration and spectrum characteristic index of the action point audio data with the corresponding reference interval duration and reference spectrum characteristic index, and determine the abnormal station in the assembly line according to the comparison result.

[0064] In the above-mentioned device for determining abnormal workstations, the sound source sampling object is determined in the following manner: obtaining the operating audio data of all sound-emitting devices at each workstation on the assembly line; extracting the frequency and amplitude of the operating audio data as operating spectrum feature indicators of the operating audio data; determining the operating audio data corresponding to the spectrum feature indicators that meet preset conditions in the operating spectrum feature indicators as target audio data, wherein each workstation corresponds to one target audio data; determining the sound-emitting device corresponding to the target audio data as the sound source sampling object, wherein adjacent workstations correspond to different sound source sampling objects.

[0065] In the determination module in the above-mentioned abnormal workstation determination device, the determination module is also used to perform spectral analysis on the vibration data to determine the amplitude threshold; from the spectrum time series diagram corresponding to the vibration data, the vibration data with an amplitude exceeding the amplitude threshold is determined as the action point data; the time point corresponding to the action point data is determined, and the time period associated with the time point is determined from the audio data; the audio data corresponding to the time period is intercepted from the audio data to obtain the action point audio data.

[0066] In the determination module in the above-mentioned abnormal workstation determination device, the determination module is also used to perform fast Fourier transform on the audio data to obtain an amplitude-frequency spectrum curve; determine the frequency band related to the action point data from the amplitude-frequency spectrum curve, and determine the average value and standard deviation of the amplitude within the frequency band; determine the spectrum domain threshold based on the average value and the standard deviation; and determine the data in the audio data that is higher than the spectrum domain threshold as the action point audio data.

[0067] In the determination module in the above-mentioned abnormal workstation determination device, the determination module is also used to perform fast Fourier transform on the target action point audio data to obtain the amplitude distribution of the audio signal of the target action point audio data at different frequencies; determine the target amplitude value within the preset frequency range from the amplitude distribution; match the target amplitude value within the preset frequency range and the corresponding frequency value to obtain a feature vector; determine the reference spectrum feature index based on the mean and standard deviation of the feature vector.

[0068] In the determination module in the above-mentioned abnormal workstation determination device, the determination module is also used to determine the time interval between two adjacent action point audio data of a preset number of groups from the target action point audio data; determine the average duration of all time intervals, and determine the average duration as the reference interval duration.

[0069] In the comparison module in the above-mentioned abnormal workstation determination device, the comparison module is also used to determine the equipment that meets the first condition in the assembly line as the first equipment, wherein the first condition is that the sampling interval duration is not within the preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach the preset value; determine the equipment that meets the second condition in the assembly line as the second equipment, wherein the second condition is that the sampling interval duration is within the preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach the preset value; determine the equipment that meets the third condition in the assembly line as the third equipment, wherein the third condition is that the sampling interval duration is not within the preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index reaches the preset value; determine the workstations corresponding to the first equipment, the second equipment and the third equipment as abnormal workstations.

[0070] It should be noted that Figure 3 The abnormal station determination device shown is used to perform Figure 2 The method for determining the abnormal workstation is shown in the figure, so the relevant explanations in the above-mentioned method for determining the abnormal workstation are also applicable to the device for determining the abnormal workstation, and will not be repeated here.

[0071] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory, and is used to execute program instructions to implement the following functions: using a sound sampler to collect audio data of a sound source sampling object, and using a vibration sampler to collect vibration data of a sound source sampling object, wherein the sound source sampling object is a device with specific spectral characteristic audio at each workstation of the assembly line; determining action point audio data of the audio data based on the vibration data, and determining a reference spectral characteristic index and a reference interval duration based on the target action point audio data of the sound source sampling object in a normal state; comparing the sampling interval duration and spectral characteristic index of the action point audio data with the corresponding reference interval duration and reference spectral characteristic index, and determining the abnormal workstation in the assembly line based on the comparison result.

[0072] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following method for determining an abnormal workstation by running the computer program: using a sound sampler to collect audio data of a sound source sampling object, and using a vibration sampler to collect vibration data of a sound source sampling object, wherein the sound source sampling object is a device with specific spectral characteristic audio at each workstation of the assembly line; determining action point audio data of the audio data based on the vibration data, and determining a reference spectral characteristic index and a reference interval duration based on the target action point audio data of the sound source sampling object in a normal state; comparing the sampling interval duration and spectral characteristic index of the action point audio data with the corresponding reference interval duration and reference spectral characteristic index, and determining the abnormal workstation in the assembly line based on the comparison result.

[0073] The embodiments of the present application also provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the method for determining an abnormal workstation in each embodiment of the present application.

[0074] The embodiments of the present application also provide a computer program, which, when executed by a processor, implements the steps of the method for determining an abnormal workstation in each embodiment of the present application.

[0075] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0076] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0077] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0078] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0079] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0080] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0081] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining an abnormal workstation, characterized in that: include: A sound sampler is used to collect audio data of a sound source sampling object, and a vibration sampler is used to collect vibration data of the sound source sampling object, wherein the sound source sampling object is a device with a specific frequency spectrum characteristic audio at each workstation of the assembly line; Determine the action point audio data of the audio data according to the vibration data, and determine the reference spectrum feature index and reference interval duration according to the target action point audio data of the sound source sampling object in a normal state; The sampling interval duration and the spectrum characteristic index of the action point audio data are compared with the corresponding reference interval duration and the reference spectrum characteristic index respectively, and the abnormal workstation in the assembly line is determined according to the comparison result.

2. The method according to claim 1, characterized in that: The sound source sampling object is determined by: Obtaining the operating audio data of all sound-generating devices at each workstation on the assembly line; Extracting the frequency and amplitude of the running audio data as running spectrum feature indicators of the running audio data; Determine the operating audio data corresponding to the spectrum characteristic index that meets the preset conditions among the operating spectrum characteristic indexes as target audio data, wherein each workstation corresponds to one target audio data; The sound device corresponding to the target audio data is determined as the sound source sampling object, wherein adjacent workstations correspond to different sound source sampling objects.

3. The method according to claim 1, characterized in that Determining the action point audio data of the audio data according to the vibration data includes: Performing spectrum analysis on normal vibration data of the sound source sampling object in a normal state to determine an amplitude threshold; From the frequency spectrum time series diagram corresponding to the vibration data, determine the vibration data with an amplitude exceeding the amplitude threshold as the action point data; Determining a time point corresponding to the action point data, and determining a time period associated with the time point from the audio data; The audio data corresponding to the time period is intercepted from the audio data to obtain the action point audio data.

4. The method according to claim 3, characterized in that: After determining the vibration data with an amplitude exceeding the amplitude threshold as the action point data, the method further includes: Performing a fast Fourier transform on the audio data to obtain an amplitude-frequency spectrum curve; Determining a frequency band associated with the action point data from the amplitude-frequency spectrum curve, and determining a mean value and a standard deviation of the amplitude within the frequency band; Determining a spectrum domain threshold according to the mean value and the standard deviation; The data in the audio data that is higher than the spectrum domain threshold is determined as the action point audio data.

5. The method according to claim 1, characterized in that: Determining a reference spectrum feature index according to the target action point audio data of the sound source sampling object in a normal state includes: Performing a fast Fourier transform on the target action point audio data to obtain an amplitude distribution of an audio signal of the target action point audio data at different frequencies; determining a target amplitude value within a preset frequency range from the amplitude distribution; Matching the target amplitude value and the corresponding frequency value within the preset frequency range to obtain a feature vector; The reference spectrum characteristic index is determined according to the average value and standard deviation of the characteristic vector.

6. The method according to claim 1, characterized in that Determining the reference interval duration according to the target action point audio data of the sound source sampling object in a normal state includes: Determine the time interval between two adjacent action point audio data of a preset number of groups from the target action point audio data; An average duration of all time intervals is determined, and the average duration is determined as the reference interval duration.

7. The method according to claim 1, characterized in that Determining the abnormal station in the assembly line according to the comparison result includes: Determine a device in the pipeline that meets a first condition as a first device, wherein the first condition is that the sampling interval duration is not within a preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach a preset value; Determine a device in the pipeline that meets a second condition as a second device, wherein the second condition is that the sampling interval duration is within a preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index does not reach a preset value; Determine a device in the pipeline that meets a third condition as a third device, wherein the third condition is that the sampling interval duration is not within a preset variation range of the reference interval duration, and the matching degree between the spectrum characteristic index and the corresponding reference spectrum characteristic index reaches a preset value; The workstations corresponding to the first device, the second device, and the third device are determined as the abnormal workstations.

8. A device for determining an abnormal workstation, characterized in that: include: A collection module, used to collect audio data of a sound source sampling object using a sound sampler, and to collect vibration data of the sound source sampling object using a vibration sampler, wherein the sound source sampling object is a device with specific frequency spectrum characteristic audio at each workstation of the assembly line; A determination module, configured to determine the action point audio data of the audio data according to the vibration data, and determine the reference spectrum feature index and the reference interval duration according to the target action point audio data of the sound source sampling object in a normal state; The comparison module is used to compare the sampling interval duration and spectrum characteristic index of the action point audio data with the corresponding reference interval duration and reference spectrum characteristic index respectively, and determine the abnormal workstation in the assembly line according to the comparison result.

9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor, connected to the memory, for executing program instructions to implement the following functions: using a sound sampler to collect audio data of a sound source sampling object, and using a vibration sampler to collect vibration data of the sound source sampling object, wherein the sound source sampling object is a device with a specific frequency spectrum characteristic audio at each workstation of the assembly line; determining the action point audio data of the audio data according to the vibration data, and determining a reference frequency spectrum characteristic index and a reference interval duration according to the target action point audio data of the sound source sampling object in a normal state; The sampling interval duration and the spectrum characteristic index of the action point audio data are compared with the corresponding reference interval duration and the reference spectrum characteristic index respectively, and the abnormal workstation in the assembly line is determined according to the comparison result.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the method for determining an abnormal workstation according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method for determining an abnormal workstation as described in any one of claims 1 to 7 is implemented.

Images