A perception monitoring system for smart factories

By using a smart factory sensing and monitoring system, neural networks are used to process sensor data and dynamically adjust the reference value of the measurement distance. This solves the problem of measurement distance fluctuation caused by environmental and equipment factors in photoelectric sensors, and enables accurate detection and early warning of photoelectric sensor faults.

CN118816994BActive Publication Date: 2024-11-26JUNHE ZHITONG (SHANDONG) BIG DATA TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411266246.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-11-26
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

In smart factories, the measurement distance of photoelectric sensors is affected by environmental and equipment factors, resulting in abnormal fluctuations between the actual measured distance and the reference value. However, such fluctuations are not enough to trigger an alarm, making it difficult to detect faults.

Method used

The intelligent factory sensing and monitoring system dynamically adjusts the reference value of the measurement distance through the first and second neural networks, and uses a long short-term memory network to process sensor data. Combining the loss function of the main loss term and the penalty term, it detects photoelectric sensor failures, switches to the backup sensor, and triggers an early warning.

Benefits of technology

It enables fault detection and early warning of photoelectric sensors, dynamically adjusts the reference value of the measurement distance, improves the accuracy and timeliness of fault detection, and ensures the accurate positioning of items on the conveyor belt.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118816994B_ABST
    Figure CN118816994B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of fault detection technology, and discloses a perception monitoring system for a smart factory, including: a first data collection module, which is used to collect first data and encode and combine it into a first data vector set; a distance prediction module, which is used to input the first data vector set into a first neural network and output a distance reference value at a corresponding time point; a fault detection module, which is used to combine the first data vector and the distance reference value into a second data vector set according to time and input it into a second neural network, output the probability of a fault occurring at each moment, and judge whether a fault occurs according to the probability; a fault elimination module, which is used to switch a spare photoelectric sensor to eliminate the fault if a fault exists; and an early warning module, which is used to trigger an early warning when a fault exists in the spare photoelectric sensor. The present invention fully considers the problem of abnormal fluctuations in the actual measured distance caused by equipment or environmental factors, and the fault detection and early warning effects are more obvious.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of fault detection technology, and more specifically, to a perception monitoring system for a smart factory. Background Art

[0002] In a smart factory in a logistics center, the production line includes a conveyor belt, a sorting robot, a sensor system, and a barcode scanning device. Since the items need to be within a limited position range to ensure that the items are scanned by the barcode scanning device, multiple photoelectric sensors are distributed on both sides of the conveyor belt to determine the actual measurement distance of the items passing through the conveyor belt by reflecting light on the surface of the items. However, the actual measurement distance of the items on the conveyor belt will be affected by many factors, such as the temperature of the photoelectric sensor, the light intensity emitted by the photoelectric sensor, the speed of the conveyor belt, the vibration frequency of the conveyor belt, the ambient temperature, and the humidity. Therefore, it is impossible to compare the actual measurement distance of the items with the fixed measurement distance reference value.

[0003] Since the internal components of the photoelectric sensor are sensitive to temperature and humidity, if the ambient temperature and humidity change greatly, the sensitivity of the internal components of the photoelectric sensor will be reduced or damaged, causing abnormal fluctuations in the actual measured distance value when the object passes through the area. However, this fluctuation may not be enough to exceed the measurement distance reference value, thereby failing to trigger an alarm.

[0004] If the speed and vibration frequency of the conveyor belt vary greatly, the photoelectric sensor's light reflection distance measurement may cause abnormal fluctuations in the actual measured distance value, but this fluctuation may not be enough to exceed the measured distance reference value, thereby failing to trigger an alarm. Summary of the invention

[0005] The present invention provides a perception monitoring system for a smart factory, which solves the technical problem that when measuring distance in the related technology, a photoelectric sensor fault is detected based on the comparison between the actual measured distance value and the measured distance reference value, and the actual measured distance causes abnormal fluctuations due to some other factors, but such fluctuations may not be sufficient to exceed the measured distance reference value, thereby making it difficult to find the fault.

[0006] The present invention provides a perception monitoring system for a smart factory, comprising the following modules:

[0007] The first data collection module is used to collect first data, and the first data includes:

[0008] The data is collected once at each sampling time point, and the first data includes monitoring parameters of the photoelectric sensor, operating parameters of the conveyor belt, and environmental parameters.

[0009] The monitoring parameters of the photoelectric sensor include: the temperature of the photoelectric sensor, the emitted light intensity, and the voltage; the operating parameters of the conveyor belt include: the speed of the conveyor belt, the vibration frequency of the conveyor belt; the environmental parameters include: light intensity, temperature, and humidity; the sampling time point refers to the moment when data is collected at a predetermined time interval, and each time interval is one minute;

[0010] The first data collected at each sampling time point is encoded into a first data vector.

[0011] As a further optimization solution of the present invention, the first data vector includes a dimension representing values ​​of all acquired parameters.

[0012] A distance prediction module is used to input a first data vector set into a first neural network, and the first neural network will output an upper limit reference value and a lower limit reference value of the distance measured by the photoelectric sensor corresponding to the v-59th to the vth sampling time point.

[0013] It should be noted that the first data vector set is composed of the first data vectors from the v-59th to the vth sampling time points, where v represents the current sampling time point. If the interval between the vth sampling time point and the sampling time point at which collection starts is less than 59, the missing first data vector is interpolated, and the interpolated first data vector is a zero vector.

[0014] As a further optimization scheme of the present invention, the first neural network uses a long short-term memory network, which includes a forget gate, an input gate, a cell state and an output gate. The calculation formula is:

[0015] ;

[0016] in, is the weight matrix of the forget gate, is the bias term of the forget gate, is the hidden state at the previous time step and the concatenation of the t-th first data vector of the first data vector set, is the Sigmoid activation function;

[0017] is equal to the first data vector at the pth sampling time point;

[0018] ;

[0019] Output of the input gate With candidate cell states The calculation formula is:

[0020] ;

[0021] ;

[0022] in, is the weight matrix of the input gate, is the bias term of the input gate, is the hyperbolic tangent activation function, is the weight matrix for calculating candidate cell states, is the bias term of the candidate cell state;

[0023] Cell state The previous cell state And the weighted synthesis of candidate cell states, the calculation formula is:

[0024] ;

[0025] in, Represents element-by-element multiplication operation;

[0026] The activation value of the output gate and the hidden state at the current time step The calculation formula is:

[0027] ;

[0028] ;

[0029] in, is the weight matrix of the output gate, is the bias term of the output gate;

[0030] The hidden state for each time step Processing is performed, and the upper limit reference value is calculated by two independent fully connected layers. and lower reference value , the calculation formula is:

[0031] ;

[0032] ;

[0033] in, and are the weight matrices of the two fully connected layers, and is the bias vector.

[0034] As a further optimization scheme of the present invention, the main loss term is used and penalty items As a loss function , the calculation formula is:

[0035] ;

[0036] ;

[0037] ;

[0038] in , are the upper and lower reference values ​​of the measured distance predicted at time step k, , are the upper and lower limit reference values ​​of the actual measured distance at time step k, is the weight coefficient, its default value is 1.

[0039] A fault detection module is used to generate a second data vector by splicing the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance at the corresponding sampling time point output by the first neural network; all the second data vectors are sorted according to the corresponding sampling time points to generate a second data vector set, the second data vector set is input into the second neural network, and the second neural network outputs a probability value of the photoelectric sensor having a fault;

[0040] As a further optimization scheme of the present invention, the method of splicing the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor measured distance at the corresponding sampling time point output by the first neural network is: splicing the upper limit reference value and the lower limit reference value of the photoelectric sensor measured distance to the first data vector.

[0041] The calculation formula of the second neural network is as follows:

[0042] ;

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] in, and denote the 1st and Lth second data vectors of the second data vector set respectively, , , , represents the trainable weight parameters, , , , represents the trainable bias parameter, represents the dot product, represents the Lth first intermediate feature, represents the Lth second intermediate feature, represents the Lth third intermediate feature, and denote the L-1th and Lth output features respectively, The value indicating whether the photoelectric sensor is faulty, tanh indicates the tanh function, Represents the Sigmoid function.

[0049] As a further optimization scheme of the present invention, is a probability value. If the probability is greater than 0.5, it means that the photoelectric sensor is faulty, otherwise it means that there is no fault.

[0050] A fault elimination module, which is used to control the actuator to switch the faulty photoelectric sensor to a spare photoelectric sensor;

[0051] As a further optimization solution of the present invention, the actuator is a turntable carrying the photoelectric sensor and a drive motor for driving the turntable, and the fault elimination module switches the spare photoelectric sensor by controlling the drive motor to rotate the turntable.

[0052] The early warning module is used to trigger an early warning if the standby photoelectric sensor fails after switching to the standby photoelectric sensor.

[0053] As a further optimization scheme of the present invention, the early warning includes light and sound warnings as well as notifying operation and maintenance personnel to conduct on-site inspection and processing.

[0054] As a further optimization solution of the present invention, a perception monitoring method of a smart factory is provided, which performs the following steps based on the perception monitoring system of the smart factory mentioned above:

[0055] Step 301, collecting monitoring parameters of the photoelectric sensor, operating parameters of the conveyor belt and environmental parameters once at each sampling time point as first data, and encoding and combining the first data collected at each sampling time point into a first data vector set;

[0056] Step 302, inputting the first data vector set into the first neural network, the first neural network will output the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance corresponding to the v-59th to the vth sampling time point;

[0057] Step 303, concatenating the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor distance measured at the corresponding sampling time point output by the first neural network to generate a second data vector; sorting all the second data vectors according to the corresponding sampling time points to generate a second data vector set, inputting the second data vector set into the second neural network, and the second neural network outputting a probability value of the photoelectric sensor being faulty;

[0058] Step 304, judging whether the photoelectric sensor is faulty according to the probability value output by the second neural network, and if faulty, controlling the actuator to switch the faulty photoelectric sensor to a spare photoelectric sensor;

[0059] Step 305: If the standby photoelectric sensor fails after switching to the standby photoelectric sensor, an early warning is triggered.

[0060] The beneficial effects of the present invention are:

[0061] The present invention outputs the measurement distance reference value at the current moment according to the monitoring parameters of the photoelectric sensor, the operating parameters of the conveyor belt, and the environmental parameters, dynamically adjusts the measurement distance reference value, and compares it with the actual measurement distance value to determine whether the actual measurement distance is abnormal. If the actual measurement distance is abnormal, it indicates that there may be a fault.

[0062] The present invention obtains the reference value of the measured distance at the current moment by calculation, and fully considers the abnormal fluctuation problem of the actual measured distance caused by equipment or environmental factors. The fault detection and early warning effects are more obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a module schematic diagram of a perception monitoring system of a smart factory of the present invention;

[0064] Figure 2 It is a flow chart of a perception monitoring method for a smart factory of the present invention. DETAILED DESCRIPTION

[0065] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0066] At least one embodiment of the present invention discloses a perception monitoring system for a smart factory, such as Figure 1 As shown, including:

[0067] The first data collection module 100 is used to collect first data, and the first data includes:

[0068] The data is collected once at each sampling time point, and the first data includes monitoring parameters of the photoelectric sensor, operating parameters of the conveyor belt, and environmental parameters.

[0069] The monitoring parameters of the photoelectric sensor include: the temperature of the photoelectric sensor, the emitted light intensity, and the voltage; the operating parameters of the conveyor belt include: the speed of the conveyor belt, the vibration frequency of the conveyor belt; the environmental parameters include: light intensity, temperature, and humidity; the sampling time point refers to the moment when data is collected at a predetermined time interval, and each time interval is one minute;

[0070] The first data collected at each sampling time point is encoded into a first data vector.

[0071] In one embodiment of the present invention, the first data vector includes dimensions representing values ​​of all acquired parameters.

[0072] The distance prediction module 200 is used to input the first data vector set into the first neural network, and the first neural network will output the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance corresponding to the v-59th to the vth sampling time point.

[0073] It should be noted that the first data vector set is composed of the first data vectors from the v-59th to the vth sampling time points, where v represents the current sampling time point. If the interval between the vth sampling time point and the sampling time point at which collection starts is less than 59, the missing first data vector is interpolated, and the interpolated first data vector is a zero vector.

[0074] In one embodiment of the present invention, the first neural network uses a long short-term memory network, which includes a forget gate, an input gate, a cell state, and an output gate. The output of the forget gate The calculation formula is:

[0075] ;

[0076] in, is the weight matrix of the forget gate, is the bias term of the forget gate, is the hidden state at the previous time step and the concatenation of the t-th first data vector of the first data vector set, is the Sigmoid activation function;

[0077] is equal to the first data vector at the pth sampling time point;

[0078] ;

[0079] Output of the input gate With candidate cell states The calculation formula is:

[0080] ;

[0081] ;

[0082] in, is the weight matrix of the input gate, is the bias term of the input gate, is the hyperbolic tangent activation function, is the weight matrix for calculating candidate cell states, is the bias term of the candidate cell state;

[0083] Cell state The previous cell state And the weighted synthesis of candidate cell states, the calculation formula is:

[0084] ;

[0085] in, Represents element-by-element multiplication operation;

[0086] The activation value of the output gate and the hidden state at the current time step The calculation formula is:

[0087] ;

[0088] ;

[0089] in, is the weight matrix of the output gate, is the bias term of the output gate;

[0090] The hidden state for each time step Processing is performed, and the upper limit reference value is calculated by two independent fully connected layers. and lower reference value , the calculation formula is:

[0091] ;

[0092] ;

[0093] in, and are the weight matrices of the two fully connected layers, and is the bias vector.

[0094] Using the main loss term and penalty items As a loss function , the calculation formula is:

[0095] ;

[0096] ;

[0097] ;

[0098] in , are the upper and lower reference values ​​of the measured distance predicted at time step k, , are the upper and lower limit reference values ​​of the actual measured distance at time step k, is the weight coefficient, its default value is 1.

[0099] A fault detection module 300 is used to generate a second data vector by splicing the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance at the corresponding sampling time point output by the first neural network; all the second data vectors are sorted according to the corresponding sampling time points to generate a second data vector set, the second data vector set is input into the second neural network, and the second neural network outputs a probability value of the photoelectric sensor having a fault;

[0100] In one embodiment of the present invention, the method of splicing the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor measured distance at the corresponding sampling time point output by the first neural network is: splicing the upper limit reference value and the lower limit reference value of the photoelectric sensor measured distance to the first data vector.

[0101] The calculation formula of the second neural network is as follows:

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] ;

[0108] in, and denote the 1st and Lth second data vectors of the second data vector set respectively, , , , represents the trainable weight parameters, , , , represents the trainable bias parameter, represents the dot product, represents the Lth first intermediate feature, represents the Lth second intermediate feature, represents the Lth third intermediate feature, and denote the L-1th and Lth output features respectively, The value indicating whether the photoelectric sensor is faulty, tanh indicates the tanh function, Represents the Sigmoid function.

[0109] In one embodiment of the present invention, is a probability value. If the probability is greater than 0.5, it means that the photoelectric sensor is faulty, otherwise it means that there is no fault.

[0110] A fault elimination module 400, which is used to control the actuator to switch the faulty photoelectric sensor to a spare photoelectric sensor;

[0111] In one embodiment of the present invention, the actuator is a turntable carrying a photoelectric sensor and a drive motor for driving the turntable, and the fault elimination module 400 switches the spare photoelectric sensor by controlling the drive motor to rotate the turntable.

[0112] The early warning module 500 is used to trigger an early warning if the standby photoelectric sensor fails after switching to the standby photoelectric sensor.

[0113] In one embodiment of the present invention, the early warning includes light and sound warnings as well as notifying operation and maintenance personnel to conduct on-site inspection and processing.

[0114] like Figure 2 As shown, in one embodiment of the present invention, a perception monitoring method for a smart factory is provided, which performs the following steps based on the perception monitoring system of the smart factory mentioned above:

[0115] Step 301, collecting monitoring parameters of the photoelectric sensor, operating parameters of the conveyor belt and environmental parameters once at each sampling time point as first data, and encoding and combining the first data collected at each sampling time point into a first data vector set;

[0116] Step 302, inputting the first data vector set into the first neural network, the first neural network will output the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance corresponding to the v-59th to the vth sampling time point;

[0117] Step 303, concatenating the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor distance measured at the corresponding sampling time point output by the first neural network to generate a second data vector; sorting all the second data vectors according to the corresponding sampling time points to generate a second data vector set, inputting the second data vector set into the second neural network, and the second neural network outputting a probability value of the photoelectric sensor being faulty;

[0118] Step 304, judging whether the photoelectric sensor is faulty according to the probability value output by the second neural network, and if faulty, controlling the actuator to switch the faulty photoelectric sensor to a spare photoelectric sensor;

[0119] Step 305: If the standby photoelectric sensor fails after switching to the standby photoelectric sensor, an early warning is triggered.

[0120] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation mode. The above-mentioned specific implementation mode is merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make more forms of equivalent embodiments, all of which are within the protection of this embodiment.

Claims

1. A perception monitoring system for a smart factory, characterized in that: Includes the following modules: A first data collection module, which is used to collect first data and encode and combine it into a first data vector set; A distance prediction module, which is used to input the first data vector set into the first neural network, and the first neural network will output the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance corresponding to the v-59th to the vth sampling time point; A fault detection module is used to generate a second data vector by splicing the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance at the corresponding sampling time point output by the first neural network; all the second data vectors are sorted according to the corresponding sampling time points to generate a second data vector set, the second data vector set is input into the second neural network, and the second neural network outputs a probability value of the photoelectric sensor having a fault; A fault elimination module, which is used to control the actuator to switch the faulty photoelectric sensor to a spare photoelectric sensor; An early warning module, which is used to trigger an early warning if the standby photoelectric sensor fails after switching to the standby photoelectric sensor; The first data in the first data collection module includes: The monitoring parameters of the photoelectric sensor, the operating parameters of the conveyor belt and the environmental parameters are collected once at each sampling time point; The monitoring parameters of the photoelectric sensor include: the temperature of the photoelectric sensor, the emitted light intensity, and the voltage; The operating parameters of the conveyor belt include: the speed of the conveyor belt, the vibration frequency of the conveyor belt; Environmental parameters include: light intensity, temperature, humidity; The sampling time point refers to the moment when data is collected at a predetermined time interval, each time interval is one minute; The first data vector set is composed of the first data vectors from the v-59th to the vth sampling time points, where v represents the current sampling time point. If the interval between the vth sampling time point and the sampling time point at which collection starts is less than 59, the missing first data vector is interpolated, and the interpolated first data vector is a zero vector.

2. The perception monitoring system of a smart factory according to claim 1, characterized in that: The calculation process of the first neural network in the distance prediction module is as follows: The first neural network uses a long short-term memory network, which includes a forget gate, an input gate, a cell state, and an output gate. The calculation formula is: ; in, is the weight matrix of the forget gate, is the bias term of the forget gate, is the hidden state at the previous time step and the concatenation of the t-th first data vector of the first data vector set, is the Sigmoid activation function; is equal to the first data vector at the pth sampling time point; ; Output of the input gate With candidate cell states The calculation formula is: ; ; in, is the weight matrix of the input gate, is the bias term of the input gate, is the hyperbolic tangent activation function, is the weight matrix for calculating candidate cell states, is the bias term of the candidate cell state; Cell state The previous cell state And the weighted synthesis of candidate cell states, the calculation formula is: ; in, Represents element-by-element multiplication operation; The activation value of the output gate The hidden state at the current time step The calculation formula is: ; ; in, is the weight matrix of the output gate, is the bias term of the output gate; The hidden state for each time step Processing is performed, and the upper limit reference value is calculated by two independent fully connected layers. and lower reference value , the calculation formula is: ; ; in, and are the weight matrices of the two fully connected layers, and is the bias vector.

3. The perception monitoring system of a smart factory according to claim 2 is characterized in that: Using the main loss term and penalty items As a loss function , the calculation formula is: ; ; ; in , are the upper and lower reference values ​​of the measured distance predicted at time step k, , are the upper and lower limit reference values ​​of the actual measured distance at time step k, is the weight coefficient, its default value is 1.

4. The perception monitoring system of a smart factory according to claim 1, characterized in that: The calculation formula of the second neural network in the fault detection module is as follows: ; ; ; ; ; ; in, and denote the 1st and Lth second data vectors of the second data vector set respectively, , , , represents the trainable weight parameters, , , , represents the trainable bias parameter, represents the dot product, represents the Lth first intermediate feature, represents the Lth second intermediate feature, represents the Lth third intermediate feature, and denote the L-1th and Lth output features respectively, The value indicating whether the photoelectric sensor is faulty, tanh indicates the tanh function, Represents the Sigmoid function.

5. The perception monitoring system of a smart factory according to claim 4, characterized in that: is a probability value. If the probability is greater than 0.5, it means that the photoelectric sensor is faulty, otherwise it means that there is no fault.

6. The perception monitoring system of a smart factory according to claim 1, characterized in that: The implementing agencies include: A turntable carrying a photoelectric sensor and a driving motor for driving the turntable, the fault elimination module switches the spare photoelectric sensor by controlling the driving motor to rotate the turntable.

7. The perception monitoring system of a smart factory according to claim 1, characterized in that: The warnings of the warning module include: Light and sound warnings are used to notify maintenance personnel to conduct on-site inspection and processing.

8. A perception monitoring method for a smart factory, characterized in that: The following steps are performed based on a perception monitoring system for a smart factory as described in any one of claims 1 to 7: Step 301, collecting monitoring parameters of the photoelectric sensor, operating parameters of the conveyor belt and environmental parameters once at each sampling time point as first data, and encoding and combining the first data collected at each sampling time point into a first data vector set; Step 302, inputting the first data vector set into the first neural network, the first neural network will output the upper limit reference value and the lower limit reference value of the photoelectric sensor measurement distance corresponding to the v-59th to the vth sampling time point; Step 303, concatenating the first data vector with the upper limit reference value and the lower limit reference value of the photoelectric sensor distance measured at the corresponding sampling time point output by the first neural network to generate a second data vector; sorting all the second data vectors according to the corresponding sampling time points to generate a second data vector set, inputting the second data vector set into the second neural network, and the second neural network outputting a probability value of the photoelectric sensor being faulty; Step 304, judging whether the photoelectric sensor is faulty according to the probability value output by the second neural network, and if faulty, controlling the actuator to switch the faulty photoelectric sensor to a spare photoelectric sensor; Step 305: If the standby photoelectric sensor fails after switching to the standby photoelectric sensor, an early warning is triggered.