Overhead Crane System Operation Abnormality Detection Method, Device, Electronic Device, and Storage Medium

Abstract

The present invention provides a method, apparatus, electronic device, and storage medium for detecting abnormal operation of a crane system. By extracting the passage characteristics of the current crane in each track section and the spatial relationship between the passage characteristics of the current crane in each track section based on the crane passage data during the execution of the current completed handling task by the current crane, a crane feature vector of the current crane is obtained. At the same time, based on the historical track passage data of each track section, the track feature vectors of each track section are respectively extracted; then, the crane feature vector of the current crane and the track feature vectors of each track section are fused to obtain a crane-track fusion feature, and based on the crane-track fusion feature, fault classification is performed to determine the probability of abnormal operation of the current crane, which can improve the detection accuracy of abnormal crane operation by combining the behavior characteristics of the crane itself and the long-term operation mode of the track section without affecting the normal task execution of the crane.

Description

Technical Field

[0001] The present invention relates to the technical field of overhead crane systems, and in particular, to a method, device, electronic device, and storage medium for detecting abnormal operation of an overhead crane system. Background Art

[0002] In a semiconductor factory, overhead cranes are one of the crucial pieces of equipment, which are used to transport raw materials, semi-finished products, and finished products on the production line. The overhead cranes move within the factory through a precise track system, ensuring efficient material transportation and production processes. It can be seen that the operating conditions of the overhead cranes directly affect the continuity and stability of the production process. Therefore, regularly inspecting and maintaining the overhead cranes not only helps extend the equipment life but also reduces the risk of failures and improves the overall safety and reliability of the system.

[0003] However, the regular inspection strategy is difficult to detect problems or potential failures that suddenly occur during the interval between two inspections in a timely manner, and it may not be able to avoid sudden failures and emergency shutdowns, thus causing temporary interruptions in the wafer handling tasks and disrupting the semiconductor preparation process. Summary of the Invention

[0004] The present invention provides a method, device, electronic device, and storage medium for detecting abnormal operation of an overhead crane system to solve the defect that the existing regular inspection strategy is difficult to detect problems or potential failures that suddenly occur during the interval between two inspections in a timely manner.

[0005] The present invention provides a method for detecting abnormal operation of an overhead crane system, including:

[0006] Based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section;

[0007] Based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through that track section;

[0008] Fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain an overhead crane-track fusion feature, and classifying faults based on the overhead crane-track fusion feature to determine the probability that the current overhead crane has an abnormal operation situation.

[0009] According to a method for detecting abnormal operation of a crane system provided by the present invention, based on the crane passing data of the current crane during the execution of the current completed handling task, extracting the passing characteristics of the current crane in each track section and the spatial relationship between the passing characteristics of the current crane in each track section, to obtain the crane feature vector of the current crane, including:

[0010] Based on the passing time, average passing speed, acceleration, deceleration and alarm situation of the current crane passing through each track section in the crane passing data, and the neighbor track numbers of each track section, constructing the passing characteristics of the current crane in each track section;

[0011] Based on the positional relationship between each track section and the passing characteristics of the current crane in each track section, constructing a crane operation feature map of the current crane; the crane operation feature map is a multi-channel two-dimensional image, and the channels of the crane operation feature map correspond to the dimensions of the passing characteristics of the current crane;

[0012] Based on a lightweight convolutional neural network, performing feature extraction on the crane operation feature map of the current crane to obtain the crane feature vector of the current crane.

[0013] According to a method for detecting abnormal operation of a crane system provided by the present invention, the lightweight convolutional neural network sequentially includes a first one-dimensional convolutional layer, a first one-dimensional max pooling layer, a second one-dimensional convolutional layer and a second one-dimensional max pooling layer.

[0014] According to a method for detecting abnormal operation of a crane system provided by the present invention, based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section, including:

[0015] Based on the order of passing, sorting the passing states of each crane passing through a track section in the historical track passing data of any track section to obtain the track passing state sequence of this track section;

[0016] Sequentially inputting the track passing state sequence of this track section into a long short-term memory network according to time steps to obtain the hidden vectors output by the long short-term memory network at each time step;

[0017] Based on the hidden vectors output by the long short-term memory network at each time step, determining the track feature vector of this track section.

[0018] According to a method for detecting abnormal operation of a crane system provided by the present invention, based on the hidden vectors output by the long short-term memory network at each time step, determining the track feature vector of this track section, including:

[0019] Perform attention transformation on the hidden vectors output by the long short-term memory network at each time step respectively with the crane feature vector of the current crane to obtain the attention weights of the hidden vectors output by the long short-term memory network at each time step;

[0020] Then, based on the difference between the value 1 and the attention weights of the hidden vectors output by the long short-term memory network at each time step, determine the screening weights of the hidden vectors output by the long short-term memory network at each time step;

[0021] Perform weighted fusion on the hidden vectors output by the long short-term memory network at each time step based on the screening weights of the hidden vectors output by the long short-term memory network at each time step to obtain the track feature vector of this track segment.

[0022] According to an abnormal operation detection method of a crane system provided by the present invention, the hidden vector output by the long short-term memory network at any time step is determined based on the passing state of the first crane input to the long short-term memory network at the any time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first crane input to the long short-term memory network at the any time step and the second crane input to the long short-term memory network at the previous time step entering this track segment.

[0023] The present invention also provides an abnormal operation detection device for a crane system, including:

[0024] A crane feature extraction unit, configured to extract the passing features of the current crane in each track segment and the spatial relationship between the passing features of the current crane in each track segment based on the crane passing data of the current crane during the execution of the current completed handling task, to obtain the crane feature vector of the current crane; wherein, the crane passing data includes the passing time, average passing speed, acceleration, deceleration, and warning situation of the current crane passing through each track segment;

[0025] A track feature extraction unit, configured to extract the track feature vectors of the respective track segments respectively based on the historical track passing data of the respective track segments; the historical track passing data of any track segment includes the passing states of the respective cranes passing through this track segment;

[0026] A crane abnormal detection unit, configured to fuse the crane feature vector of the current crane and the track feature vectors of the respective track segments to obtain a crane-track fusion feature, and perform fault classification based on the crane-track fusion feature to determine the probability that the current crane has an abnormal operation situation.

[0027] An abnormal operation detection device for an overhead crane system provided by the present invention extracts the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section based on the overhead crane passing data during the execution of the current completed handling task by the current overhead crane, and obtains the overhead crane feature vector of the current overhead crane, including:

[0028] Based on the passing time, average passing speed, acceleration, deceleration, and warning situation of the current overhead crane passing through each track section in the overhead crane passing data, and the neighbor track numbers of each track section, construct the passing characteristics of the current overhead crane in each track section;

[0029] Based on the positional relationship between each track section and the passing characteristics of the current overhead crane in each track section, construct an overhead crane operation feature map of the current overhead crane; the overhead crane operation feature map is a multi-channel two-dimensional image, and the channels of the overhead crane operation feature map correspond to the dimensions of the passing characteristics of the current overhead crane;

[0030] Based on a lightweight convolutional neural network, perform feature extraction on the overhead crane operation feature map of the current overhead crane to obtain the overhead crane feature vector of the current overhead crane.

[0031] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the overhead crane system operation abnormal detection method as described in any one of the above.

[0032] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the overhead crane system operation abnormal detection method as described in any one of the above.

[0033] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the overhead crane system operation abnormal detection method as described in any one of the above.

[0034] The overhead crane system operation anomaly detection method, device, electronic device, and storage medium provided by the present invention extract the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section based on the overhead crane passing data during the execution of the current completed handling task by the current overhead crane, obtain the overhead crane feature vector of the current overhead crane, and at the same time extract the track feature vectors of each track section based on the historical track passing data of each track section; then fuse the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane - track fusion feature, and perform fault classification based on the overhead crane - track fusion feature to determine the probability that the current overhead crane has an abnormal operation situation. It can perform precise analysis using the passing data after the task is completed and the track data of the track section without affecting the normal task execution of the overhead crane. By combining the behavior characteristics of the overhead crane itself and the long - term operation mode of the track section, the detection accuracy of the abnormal operation of the overhead crane is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1 is a schematic flowchart of the overhead crane system operation anomaly detection method provided by the present invention;

[0037] Figure 2 is a schematic flowchart of the overhead crane feature extraction method provided by the present invention;

[0038] Figure 3 is a schematic structural diagram of the overhead crane system operation anomaly detection device provided by the present invention;

[0039] Figure 4 is a schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0041] Since the regular maintenance strategy is difficult to detect problems or potential faults that suddenly occur during the interval between two maintenance operations of the equipment in a timely manner, sudden failures and emergency shutdowns may not be avoided, resulting in the temporary interruption of the wafer handling task and disrupting the semiconductor manufacturing process. Therefore, it is necessary to perform anomaly detection on the overhead crane after it finishes the current handling task. When the probability of its abnormal operation is relatively high, intervene in advance to conduct a safety inspection on the corresponding overhead crane to ensure that the overhead crane will not have an emergency failure during the execution of the handling task.

[0042] For this reason, the present invention provides a method for detecting abnormal operation of an overhead crane system. Figure 1 It is a schematic flow chart of the method for detecting abnormal operation of the overhead crane system provided by the present invention, as Figure 1 shown, the method includes:

[0043] Step 110: Based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, extract the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section, so as to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration and alarm situation of the current overhead crane passing through each track section;

[0044] Step 120: Based on the historical track passing data of each track section, extract the track feature vectors of each track section respectively; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section;

[0045] Step 130: Integrate the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane - track integrated feature, and perform fault classification based on the overhead crane - track integrated feature to determine the probability of the current overhead crane having an abnormal operation situation.

[0046] Here, after the current overhead crane completes the current handling task, the system will collect a series of passing data related to its operation process. These passing data mainly include parameters such as the passing time, average passing speed, acceleration and deceleration when the current overhead crane passes through each track section during the execution of the current completed handling task, and also include information such as whether an alarm occurs, the type of alarm and the track position where the alarm occurs. These data can be obtained through sensors, control modules or historical data recording systems deployed along the track or on the overhead crane body to ensure that the passing data has high accuracy and integrity.

[0047] After the acquisition of the overhead crane passing data is completed, it will enter the feature extraction stage of the overhead crane itself. For the overhead crane passing data of the current overhead crane, the passing features can be extracted in units of track segments, which may specifically include the deviation between the passing time of the overhead crane on a certain track segment and the historical average passing time, the fluctuation of acceleration and deceleration, the smoothness of speed change, and the density of alarm records, etc. Considering that there is a certain correlation in the operating states of the overhead crane on adjacent tracks, especially when the overhead crane has an abnormal operation, the operating states of the overhead crane on the adjacent track segments it passes through all show a certain abnormality. Therefore, in some embodiments, in addition to the passing time, average passing speed, acceleration, deceleration, and alarm conditions of the current overhead crane passing through each track segment in the overhead crane passing data, the neighbor track numbers of each track segment can also be incorporated into the passing features of the current overhead crane on each track segment, thereby helping to further extract the correlation between the passing features of adjacent track segments from the passing features of the current overhead crane on each track segment based on the spatial relationship between the passing features, such as information on acceleration mutation, speed trend change, and continuity of abnormal states. These features can not only reflect the current operating state of the overhead crane on a single track segment but also reflect the operating continuity and stability of the overhead crane in the entire track system. Finally, this series of features will be integrated into a multi-dimensional overhead crane feature vector to comprehensively represent the operating conditions of the current overhead crane during the completion of the current handling task.

[0048] In some embodiments, as Figure 2 shown, the passing features of the current overhead crane on each track segment and the spatial relationship between the passing features of the current overhead crane on each track segment can be extracted based on the following steps to obtain the overhead crane feature vector of the current overhead crane:

[0049] Step 210, based on the passing time, average passing speed, acceleration, deceleration, and alarm conditions of the current overhead crane passing through each track segment in the overhead crane passing data, and the neighbor track numbers of each track segment, construct the passing features of the current overhead crane on each track segment;

[0050] Step 220, based on the positional relationship between each track segment and the passing features of the current overhead crane on each track segment, construct the overhead crane operation feature map of the current overhead crane; the overhead crane operation feature map is a multi-channel two-dimensional image, and the channels of the overhead crane operation feature map correspond to the dimensions of the passing features of the current overhead crane;

[0051] Step 230, perform feature extraction on the overhead crane operation feature map of the current overhead crane based on a lightweight convolutional neural network to obtain the overhead crane feature vector of the current overhead crane.

[0052] Specifically, in order to more comprehensively and accurately model the characteristics of the overhead crane's operating state and thus improve the accuracy of subsequent anomaly detection, in this embodiment, by deeply mining the track passing behavior of the overhead crane during the process of completing a handling task and combining the spatial correlation relationship between track segments, the overhead crane feature vector of the current overhead crane is obtained.

[0053] Among them, after the overhead crane completes a handling task, it will automatically call the overhead crane passing data in this task. This data records the operating information of all track segments that the overhead crane passes through from the task start point to the end point in sequence, including the passing time, average speed, acceleration, deceleration, and whether there has been an operating alarm on this segment. At the same time, the system will also obtain the structural information of each track segment in the entire track network, that is, the neighbor track numbers of this track segment, which are usually represented as a set of numbers of the front and rear connected track segments. Based on the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track segment, as well as the neighbor track numbers of each track segment, the passing characteristics of the current overhead crane in each track segment are constructed. In some embodiments, for any track segment, the passing characteristics of the current overhead crane in this track segment usually consist of the following items: actual passing time, average speed per unit length, start and end point speed difference (used to calculate acceleration), whether there is a deceleration process, whether an alarm is triggered, neighbor track numbers, and other optional characteristics such as track load factor, etc.

[0054] After obtaining the passing characteristics of the current overhead crane in all track segments, the spatial position relationship between each track segment will be further considered, such as the connection direction, arrangement order, or position layout on the two-dimensional plane between track segments. Combining these structural information, an overhead crane operation feature map representing the overhead crane passing path is constructed. Among them, each track segment in the overhead crane system can be regarded as a node to construct an undirected graph, and then calculate the sum of the distances from each node in this undirected graph to all other nodes, and thus select the node with the smallest sum of distances as the geometric center node, and establish a mapping between this node and the center position of a two-dimensional image with a preset size. Subsequently, starting from the geometric center node, all neighbor nodes of the current node are mapped to the four-neighborhood of the position of the current node in the two-dimensional image according to the actual position relationship between the track corresponding to the current node and the track corresponding to the neighbor node. After establishing the mapping relationship between all nodes and the pixel points in the two-dimensional image, the passing characteristics corresponding to each node (i.e., track segment) can be used as the pixel values of the corresponding pixel points to obtain the overhead crane operation feature map. It should be noted that for the track segments that the current overhead crane has not passed through during the handling process, their pixel values are set to preset values.

[0055] It can be seen that each pixel in the overhead crane operation feature map corresponds to an orbital segment in the overhead crane system. For the orbital segments passed by the current overhead crane during the handling process, the corresponding pixel value is the passing feature of the current overhead crane in that orbital segment. Therefore, the overhead crane operation feature map is a multi-channel two-dimensional image, and the channels of the overhead crane operation feature map correspond to the dimensions of the passing features of the current overhead crane. Since the positions of adjacent orbital segments have local continuity in this two-dimensional image, it is convenient to capture the change pattern of the overhead crane operation state in space subsequently. Next, the constructed overhead crane operation feature map is input into a lightweight convolutional neural network (lightweight CNN) for processing to obtain the overhead crane feature vector of the current overhead crane. The selected lightweight convolutional network has a small parameter scale and low computational complexity to meet the real-time requirements of the overhead crane system. In some embodiments, the lightweight convolutional neural network sequentially includes a first one-dimensional convolutional layer, a first one-dimensional max pooling layer, a second one-dimensional convolutional layer, and a second one-dimensional max pooling layer.

[0056] Through this method, this embodiment not only realizes the accurate modeling of the operation state of the overhead crane in the track system, but also effectively integrates the spatial relationship between orbital segments. Combined with the subsequent orbital feature vectors, on the one hand, it helps to distinguish the abnormalities of the overhead crane itself, the common abnormalities caused by certain path combinations of the track system (that is, the abnormal behaviors caused by certain specific track path combinations, which are not caused by the abnormalities of a single orbital segment or the overhead crane, but are jointly caused by the combination order, structural connection method or operation habits of these orbital segments), and the abnormalities of orbital segments. In addition, since a lightweight convolutional neural network is used as the core model, this solution greatly improves the processing efficiency and system response speed on the premise of ensuring the detection accuracy.

[0057] At the same time, the embodiment of the present invention will also retrieve the historical track passing data of each orbital segment stored in the background database and extract the long-term operation features of each orbital segment respectively. Among them, the historical track passing data usually includes the passing states of multiple overhead cranes passing through this orbital segment at different times, including its speed distribution and passing time distribution, etc. Based on these historical track passing data, an orbital feature vector describing the operation mode of each orbital segment can be constructed. After obtaining the overhead crane feature vector of the current overhead crane and the orbital feature vectors of all orbital segments, these two types of information can be fused to obtain the overhead crane-orbital fusion feature. The fusion process can adopt methods such as feature splicing, weighted combination, and vector cross, and the embodiment of the present invention does not make specific limitations on this. This overhead crane-orbital fusion feature not only reflects the operation characteristics of the overhead crane itself, but also reflects the potential abnormal tendency of its operation environment.

[0058] In some embodiments, when extracting the track feature vectors of each track segment, in order to introduce time dynamics, the passing states of each overhead crane passing through the track segment in the historical track passing data of any track segment can be sorted based on the passing sequence to obtain the track passing state sequence of the track segment. Subsequently, the track passing state sequence of the track segment is sequentially input into the long short-term memory network according to time steps to obtain the hidden vectors output by the long short-term memory network at each time step, and based on the hidden vectors output by the long short-term memory network at each time step, the track feature vector of the track segment is determined. Among them, the long short-term memory network can effectively model the long-term dependencies existing in the track passing state sequence and learn the state evolution of the track segment at different time periods.

[0059] Based on the overall distribution of the hidden vectors at each time step, the features therein can be further fused to form the track feature vector of the track segment. In some embodiments, the hidden vectors output by the long short-term memory network at each time step can be respectively subjected to an attention transformation with the overhead crane feature vector of the current overhead crane to obtain the attention weights of the hidden vectors output by the long short-term memory network at each time step. Then, based on the difference between the value 1 and the attention weights of the hidden vectors output by the long short-term memory network at each time step, the screening weights of the hidden vectors output by the long short-term memory network at each time step are determined. It should be noted here that the special design of calculating the difference between the value 1 and the attention weights as the screening weights of the corresponding hidden vectors is to retain the features with a low correlation with the overhead crane feature vector. The reason is that the possibility of simultaneous existence of overhead crane anomalies and track anomalies is relatively low, and the manifestations of overhead crane anomalies and track anomalies are different. If it is an overhead crane anomaly, the hidden vectors extracted by the long short-term memory network should be different from the overhead crane feature vector because the passing data of other overhead cranes is normal; if it is a track anomaly, the overhead crane feature vector should also be different from the hidden vectors extracted by the long short-term memory network because the passing data of this overhead crane is normal. Subsequently, the hidden vectors output by the long short-term memory network at each time step are weighted and fused based on the screening weights of the hidden vectors output by the long short-term memory network at each time step to obtain the track feature vector of the track segment.

[0060] In some other embodiments, in order to more accurately extract the operation mode of the track segment, the hidden vector output by the long short-term memory network at the current time step can be determined based on the passing state of the first overhead crane input to the long short-term memory network at any time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first overhead crane input to the long short-term memory network at the current time step and the second overhead crane input to the long short-term memory network at the previous time step when entering the track segment. That is, at any time step, the inputs to the long short-term memory network include three: the passing state of the first overhead crane, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first overhead crane and the second overhead crane input to the long short-term memory network at the previous time step when entering the track segment. Here, considering that the operation states of the overhead cranes entering the track segment successively within a short period of time will interfere with each other. For example, if the previous overhead crane brakes, etc., it will also affect the running speed of the following overhead crane, but this is not caused by the track segment. Therefore, by additionally introducing the time difference between the first overhead crane input to the long short-term memory network at the current time step and the second overhead crane input to the long short-term memory network at the previous time step when entering the track segment, the long short-term memory network can be helped to learn this operation mode, which is helpful for distinguishing operation interference from track segment anomalies.

[0061] Subsequently, based on the fused feature of the overhead crane and the track, fault classification is carried out to determine the probability that the current overhead crane has an abnormal operation situation. Here, a fault detection model can be constructed based on traditional machine learning algorithms (such as support vector machines, random forests, etc.) or based on deep neural network structures such as convolutional neural networks (CNNs), graph neural networks (GNNs), etc. The model analyzes and judges the fused feature of the overhead crane and the track, and outputs the probability value indicating whether the current overhead crane has an abnormal operation during the execution of this task. The higher this probability value, the greater the possibility of abnormal operation of the overhead crane. If the probability that the current overhead crane has an abnormal operation situation is higher than the preset threshold, it can be considered that the overhead crane may be in an abnormal state, and the corresponding emergency handling mechanism is triggered. These mechanisms can include: automatically sending an abnormal alarm message to the monitoring platform, suspending the scheduling task of the overhead crane, recording the operation log of this time for subsequent retrospective analysis, and so on.

[0062] Through the above method, the overhead crane system operation anomaly detection method proposed by the embodiments of the present invention extracts the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section based on the overhead crane passing data during the execution of the current completed handling task by the current overhead crane, obtains the overhead crane feature vector of the current overhead crane, and at the same time extracts the track feature vectors of each track section based on the historical track passing data of each track section; then fuses the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane - track fusion feature, and classifies faults based on the overhead crane - track fusion feature to determine the probability that the current overhead crane has an abnormal operation situation. It can accurately analyze using the passing data after the task is completed and the track data of the track section without affecting the normal task execution of the overhead crane. By combining the behavior characteristics of the overhead crane itself and the long - term operation mode of the track section, the detection accuracy of the overhead crane operation anomaly is improved.

[0063] Next, the overhead crane system operation anomaly detection device provided by the present invention will be described. The overhead crane system operation anomaly detection device described below can be mutually referred to the overhead crane system operation anomaly detection method described above.

[0064] Based on any of the above embodiments, Figure 3 is a schematic structural diagram of the overhead crane system operation anomaly detection device provided by the present invention. As Figure 3 shown, the device includes:

[0065] An overhead crane feature extraction unit 310, configured to extract the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section based on the overhead crane passing data during the execution of the current completed handling task by the current overhead crane, and obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section;

[0066] A track feature extraction unit 320, configured to extract the track feature vectors of each track section respectively based on the historical track passing data of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section;

[0067] An overhead crane anomaly detection unit 330, configured to fuse the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane - track fusion feature, and classify faults based on the overhead crane - track fusion feature to determine the probability that the current overhead crane has an abnormal operation situation.

[0068] The device provided by the embodiment of the present invention extracts the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section based on the overhead crane passing data during the execution of the current completed handling task by the current overhead crane, obtains the overhead crane feature vector of the current overhead crane, and at the same time extracts the track feature vectors of each track section based on the historical track passing data of each track section; then fuses the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane - track fusion feature, and classifies faults based on the overhead crane - track fusion feature to determine the probability of abnormal operation of the current overhead crane. It can perform accurate analysis using the passing data after the task is completed and the track data of the track section without affecting the normal task execution of the overhead crane. By combining the self - behavior characteristics of the overhead crane and the long - term operation mode of the track section, the detection accuracy of abnormal operation of the overhead crane is improved.

[0069] Based on any of the above embodiments, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section based on the overhead crane passing data during the execution of the current completed handling task by the current overhead crane, and obtaining the overhead crane feature vector of the current overhead crane includes:

[0070] Based on the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section in the overhead crane passing data, and the neighbor track numbers of each track section, construct the passing characteristics of the current overhead crane in each track section;

[0071] Based on the positional relationship between each track section and the passing characteristics of the current overhead crane in each track section, construct the overhead crane operation feature map of the current overhead crane; the overhead crane operation feature map is a multi - channel two - dimensional image, and the channels of the overhead crane operation feature map correspond to the dimensions of the passing characteristics of the current overhead crane;

[0072] Based on a lightweight convolutional neural network, perform feature extraction on the overhead crane operation feature map of the current overhead crane to obtain the overhead crane feature vector of the current overhead crane.

[0073] Based on any of the above embodiments, the lightweight convolutional neural network sequentially includes a first one - dimensional convolutional layer, a first one - dimensional max - pooling layer, a second one - dimensional convolutional layer, and a second one - dimensional max - pooling layer.

[0074] Based on any of the above embodiments, extracting the track feature vectors of each track section based on the historical track passing data of each track section includes:

[0075] Based on the order of passing, sort the passing states of each overhead crane passing through a track section in the historical track passing data of any track section to obtain the track passing state sequence of the track section;

[0076] The track passing state sequence of this track segment is input into the long short-term memory network in time steps in sequence, and the hidden vectors output by the long short-term memory network at each time step are obtained;

[0077] Based on the hidden vectors output by the long short-term memory network at each time step, the track feature vector of this track segment is determined.

[0078] Based on any of the above embodiments, the determining of the track feature vector of this track segment based on the hidden vectors output by the long short-term memory network at each time step includes:

[0079] The hidden vectors output by the long short-term memory network at each time step are respectively subjected to attention transformation with the crane feature vector of the current crane to obtain the attention weights of the hidden vectors output by the long short-term memory network at each time step;

[0080] Then, based on the difference between the value 1 and the attention weights of the hidden vectors output by the long short-term memory network at each time step, the screening weights of the hidden vectors output by the long short-term memory network at each time step are determined;

[0081] The hidden vectors output by the long short-term memory network at each time step are weighted and fused based on the screening weights of the hidden vectors output by the long short-term memory network at each time step to obtain the track feature vector of this track segment.

[0082] Based on any of the above embodiments, the hidden vector output by the long short-term memory network at any time step is determined based on the passing state of the first crane input into the long short-term memory network at this time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first crane input into the long short-term memory network at this time step and the second crane input into the long short-term memory network at the previous time step entering this track segment.

[0083] Figure 4 It is a schematic structural diagram of the electronic device provided by the present invention, as Figure 4As shown in the figure, the electronic device may include: a processor 410, a memory 420, a communication interface 430, and a communication bus 440. Among them, the processor 410, the memory 420, and the communication interface 430 complete communication with each other through the communication bus 440. The processor 410 may call the logical instructions in the memory 420 to execute the abnormal operation detection method for the overhead crane system. The method includes: based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section; based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section; fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane track fusion feature, and based on the overhead crane track fusion feature, performing fault classification to determine the probability that the current overhead crane has an abnormal operation situation.

[0084] In addition, when the logical instructions in the above-mentioned memory 420 are implemented in the form of software function units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0085] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the overhead crane system operation anomaly detection method provided by the above-mentioned various methods. The method includes: based on the overhead crane passing data during the current overhead crane's execution of the current completed handling task, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section; based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section; fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain an overhead crane track fusion feature, and based on the overhead crane track fusion feature, performing fault classification to determine the probability that the current overhead crane has an abnormal operation situation.

[0086] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the overhead crane system operation anomaly detection method provided by the above-mentioned various methods. The method includes: based on the overhead crane passing data during the current overhead crane's execution of the current completed handling task, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section; based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section; fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain an overhead crane track fusion feature, and based on the overhead crane track fusion feature, performing fault classification to determine the probability that the current overhead crane has an abnormal operation situation.

[0087] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0088] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A method for detecting abnormal operation of a crane system, characterized in that, Including: Based on the crane passing data of the current crane during the execution of the current completed handling task, extracting the passing characteristics of the current crane in each track section and the spatial relationship between the passing characteristics of the current crane in each track section to obtain the crane feature vector of the current crane; wherein, the crane passing data includes the passing time, average passing speed, acceleration, deceleration and alarm situation of the current crane passing through each track section. Based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each crane passing through this track section. Fusing the crane feature vector of the current crane and the track feature vectors of each track section to obtain the crane-track fusion feature, and based on the crane-track fusion feature, performing fault classification to determine the probability that the current crane has an abnormal operation situation. The extracting the track feature vectors of each track section based on the historical track passing data of each track section includes: Sorting the passing states of each crane passing through any track section in the historical track passing data of this track section according to the passing sequence to obtain the track passing state sequence of this track section. Sequentially inputting the track passing state sequence of this track section into the long short-term memory network according to time steps to obtain the hidden vectors output by the long short-term memory network at each time step. Based on the hidden vectors output by the long short-term memory network at each time step, determining the track feature vector of this track section.

2. The abnormal operation detection method of the overhead crane system according to claim 1, wherein, The extracting the passing characteristics of the current crane in each track section and the spatial relationship between the passing characteristics of the current crane in each track section based on the crane passing data of the current crane during the execution of the current completed handling task to obtain the crane feature vector of the current crane includes: Based on the passing time, average passing speed, acceleration, deceleration and alarm situation of the current crane passing through each track section in the crane passing data, and the neighbor track numbers of each track section, constructing the passing characteristics of the current crane in each track section. Based on the positional relationship between each track section and the passing characteristics of the current crane in each track section, constructing the crane operation feature map of the current crane; the crane operation feature map is a multi-channel two-dimensional image, and the channels of the crane operation feature map correspond to the dimensions of the passing characteristics of the current crane. Based on the lightweight convolutional neural network, performing feature extraction on the crane operation feature map of the current crane to obtain the crane feature vector of the current crane.

3. The abnormal operation detection method of the overhead crane system according to claim 2, characterized in that The lightweight convolutional neural network sequentially includes a first one-dimensional convolutional layer, a first one-dimensional max pooling layer, a second one-dimensional convolutional layer and a second one-dimensional max pooling layer.

4. The abnormal operation detection method of the overhead crane system according to claim 1, wherein The determining the track feature vector of this track section based on the hidden vectors output by the long short-term memory network at each time step includes: Performing attention transformation on the hidden vectors output by the long short-term memory network at each time step respectively with the crane feature vector of the current crane to obtain the attention weights of the hidden vectors output by the long short-term memory network at each time step. Then, based on the difference between the numerical value 1 and the attention weights of the hidden vectors output by the long short-term memory network at each time step, determine the screening weights of the hidden vectors output by the long short-term memory network at each time step; Based on the screening weights of the hidden vectors output by the long short-term memory network at each time step, perform weighted fusion on the hidden vectors output by the long short-term memory network at each time step to obtain the track feature vector of this track segment.

5. The abnormal operation detection method of the overhead crane system according to claim 1, characterized in that, The hidden vector output by the long short-term memory network at any time step is determined based on the passing state of the first crane input to the long short-term memory network at the any time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first crane input to the long short-term memory network at the any time step and the second crane input to the long short-term memory network at the previous time step entering this track segment.

6. An abnormal operation detection device for an overhead crane system, characterized in that It includes: A crane feature extraction unit, configured to extract the passing features of the current crane in each track segment and the spatial relationship between the passing features of the current crane in each track segment based on the crane passing data of the current crane during the execution of the current completed handling task, so as to obtain the crane feature vector of the current crane; wherein, the crane passing data includes the passing time, average passing speed, acceleration, deceleration, and warning situation of the current crane passing through each track segment; A track feature extraction unit, configured to extract the track feature vectors of the respective track segments based on the historical track passing data of the respective track segments; the historical track passing data of any track segment includes the passing states of the respective cranes passing through this track segment; A crane anomaly detection unit, configured to fuse the crane feature vector of the current crane and the track feature vectors of the respective track segments to obtain a crane-track fusion feature, and perform fault classification based on the crane-track fusion feature to determine the probability that the current crane has an abnormal operation situation; The extracting the track feature vectors of the respective track segments based on the historical track passing data of the respective track segments includes: Sort the passing states of the respective cranes passing through any track segment in the historical track passing data of this track segment based on the passing sequence to obtain the track passing state sequence of this track segment; Sequentially input the track passing state sequence of this track segment into the long short-term memory network according to time steps to obtain the hidden vectors output by the long short-term memory network at each time step; Based on the hidden vectors output by the long short-term memory network at each time step, determine the track feature vector of this track segment.

7. The abnormal operation detection device for the overhead crane system according to claim 6, wherein The extracting the passing features of the current crane in each track segment and the spatial relationship between the passing features of the current crane in each track segment based on the crane passing data of the current crane during the execution of the current completed handling task to obtain the crane feature vector of the current crane includes: Based on the passing time, average passing speed, acceleration, deceleration, and warning situation of the current crane passing through each track segment in the crane passing data, and the neighbor track numbers of the respective track segments, construct the passing features of the current crane in the respective track segments; Construct a crane operation feature map of the current crane based on the positional relationship between the respective track segments and the passing characteristics of the current crane on the respective track segments; the crane operation feature map is a multi-channel two-dimensional image, and the channels of the crane operation feature map correspond to the dimensions of the passing characteristics of the current crane; Based on a lightweight convolutional neural network, perform feature extraction on the crane operation feature map of the current crane to obtain a crane feature vector of the current crane.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the crane system operation anomaly detection method according to any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the crane system operation anomaly detection method according to any one of claims 1 to 5.

Images