Crown block system operation abnormity detection method and device, electronic equipment and storage medium

Abstract

The invention provides a crown block system operation abnormity detection method, a crown block system operation abnormity detection device, electronic equipment and a storage medium. Extracting passage characteristics of the current crown block in each track section and a spatial relationship between the passage characteristics of the current crown block in each track section to obtain a crown block characteristic vector of the current crown block, and extracting a track characteristic vector of each track section based on historical track passage data of each track section; and fusing the crown block feature vector of the current crown block and the track feature vector of each track section to obtain a crown block track fusion feature, carrying out fault classification based on the crown block track fusion feature, and determining the probability that the current crown block has an abnormal operation condition. By combining the behavior characteristics of the crown block and the long-term operation mode of the track section, the detection accuracy of the abnormal operation of the crown block is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of overhead crane systems, and in particular, to a method, device, electronic device, and storage medium for detecting abnormal operation of an overhead crane system. Background Art

[0002] In a semiconductor factory, overhead cranes are one of the crucial devices, which are used to transport raw materials, semi-finished products, and finished products on the production line. The overhead cranes move within the factory through a precise track system, ensuring efficient material transportation and production processes. It can be seen that the operating conditions of the overhead cranes directly affect the continuity and stability of the production process. Therefore, regularly overhauling and maintaining the overhead cranes not only helps to extend the equipment life, but also reduces the risk of failures and improves the overall safety and reliability of the system.

[0003] However, the regular overhaul strategy is difficult to detect problems or potential failures that suddenly occur during the interval between two overhauls in a timely manner, and may not be able to avoid sudden failures and emergency shutdowns, thus causing temporary interruptions in the wafer handling tasks and disrupting the semiconductor preparation process. Summary of the Invention

[0004] The present invention provides a method, device, electronic device, and storage medium for detecting abnormal operation of an overhead crane system to solve the defect that the existing regular overhaul strategy is difficult to detect problems or potential failures that suddenly occur during the interval between two overhauls in a timely manner.

[0005] The present invention provides a method for detecting abnormal operation of an overhead crane system, including: Based on the overhead crane passage data of the current overhead crane during the execution of the current completed handling task, extracting the passage characteristics of the current overhead crane in each track section and the spatial relationship between the passage characteristics of the current overhead crane in each track section to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passage data includes the passage time, average passage speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section; Based on the historical track passage data of each track section, respectively extracting the track feature vectors of each track section; the historical track passage data of any track section includes the passage states of each overhead crane passing through the track section; Fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane - track fusion feature, and classifying faults based on the overhead crane - track fusion feature to determine the probability of the current overhead crane having an abnormal operation situation.

[0006] According to a method for detecting abnormal operation of a crane system provided by the present invention, based on the crane passing data of the current crane during the execution of the current completed handling task, the passing characteristics of the current crane in each track section and the spatial relationship between the passing characteristics of the current crane in each track section are extracted to obtain the crane feature vector of the current crane, including: Based on the passing time, average passing speed, acceleration, deceleration and alarm situation of the current crane passing through each track section in the crane passing data, and the neighbor track numbers of each track section, the passing characteristics of the current crane in each track section are constructed; Based on the positional relationship between each track section and the passing characteristics of the current crane in each track section, a crane operation feature map of the current crane is constructed; the crane operation feature map is a multi-channel two-dimensional image, and the channels of the crane operation feature map correspond to the dimensions of the passing characteristics of the current crane; Based on a lightweight convolutional neural network, feature extraction is performed on the crane operation feature map of the current crane to obtain the crane feature vector of the current crane.

[0007] According to a method for detecting abnormal operation of a crane system provided by the present invention, the lightweight convolutional neural network sequentially includes a first one-dimensional convolutional layer, a first one-dimensional max pooling layer, a second one-dimensional convolutional layer and a second one-dimensional max pooling layer.

[0008] According to a method for detecting abnormal operation of a crane system provided by the present invention, based on the historical track passing data of each track section, the track feature vectors of each track section are respectively extracted, including: Based on the order of passing, the passing states of each crane passing through the track section in the historical track passing data of any track section are sorted to obtain the track passing state sequence of the track section; The track passing state sequence of the track section is sequentially input into the long short-term memory network according to time steps to obtain the hidden vectors output by the long short-term memory network at each time step; Based on the hidden vectors output by the long short-term memory network at each time step, the track feature vector of the track section is determined.

[0009] According to a method for detecting abnormal operation of a crane system provided by the present invention, based on the hidden vectors output by the long short-term memory network at each time step, determining the track feature vector of the track section includes: The hidden vectors output by the long short-term memory network at each time step are respectively subjected to attention transformation with the crane feature vector of the current crane to obtain the attention weights of the hidden vectors output by the long short-term memory network at each time step; Then, based on the difference between the numerical value 1 and the attention weights of the hidden vectors output by the long short-term memory network at each time step, the screening weights of the hidden vectors output by the long short-term memory network at each time step are determined. Based on the screening weights of the hidden vectors output by the long short-term memory network at each time step, the hidden vectors output by the long short-term memory network at each time step are weighted and fused to obtain the track feature vector of this track segment.

[0010] According to an abnormal operation detection method of a crane system provided by the present invention, the hidden vector output by the long short-term memory network at any time step is determined based on the passing state of the first crane input to the long short-term memory network at the any time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first crane input to the long short-term memory network at the any time step and the second crane input to the long short-term memory network at the previous time step when entering this track segment.

[0011] The present invention also provides an abnormal operation detection device for a crane system, including: A crane feature extraction unit, configured to extract the passing features of the current crane in each track segment and the spatial relationship between the passing features of the current crane in each track segment based on the crane passing data of the current crane during the execution of the current completed handling task, so as to obtain the crane feature vector of the current crane; wherein, the crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current crane passing through each track segment. A track feature extraction unit, configured to extract the track feature vectors of the respective track segments based on the historical track passing data of the respective track segments; the historical track passing data of any track segment includes the passing states of the respective cranes passing through this track segment. A crane abnormal detection unit, configured to fuse the crane feature vector of the current crane and the track feature vectors of the respective track segments to obtain a crane-track fusion feature, and perform fault classification based on the crane-track fusion feature to determine the probability that the current crane has an abnormal operation situation.

[0012] According to an abnormal operation detection device for a crane system provided by the present invention, the extracting the passing features of the current crane in each track segment and the spatial relationship between the passing features of the current crane in each track segment based on the crane passing data of the current crane during the execution of the current completed handling task, so as to obtain the crane feature vector of the current crane includes: Based on the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current crane passing through each track segment in the crane passing data, and the neighbor track numbers of the respective track segments, the passing features of the current crane in the respective track segments are constructed. Construct a crane operation feature map of the current crane based on the positional relationship between the respective track segments and the passing characteristics of the current crane on the respective track segments; the crane operation feature map is a multi-channel two-dimensional image, and the channels of the crane operation feature map correspond to the dimensions of the passing characteristics of the current crane; Extract features from the crane operation feature map of the current crane based on a lightweight convolutional neural network to obtain a crane feature vector of the current crane.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the crane system operation anomaly detection method as described in any one of the above.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the crane system operation anomaly detection method as described in any one of the above.

[0015] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the crane system operation anomaly detection method as described in any one of the above.

[0016] The crane system operation anomaly detection method, device, electronic device, and storage medium provided by the present invention extract the passing characteristics of the current crane on each track segment and the spatial relationship between the passing characteristics of the current crane on each track segment based on the crane passing data of the current crane during the execution of the current completed handling task, obtain a crane feature vector of the current crane, and at the same time, based on the historical track passing data of each track segment, extract the track feature vectors of each track segment respectively; then fuse the crane feature vector of the current crane and the track feature vectors of each track segment to obtain a crane-track fusion feature, and classify faults based on the crane-track fusion feature to determine the probability of the current crane having an abnormal operation situation. It can accurately analyze using the passing data after the task is completed and the track data of the track segment without affecting the normal task execution of the crane. By combining the behavior characteristics of the crane itself with the long-term operation mode of the track segment, the detection accuracy of the crane operation anomaly is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart of the method for detecting abnormal operation of the overhead crane system provided by the present invention; Figure 2 It is a schematic flowchart of the method for extracting overhead crane features provided by the present invention; Figure 3 It is a schematic structural diagram of the device for detecting abnormal operation of the overhead crane system provided by the present invention; Figure 4 It is a schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0020] Since the regular maintenance strategy is difficult to detect problems or potential faults that suddenly occur during the interval between two maintenance operations of the equipment in a timely manner, sudden failures and emergency shutdowns may not be avoided, thereby causing the temporary interruption of the wafer handling task and disrupting the semiconductor manufacturing process. Therefore, it is necessary to perform abnormal detection on the overhead crane after it finishes the current handling task. When the probability of its abnormal operation is relatively high, intervene in advance to perform a safety inspection on the corresponding overhead crane to ensure that the overhead crane will not have an emergency failure during the execution of the handling task.

[0021] In view of this, the present invention provides a method for detecting abnormal operation of an overhead crane system. Figure 1 It is a schematic flowchart of the method for detecting abnormal operation of the overhead crane system provided by the present invention. As Figure 1 shown, the method includes: Step 110: Based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, extract the passing features of the current overhead crane in each track section and the spatial relationship between the passing features of the current overhead crane in each track section to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration and alarm situation of the current overhead crane passing through each track section; Step 120: Based on the historical track passing data of each track section, extract the track feature vectors of each track section respectively; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section; Step 130: Integrate the crane feature vector of the current crane and the track feature vectors of the respective track segments to obtain a crane-track integrated feature, and perform fault classification based on the crane-track integrated feature to determine the probability of abnormal operation of the current crane.

[0022] Here, after the current crane completes the current handling task, the system will collect a series of passing data related to its operation process. These passing data mainly include parameters such as the passing time, average passing speed, acceleration, and deceleration when the current crane passes through each track segment during the execution of the current completed handling task. At the same time, it also includes information such as whether an alarm occurs, the type of alarm, and the track position where the alarm occurs. These data can be obtained through sensors, control modules, or historical data recording systems deployed along the track or on the crane body to ensure the high accuracy and integrity of the passing data.

[0023] After the collection of the crane passing data is completed, it will enter the feature extraction stage of the crane itself. For the passing data of the current crane, its passing features can be extracted in units of track segments, specifically including the deviation between the passing time of the crane on a certain track segment and the historical average passing time, the fluctuation of acceleration and deceleration, the smoothness of speed change, and the density of alarm records, etc. Considering that there is a certain correlation in the operating states of the crane on adjacent tracks, especially when the crane has an abnormal operation, the operating states of the crane on the adjacent track segments it passes through all show a certain abnormality. Therefore, in some embodiments, in addition to the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current crane passing through each track segment in the crane passing data, the neighbor track numbers of each track segment can also be incorporated into the passing features of the current crane on each track segment, which helps to further extract the correlation between the passing features of adjacent track segments from the passing features of the current crane on each track segment based on the spatial relationship between the passing features, such as information on acceleration mutation, speed trend change, and continuity of abnormal states. These features can not only reflect the current operating state of the crane on a single track segment but also reflect the operating continuity and stability of the crane in the entire track system. Finally, this series of features will be integrated into a multi-dimensional crane feature vector to comprehensively represent the operation of the current crane during the completion of the current completed handling task.

[0024] In some embodiments, as Figure 2 shown, the passing features of the current crane on each track segment and the spatial relationship between the passing features of the current crane on each track segment can be extracted based on the following steps to obtain the crane feature vector of the current crane: Step 210: Based on the passing time, average passing speed, acceleration, deceleration, and warning situation of the current crane passing through each track section in the crane passing data, and the neighbor track numbers of each track section, construct the passing characteristics of the current crane in each track section. Step 220: Based on the positional relationship between each track section and the passing characteristics of the current crane in each track section, construct a crane operation characteristic map of the current crane; the crane operation characteristic map is a multi-channel two-dimensional image, and the channels of the crane operation characteristic map correspond to the dimensions of the passing characteristics of the current crane. Step 230: Based on a lightweight convolutional neural network, extract features from the crane operation characteristic map of the current crane to obtain a crane feature vector of the current crane.

[0025] Specifically, in order to more comprehensively and accurately model the characteristics of the crane operation state to improve the accuracy of subsequent anomaly detection, in this embodiment, by deeply mining the track passing behavior of the crane during a handling task and combining the spatial correlation relationship between track sections, the crane feature vector of the current crane is obtained.

[0026] Among them, after the crane completes a handling task, it will automatically call the crane passing data in this task, which records the operation information of all track sections passed by the crane from the task start point to the end point, including the passing time, average speed, acceleration, deceleration, and whether there has been an operation warning on this section. At the same time, the system will also obtain the structural information of each track section in the entire track network, that is, the neighbor track numbers of this track section, usually represented as a set of numbers of the front and rear connected track sections. Based on the passing time, average passing speed, acceleration, deceleration, and warning situation of the current crane passing through each track section, and the neighbor track numbers of each track section, construct the passing characteristics of the current crane in each track section. In some embodiments, for any track section, the passing characteristics of the current crane in this track section usually consist of the following items: actual passing time, average speed per unit length, speed difference between the start and end points (used to calculate acceleration), whether there is a deceleration process, whether an alarm is triggered, neighbor track numbers, and other optional characteristics such as track load factor, etc.

[0027] After obtaining the passing characteristics of the current overhead crane in all track sections, the spatial position relationship between the track sections will be further considered, such as the connection direction, arrangement order, or position layout on the two-dimensional plane between the track sections. Combining this structural information, an overhead crane operation feature map representing the passing path of the overhead crane is constructed. Among them, each track section in the overhead crane system can be regarded as a node to construct an undirected graph, and then the sum of the distances from each node in the undirected graph to all other nodes is calculated, so as to select the node with the smallest sum of distances as the geometric center node, and map this node to the center position of a two-dimensional image with a preset size. Subsequently, starting from the geometric center node, all neighbor nodes of the current node are mapped to the four-neighborhood of the position of the current node in the two-dimensional image according to the actual position relationship between the track corresponding to the corresponding node and the track corresponding to the current node. After establishing the mapping relationship between all nodes and the pixel points in the two-dimensional image, the passing characteristics corresponding to each node (i.e., track section) can be used as the pixel value of the corresponding pixel point to obtain the overhead crane operation feature map. It should be noted that for the track sections not passed by the current overhead crane during the handling process, their pixel values are set to preset values.

[0028] It can be seen that each pixel in the overhead crane operation feature map corresponds to a track section in the overhead crane system. For the track sections passed by the current overhead crane during the handling process, the corresponding pixel value is the passing characteristic of the current overhead crane in this track section. Therefore, the overhead crane operation feature map is a multi-channel two-dimensional image, and the channels of the overhead crane operation feature map correspond to the dimensions of the passing characteristics of the current overhead crane. Since the positions of adjacent track sections have local continuity in this two-dimensional image, it is convenient to capture the change pattern of the overhead crane operation state in space subsequently. Next, the constructed overhead crane operation feature map is input into a lightweight convolutional neural network (lightweight CNN) for processing to obtain the overhead crane feature vector of the current overhead crane. The selected lightweight convolutional network has a small parameter scale and low computational complexity to meet the real-time requirements of the overhead crane system. In some embodiments, the lightweight convolutional neural network sequentially includes a first one-dimensional convolutional layer, a first one-dimensional max pooling layer, a second one-dimensional convolutional layer, and a second one-dimensional max pooling layer.

[0029] Through this method, in this embodiment, not only the accurate modeling of the operating state of the overhead crane in the track system is achieved, but also the spatial relationship between track segments is effectively integrated. Combined with the subsequent track feature vectors, on the one hand, it helps to distinguish the anomalies of the overhead crane itself, the common anomalies caused by certain path combinations of the track system (i.e., the abnormal behaviors caused by certain specific track path combinations, which are not caused by the anomalies of a single track segment or the overhead crane, but are jointly caused by the combination order, structural connection method or operating habits of these track segments), and the anomalies of track segments. In addition, due to the use of a lightweight convolutional neural network as the core model, this solution greatly improves the processing efficiency and system response speed while ensuring the detection accuracy.

[0030] At the same time, the embodiment of the present invention will also retrieve the historical track passing data of each track segment stored in the background database, and extract the long-term operating characteristics of each track segment respectively. Among them, the historical track passing data usually includes the passing states of multiple overhead cranes passing through this track segment at different times, including their speed distribution and passing time distribution, etc. Based on these historical track passing data, a track feature vector describing the operating mode of each track segment can be constructed. After obtaining the overhead crane feature vector of the current overhead crane and the track feature vectors of all track segments, these two types of information can be fused to obtain the overhead crane-track fusion feature. The fusion process can adopt methods such as feature splicing, weighted combination, and vector cross, and the embodiment of the present invention does not make specific limitations on this. This overhead crane-track fusion feature not only reflects the operating characteristics of the overhead crane itself, but also reflects the potential abnormal tendency of its operating environment.

[0031] In some embodiments, when extracting the track feature vectors of each track segment, in order to introduce time dynamic characteristics, the passing states of each overhead crane passing through this track segment in the historical track passing data of any track segment can be sorted based on the passing sequence to obtain the track passing state sequence of this track segment. Subsequently, the track passing state sequence of this track segment is sequentially input into the long short-term memory network according to time steps, and the hidden vectors output by the long short-term memory network at each time step are obtained, and based on the hidden vectors output by the long short-term memory network at each time step, the track feature vector of this track segment is determined. Among them, the long short-term memory network can effectively model the long-term dependence relationships existing in the track passing state sequence and learn the state evolution of the track segment at different time periods.

[0032] Based on the overall distribution of the hidden vectors at each time step, the features therein can be further fused to form the track feature vector of this track segment. In some embodiments, the hidden vectors output by the long short-term memory network at each time step can be respectively subjected to an attention transformation with the crane feature vector of the current crane to obtain the attention weights of the hidden vectors output by the long short-term memory network at each time step. Then, based on the difference between the value 1 and the attention weights of the hidden vectors output by the long short-term memory network at each time step, the screening weights of the hidden vectors output by the long short-term memory network at each time step are determined. It should be noted here that the special design of calculating the difference between the value 1 and the attention weights as the screening weights of the corresponding hidden vectors is to retain the features with a relatively low correlation with the crane feature vector. The reason is that the possibility of the coexistence of crane anomalies and track anomalies is relatively low, and the manifestations of crane anomalies and track anomalies are different. If it is a crane anomaly, the hidden vectors extracted by the long short-term memory network should be different from the crane feature vector because the passing data of other cranes is normal; if it is a track anomaly, the crane feature vector should also be different from the hidden vectors extracted by the long short-term memory network because the passing data of this crane is normal. Subsequently, the hidden vectors output by the long short-term memory network at each time step are weighted and fused based on the screening weights of the hidden vectors output by the long short-term memory network at each time step to obtain the track feature vector of this track segment.

[0033] In some other embodiments, in order to more accurately extract the operation mode of the track segment, the passing state of the first crane input to the long short-term memory network at any time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first crane input to the long short-term memory network at this time step and the second crane input to the long short-term memory network at the previous time step entering this track segment can be used to determine the hidden vector output by the long short-term memory network at this time step. That is, at any time step, the inputs to the long short-term memory network include three: the passing state of the first crane, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first crane and the second crane input to the long short-term memory network at the previous time step entering this track segment. Here, considering that the operation states of the cranes entering this track segment successively within a short period of time will interfere with each other. For example, if the previous crane brakes, etc., it will also affect the running speed of the crane following closely behind, but this is not caused by the track segment. Therefore, by additionally introducing the time difference between the first crane input to the long short-term memory network at this time step and the second crane input to the long short-term memory network at the previous time step entering this track segment, it can help the long short-term memory network learn this operation mode and contribute to distinguishing operation interference from track segment anomalies.

[0034] Subsequently, based on the crane rail fusion features, fault classification is performed to determine the probability of abnormal operation of the current crane. Here, a fault detection model can be constructed based on traditional machine learning algorithms (such as support vector machines, random forests, etc.) or based on deep neural network structures such as convolutional neural networks (CNNs), graph neural networks (GNNs), etc. The model analyzes and judges the crane rail fusion features and outputs the probability value indicating whether there is abnormal operation of the current crane during the execution of the current task. The higher this probability value, the greater the likelihood of abnormal operation of the crane. If the probability of abnormal operation of the current crane is higher than the preset threshold, it can be considered that the crane may be in an abnormal state, and corresponding emergency handling mechanisms are triggered. These mechanisms can include: automatically sending abnormal alarm information to the monitoring platform, pausing the scheduling task of the crane, recording the current operation log for subsequent retrospective analysis, and so on.

[0035] Through the above method, the crane system operation anomaly detection method proposed in the embodiment of the present invention extracts the passing characteristics of the current crane in each track section and the spatial relationship between the passing characteristics of the current crane in each track section based on the crane passing data during the execution of the current completed handling task by the current crane, obtains the crane feature vector of the current crane, and at the same time extracts the track feature vectors of each track section based on the historical track passing data of each track section; then fuses the crane feature vector of the current crane and the track feature vectors of each track section to obtain the crane rail fusion feature, and performs fault classification based on the crane rail fusion feature to determine the probability of abnormal operation of the current crane. It can perform accurate analysis using the passing data after the task is completed and the track data of the track section without affecting the normal task execution of the crane. By combining the behavior characteristics of the crane itself and the long-term operation mode of the track section, the detection accuracy of crane operation anomalies is improved.

[0036] Next, a description is given of the crane system operation anomaly detection device provided by the present invention. The crane system operation anomaly detection device described below can be mutually referred to the crane system operation anomaly detection method described above.

[0037] Based on any of the above embodiments, Figure 3 is a schematic structural diagram of the crane system operation anomaly detection device provided by the present invention. As Figure 3 shown, the device includes: A crane feature extraction unit 310, configured to extract the passing characteristics of the current crane in each track section and the spatial relationship between the passing characteristics of the current crane in each track section based on the crane passing data during the execution of the current completed handling task by the current crane, and obtain the crane feature vector of the current crane; wherein, the crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current crane passing through each track section. An orbit feature extraction unit 320 is configured to extract orbit feature vectors of each of the orbit segments respectively based on the historical orbit passing data of each of the orbit segments; the historical orbit passing data of any one orbit segment includes the passing states of each overhead crane passing through the orbit segment. An overhead crane anomaly detection unit 330 is configured to fuse the overhead crane feature vector of the current overhead crane and the orbit feature vectors of each of the orbit segments to obtain an overhead crane-orbit fusion feature, and perform fault classification based on the overhead crane-orbit fusion feature to determine the probability that the current overhead crane has an abnormal operation condition.

[0038] The device provided by the embodiment of the present invention extracts the passing features of the current overhead crane in each orbit segment and the spatial relationship between the passing features of the current overhead crane in each orbit segment based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, so as to obtain the overhead crane feature vector of the current overhead crane. At the same time, based on the historical orbit passing data of each orbit segment, the orbit feature vectors of each orbit segment are extracted respectively. Then, the overhead crane feature vector of the current overhead crane and the orbit feature vectors of each orbit segment are fused to obtain an overhead crane-orbit fusion feature, and fault classification is performed based on the overhead crane-orbit fusion feature to determine the probability that the current overhead crane has an abnormal operation condition. It can perform accurate analysis by using the passing data after the task is completed and the orbit data of the orbit segment without affecting the normal task execution of the overhead crane. By combining the behavior characteristics of the overhead crane itself and the long-term operation mode of the orbit segment, the detection accuracy of the abnormal operation of the overhead crane is improved.

[0039] Based on any of the above embodiments, the extracting the passing features of the current overhead crane in each orbit segment and the spatial relationship between the passing features of the current overhead crane in each orbit segment based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, so as to obtain the overhead crane feature vector of the current overhead crane includes: Based on the passing time, average passing speed, acceleration, deceleration and alarm situation of the current overhead crane passing through each orbit segment in the overhead crane passing data, and the neighbor orbit numbers of each orbit segment, construct the passing features of the current overhead crane in each orbit segment. Based on the positional relationship between each orbit segment and the passing features of the current overhead crane in each orbit segment, construct an overhead crane operation feature map of the current overhead crane; the overhead crane operation feature map is a multi-channel two-dimensional image, and the channels of the overhead crane operation feature map correspond to the dimensions of the passing features of the current overhead crane. Based on a lightweight convolutional neural network, perform feature extraction on the overhead crane operation feature map of the current overhead crane to obtain the overhead crane feature vector of the current overhead crane.

[0040] Based on any of the above embodiments, the lightweight convolutional neural network sequentially includes a first one-dimensional convolutional layer, a first one-dimensional max pooling layer, a second one-dimensional convolutional layer, and a second one-dimensional max pooling layer.

[0041] Based on any of the above embodiments, extracting the track feature vectors of each track segment based on the historical track passing data of each track segment respectively includes: Sorting the passing states of each overhead crane passing through a track segment in the historical track passing data of any track segment based on the order of passing to obtain the track passing state sequence of this track segment; Sequentially inputting the track passing state sequence of this track segment into the long short-term memory network according to time steps to obtain the hidden vectors output by the long short-term memory network at each time step; Based on the hidden vectors output by the long short-term memory network at each time step, determine the track feature vector of this track segment.

[0042] Based on any of the above embodiments, determining the track feature vector of this track segment based on the hidden vectors output by the long short-term memory network at each time step includes: Performing an attention transformation on the hidden vectors output by the long short-term memory network at each time step respectively with the overhead crane feature vector of the current overhead crane to obtain the attention weights of the hidden vectors output by the long short-term memory network at each time step; Then, based on the difference between the value 1 and the attention weights of the hidden vectors output by the long short-term memory network at each time step, determine the screening weights of the hidden vectors output by the long short-term memory network at each time step; Based on the screening weights of the hidden vectors output by the long short-term memory network at each time step, perform weighted fusion on the hidden vectors output by the long short-term memory network at each time step to obtain the track feature vector of this track segment.

[0043] Based on any of the above embodiments, the hidden vector output by the long short-term memory network at any time step is determined based on the passing state of the first overhead crane input to the long short-term memory network at this time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first overhead crane input to the long short-term memory network at this time step and the second overhead crane input to the long short-term memory network at the previous time step entering this track segment.

[0044] Figure 4 is a schematic structural diagram of the electronic device provided by the present invention, as Figure 4As shown in the figure, the electronic device may include: a processor 410, a memory 420, a communications interface 430, and a communication bus 440. Among them, the processor 410, the memory 420, and the communication interface 430 complete communication with each other through the communication bus 440. The processor 410 may call the logical instructions in the memory 420 to execute the abnormal detection method for the overhead crane system. The method includes: based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section; based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section; fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain the overhead crane track fusion feature, and based on the overhead crane track fusion feature, performing fault classification to determine the probability of the current overhead crane having an abnormal operation situation.

[0045] In addition, when the logical instructions in the above-mentioned memory 420 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0046] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the overhead crane system operation anomaly detection method provided by the above-mentioned various methods. The method includes: based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section, to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section; based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section; fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain an overhead crane-track fusion feature, and based on the overhead crane-track fusion feature, performing fault classification to determine the probability that the current overhead crane has an operation anomaly situation.

[0047] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the overhead crane system operation anomaly detection method provided by the above-mentioned various methods. The method includes: based on the overhead crane passing data of the current overhead crane during the execution of the current completed handling task, extracting the passing characteristics of the current overhead crane in each track section and the spatial relationship between the passing characteristics of the current overhead crane in each track section, to obtain the overhead crane feature vector of the current overhead crane; wherein, the overhead crane passing data includes the passing time, average passing speed, acceleration, deceleration, and alarm situation of the current overhead crane passing through each track section; based on the historical track passing data of each track section, respectively extracting the track feature vectors of each track section; the historical track passing data of any track section includes the passing states of each overhead crane passing through the track section; fusing the overhead crane feature vector of the current overhead crane and the track feature vectors of each track section to obtain an overhead crane-track fusion feature, and based on the overhead crane-track fusion feature, performing fault classification to determine the probability that the current overhead crane has an operation anomaly situation.

[0048] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0049] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting abnormal operation of an overhead crane system, characterized in that: include: Based on the overhead crane travel data of the current overhead crane in the process of executing the currently completed handling task, the travel characteristics of the current overhead crane in each track segment and the spatial relationship between the travel characteristics of the current overhead crane in each track segment are extracted to obtain the overhead crane feature vector of the current overhead crane; wherein the overhead crane travel data includes the travel time, average travel speed, acceleration, deceleration and alarm status of the current overhead crane passing through each track segment; Based on the historical track traffic data of each track segment, the track feature vectors of each track segment are extracted respectively; the historical track traffic data of any track segment includes the traffic status of each overhead crane passing through the track segment; The crane feature vector of the current crane and the track feature vectors of the track segments are integrated to obtain the crane track fusion feature, and fault classification is performed based on the crane track fusion feature to determine the probability of abnormal operation of the current crane.

2. The method for detecting abnormal operation of an overhead crane system according to claim 1, characterized in that: The method of extracting the passage characteristics of the current overhead crane in each track segment and the spatial relationship between the passage characteristics of the current overhead crane in each track segment based on the overhead crane passage data of the current overhead crane in the process of executing the currently completed handling task, and obtaining the overhead crane feature vector of the current overhead crane, includes: Based on the travel time, average travel speed, acceleration, deceleration and alarm status of the current overhead crane passing through each track segment in the overhead crane travel data, and the neighbor track number of each track segment, construct the travel characteristics of the current overhead crane in each track segment; Based on the positional relationship between the various track segments and the passage characteristics of the current overhead crane on the various track segments, a crane operation characteristic map of the current overhead crane is constructed; the crane operation characteristic map is a multi-channel two-dimensional image, and the channels of the crane operation characteristic map correspond to the dimensions of the passage characteristics of the current overhead crane; Based on the lightweight convolutional neural network, feature extraction is performed on the crane operation feature map of the current crane to obtain the crane feature vector of the current crane.

3. The method for detecting abnormal operation of an overhead crane system according to claim 2, characterized in that: The lightweight convolutional neural network sequentially includes a first one-dimensional convolutional layer, a first one-dimensional maximum pooling layer, a second one-dimensional convolutional layer, and a second one-dimensional maximum pooling layer.

4. The method for detecting abnormal operation of an overhead crane system according to claim 1, characterized in that: The extracting the track feature vectors of the respective track segments based on the historical track traffic data of the respective track segments respectively comprises: Based on the order of passage, the passage states of each overhead crane passing through any track section in the historical track passage data of the track section are sorted to obtain the track passage state sequence of the track section; Inputting the track traffic state sequence of the track segment into the long short-term memory network in turn according to the time step, and obtaining the hidden vector output by the long short-term memory network at each time step; Based on the hidden vector output by the long short-term memory network at each time step, a track feature vector of the track segment is determined.

5. The method for detecting abnormal operation of an overhead crane system according to claim 4, characterized in that: The step of determining the track feature vector of the track segment based on the hidden vector output by the long short-term memory network at each time step includes: Performing attention transformation on the hidden vector output by the long short-term memory network at each time step and the crane feature vector of the current crane, respectively, to obtain the attention weight of the hidden vector output by the long short-term memory network at each time step; Then, based on the difference between the value 1 and the attention weight of the hidden vector output by the long short-term memory network at each time step, the screening weight of the hidden vector output by the long short-term memory network at each time step is determined; Based on the screening weights of the hidden vectors output by the long short-term memory network at each time step, the hidden vectors output by the long short-term memory network at each time step are weightedly fused to obtain the track feature vector of the track segment.

6. The method for detecting abnormal operation of an overhead crane system according to claim 4, characterized in that: The hidden vector output by the long short-term memory network at any time step is determined based on the traffic status of the first train input into the long short-term memory network at any time step, the hidden vector output by the long short-term memory network at the previous time step, and the time difference between the first train input into the long short-term memory network at any time step and the second train input into the long short-term memory network at the previous time step entering the track segment.

7. A device for detecting abnormal operation of an overhead crane system, characterized in that: include: The overhead crane feature extraction unit is used to extract the passage characteristics of the current overhead crane in each track segment and the spatial relationship between the passage characteristics of the current overhead crane in each track segment based on the overhead crane passage data of the current overhead crane in the process of executing the currently completed handling task, so as to obtain the overhead crane feature vector of the current overhead crane; wherein the overhead crane passage data includes the passage time, average passage speed, acceleration, deceleration and alarm status of the current overhead crane passing through each track segment; A track feature extraction unit, configured to extract track feature vectors of each track segment based on historical track traffic data of each track segment; the historical track traffic data of any track segment includes the traffic status of each overhead crane passing through the track segment; The crane anomaly detection unit is used to fuse the crane feature vector of the current crane and the track feature vectors of each track segment to obtain the crane track fusion feature, and perform fault classification based on the crane track fusion feature to determine the probability of the current crane having an abnormal operation.

8. The device for detecting abnormal operation of an overhead travelling vehicle system according to claim 7, characterized in that: The method of extracting the passage characteristics of the current overhead crane in each track segment and the spatial relationship between the passage characteristics of the current overhead crane in each track segment based on the overhead crane passage data of the current overhead crane in the process of executing the currently completed handling task, and obtaining the overhead crane feature vector of the current overhead crane, includes: Based on the travel time, average travel speed, acceleration, deceleration and alarm status of the current overhead crane passing through each track segment in the overhead crane travel data, and the neighbor track number of each track segment, construct the travel characteristics of the current overhead crane in each track segment; Based on the positional relationship between the various track segments and the passage characteristics of the current overhead crane on the various track segments, a crane operation characteristic map of the current overhead crane is constructed; the crane operation characteristic map is a multi-channel two-dimensional image, and the channels of the crane operation characteristic map correspond to the dimensions of the passage characteristics of the current overhead crane; Based on the lightweight convolutional neural network, feature extraction is performed on the crane operation feature map of the current crane to obtain the crane feature vector of the current crane.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting abnormal operation of an overhead crane system as claimed in any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting abnormal operation of an overhead crane system as claimed in any one of claims 1 to 6 is implemented.

Images