Strip threading control method, system, equipment and medium for unwinder of strip processing line

Abstract

The present invention discloses a method, system, device and medium for controlling the threading of a coiler for a strip processing line. The method comprises obtaining observation samples, pre-processing them and then re-sampling them, extracting the operation characteristics of the threading of the coiler as an observation sample set; using the observation sample set to train a feature encoder to extract expert operation characteristic data of the threading of the coiler; constructing an automatic threading neural network model for the coiler; calculating the best matching method for migrating the extracted expert operation characteristic data to the automatic threading neural network, and using the matching value of the matching method as a reward function for training the automatic threading neural network model for the coiler; using the reward function to train the automatic threading neural network model for the coiler; using the expert operation characteristics as the basic strategy of the automatic threading neural network model for iteration, learning the optimal threading operation strategy for the coiler, and obtaining the optimal threading operation. The system comprises a module corresponding to the method, and the device comprises a unit corresponding to the method.

Description

Technical Field

[0001] The present invention relates to the technical field of steel smelting equipment, and in particular to a method, system, equipment and medium for controlling the threading of a coiler of a strip processing line. Background Art

[0002] Uncoiler is a commonly used equipment in the steel, metallurgy and other industries, used to unwind coils. Uncoiler threading is the operation of connecting the coil to the uncoiler for unwinding. Uncoiler threading is a complex and extremely delicate process in the strip processing line. At present, most of the automatic control systems of uncoiler threading processing lines require manual on-site operation to complete this process, but manual operation is inefficient and there are safety hazards in the threading process.

[0003] In view of the problem that manual on-site threading of the uncoiler is inefficient and poses a safety hazard, many manufacturers have developed auxiliary threading devices to realize automatic threading of the uncoiler. However, the auxiliary threading device is usually composed of complex mechanical components and hydraulic systems. The complex mechanical components and hydraulic systems will bring many unpredictable problems to the threading operation, such as equipment failure, mechanical debugging, and jamming during threading. In the event of abnormal threading, manual intervention is required to handle it, and multiple groups of related operators are required to collaborate to complete the threading process, which is time-consuming and labor-intensive, with high labor costs and low threading efficiency. In particular, when the mechanical components are deformed by impact or the hydraulic system is abnormal, additional manual processing is required. Moreover, manual threading is heavily dependent on the operating experience of the staff. When the shape of the coil being threaded changes greatly, the threading efficiency will be affected. In particular, when the produced coil is a medium and thick plate, due to the large stress of the coil, it will also pose a serious potential threat to the personal safety of on-site staff. Summary of the invention

[0004] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a method, system, equipment and medium for controlling the threading of a strip machine of a strip processing line. By using a trained automatic threading neural network model to automatically thread the strip of the uncoiler, the automatic threading of the strip of the uncoiler can be achieved without changing the existing production equipment. The threading result is accurate, the threading efficiency is improved, the safety of the operators is guaranteed, and the threading cost of the uncoiler is reduced.

[0005] The technical solution of the present invention is as follows:

[0006] In a first aspect, the present invention provides a strip threading control method for a decoiler of a strip processing line, comprising:

[0007] Acquiring observation samples, wherein the observation samples include operation data of operators during the threading process of the uncoiler and motion data of the uncoiler;

[0008] Preprocessing the acquired observation samples;

[0009] Resample the preprocessed observation samples and extract the operation characteristics of the unwinder threading as the observation sample set;

[0010] Using the observation sample set to train a feature encoder offline in a self-learning manner, so that the feature encoder extracts expert operation feature data of the unwinder threading;

[0011] Construct a neural network model for automatic tape threading of the uncoiler, which adopts a deep deterministic policy gradient algorithm network structure;

[0012] Calculate the best matching method for migrating the extracted expert operation feature data to the automatic threading neural network, and use the matching value of the best matching method as the reward function for the training operation of the automatic threading neural network model of the unwinder;

[0013] The reward function is used to train the neural network model for automatic threading of the unwinder;

[0014] The expert operation characteristics are used as the basic strategy of the automatic threading neural network model for iteration, and the optimal threading operation strategy of the uncoiler is learned to obtain the optimal threading operation of the uncoiler.

[0015] Furthermore, in the above-mentioned uncoiler threading control method applicable to a strip processing line, preprocessing the acquired observation samples includes:

[0016] The operation data of the operator during the uncoiler threading process and the action data of the uncoiler are time-series aligned according to the collection time.

[0017] Furthermore, in the above-mentioned uncoiler threading control method applicable to the strip processing line, resampling is performed in the preprocessed observation samples, and the operation characteristics of the operator of the uncoiler threading are extracted as the observation sample set, including:

[0018] At time t, the image frame data of the video stream data in the observation sample is v t , the equipment action status data and monitoring status data are combined into s t , the device operation action data is a t , then after the time series is aligned, the observed sample data (v t ,s t ,a t ), to generate observation sample data (v t ,s t ,a t ) resamples at time intervals to obtain the observed sample data set D = {(v t ,s t ,a t), where time t can be replaced by any time.

[0019] Furthermore, in the above-mentioned strip threading control method for a coiler of a strip processing line, offline training of a feature encoder by self-learning using the observation sample set includes:

[0020] The feature encoder includes a first deep neural network model and a second deep neural network model that are independent of each other. The observation sample set is fed into the first deep neural network model and the second deep neural network model for training, and the image frame data in the observation sample set is converted into latent variables in a low-dimensional vector space, wherein:

[0021] The first deep neural network model is trained using a noise contrast loss-based loss function, where the noise contrast loss-based loss function is set to:

[0022]

[0023] in, represents the loss function based on noise contrast loss, -E D [·] represents the mathematical expectation of the relevant statistics calculated for the observed sample set D, e represents the base of the natural logarithm function, z t Indicates that the image frame data in the observation sample set is converted into latent variables in the low-dimensional vector space. represents the extracted positive example latent variable, i represents the number of samples in the sample data set, and n represents the size of the sample data set;

[0024] The second deep neural network model is trained using a loss function based on average loss, and the loss function based on average loss is set to:

[0025]

[0026] in, represents the loss function based on the average loss, s t represents the action status data and monitoring status data of the uncoiler at time t, s t+k represents the action state data and monitoring state data of the uncoiler at time t+k, where k represents a settable coefficient, which is used to adjust the time granularity of the second deep neural network model's perception of changes in the equipment action state data and monitoring state data;

[0027] Among them, the loss function based on noise contrast loss and the loss function based on average loss form a combined loss function, and the combined loss function includes:

[0028]

[0029] Lf represents the combined loss function formed by the loss function based on noise contrast loss and the loss function based on average loss, and λ represents the loss function based on noise contrast loss. The proportional coefficient is a real number, and its value is greater than 0 and less than 1.

[0030] Furthermore, in the above-mentioned unwinder threading control method applicable to the strip processing line, the best matching method of calculating the extracted expert operation feature data to migrate to the automatic threading neural network, and using the matching value of the best matching method as the reward function of the automatic threading neural network training operation strategy includes:

[0031] Through self-supervised learning, the image frame data in the sample data observed at any time is converted into latent variables in the low-dimensional vector space to obtain the expert operation feature set;

[0032] The positive samples in the expert operation feature set are selected as basic samples to train the automatic threading neural network model of the uncoiler online, and the optimal operation of the automatic threading of the uncoiler is obtained.

[0033] The implicit action representation in the extracted observation sample data and the action representation in the expert threading operation feature set;

[0034] Comparing the extracted action representations, obtaining the best fitting method between the implicit action representation corresponding to the image frame data in the observed sample data and the action representation in the expert operation feature set;

[0035] The matching value corresponding to the best fitting method is used as a reward function to train the automatic threading neural network model of the unwinder;

[0036] The reward function is determined by the following formula:

[0037]

[0038] Among them, r OT (z t ) represents the best matching value of the extracted expert operation feature data transferred to the automatic threading neural network, j represents the column index of the transmission cost matrix c, n represents the number of action representation samples during the expert threading operation, c tj Represented as the t-th row and j-th column element of the transmission cost matrix c, represents the matrix of best fit values.

[0039] Furthermore, in the above-mentioned uncoiler threading control method applicable to the strip processing line, the uncoiler automatic threading neural network model adopts a deep deterministic policy gradient algorithm network structure, and the deep deterministic policy gradient algorithm network structure includes the loss function of two Critic networks and the loss function of two Actor networks, wherein:

[0040] The loss functions of the two critic networks in the deep deterministic policy gradient algorithm network structure used in the automatic tape threading neural network model of the unwinder are set as:

[0041] L θ1 =E (z,a′)～D [(Q θ1 (z′,s′,a′)-y) 2 ];

[0042] L θ2 =E (z,a′)～D [(Q θ2 (z′,s′,a′)-y) 2 ];

[0043] Among them, L θ1 Represents the current network loss function of Critic, L θ2 represents the Critic target network loss function, z represents the current state, a′ represents the action taken on the current state, D represents the training buffer for online training of the deep deterministic policy gradient algorithm network structure, E (z,a′)～D It represents the expectation in the training buffer process D after calculating the current state z and taking action a′; Q θ1 and Q θ2 Respectively represent the action value, z′, s′, a′ respectively represent the threading action representation state at the next moment, the uncoiler equipment state monitored at the next moment, and the action taken for the threading action state at the next moment, y is the target value of the deep deterministic policy gradient algorithm network structure, where the target value y of the deep deterministic policy gradient algorithm network structure is calculated by the following formula. z represents the current threading action representation state, a′ represents the action taken for the current state, D represents the training buffer for online training of the deep deterministic policy gradient algorithm network structure, E (z,a′)～D It represents the expectation in the training buffer process D after calculating the current state z and taking action a′; Q θ1 and Q θ2 Respectively represent the action value, z′, s′, a′ respectively represent the threading action representation state at the next moment, the equipment state (including video data) of the unwinder monitored at the next moment, and the action taken for the threading action state at the next moment, y represents the target value of the deep deterministic policy gradient algorithm network structure, where the target value y of the deep deterministic policy gradient algorithm network structure is calculated by the following formula:

[0044]

[0045] Among them, γ represents the reinforcement learning discount coefficient, i represents the training sample index, n represents the size of the training sample set, and rt+i represents the reward function at time t+i, minQ θ1 Represents the action value Q θ1 Take the minimum value, z′ t+n ,s′ t+n ,a′ t+n They respectively represent the threading action status at time t+n, the status of the uncoiler and other equipment monitored at time t+n, and the action taken by the threading action status at time t+n, minQ θ2 Denotes the action value Q θ2 Take the minimum value;

[0046] The loss function of the two identical Actor networks in the deep deterministic policy gradient algorithm network structure is set to:

[0047]

[0048] in, Represents the loss function of the Actor network in the deep deterministic policy gradient algorithm network structure.

[0049] Furthermore, in the above-mentioned uncoiler threading control method applicable to the strip processing line, the expert operation characteristics are used as the basic strategy of the automatic threading neural network model for iteration, and the optimal threading operation strategy of the uncoiler is learned. The optimal threading operation of the uncoiler includes:

[0050] The expert operation feature data of the unwinder threading is extracted by the feature encoder as the input of the Actor network and the Critic network. During the iteration of the automatic threading neural network model, the expert operation feature data is extracted by the feature encoder from the observed sample data.

[0051] After the extracted expert operation feature data is input into the Actor network for processing, the uncoiler threading action data is output, and after the uncoiler threading action data is executed, new uncoiler status data is generated;

[0052] The new state data generated is compared through the Critic network, and the weight parameters of the Actor network and the Critic network and the current threading operation are updated according to the reward function until the optimal operation that is close to the expert operation characteristics is achieved.

[0053] In a second aspect, the present invention further provides a strip threading control system for a coiler of a strip processing line, comprising:

[0054] An acquisition module, the acquisition module is used to acquire observation samples, the acquired observation samples include operation data of operators during the threading process of the uncoiler and action data of the uncoiler;

[0055] A preprocessing module, wherein the preprocessing module is used to preprocess the acquired observation samples;

[0056] A feature extraction module, wherein the feature extraction module is used to resample the preprocessed observation samples and extract the operation features of the unwinder threading as an observation sample set;

[0057] An offline training module, wherein the offline training module is used to use the observation sample set to train the feature encoder offline in a self-learning manner, so that the feature encoder extracts expert operation feature data of the unwinder threading;

[0058] A network model building module, wherein the network model building module is used to build a neural network model for automatic tape threading of an unwinder;

[0059] A calculation module, the calculation module is used to calculate the best matching method for migrating the extracted expert operation feature data to the automatic threading neural network, and use the matching value of the best matching method as a reward function for the automatic threading neural network training operation strategy;

[0060] A training module, wherein the training module is used to train an automatic threading neural network model of an unwinder using a reward function;

[0061] The iterative module is used to iterate based on the expert operation characteristics as the basic strategy of the automatic threading neural network model, learn the optimal threading operation strategy of the uncoiler, and obtain the optimal threading operation of the uncoiler.

[0062] In a third aspect, the present invention further provides a strip threading control device for a uncoiler of a strip processing line, comprising:

[0063] A collection unit, the collection unit is used to collect the operation data of the operator during the threading process of the uncoiler and the action data of the uncoiler;

[0064] A processing unit, the processing unit is used to pre-process the data collected by the collection unit;

[0065] A control unit is provided in which an automatic tape-threading neural network model and a feature encoder are configured. The control unit is used for offline training of the operation feature encoder by self-supervised learning so that the feature encoder extracts the expert operation feature data of the tape-threading of the uncoiler; at the same time, the control unit is used for calculating the best matching mode of migrating the extracted expert operation feature data to the automatic tape-threading neural network model, and using the matching value of the two as the reward function of the automatic tape-threading neural network training operation strategy; and using the sample data and the reward function to perform unsupervised training on the automatic tape-threading neural network model of the uncoiler, and at the same time, the control unit uses the expert operation feature as a basis to iterate the uncoiler tape-threading neural network model, so that the automatic tape-threading neural network model outputs the optimal tape-threading operation of the uncoiler.

[0066] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and one or more of the programs can be executed by one or more processors to implement the control method as described above.

[0067] The main advantages of the technical solution of the present invention are as follows:

[0068] The present invention is suitable for the uncoiler tape threading control method of the strip processing line, which uses the obtained optimal tape threading operation of the uncoiler as the tape threading action of the uncoiler in the automatic control system of the strip processing line, and uses the trained automatic tape threading neural network model to perform automatic tape threading of the uncoiler. Under the premise of not changing the existing production equipment, the automatic tape threading of the uncoiler can be realized, the tape threading result is accurate, the safety of the operating personnel is guaranteed, the tape threading cost of the uncoiler is reduced, and the tape threading efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0070] Figure 1 It is a flow chart of a strip threading control method for an uncoiler applicable to a strip processing line according to an embodiment of the present invention;

[0071] Figure 2 A schematic flow chart of a method for controlling the threading of a coiler of a strip processing line applicable to an embodiment of the present invention, wherein the extracted expert operation feature data is transferred to the best matching mode of an automatic threading neural network, and the matching value of the best matching mode is used as a reward function for the training operation strategy of the automatic threading neural network;

[0072] Figure 3A schematic diagram of a process for iterating an automatic threading neural network model using expert operation feature data in a threading control method for a coiler applicable to a strip processing line according to an embodiment of the present invention to obtain an optimal threading operation of the coiler;

[0073] Figure 4 A schematic diagram of a strip threading control device for an uncoiler of a strip processing line provided by an embodiment of the present invention;

[0074] Figure 5 A schematic diagram of a strip threading control system for a decoiler of a strip processing line provided in one embodiment of the present invention.

[0075] Description of reference numerals:

[0076] 10. Acquisition module; 20. Preprocessing module; 30. Feature extraction module; 40. Offline training module; 50. Network model building module; 60. Calculation module; 70. Training module; 80. Iteration module;

[0077] 100, acquisition unit; 200, data processing unit; 300, control unit. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0079] The following is combined with Figure 1-5 , describes in detail the technical solution provided by the embodiments of the present invention.

[0080] First of all, it should be explained that the uncoiler in the present invention refers to a special equipment used for unwinding strip steel coils in a strip processing line, and is suitable for the uncoiler threading control of a coil processing production line, wherein the above-mentioned coil refers to continuously rolled steel. Therefore, the uncoiler threading control method applicable to a strip processing line of the present invention is particularly suitable for the uncoiler threading control of a strip processing production line. In addition, the uncoiler threading of the present invention is not only suitable for steel coils or strip steel, but also suitable for the uncoiler threading control of metal coils such as copper coils and aluminum coils. The present invention takes the uncoiler threading control applicable to a strip processing line as an example to illustrate the technical solution of the present invention.

[0081] In order to more clearly describe the strip threading process of the uncoiler of the present invention, the scenario of the strip threading process on the uncoiler is briefly described by taking the scenario of the strip threading from the uncoiler to the entrance in the strip processing line as an example in combination with the actual strip threading operation of the uncoiler:

[0082] After the steel coil is rolled up by the rolling system, it is placed on the uncoiler drum. To guide the strip from the uncoiler to the shear at the entrance of the processing line, it is necessary to first determine the position of the strip head on the drum (under normal circumstances, the steel coil is automatically rolled up by the steel coil trolley after being unbundled. Therefore, in order to ensure that the steel coil is not loose, the strip head is located above the support roller on the steel coil trolley and is squeezed by the support roller. In this way, after rolling up, the strip head is located at 5 o'clock or 7 o'clock below the drum). When the strip head is below the drum, it is necessary to rotate the uncoiler drum clockwise to adjust the strip head position to the 10 o'clock position of the drum (in this process, the rotation direction and adjustment position of the drum are actually related to the unwinding method of the uncoiler. The above unwinding is taken as an example in this embodiment). When adjusting the position of the strip head, the uncoiler pressure roller needs to keep pressing down; after adjusting the position of the strip head, it is necessary to lift the uncoiler guide plate to the appropriate position and extend it out. The actual extended position may be due to The coil diameters of steel coils vary, which requires staff to judge. After the uncoiler guide plate is adjusted to the appropriate position, the uncoiler drum motor is driven to convey the strip forward. During this process, since the strip is unfolded from a curled state, it will have a large curvature, and the uncoiler bending roller is required to initially straighten the strip. The specific degree of pressing down requires staff to judge. After the initial straightening is completed, the strip enters the pinch roller, which is driven by the pinch roller motor and the uncoiler drum motor to convey the strip forward at the same time. Thereafter, the strip passes through the side guide device, which adjusts the center position of the strip in the width direction through a centering action and then sends it to the straightener. After the strip enters the straightener for a certain length (the specific length can be set as a parameter), the straightener is pressed down and the strip head is flattened by reverse straightening (motor driven). The degree of flattening requires on-site observation and judgment by staff. After the flattening is completed, the strip enters the entrance shearing machine area and the strip threading is completed.

[0083] According to the above-described threading operation process, it can be found that during the entire threading process, the staff needs to always pay attention to whether the strip head is stuck in the processing line channel due to bending, and also need to pay attention to the steel coil on the uncoiler to prevent it from loosening, and ensure that the strip in the conveying part maintains a certain tension state.

[0084] To this end, an embodiment of the present invention provides a method for controlling the threading of a coiler of a strip processing line to realize the threading control of the coiler. The core idea includes: forming an offline training sample based on the video information of the threading process of the on-site staff operating the coiler obtained by the visual system of the strip processing line and the action and status information of the relevant equipment during the threading process, and using this sample to obtain the target representation of the operation process through self-learning, and then using these target features to train the artificial neural network to learn threading, and finally realizing unmanned threading control through an automatic threading artificial neural network intelligent agent with the optimal threading method, so that the threading result is accurate, the safety of the operators is guaranteed, the threading cost of the coiler is reduced, and the threading efficiency is improved.

[0085] In this embodiment, strip steel is taken as an example below to illustrate how the uncoiler threading control method applicable to a strip steel processing line provided by the present invention is used for the uncoiler threading control applicable to a strip steel processing line of a strip steel processing production line, but it is not intended to provide a sole limitation thereto.

[0086] Secondly, in order to more clearly understand and explain the technical solutions provided by the embodiments of the present invention, the technical terms involved in the embodiments of the present invention are further explained below. Specifically, the technical terms involved in the present invention include:

[0087] Artificial neural network: Artificial neural network is a hot research topic in the field of artificial intelligence since the 1980s. It abstracts the human brain nervous system from the perspective of information processing, establishes an artificial neuron model, and forms different networks according to different connection methods, which are applied to intelligent learning, speech processing, etc.

[0088] Deep learning: An artificial neural network structure with multiple layers of hidden units has more intelligent analytical learning capabilities than simple neural networks and can recognize data such as text, images, and sounds.

[0089] Self-supervised learning: A machine learning technique that uses unsupervised learning to accomplish tasks that normally require supervised learning. Self-supervised models do not rely on manually labeled datasets, but instead generate implicit labels from unstructured data to accomplish learning tasks.

[0090] Noise contrast loss function: used to calculate local loss based on the local similarity matrix.

[0091] Optimal transport: Optimal transport is a mathematical theory and computational method used to describe the distance or correspondence between the probability distributions of two random variables. Its core idea is how to transport a set of resources from one location to another in the best way.

[0092] Reinforcement learning: Reinforcement learning is a machine learning method that learns through interaction and feedback with the environment. Its goal is to find an optimal strategy that enables the agent to obtain the maximum reward or achieve a specific goal in a continuous time series.

[0093] Actor-Critic algorithm: A reinforcement learning algorithm that combines policy learning and action-value function estimation to form a closed-loop feedback mechanism, improving the stability and flexibility of reinforcement learning.

[0094] DDPG: Deep Deterministic Policy Gradient (DDPG), an Actor-Critic based deep reinforcement learning algorithm that can be used for model-free policy learning of continuous actions.

[0095] Feature encoder: refers to a learning-free algorithm, which is a neural network architecture that learns the compressed representation of data through an encoder and a decoder. The encoder compresses the input data into a low-dimensional feature vector, and the decoder restores it to the original data. The purpose of the feature encoder is to minimize the difference between the encoder and the decoder, thereby learning the important features of the data.

[0096] First, refer to Figure 1 An embodiment of the present invention provides a strip threading control method for a uncoiler of a strip processing line, the method comprising the following steps S1-S8:

[0097] Step S1, obtaining observation samples, where the observation samples include the operation data of the operator during the threading process of the uncoiler and the action data of the uncoiler.

[0098] Specifically, the operating data of the operator during the strip threading process of the uncoiler and the movement data of the uncoiler can be obtained through a visual system or an image system. The acquired operating data of the operator and the movement data of the uncoiler include but are not limited to video information of all actions of the operator during the strip threading process and video information of all actions and equipment status of the uncoiler.

[0099] In order to enable the uncoiler threading control method for strip steel processing line of the present invention to realize accurate, efficient and safe automatic threading of metal coils such as strip steel, the acquired video information should include the threading operation process of multiple strip steels of different specifications, and the video information should include the threading operation process completed by different staff at different times. At the same time, the video information includes the operation of the operator to operate the forward and reverse rotation of the uncoiler, the lifting and pressing of the uncoiler pressure roller, the lifting and pressing of the uncoiler bending roller, the lifting and pressing of the uncoiler guide plate, the extension and retraction of the uncoiler guide plate, the opening and closing of the pinch roller, the centering opening of the side guide device, the opening and closing of the straightening roller of the straightening machine, and the forward and reverse rotation of the straightening roller of the straightening machine. At the same time, in order to obtain more data that is beneficial to the threading of metal coils such as strip steel on the uncoiler, further improve the threading efficiency of the uncoiler, ensure the safety of the operators during the threading process, and ensure good threading quality during the threading process, during the operation of the staff, the video shot by the cameras at various locations should be used as the main operation observation angle as much as possible, and the on-site human eye observation angle should be used as the auxiliary operation observation angle.

[0100] As an example, video information is acquired by multiple cameras arranged from the uncoiler to the entrance shearing machine. The video information includes the actions of the operator operating the forward and reverse rotation of the uncoiler, the lifting and pressing of the uncoiler pressure roller, the lifting and pressing of the uncoiler bending roller, the lifting and pressing of the uncoiler guide plate, the extension and retraction of the uncoiler guide plate, the opening and closing of the pinch roller, the centering opening of the side guide device, the opening and closing of the straightening roller of the straightening machine, and the forward and reverse rotation of the straightening roller of the straightening machine during the threading process. These operating actions are located at different stages of the process of the strip from the uncoiler to the entrance shearing machine on the processing line. Specifically, it is a series of actions that are connected and operated continuously. In order to avoid omissions in the process of acquiring video information and ensure that the acquired video information is comprehensive, the video information is divided into different action sets according to the section where the strip is located. In the embodiment of the present invention, the video information can be divided into three parts: the uncoiler action set, the pinch roller action set, and the straightening machine action set for acquisition.

[0101] Specifically, the relevant equipment action and status information includes the action information of all equipment from the starting point to the end point of the strip when the staff implements the strip threading operation. These action information include but are not limited to: forward and reverse rotation of the uncoiler, lifting and pressing down of the uncoiler pressure roller, lifting and pressing down of the uncoiler bending roller, lifting and pressing down of the uncoiler guide plate, extending and retracting of the uncoiler guide plate, as well as opening and closing of the pinch roller, centering opening of the side guide device, opening and closing of the straightening roller of the straightener, and forward and reverse rotation of the straightening roller of the straightener; the status information includes the equipment action status and monitoring status, wherein the action status corresponds to the above-mentioned action, and the monitoring status includes: the pressure difference of the uncoiler bending roller downward hydraulic cylinder, the pressure difference of the uncoiler guide plate extension and retraction hydraulic cylinder, the uncoiler torque, the pinch roller torque, the straightening roller depression amount of the straightener, and the straightening roller torque of the straightener.

[0102] In order to make the actions of the operators and the actions and status data of the uncoiler in the acquired video information clear and capture enough features, the camera can use a high-speed industrial camera. For areas where the surface deformation of the strip needs to be judged, such as the exit of the uncoiler and the entrance of the straightener, two cameras can be installed to improve the accuracy of the acquired information and facilitate the acquisition of more effective information. After the information is acquired, the global image of the strip is obtained by image stitching.

[0103] It should be noted that, from the perspective of the embodiments of the present invention, the present invention hopes to train an artificial neural network agent by imitating or learning the actual operation process of the staff on the strip processing line, so as to autonomously complete the complex process of coiling the uncoiler, so a complete learning sample is extremely necessary. The requirement for samples is not that the more the better, but that the details of each operation action should be covered as much as possible, and the numerical parameters of these operation actions should be effectively recorded. Otherwise, a bad result may be that after deployment, the action strategy distribution deviation occurs because the trained strategy falls into the local optimum.

[0104] In the embodiment of the present invention, the visual system and the automated control system should calibrate the system time to ensure that all data have a unified time reference. A simple processing method is to provide a stateful data fusion unit in the method of the present invention, which can provide clock synchronization based on the Network Time Protocol (NTP), and the visual system and the automated control system can use this clock reference to calibrate the time.

[0105] Step S2: pre-process the acquired observation samples.

[0106] The preprocessing includes aligning the operation data of the operator during the threading process of the uncoiler and the action data of the uncoiler according to the collection time.

[0107] Specifically, since the video stream data is acquired by the visual system, the video data is actually a series of image frames, and each frame of the image can be described by a three-dimensional matrix or tensor; the motion information data comes from the automated control system of the strip processing line, which is usually some structured data, which is specifically expressed in scalar form or can be processed into an array or one-dimensional matrix during the acquisition process. Since the data structures of these two parts are very different and are acquired at different acquisition frequencies, it is necessary to align and fuse these two aspects of data. The acquired video stream and motion information stream need to be time-sequentially aligned according to the acquisition time, and resampled to generate a usable observation sample set.

[0108] Step S3: resample the preprocessed observation samples and extract the operation characteristics of the unwinder threading as the observation sample set.

[0109] Specifically, the observation sample set extraction includes the following methods:

[0110] In some optional implementations of this embodiment, for any time, take time t as an example, at time t, for the video stream data in the observation sample set, the image frame data is v t , the equipment action status data and monitoring status data are combined into s t , the device operation action data is a t , then after the time series is aligned, the observed sample data (v t ,s t ,a t ), to generate observation sample data (v t ,s t ,a t ) resamples at the preset time interval to obtain the observed sample data set D = {(v t ,s t ,a t)}, where t can be replaced by any time, such as t = 1, 2, …, n.

[0111] In some optional implementations of this embodiment, the above-mentioned preset time includes any time between 50ms and 200ms, preferably any time between 80ms and 120ms, and most preferably 80ms, 90ms, 100ms, 110ms and 120ms.

[0112] In some achievable methods, the processed sample data set is stored in the form of a file. The graphic frame is stored in an image file format, which can be a .jpg, .png, etc. format; the state and action data are stored in an independent .csv format. When needed in subsequent training or use, the data loader provided by the deep learning framework or deep learning algorithm used can be used for loading.

[0113] Step S4: using the observed sample set to train the feature encoder offline in a self-learning manner, so that the feature encoder extracts the expert operation feature data of the unwinder threading.

[0114] Specifically, the feature encoder is trained offline by self-learning using the observed sample set, including:

[0115] The feature encoder includes a first deep neural network model and a second deep neural network model that are independent of each other. The observation sample set is fed into the first deep neural network model and the second deep neural network model for training, and the image frame data in the observation sample set is converted into latent variables in a low-dimensional vector space, wherein,

[0116] The first deep neural network model is trained using a noise contrast loss-based loss function, where the noise contrast loss-based loss function is set to:

[0117]

[0118] in, represents the loss function based on noise contrast loss, -E D [·] represents the mathematical expectation of the relevant statistics calculated for the observed sample set D, e represents the base of the natural logarithm function, z t Indicates that the image frame data in the observation sample set is converted into latent variables in the low-dimensional vector space. represents the extracted positive example latent variable, i represents the number of samples in the sample data set, and n represents the size of the sample data set;

[0119] The second deep neural network model is trained using a loss function based on average loss, and the loss function based on average loss is set to:

[0120]

[0121] in, represents the loss function based on the average loss, s t represents the action status data and monitoring status data of the uncoiler at time t, s t+k represents the action state data and monitoring state data of the uncoiler at time t+k, where k represents a settable coefficient, which is used to adjust the time granularity of the second deep neural network model's perception of changes in the equipment action state data and monitoring state data;

[0122] Among them, the loss function based on noise contrast loss and the loss function based on average loss form a combined loss function, and the combined loss function includes:

[0123]

[0124] L f represents the combined loss function formed by the loss function based on noise contrast loss and the loss function based on average loss, and λ represents the loss function based on noise contrast loss. The proportional coefficient is a real number whose value is greater than 0 and less than 1.

[0125] More specifically, in the embodiment of the present invention, the sample data after time alignment and resampling still belongs to the high-dimensional vector space and cannot be used as the operation action feature for strategy training. The present invention uses self-learning to reduce the dimension of these observation data and extract the implicit variable information, thereby obtaining the implicit action annotation information. Moreover, the implicit variables are located in the low-dimensional vector space, which can more effectively characterize the data features. Specifically, for the observation sample set D={(v t ,s t ,a t}, t=1,2,…,n, which contains a positive sample The remaining n-1 are negative samples, and the extracted latent variable is defined as z t , the corresponding positive implicit variable is The design consists of two independent deep neural networks and The composition of the operational feature encoder Among them, θ, For two neural networks and The weight parameter is used to feed the observation sample into and Training can be performed so that the observed sample data v t Transformed into a latent variable z in a low-dimensional vector space t , this process can be expressed as For operating feature encoder The training adopts a combination of the noise contrast loss function and the L1 average loss function. in, is a loss function based on noise contrast loss, used to train deep neural networks is a loss function based on L1 average loss, used to train deep neural networks k is a configurable coefficient used to adjust the neural network For state s t Temporal granularity of change perception. Trained operational feature encoder Applied to the subsequent online operation strategy training to generate online features D′={(z′ t ,s′ t ,a′ t )}.

[0126] In the embodiment of the present invention, considering the limitation of computing power, the extracted observation sample set containing positive samples and the subsequent feature encoder The same observation sample data set D is used for training. In training, the loss function of L1 average loss is middle.

[0127] In some optional implementations, the constant k is set to 3.

[0128] As an example, for the above three sets of operation actions, 16 groups of video data and state data, each lasting 90 seconds, are extracted for training.

[0129] In some optional implementations of this embodiment, the image converted from the original video is processed using the Imgaug image enhancement library in Python language, and the generated sample image frames are subjected to enhancement processing such as cropping, scaling, color conversion, and random erasing.

[0130] In the embodiment of the present invention, the obtained observation sample data set or the image after video conversion includes not only the strip steel, uncoiler, pinch roller, straightener and other equipment involved in the target operation action, but also the background image, and there will be some interference elements irrelevant to the threading operation in the background image. Therefore, the target operation action is segmented from the video image, and the strip steel, equipment and background elements involved in the implementation of the operation action are distinguished, and then the background in the video image is eliminated, so that the sample image only contains the target action object, which can more effectively train the operation action features.

[0131] Therefore, in an embodiment of the present invention, an image segmentation algorithm can be selected to perform image segmentation processing on the image after video conversion, so as to segment out the target object related to the strap-on operation action in the image, and then eliminate irrelevant background information, so as to remove the noise interference that may be caused by the background information to the operation feature training.

[0132] As an example, in the present invention, a residual neural network with 50 layers is selected as the operational feature encoder Backbone network The output layer is connected to the pooling layer to downsample to a feature vector of 2048*1, and a multi-layer perceptron network is selected as Eventually The output vector of is projected to extract a 256*1 feature vector.

[0133] Step S5: construct a neural network model for automatic tape threading of the uncoiler, and the neural network model for automatic tape threading of the uncoiler adopts a deep deterministic policy gradient algorithm network structure (DDPG).

[0134] Specifically, the network structure of the deep deterministic policy gradient algorithm includes the loss functions of two Critic networks and the loss functions of two Actor networks, where:

[0135] The loss functions of the two critic networks in the deep deterministic policy gradient algorithm network structure used in the automatic tape threading neural network model of the unwinder are set as:

[0136] L θ1 =E (z,a′)～D [(Q θ1 (z′,s′,a′)-y) 2 ];

[0137] L θ2 =E (z,a′)～D [(Q θ2 (z′,s′,a′)-y) 2 ];

[0138] Among them, L θ1 Represents the current network loss function of Critic, L θ2 represents the Critic target network loss function, z represents the current state of the wear action representation, a′ represents the action taken on the current state, D represents the training buffer for online training of the deep deterministic policy gradient algorithm network structure, E (z,a′)～D It represents the expectation in the training buffer process D after calculating the current state z and taking action a′; Q θ1 and Q θ2Respectively represent the action value, z′, s′, a′ respectively represent the representation state of the tape threading action at the next moment, the state of the equipment such as the uncoiler monitored at the next moment (including video data), and the action taken for the action state of the tape threading action at the next moment, y is the target value of the deep deterministic policy gradient algorithm network structure, where the target value y of the deep deterministic policy gradient algorithm network structure is calculated by the following formula:

[0139]

[0140] Among them, γ represents the reinforcement learning discount coefficient, i represents the training sample index, n represents the size of the training sample set, and r t+i represents the reward function at time t+i, minQ θ1 Represents the action value Q θ1 Take the minimum value, z′ t+n ,s′ t+n ,a′ t+n They respectively represent the threading action status at time t+n, the status of the uncoiler and other equipment monitored at time t+n (including video data), and the action taken by the threading action status at time t+n, minQ θ2 Denotes the action value Q θ2 Take the minimum value;

[0141] The loss function of the two Actor networks in the deep deterministic policy gradient algorithm network structure is set to:

[0142]

[0143] in, Represents the loss function of the Actor network in the deep deterministic policy gradient algorithm network structure.

[0144] In the embodiment of the present invention, the deep reinforcement learning network based on the Actor-Critic mechanism adopts the DDPG deep reinforcement learning network as the backbone network. In order to adapt to the operation strategy migration, the embodiment of the present invention makes the following improvements to it: for a reinforcement learning training round (episode, including action, state (z′ t ,s′ t ,a′ t ), where t = 1, ..., T), first randomly initialize the weight parameters of the policy network (used to output the modified policy π′) and the critic network (used to output the action value Q θ1 and Q θ2 ) weight parameter; through the basic strategy π b and the current operating characteristic z′ t Get basic actions By modifying the strategy π rand the current operating characteristic z′ t And the basic actions obtained Get corrective action Then make the output action Update network weight parameters.

[0145] Step S6: Calculate the best matching method for migrating the extracted expert operation feature data to the automatic threading neural network, and use the matching value of the best matching method as the reward function for the training operation of the automatic threading neural network model of the unwinder.

[0146] Specifically, combined Figure 2 , calculating the best matching method of migrating the extracted expert operation feature data to the automatic threading neural network, and using the matching value of the best matching method as the reward function of the automatic threading neural network training operation strategy includes the following steps S61-S65:

[0147] Step S61: converting the image frame data in the sample data observed at any time into latent variables in the low-dimensional vector space through self-supervised learning to obtain an expert operation feature set;

[0148] Step S62: Select positive samples in the expert operation feature set as basic samples to train the automatic threading neural network model of the uncoiler online, and obtain the optimal operation of the automatic threading of the uncoiler;

[0149] Step S63: extracting implicit action representations in the observed sample data and action representations in the expert wearing operation feature set;

[0150] Step S64: comparing the extracted action representations to obtain the best fitting mode between the implicit action representation corresponding to the image frame data in the observed sample data and the action representation in the expert operation feature set;

[0151] Specifically, the above representation refers to a vector that can represent the characteristics of the data obtained after training the data.

[0152] Step S65: using the matching value corresponding to the best fitting method as a reward function to train the automatic threading neural network model of the unwinder;

[0153] Among them, the reward function is determined by the following formula:

[0154]

[0155] Among them, r OT (z t ) represents the best matching value of the extracted expert operation feature data transferred to the automatic threading neural network, j represents the column index of the transmission cost matrix c, n represents the number of action representation samples during the expert threading operation, c tjRepresented as the t-th row and j-th column element of the transmission cost matrix c, represents the matrix of best fit values.

[0156] In the embodiment of the present invention, the automatic threading neural network f is trained by deep reinforcement learning. π Specifically, by constructing a deep reinforcement learning neural network f π , select the expert operation feature set D E The strategy corresponding to the positive sample in is used as the basic strategy N represents the strategy samples corresponding to N groups of operations; from the basic strategy π b Start with online training of automatic threading neural network f π , get the optimal operation strategy π * .

[0157] But in the automatic threading neural network f π During training, it is impossible to explicitly specify the value function corresponding to a certain operation strategy, so it is impossible to directly perform gradient learning on the threading operation strategy. Therefore, this method automatically threading the neural network f through online training π When the operational feature encoder is used The extracted latent action representation z′ t and expert operating characteristic D E Action representation in z t Compare and find z′ t and z t The most ideal fitting method is used, and the degree of matching between the two under this fitting method is used as the reward function to train the operation strategy.

[0158] The following is a detailed description of the belt-threading action:

[0159] For a certain threading process operation, the action trajectory containing the expert operation features in the obtained observation sample is T v = {v 1 ,v 2 ,…,v n}, after feature encoder Then obtain its low-dimensional spatial feature representation T z ={z 1 ,z 2 ,…,z n}; Online training of automatic threading neural network f π When , the action trajectory executed by its output operation strategy is Feature Encoder After processing, its low-dimensional space feature representation is Let the optimal transmission cost matrix be c, c ijis the element in the i-th row and j-th column of the matrix, which represents the action representation z during the expert's wearing operation. i The random variable distribution represented by the action representation generated by the operation strategy during the automatic threading neural network training The cost of the random variable distribution represented by can be calculated using the cosine similarity between the two. Then z is calculated by the optimal transmission solver t and The most ideal fitted value matrix Thus, we can obtain the action representation sample z during the expert belt-threading operation t and the action representations produced by the operation strategy during the training process The matching degree of , which is used as the reward function of the stage of the training process, is

[0160] In the embodiment of the present invention, the above-mentioned calculation of the optimal fitting value matrix The optimal transport solver used can be the Sinkhorn-Knopp algorithm, which solves the optimal value through iterative calculation

[0161] Step S7: using the reward function to train the automatic threading neural network model of the unwinder.

[0162] Specifically, using the reward function to train the automatic tape threading neural network model of the uncoiler includes: using a feature encoder to extract expert operation feature data of the uncoiler threading as input, using the optimal threading error as output, and using the reward function to train the automatic tape threading neural network model of the uncoiler by adopting a gradient learning algorithm.

[0163] Step S8: Iterate the expert operation features as the basic strategy of the automatic threading neural network model to learn the optimal threading operation strategy of the uncoiler and obtain the optimal threading operation of the uncoiler.

[0164] Specifically, combined Figure 3 , the expert operation characteristics are used as the basic strategy of the automatic threading neural network model to iterate, learn the optimal threading operation strategy of the uncoiler, and obtain the optimal threading operation of the uncoiler, including the following steps S81-step S83:

[0165] Step S81: extracting expert operation feature data of the unwinder threading with a feature encoder as input to the Actor network and the Critic network, and extracting expert operation feature data from the observed sample data through the feature encoder during the iteration of the automatic threading neural network model;

[0166] Step S82: After the extracted expert operation feature data is input into the Actor network for processing, the uncoiler threading action data is output, and after the uncoiler threading action data is executed, new uncoiler state data is generated;

[0167] Step S83: After comparing the generated new state data through the Critic network, the weight parameters of the Actor network and the Critic network and the current threading operation are updated according to the reward function until the optimal operation that is close to the expert operation characteristics is achieved.

[0168] In the embodiment of the present invention, the online training of the automatic threading neural network learns the optimal threading operation strategy based on the expert threading operation characteristics, including: using the deep reinforcement learning network of the Actor-Critic mechanism based on the above-mentioned expert threading operation process action representation sample z t and the action representation of the training process The matching degree of the reward function is used to train the automatic threading neural network. The automatic threading neural network contains the above feature encoder Network, Actor network and Critic network, and feature encoder The output of the network is used as the input of the Actor network and the Critic network. During the training process, the video information and device status information from the visual system are passed through the feature encoder. Generate an operation state (z′ t ,s′ t ,a′ t ), these states are processed by the Actor network and the threading action is output. After the threading action is executed, a new state (z′) is generated. t ,s′ t ,a′ t ), these new states are compared through the Critic network and the weight parameters of the Actor network and the Critic network and the current operation strategy are updated according to the above reward function until the optimal operation strategy that is close to the expert operation characteristics is reached.

[0169] In summary, the uncoiler threading control method suitable for a strip processing line of the present invention uses the obtained optimal threading operation of the uncoiler as the threading action of the uncoiler in the automatic control system of the strip processing line, and performs automatic threading of the uncoiler by using a trained automatic threading neural network model. Without changing the existing production equipment, automatic threading of the uncoiler can be achieved, the threading result is accurate, the safety of the operators is guaranteed, the threading cost of the uncoiler is reduced, and the threading efficiency is improved.

[0170] It should be noted that for copper coils and aluminum coils, that is, aluminum strips and copper strips, the same method is used to thread the strips on the uncoiler in the present invention, and the uncoiler is controlled in the same way as the strip steel, which will not be illustrated one by one in this embodiment.

[0171] Second, combining Figure 5 As shown, the present invention also provides a strip threading control system for a strip processing line, comprising: an acquisition module 10, a preprocessing module 20, a feature extraction module 30, an offline training module 7040, a network model building module 50, a calculation module 60, a training module 70 and an iteration module 80, wherein:

[0172] The acquisition module 10 is used to acquire observation samples, and the acquired observation samples include the operation data of the operator during the threading process of the uncoiler and the action data of the uncoiler; the preprocessing module 20 is used to preprocess the acquired observation samples; the feature extraction module 30 is used to resample the preprocessed observation samples and extract the operation features of the uncoiler threading as the observation sample set; the offline training module 7040 is used to use the observation sample set to offline train the feature encoder in a self-learning manner so that the feature encoder extracts the expert operation feature data of the uncoiler threading; the network model construction module 50 is used to construct the automatic threading neural network model of the uncoiler; the calculation module 60 is used to calculate the best matching method for migrating the extracted expert operation feature data to the automatic threading neural network, and use the matching value of the best matching method as the reward function of the automatic threading neural network training operation strategy; the training module 70 is used to train the automatic threading neural network model of the uncoiler using the reward function; the iteration module 80 is used to iterate the automatic threading neural network model with the expert operation feature data to obtain the optimal threading operation of the uncoiler.

[0173] Thirdly, combining Figure 4 As shown, the present invention also provides a strip threading control device for a decoiler of a strip processing line, comprising: a collection unit 100, a data processing unit 200 and a control unit 300, wherein:

[0174] The acquisition unit 100 is used to collect the operation data of the operators and the action data of the uncoiler during the threading process of the uncoiler; the data processing unit 200 is used to pre-process the data collected by the acquisition unit 100; the control unit 300 is configured with an automatic threading neural network model and a feature encoder for the uncoiler, and the control unit 300 is used to train the operation feature encoder offline by self-supervised learning, so that the feature encoder extracts the expert operation feature data of the uncoiler threading; at the same time, the control unit 300 is used to calculate the best matching method for migrating the extracted expert operation feature data to the automatic threading neural network model, and use the matching value of the two as the reward function of the automatic threading neural network training operation strategy; and use the sample data and the reward function to perform unsupervised training on the automatic threading neural network model of the uncoiler, and at the same time, the control unit 300 uses the expert operation features as the basis to iterate the uncoiler threading neural network model, so that the automatic threading neural network model outputs the optimal threading operation of the uncoiler.

[0175] Specifically, the operating data of the operator during the strip threading process of the uncoiler and the movement data of the uncoiler can be obtained through a visual system or an image system. The acquired operating data of the operator and the movement data of the uncoiler include but are not limited to video information of all actions of the operator during the strip threading process and video information of all actions and equipment status of the uncoiler.

[0176] In order to enable the uncoiler threading control method for strip steel processing line of the present invention to realize accurate, efficient and safe automatic threading of metal coils such as strip steel, the acquired video information should include the threading operation process of multiple strip steels of different specifications, and the video information should include the threading operation process completed by different staff at different times. At the same time, the video information includes the operation of the operator to operate the forward and reverse rotation of the uncoiler, the lifting and pressing of the uncoiler pressure roller, the lifting and pressing of the uncoiler bending roller, the lifting and pressing of the uncoiler guide plate, the extension and retraction of the uncoiler guide plate, the opening and closing of the pinch roller, the centering opening of the side guide device, the opening and closing of the straightening roller of the straightening machine, and the forward and reverse rotation of the straightening roller of the straightening machine. At the same time, in order to obtain more data that is beneficial to the threading of metal coils such as strip steel on the uncoiler, further improve the threading efficiency of the uncoiler, ensure the safety of the operators during the threading process, and ensure good threading quality during the threading process, during the operation of the staff, the video shot by the cameras at various locations should be used as the main operation observation angle as much as possible, and the on-site human eye observation angle should be used as the auxiliary operation observation angle.

[0177] As an example, video information is acquired by multiple cameras arranged from the uncoiler to the entrance shearing machine. The video information includes the actions of the operator operating the forward and reverse rotation of the uncoiler, the lifting and pressing of the uncoiler pressure roller, the lifting and pressing of the uncoiler bending roller, the lifting and pressing of the uncoiler guide plate, the extension and retraction of the uncoiler guide plate, the opening and closing of the pinch roller, the centering opening of the side guide device, the opening and closing of the straightening roller of the straightening machine, and the forward and reverse rotation of the straightening roller of the straightening machine during the threading process. These operating actions are located at different stages of the process of the strip from the uncoiler to the entrance shearing machine on the processing line. Specifically, it is a series of actions that are connected and operated continuously. In order to avoid omissions in the process of acquiring video information and ensure that the acquired video information is comprehensive, the video information is divided into different action sets according to the section where the strip is located. In the embodiment of the present invention, the video information can be divided into three parts: the uncoiler action set, the pinch roller action set, and the straightening machine action set for acquisition.

[0178] Specifically, the relevant equipment action and status information includes the action information of all equipment from the starting point to the end point of the strip when the staff implements the strip threading operation. These action information include but are not limited to: forward and reverse rotation of the uncoiler, lifting and pressing down of the uncoiler pressure roller, lifting and pressing down of the uncoiler bending roller, lifting and pressing down of the uncoiler guide plate, extending and retracting of the uncoiler guide plate, as well as opening and closing of the pinch roller, centering opening of the side guide device, opening and closing of the straightening roller of the straightener, and forward and reverse rotation of the straightening roller of the straightener; the status information includes the equipment action status and monitoring status, wherein the action status corresponds to the above-mentioned action, and the monitoring status includes: the pressure difference of the uncoiler bending roller downward hydraulic cylinder, the pressure difference of the uncoiler guide plate extension and retraction hydraulic cylinder, the uncoiler torque, the pinch roller torque, the straightening roller depression amount of the straightener, and the straightening roller torque of the straightener.

[0179] In order to make the actions of the operators and the actions and status data of the uncoiler in the acquired video information clear and capture enough features, the camera can use a high-speed industrial camera. For areas where the surface deformation of the strip needs to be judged, such as the exit of the uncoiler and the entrance of the straightener, two cameras can be installed to improve the accuracy of the acquired information and facilitate the acquisition of more effective information. After the information is acquired, the global image of the strip is obtained by image stitching.

[0180] As an example, in order to facilitate the statistics of the operating data of the operators during the threading process of the uncoiler and the motion data of the uncoiler obtained in the present invention, the acquisition unit 100 can be set as a first acquisition unit 100 and a second acquisition unit 100, the first acquisition unit 100 is used to collect the operating data of the operators during the threading process of the uncoiler, and the second acquisition unit 100 is used to collect the motion data of the uncoiler during the operating process of the operators during the threading process of the uncoiler.

[0181] The above-described apparatus / system / equipment, and the module / unit embodiments corresponding to or included in the apparatus / system / equipment are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present embodiment without creative effort.

[0182] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0183] In the fourth aspect, according to the above-mentioned uncoiler threading control method applicable to the strip processing line, an embodiment of the present invention also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned control method.

[0184] It should be understood that the specific features, operations and details described hereinabove about the method of the present invention may also be similarly applied to the device and system of the present invention, or, vice versa. In addition, each step of the method of the present invention described above may be performed by the corresponding parts or units of the device or system of the present invention.

[0185] It should be understood that each module / unit of the device of the present invention can be implemented in whole or in part by software, hardware, firmware or a combination thereof. Each module / unit can be embedded in the processor of the electronic device or independent of the processor in the form of hardware or firmware, or can be stored in the memory of the electronic device in the form of software for the processor to call to perform the operation of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0186] It will be appreciated by those skilled in the art that the method steps of the present invention may be performed by instructing related hardware such as an electronic device or a processor through a computer program, and the computer program may be stored in a non-temporary computer-readable storage medium, which causes the steps of the present invention to be performed when the computer program is executed. Depending on the circumstances, any reference to memory, storage, or other media herein may include non-volatile or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0187] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, "front", "back", "left", "right", "upper" and "lower" in this article are all referenced to the placement state shown in the accompanying drawings.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A strip threading control method for an unwinder of a strip processing line, characterized in that: include: Acquiring observation samples, wherein the observation samples include operation data of operators during the threading process of the uncoiler and motion data of the uncoiler; Preprocessing the acquired observation samples; Resample the preprocessed observation samples and extract the operation characteristics of the unwinder threading as the observation sample set; Using the observation sample set to train a feature encoder offline in a self-learning manner, so that the feature encoder extracts expert operation feature data of the unwinder threading; Construct a neural network model for automatic tape threading of the uncoiler, which adopts a deep deterministic policy gradient algorithm network structure; Calculate the best matching method for migrating the extracted expert operation feature data to the automatic threading neural network, and use the matching value of the best matching method as the reward function for the training operation of the automatic threading neural network model of the unwinder; The reward function is used to train the neural network model for automatic threading of the unwinder; The expert operation characteristics are used as the basic strategy of the automatic threading neural network model for iteration, and the optimal threading operation strategy of the uncoiler is learned to obtain the optimal threading operation of the uncoiler.

2. The strip threading control method for an unwinder of a strip processing line according to claim 1, characterized in that: The preprocessing of the acquired observation samples includes: The operation data of the operator during the uncoiler threading process and the action data of the uncoiler are time-series aligned according to the collection time.

3. The strip threading control method for an unwinder of a strip processing line according to claim 1, characterized in that: Resample the preprocessed observation samples and extract the operating characteristics of the operators threading the unwinder as the observation sample set, including: At time t, the image frame data of the video stream data in the observation sample is v t , the equipment action status data and monitoring status data are combined into s t , the device operation action data is a t , then after the time series is aligned, the observed sample data (v t ,s t ,a t ), to generate observation sample data (v t ,s t ,a t ) resamples at time intervals to obtain the observed sample data set D = {(v t ,s t ,a t ), where time t can be replaced by any time.

4. The strip threading control method for an unwinder of a strip processing line according to claim 1, characterized in that: Offline training of the feature encoder by self-learning using the observed sample set includes: The feature encoder includes a first deep neural network model and a second deep neural network model that are independent of each other. The observation sample set is fed into the first deep neural network model and the second deep neural network model for training, and the image frame data in the observation sample set is converted into latent variables in a low-dimensional vector space, wherein: The first deep neural network model is trained using a noise contrast loss-based loss function, where the noise contrast loss-based loss function is set to: in, represents the loss function based on noise contrast loss, -E D [·] represents the mathematical expectation of the relevant statistics calculated for the observed sample set D, e represents the base of the natural logarithm function, z t Indicates that the image frame data in the observation sample set is converted into latent variables in the low-dimensional vector space. represents the extracted positive example implicit variable, i represents the number of the sample in the observed sample set, and n represents the size of the sample data set; The second deep neural network model is trained using a loss function based on average loss, and the loss function based on average loss is set to: in, represents the loss function based on the average loss, s t represents the action status data and monitoring status data of the uncoiler at time t, s t+k represents the action state data and monitoring state data of the uncoiler at time t+k, where k represents a settable coefficient, which is used to adjust the time granularity of the second deep neural network model's perception of changes in the equipment action state data and monitoring state data; Among them, the loss function based on noise contrast loss and the loss function based on average loss form a combined loss function, and the combined loss function includes: L f represents the combined loss function formed by the loss function based on noise contrast loss and the loss function based on average loss, and λ represents the loss function based on noise contrast loss. The proportional coefficient is a real number, and its value is greater than 0 and less than 1.

5. The strip threading control method for an unwinder of a strip processing line according to claim 1, characterized in that: Calculating the best matching method of migrating the extracted expert operation feature data to the automatic threading neural network, and using the matching value of the best matching method as the reward function of the automatic threading neural network training operation strategy includes: Through self-supervised learning, the image frame data in the sample data observed at any time is converted into latent variables in the low-dimensional vector space to obtain the expert operation feature set; The positive samples in the expert operation feature set are selected as basic samples to train the automatic threading neural network model of the uncoiler online, and the optimal operation of the automatic threading of the uncoiler is obtained. The implicit action representation in the extracted observation sample data and the action representation in the expert threading operation feature set; Comparing the extracted action representations, obtaining the best fitting method between the implicit action representation corresponding to the image frame data in the observed sample data and the action representation in the expert operation feature set; The matching value corresponding to the best fitting method is used as a reward function to train the automatic threading neural network model of the unwinder; The reward function is determined by the following formula: Among them, r OT (z t ) represents the best matching value of the extracted expert operation feature data transferred to the automatic threading neural network, j represents the column index of the transmission cost matrix c, n represents the number of action representation samples during the expert threading operation, c tj Represented as the t-th row and j-th column element of the transmission cost matrix c, represents the matrix of best fit values.

6. The strip threading control method for an unwinder of a strip processing line according to claim 1, characterized in that: The neural network model of the automatic tape threading machine adopts a deep deterministic policy gradient algorithm network structure. The deep deterministic policy gradient algorithm network structure includes the loss function of two Critic networks and the loss function of two identical Actor networks, where: The loss functions of the two critic networks in the deep deterministic policy gradient algorithm network structure used in the automatic tape threading neural network model of the unwinder are set as: L θ1 =E (z,a′)～D [(Q θ1 (z′,s′,a′)-y) 2 ]; L θ2 =E (z,a′)～D [(Q θ2 (z′,s′,a′)-y) 2 ]; Among them, L θ1 Represents the current network loss function of Critic, L θ2 represents the Critic target network loss function, z represents the current wear action representation state, a′ represents the action taken on the current state, D represents the training buffer for online training of the deep deterministic policy gradient algorithm network structure, E (z,a′)～D It represents the expectation in the training buffer process D after calculating the current state z and taking action a′; Q θ1 and Q θ2 They represent the action value respectively, z′, s′, a′ represent the threading action representation state at the next moment, the uncoiler equipment state monitored at the next moment and the action taken for the threading action state at the next moment respectively, y is the target value of the deep deterministic policy gradient algorithm network structure, where the target value y of the deep deterministic policy gradient algorithm network structure is calculated by the following formula: Among them, γ represents the reinforcement learning discount coefficient, i represents the training sample index, n represents the size of the training sample set, and r t+i represents the reward function at time t+i, minQ θ1 Represents the action value Q θ1 Take the minimum value, z′ t+n ,s′ t+n ,a′ t+n They respectively represent the threading action status at time t+n, the monitored uncoiler equipment status, and the action taken by the threading action status at time t+n, minQ θ2 Denotes the action value Q θ2 Take the minimum value; The loss functions of the two identical Actor networks in the deep deterministic policy gradient algorithm network structure are set to: in, Represents the loss function of the Actor network in the deep deterministic policy gradient algorithm network structure.

7. The strip threading control method for an unwinder of a strip processing line according to claim 6, characterized in that: The expert operation characteristics are used as the basic strategy of the automatic threading neural network model for iteration to learn the optimal threading operation strategy of the uncoiler. The optimal threading operation of the uncoiler includes: The expert operation feature data of the unwinder threading is extracted by the feature encoder as the input of the Actor network and the Critic network. During the iteration of the automatic threading neural network model, the expert operation feature data is extracted from the observed sample data through the feature encoder. After the extracted expert operation feature data is input into the Actor network for processing, the uncoiler threading action data is output, and after the uncoiler threading action data is executed, new uncoiler status data is generated; The new state data generated is compared through the Critic network, and the weight parameters of the Actor network and the Critic network and the current threading operation are updated according to the reward function until the optimal operation that is close to the expert operation characteristics is achieved.

8. A strip threading control system for a coiler in a strip processing line, characterized in that: include: An acquisition module, the acquisition module is used to acquire observation samples, the acquired observation samples include operation data of operators during the threading process of the uncoiler and action data of the uncoiler; A preprocessing module, wherein the preprocessing module is used to preprocess the acquired observation samples; A feature extraction module, wherein the feature extraction module is used to resample the preprocessed observation samples and extract the operation features of the unwinder threading as an observation sample set; An offline training module, wherein the offline training module is used to use the observation sample set to offline train the feature encoder in a self-learning manner, so that the feature encoder extracts expert operation feature data of the unwinder threading; A network model building module, wherein the network model building module is used to build a neural network model for automatic tape threading of an unwinder; A calculation module, the calculation module is used to calculate the best matching method for migrating the extracted expert operation feature data to the automatic threading neural network, and use the matching value of the best matching method as a reward function for the automatic threading neural network training operation strategy; A training module, wherein the training module is used to train an automatic threading neural network model of an unwinder using a reward function; The iterative module is used to iterate based on the expert operation characteristics as the basic strategy of the automatic threading neural network model, learn the optimal threading operation strategy of the uncoiler, and obtain the optimal threading operation of the uncoiler.

9. A strip threading control device for a uncoiler of a strip processing line, characterized in that: include: A collection unit, the collection unit is used to collect the operation data of the operator during the threading process of the uncoiler and the action data of the uncoiler; A processing unit, the processing unit is used to pre-process the data collected by the collection unit; A control unit is provided in which an automatic tape-threading neural network model and a feature encoder are configured. The control unit is used for offline training of the operation feature encoder by self-supervised learning so that the feature encoder extracts the expert operation feature data of the tape-threading of the uncoiler; at the same time, the control unit is used for calculating the best matching mode of migrating the extracted expert operation feature data to the automatic tape-threading neural network model, and using the matching value of the two as the reward function of the automatic tape-threading neural network training operation strategy; and using the sample data and the reward function to perform unsupervised training on the automatic tape-threading neural network model of the uncoiler, and at the same time, the control unit uses the expert operation feature as a basis to iterate the uncoiler tape-threading neural network model, so that the automatic tape-threading neural network model outputs the optimal tape-threading operation of the uncoiler.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the control method according to any one of claims 1 to 7.

Images