Dross position estimation device and dross position estimation method

Abstract

To provide a device that is effective in detecting whether dross needs to be removed with high reliability.SOLUTION: A dross position estimation device 30 comprises: a camera 40 which photographs the liquid level of a liquid plating material contained in a plating pot 2 at a continuous plating device 1 passing a steel strip continuously through the plating material in the plating pot 2; a displacement map creation part 114 which creates a displacement map representing a displacement vector of a photographed object in each of a plurality of areas in the visual field of the camera 40 based upon moving image data including a plurality of images photographed by the camera 40 in time series; and an estimation part 122 which estimates a border line between a region of the liquid level covered with dross and a region not covered with the dross based upon an estimation model created through machine learning to represent the relation between input data including the displacement map and the border line, and input data including the newly created displacement map.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a dross position estimation device and a dross position estimation method. [Background technology]

[0002] Patent Document 1 discloses a dross detection system including an infrared detection device that detects infrared rays radiated from the surface of the molten metal, a measurement unit configured to control the infrared detection device to measure the infrared intensity of the infrared rays radiated from the surface of the molten metal over a predetermined period of time, and a detection unit configured to detect dross based on a change over time in the measured infrared intensity caused by rippling on the surface of the molten metal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-14632 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an apparatus that is effective in reliably detecting the need for dross removal. [Means for solving the problem]

[0005] A dross position estimation device according to one aspect of the present disclosure, in a continuous plating apparatus in which a steel strip is continuously passed through liquid plating material contained in a plating pot, comprises a camera that photographs the liquid surface of the plating material in the plating pot, a displacement map generation unit that generates a displacement map representing the displacement vector of the photographed object in each of multiple areas in the camera's field of view based on video data including multiple images photographed in time series by the camera, and an estimation unit that estimates the boundary line based on input data including the displacement map, an estimation model generated by machine learning to represent the relationship between the boundary line between areas of the liquid surface covered by dross and areas not covered by dross, and input data including the newly generated displacement map.

[0006] In Patent Document 1, it is possible to detect whether or not dross is present at a specific position. However, even if there is no dross at a specific position, there is a possibility that dross has approached the steel strip at another position, so the need for dross removal cannot be detected with high reliability only by detection at a specific position. To detect the position of dross over a wide range, it is effective to use an image captured over a wide range, but when the brightness difference between the area covered with dross and the area not covered with dross is small, the boundary line cannot be detected with high reliability. According to the dross position estimation device of the present disclosure, a displacement map representing the displacement vector of the photographed object within each of multiple areas in the field of view of the camera is generated, and the boundary line is estimated based on input data including the displacement map, an estimation model generated by machine learning to represent the relationship with the boundary line, and new input data including the newly generated displacement map. Even when the brightness difference between the area covered with dross and the area not covered with dross is small, the difference in displacement vector appears clearly. Therefore, based on the estimation model generated by including the displacement map in the input data and the new input data including the newly generated displacement map, the boundary line can be estimated with high reliability over a wide range. Therefore, this dross position estimation device is effective in detecting the need for dross removal with high reliability.

[0007] The apparatus may further include a determination unit that determines whether or not dross needs to be removed based on difference data that indicates the difference between a predetermined reference line and the estimated boundary line. By determining the difference between the boundary line estimated with high reliability and the reference line, it is possible to determine with high reliability whether or not dross needs to be removed.

[0008] The apparatus may further include a standard setting unit that sets a reference line based on the result of estimating the boundary line, and the determining unit may determine whether or not dross removal is required based on difference data that indicates the difference between the set reference line and the estimated boundary line. The result of estimating the boundary line may also be utilized in setting the reference line, thereby improving ease of use.

[0009] The apparatus may further include a reference setting unit that sets a reference line based on the operating conditions including at least the feed speed of the steel strip, a reference profile that is predetermined to represent a relationship with the reference line, and newly acquired operating conditions, and the judgment unit may judge whether or not dross removal is required based on the difference between the set reference line and the estimated boundary line. The boundary line may also change depending on the operating conditions. In response to this, by setting the reference line depending on the operating conditions, it is possible to more reliably judge whether or not dross removal is required.

[0010] The apparatus may further include a profile generating unit that generates a reference profile based on a plurality of records each including an operating condition including at least the feed speed of the steel strip and an estimated result of the boundary line, and the reference setting unit may set the reference line based on the generated reference profile and the newly acquired operating condition. The estimated result of the boundary line may also be utilized in generating the reference profile, improving ease of use.

[0011] The apparatus may further include a drawing unit that draws a map image of the liquid surface including the boundary line viewed from above based on the estimated boundary line. By graphically showing the boundary line, it is possible to further improve usability.

[0012] The image processing device may further include a brightness map generating unit that generates a brightness map representing the brightness of each of the multiple areas based on the video data, and the estimation unit may estimate the boundary line based on input data further including the brightness map, an estimation model generated by machine learning so as to represent a relationship with the boundary line, and new input data further including the brightness map. The boundary line can be estimated with higher reliability.

[0013] The apparatus may further include a tool recognition unit that recognizes the position of a tool for work around the plating pot, such as work to remove dross, based on the video data, and a correction unit that corrects the difference data so as to exclude a difference occurring within a predetermined range including the position of the tool from the difference between the reference line and the estimated boundary line, and the determination unit may determine whether or not dross removal is required based on the corrected difference data. This makes it possible to determine whether or not dross removal is required with higher reliability.

[0014] The determining unit may determine whether or not dross removal is required based on the speed of change of the boundary line, thereby making it possible to determine with higher reliability whether or not dross removal is required.

[0015] The apparatus may further include a correct answer data acquisition unit that acquires correct answer data indicating the boundary line visually recognized based on the video data, a storage unit that stores learning records in a database in which the input data and the correct answer data are associated with each other, and a model generation unit that generates an estimation model by machine learning based on the multiple learning records stored in the database. By further performing processes from storing the learning records to generating the estimation model, the usability of the dross position estimation device is further improved.

[0016] The model generation unit may generate the estimation model by deep learning, which makes it possible to estimate the boundary line with higher reliability.

[0017] A dross position estimation method according to another aspect of the present disclosure includes, in a continuous plating apparatus in which a steel strip is continuously passed through a liquid plating material contained in a plating pot, acquiring video data including a plurality of images captured in a time series by a camera that captures the liquid surface of the plating material in the plating pot, generating a displacement map representing the displacement vector of the captured object in each of a plurality of areas in the field of view of the camera based on the video data, acquiring ground truth data indicating a boundary line between an area of ​​the liquid surface that is covered by dross and an area that is not covered by dross, as viewed based on the video data, accumulating in a database learning records that correspond to input data including the displacement map and the ground truth data, generating an estimation model representing the relationship between the input data and the boundary line based on the plurality of learning records accumulated in the database, generating a new displacement map based on the newly acquired video data, and estimating the boundary line based on the new input data including the newly generated displacement map and the estimation model. Effect of the Invention

[0018] According to the present disclosure, it is possible to provide an apparatus that is effective in reliably detecting the need for dross removal. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of a plating system. [Diagram 2] FIG. 2 is a block diagram illustrating a functional configuration of the control system. [Diagram 3] FIG. 10 is a diagram illustrating a schematic example of a displacement map. [Figure 4] FIG. 1 is a diagram illustrating a schematic example of an estimation model. [Diagram 5] 13A and 13B are schematic diagrams illustrating boundary line drawing results; [Figure 6] FIG. 13 is a block diagram showing a modified example of the control system. [Figure 7] FIG. 2 is a block diagram illustrating a hardware configuration of the control system. [Figure 8] 1 is a flowchart illustrating a procedure for generating an estimation model. [Figure 9] 13 is a flowchart illustrating a procedure for generating a reference profile. [Figure 10] 1 is a flowchart illustrating a procedure for determining whether or not dross removal is required. [Figure 11] 1 is a flowchart illustrating a procedure for determining whether or not dross removal is required. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the description, the same elements or elements having the same functions are denoted by the same reference numerals, and duplicated description will be omitted.

[0021] [Plating Equipment] The plating apparatus 1 shown in FIG. 1 is a continuous plating apparatus that applies a plating process to a steel strip 9 fed along a predetermined pass line PL. For example, the plating apparatus 1 includes a plating pot 2, a plurality of rolls 3, a snout 7, a gas wiping device 10, and a control device 20. The plating pot 2 contains a liquid plating material containing plating components such as molten zinc. The plating material may further contain molten aluminum as a plating component. Hereinafter, the plating material contained in the plating pot 2 is also referred to as a "plating bath."

[0022] The multiple rolls 3 are lined up along a pass line PL that passes through the coating bath in the coating pot 2, and the steel strip 9 is aligned along the pass line PL. Each of the multiple rolls 3 extends perpendicular to the pass line PL. The steel strip 9 is bent around each of the multiple rolls 3 to follow the pass line PL. Each of the multiple rolls 3 is driven to rotate about an axis that is aligned along the width direction of the steel strip 9 (direction perpendicular to the pass line PL).

[0023] The multiple rolls 3 include a sink roll 4 and two support rolls 5, 6. The sink roll 4 is located in the coating bath contained in the coating pot 2. At the sink roll 4, the steel strip 9 that has entered the coating bath is folded back from bottom to top. Hereinafter, the surface of the steel strip 9 in contact with the sink roll 4 is referred to as the "back surface 9b," and the opposite surface is referred to as the "front surface 9a."

[0024] The support roll 6 is located above the sink roll 4 in the plating bath contained in the plating pot 2. The support roll 6 comes into contact with the front surface 9a of the steel strip 9 which flows upward through the sink roll 4. At the support roll 6, the steel strip 9 is gently turned back from the back direction (the direction in which the back surface 9b faces) to the front direction (the direction in which the front surface 9a faces) while flowing upward.

[0025] The support roll 5 is located above the support roll 6 in the plating bath contained in the plating pot 2. The support roll 5 comes into contact with the back surface 9b of the steel strip 9 which passes through the support roll 6 and moves upward. At the support roll 5, the steel strip 9 is gently turned back from the front direction to the back direction while moving upward.

[0026] The arrangement of the support rolls 5, 6 is an example and can be changed as appropriate. For example, the support roll 5 in contact with the back surface 9b may be arranged lower than the support roll 6 in contact with the front surface 9a.

[0027] The snout 7 is, for example, a square cylindrical member having a rectangular cross-sectional shape, and surrounds the steel strip 9 around the pass line PL as it advances toward the plating bath.

[0028] The gas wiping device 10 removes excess plating material from the steel strip 9 by spraying gas onto the front surface 9a and the back surface 9b. The gas may be air or an inert gas such as nitrogen gas. For example, the gas wiping device 10 has a gas wiping nozzle 11 and a gas wiping nozzle 12. The gas wiping nozzle 11 discharges gas toward the front surface 9a from a slit-shaped outlet extending in the width direction of the steel strip 9. The gas wiping nozzle 12 discharges gas toward the back surface 9b from a slit-shaped outlet extending in the width direction of the steel strip 9.

[0029] The control device 20 controls the pressure at which gas is supplied to the gas wiping nozzles 11, 12 (hereinafter referred to as "wiping pressure"), and the feed speed of the steel strip 9 by the sink roll 4 and the support rolls 5, 6. The control device 20 may further control the opening width of the slits of the gas wiping nozzles 11, 12 in accordance with the width of the steel strip 9.

[0030] [Dross position estimation device] In the above-mentioned coating apparatus 1, impurities called "dross" are generated in the coating bath. Dross is a fine lump formed by alloying components eluted from the steel strip 9 with coating components such as molten zinc. Some of the dross settles to the bottom of the coating bath, and other parts of the dross rise to the surface of the coating bath. The dross that settles to the bottom of the coating bath is called "bottom dross," and the dross that rises to the surface of the coating bath is called "top dross."

[0031] The top-dross DRS is removed by an operator (person). For example, the operator uses a tool 8 such as a spoon to scoop out the top-dross DRS from the coating bath. To prevent the top-dross DRS from solidifying and becoming difficult to remove, or from adhering to the steel strip 9 and degrading the quality of the steel strip 9, the top-dross DRS removal work needs to be performed in a timely manner. To this end, the accumulation status of the top-dross DRS needs to be monitored frequently, which is a burden for the operator.

[0032] The above-mentioned Patent Document 1 discloses a system that detects dross below a sensor based on the results of detection by the sensor of far-infrared rays emitted from the surface of the molten metal. According to Patent Document 1, it is possible to detect whether or not dross is present at a specific position (for example, below the sensor). However, even if there is no dross at the specific position, there is a possibility that dross has come close to the steel strip at another position, so whether or not dross needs to be removed cannot be detected with high reliability only by detection at a specific position.

[0033] In order to monitor the accumulation status of the top dross, the plating apparatus 1 further includes a dross position estimation device 30. The dross position estimation device 30 includes a camera 40 that captures an image of the liquid surface of the plating bath in the plating pot 2, and a calculation device 100. The calculation device 100 is configured to execute the following operations: generate a displacement map representing the displacement vector of the captured object in each of a plurality of areas in the field of view of the camera 40 based on video data including a plurality of images captured in time series by the camera 40; and estimate the position of the boundary line based on an estimation model generated by machine learning so as to represent the relationship between input data including the displacement map, an area of ​​the liquid surface covered with dross (top dross) and an area not covered with dross, and input data including the newly generated displacement map.

[0034] In order to detect the position of dross over a wide range, it is effective to use an image captured over a wide range. However, when the brightness difference between the area covered with dross and the area not covered with dross is small, the boundary line cannot be detected with high reliability. According to the dross position estimation device of the present disclosure, a displacement map representing the displacement vector of the photographed object in each of the multiple areas in the field of view of the camera is generated, and the boundary line is estimated based on input data including the displacement map, an estimation model generated by machine learning to represent the relationship with the boundary line, and new input data including the newly generated displacement map. Even when the brightness difference between the area covered with dross and the area not covered with dross is small, the difference in the displacement vector appears clearly. Therefore, the boundary line over a wide range can be estimated with high reliability based on the estimation model generated by including the displacement map in the input data and the new input data including the newly generated displacement map. Therefore, this dross position estimation device is effective in detecting the necessity of removing dross with high reliability.

[0035] In this application, the term "camera" includes a video camera capable of shooting moving images. The camera may be a camera that supports visible light, or a camera that supports light in a wavelength band other than the visible light wavelength band (e.g., infrared light).

[0036] The dross position estimation device 30 may include a plurality of partial cameras 41, 42, 43 as the camera 40. The partial cameras 41, 42, 43 face the liquid surface from above (including diagonally above) with different fields of view, and each captures the liquid surface. Each of the partial cameras 41, 42, 43 includes an imaging element such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor, and an optical system that forms an image of light from the field of view on the imaging element. Capturing the image of the liquid surface includes capturing an image of the liquid surface covered with top dross. The camera 40 may include an actuator 40a that changes the field of view direction of the camera 40. By changing the field of view direction with the actuator 40a, a wider range can be captured.

[0037] Estimating the boundary line includes, but is not limited to, calculating the coordinates of each of a plurality of points along the boundary line. For example, deriving the coefficients of a function that approximates the boundary line is also included in estimating the boundary line. Expressing the boundary line as a combination of a plurality of figures and calculating the coefficients of the function of each figure is also included in estimating the boundary line.

[0038] As shown in FIG. 2, the arithmetic device 100 has, as functional components (hereinafter referred to as "functional blocks"), a recording unit 111, a video data storage unit 112, a brightness map generation unit 113, a displacement map generation unit 114, an input data generation unit 115, a correct answer data acquisition unit 116, an accumulation unit 117, a database 118, and a model generation unit 119. The recording unit 111 acquires a plurality of images photographed in time series by the camera 40. For example, the recording unit 111 causes the camera 40 to repeat photographing at a predetermined cycle, and acquires a plurality of images obtained by the repetition over a period of a predetermined length. The recording unit 111 stores the acquired plurality of images in time series in the video data storage unit 112. As a result, video data including a plurality of images in time series is held at least temporarily in the video data storage unit 112.

[0039] Video data is data that allows the displacement of elements included in an image to be visually recognized by sequentially displaying multiple images in chronological order. The displacement of elements included in an image is visually recognized by video data because an element included in a previous image is also included in a subsequent image at a position different from that in the previous image. Hereinafter, a common element included in both a previous image and a subsequent image at different positions is referred to as a "common displacement element." If the above-mentioned predetermined period is too long and the previous image and the subsequent image do not include a common displacement element, the multiple images obtained by repeating shooting at the predetermined period do not correspond to video data. The predetermined period for acquiring multiple images corresponding to video data is, for example, 0.1 seconds or less, may be 0.06 seconds or less, or may be 0.03 seconds or less.

[0040] The recording unit 111 may synthesize images captured by each of the partial cameras 41, 42, and 43 to generate one image, and store the generated one image in chronological order in the video data storage unit 112. For example, the recording unit 111 may convert images captured by each of the partial cameras 41, 42, and 43 into images from a common viewpoint based on the positions and orientations of each of the partial cameras 41, 42, and 43 in a common three-dimensional coordinate system, and synthesize the conversion result with the image of the plating apparatus 1. For example, the recording unit 111 may convert images captured by each of the partial cameras 41, 42, and 43 into images from a common viewpoint located vertically above the plating pot 2, and synthesize the conversion result with the image of the plating apparatus 1 viewed from vertically above the plating pot 2.

[0041] When the camera 40 includes the actuator 40a described above, the recording unit 111 may synthesize multiple images taken by the camera 40 in multiple postures to generate a single image, and store the generated single image in chronological order in the video data storage unit 112.

[0042] Returning to FIG. 2, the luminance map generating unit 113 generates a luminance map based on the video data stored in the video data storage unit 112. The luminance map is data representing the luminance for each of a plurality of areas. The plurality of areas are, for example, a plurality of areas obtained by subdividing the field of view of the camera 40 (within the image obtained by the camera 40) in a matrix shape. The granularity of the subdivision within the field of view of the camera 40 may be determined so that the boundary line can be approximated with a desired reliability by the boundaries between the areas. For example, the granularity of the subdivision may be determined so that the size of each area is an allowable error for the estimated position of the boundary line. For example, the luminance map generating unit 113 generates a luminance map including the luminance values ​​of each of the plurality of areas. The luminance map generating unit 113 may calculate the luminance value of each of the plurality of areas based on the luminance values ​​of a plurality of pixels included in the area. For example, the luminance map generating unit 113 may calculate the average value of the luminance values ​​of a plurality of pixels as the luminance value of each of the plurality of areas.

[0043] The luminance map generating unit 113 may calculate the luminance value of each of the multiple areas based on the luminance values ​​in multiple images of the video data. For example, the luminance map generating unit 113 may calculate the average value of the luminance values ​​in the multiple images as the luminance value of each of the multiple areas.

[0044] The luminance map generating unit 113 may generate a luminance map for each of the above-mentioned multiple colors. For example, the luminance map generating unit 113 calculates a luminance value of R (Red) for each of the multiple areas to generate a luminance map of R. For example, the luminance map generating unit 113 calculates a luminance value of R based on the luminance value of a pixel for R included in each of the multiple areas. Similarly, the luminance map generating unit 113 calculates a luminance value of G (Green) for each of the multiple areas to generate a luminance map of G, and calculates a luminance value of B (Blue) for each of the multiple areas to generate a luminance map of B.

[0045] The displacement map generating unit 114 generates a displacement map based on the video data stored in the video data storage unit 112. The displacement map is data that indicates, for each of a plurality of areas, the displacement vector of the subject within the area. The displacement vector includes information on the movement direction and movement distance of the subject within the area. The plurality of areas are, for example, a plurality of areas obtained by subdividing the field of view of the camera 40 (within the image obtained by the camera 40) in a matrix form. The granularity of the subdivision within the field of view of the camera 40 may be determined so that the boundary line can be approximated with a desired reliability by the boundaries between the areas.

[0046] The multiple areas for generating the displacement map and the multiple areas for generating the brightness map may be the same or different from each other. For example, the displacement map generating unit 114 generates a displacement map based on a comparison between an image of the plating apparatus 1 included in the video data as a reference image and an image immediately before the reference image in time series or an image immediately after the reference image in time series as a comparison image. For example, the displacement map generating unit 114 generates a displacement map by performing the following process on each of the multiple areas. Process 1) In a reference image, a common displacement element is determined within an area. Process 2) Calculate a displacement vector of the area based on the difference between the position of the common displacement element in the reference image and the position of the common displacement element in the comparison image.

[0047] The displacement map generating unit 114 may generate a displacement map using a plurality of images included in the video data as reference images. For example, the brightness map generating unit 113 may perform the above-mentioned process on each of the plurality of reference images to calculate a plurality of displacement vectors for each of the plurality of areas, and may calculate the displacement vector for each of the plurality of areas by averaging the calculated plurality of displacement vectors.

[0048] Fig. 3 is a diagram illustrating a schematic example of a displacement map. The displacement map 200 shown in Fig. 3 represents a displacement vector 211 for each of a plurality of areas 210. Each of the plurality of areas 210 is an area obtained by partitioning the field of view of the camera 40 in a matrix shape. Each of the plurality of areas 210 has a plurality of pixels arranged in a matrix shape.

[0049] The displacement vector 211 indicates the direction and amount of displacement of the common displacement element in the image. In FIG. 3, the dashed line indicates the boundary line between the area of ​​the liquid surface covered with dross and the area not covered with dross. The liquid surface covered with dross is less likely to flow than the liquid surface not covered with dross. Therefore, the displacement vector 211 of the area 210A covered with dross is smaller than that of the area 210B not covered with dross. In the figure, a flow in a direction away from the steel strip 9 occurs on the liquid surface not covered with dross, and thus a displacement vector 211 in a direction away from the steel strip 9 occurs in each of the multiple areas 210B. In contrast, the displacement vector 211 does not substantially occur in the multiple areas 210B. Therefore, the boundary line between the area covered with dross and the area not covered with dross is likely to appear in the displacement map 200 due to the boundary between the multiple areas 210A and the multiple areas 210B.

[0050] Returning to Fig. 2, the input data generation unit 115 generates input data based on the video data. For example, the input data generation unit 115 generates input data including at least the displacement map generated by the displacement map generation unit 114. The input data generation unit 115 generates input data including the displacement map generated by the displacement map generation unit 114 and a luminance map generated by the input data generation unit 115. When the luminance map generation unit 113 generates a luminance map for each of the multiple colors, the input data generation unit 115 may generate input data including the displacement map generated by the displacement map generation unit 114 and the luminance map generated by the luminance map generation unit 113 for each of the multiple colors.

[0051] The correct data acquisition unit 116 acquires correct data indicating the boundary line visually recognized based on the video data. For example, the correct data acquisition unit 116 may acquire data indicating the boundary position recognized by the operator based on the playback result of the video data stored in the video data storage unit 112, based on an input to a user interface 196 (described later).

[0052] The storage unit 117 stores in the database 118 a learning record in which the input data generated by the input data generation unit 115 is associated with the correct answer data acquired by the correct answer data acquisition unit 116. For example, the storage unit 117 generates a learning record by associating input data based on the same video data with the correct answer data, and stores the learning record in the database 118.

[0053] The model generation unit 119 generates the estimation model by machine learning based on a plurality of learning records stored in the database 118, and stores the generated estimation model in the model storage unit 121. The estimation model generated by the model generation unit 119 represents at least the relationship between the displacement map and the boundary line. When the input data includes both the displacement map and the brightness map, the estimation model generated by the model generation unit 119 represents the relationship between both the displacement map and the brightness map and the boundary line.

[0054] For example, the model generation unit 119 generates an estimation model that expresses the relationship between the input data and the boundary line by a multi-stage input / output relationship. The input / output relationship may be expressed by a case classification process that classifies the input and generates an output according to the case classification result, or a probabilistic case classification process. As an example, the model generation unit 119 may generate the estimation model by deep learning.

[0055] FIG. 4 is a diagram illustrating a schematic example of an estimation model generated by deep learning. The estimation model 300 shown in FIG. 4 is a neural network, and includes an input layer 311, one or more output layers 313, and an intermediate layer 312. The input layer 311 outputs an input vector to the next output layer 313. The output layer 313 converts the input from the previous layer by an activation function and outputs it to the next layer. The intermediate layer 312 converts the input from the output layer 313, which is the farthest from the input layer 311, by an activation function and outputs the conversion result as an output vector. In the estimation model 300, the activation function in each output layer 313 and the activation function in the intermediate layer 312 are an example of the above-mentioned multi-stage input / output relationship. According to the process of converting the input by the activation function, the conversion result changes depending on the input. In other words, since the conversion result changes depending on the input case, the conversion by the activation function is an example of the above-mentioned case classification process or probabilistic case classification process.

[0056] Returning to FIG. 2, the arithmetic device 100 further includes an estimation unit 122. The estimation unit 122 estimates a boundary line based on the estimation model stored in the model storage unit 121 and input data newly generated by the input data generation unit 115. "Newly generated" means that the input data is generated after the estimation model is generated by the model generation unit 119. Hereinafter, the input data newly generated by the input data generation unit 115 is referred to as "new input data."

[0057] The new input data includes a displacement map newly generated by the displacement map generation unit 114. The newly generated displacement map is a displacement map generated by the displacement map generation unit 114 based on video data acquired by the recording unit 111 via the camera 40 after the estimation model is generated by the model generation unit 119. The new input data may further include a luminance map newly generated by the luminance map generation unit 113. The newly generated luminance map is a luminance map generated by the luminance map generation unit 113 based on video data acquired by the recording unit 111 via the camera 40 after the estimation model is generated by the model generation unit 119.

[0058] For example, the estimation unit 122 inputs new input data to the estimation model stored in the model storage unit 121, and estimates the boundary line based on the output of the estimation model according to the input of the new input data. When the estimation model is generated to represent the relationship between the boundary line and input data further including a luminance map, the estimation unit 122 estimates the boundary line based on the new input data further including the luminance map and the estimation model.

[0059] The arithmetic device 100 may further include a difference data generating unit 131 and a determination unit 132. The difference data generating unit 131 generates difference data between a predetermined reference line and an estimated boundary line. The reference line may be represented by, for example, coordinates of multiple positions aligned along the reference line. The reference line may be represented by coefficients of a function that approximates the reference line. The reference line may be represented by coefficients of a function that represents each of multiple figures that become the reference line by synthesis.

[0060] The difference data may include, for example, the difference between the boundary line and the reference line at multiple locations aligned in the circumferential direction around the steel strip 9. The difference between the boundary line and the reference line may include a sign indicating whether the boundary line is located inside the reference line (closer to the steel strip 9). For example, the difference between the boundary line and the reference line may be calculated to be a positive value when the boundary line is located inside the reference line and a negative value when the boundary line is located outside the reference line. The difference between the boundary line and the reference line may be calculated to be a positive value when the boundary line is located outside the reference line and a positive value when the boundary line is located outside the reference line. In the following, when illustrating a process based on the difference between the boundary line and the reference line, a case will be illustrated in which the difference is a positive value when the boundary line is located on the position side of the reference line.

[0061] The difference data may be a statistical value of the differences at a plurality of points, for example, the maximum value, average value, or median value of the differences at a plurality of points.

[0062] The determination unit 132 determines whether or not dross removal is necessary based on the difference data. For example, the determination unit 132 determines whether or not dross removal is necessary based on a comparison between the difference data and a predetermined difference threshold. When the difference data includes differences at multiple locations, the determination unit 132 may determine that dross removal is necessary when the difference at at least one of the multiple locations exceeds the difference threshold, and may determine that dross removal is not necessary when the differences at all of the multiple locations are below the difference threshold.

[0063] The determination unit 132 may determine whether or not dross removal is necessary based on a comparison between a statistical value (e.g., a maximum value, an average value, or a median value) of differences at multiple locations and a difference threshold. For example, the determination unit 132 may determine that dross removal is necessary when the average value of the differences at multiple locations exceeds the difference threshold, and may determine that dross removal is not necessary when the average value of the differences at multiple locations is below the difference threshold.

[0064] The determination unit 132 may determine whether or not dross removal is necessary based on the change speed of the boundary line. The change speed of the boundary line represents, for example, the displacement of the boundary line per unit time at at least one location. For example, the determination unit 132 may determine whether or not dross removal is necessary based on a comparison between the absolute value of the change speed of the boundary line and a predetermined speed threshold. For example, the determination unit 132 may determine that dross removal is necessary when the absolute value of the change speed of the boundary line exceeds the speed threshold, and may determine that dross removal is not necessary when the absolute value of the change speed of the boundary line is below the speed threshold.

[0065] The determination unit 132 may calculate the speed of change of the boundary line starting from the time when the operator finishes the dross removal work, and may determine whether or not the dross needs to be removed based on the calculated speed of change. The determination unit 132 may detect the end of the dross removal work based on the retraction of the tool 8 from within the image, or may detect the end of the dross removal work based on an input to the user interface 196 by the operator.

[0066] When it is determined that a dross removal operation is required, the determination unit 132 may notify the operator of the determination result by, for example, displaying the result on a user interface 196 (described later).

[0067] The arithmetic device 100 may further include a drawing unit 133. The drawing unit 133 draws a map image of the liquid surface including the boundary line viewed from above on a user interface 196 (described later) or the like, based on the boundary line estimated by the estimation unit 122. By graphically showing the boundary line, it is possible to further improve usability. The drawing unit 133 may draw a map image further including a reference line.

[0068] Fig. 5 is a schematic diagram illustrating a map image. The map image 400 shown in Fig. 5 includes a region 413 covered with dross, a region 414 not covered with dross, a boundary line 411 between the region 413 and the region 414, and a reference line 412. The map image 400 may further include a steel plate cross section 415 indicating the position of the steel strip 9.

[0069] Returning to FIG. 2, the calculation device 100 may further include a reference setting unit 134. The reference setting unit 134 sets the above-mentioned reference line. The reference setting unit 134 may set the reference line based on the estimation result of the boundary line. For example, the reference setting unit 134 may set the estimation result of the boundary line estimated immediately before the operator determines that the dross needs to be removed based on a map image or the like as the reference line. The difference data generation unit 131 may generate difference data representing the difference between the difference data set by the reference setting unit 134 and the boundary line. The determination unit 132 may determine whether or not the dross needs to be removed based on the difference data representing the difference between the difference data set by the reference setting unit 134 and the boundary line.

[0070] The reference setting unit 134 may set a reference line based on the operating conditions of the steel strip 9. For example, the calculation device 100 may further have an operating condition acquisition unit 135. The operating condition acquisition unit 135 acquires the operating conditions of the plating apparatus 1 from the control device 20 or the like. The operating conditions include at least the feed speed of the steel strip 9. The operating conditions may further include the above-mentioned wiping pressure and the width of the steel strip 9. The operating conditions may further include the zinc bath temperature and the zinc bath components. The reference setting unit 134 may set a reference line based on the operating conditions acquired by the operating condition acquisition unit 135.

[0071] For example, the reference setting unit 134 sets a reference line based on a reference profile that is determined in advance to represent the relationship between the operating conditions and the reference line, and on newly acquired operating conditions.

[0072] 6, the calculation device 100 may further include a reference data accumulation unit 141, a reference database 142, a profile generation unit 143, and a profile storage unit 144. The reference data accumulation unit 141 repeatedly stores the operating conditions acquired by the operating condition acquisition unit 135 and the estimation results of the boundary line by the estimation unit 122 in the reference database 142 in association with each other. As a result, a plurality of records each including the operating conditions and the estimation results of the boundary line are accumulated in the reference database 142.

[0073] The profile generating unit 143 generates the reference profile based on a plurality of records stored in the reference database 142, and stores the reference profile in the profile storage unit 144. The profile generating unit 143 may generate the reference profile expressed as a lookup table that discretely expresses the relationship between the operating conditions and the reference line, or may generate the reference profile that expresses the relationship between the operating conditions and the reference line as a function.

[0074] In a configuration in which the calculation device 100 includes a reference data accumulation unit 141, a reference database 142, a profile generation unit 143, and a profile storage unit 144, the reference setting unit 134 may set a reference line based on the reference profile stored in the profile storage unit 144 and newly acquired operating conditions.

[0075] The arithmetic device 100 may further include a tool recognition unit 146 and a correction unit 147. The tool recognition unit 146 recognizes the position of a tool (e.g., the above-mentioned tool 8) for work around a plating pot, such as work to remove dross, based on the video data stored in the video data storage unit 112. For example, the tool recognition unit 146 may recognize the tool 8 by image processing such as pattern matching, or may recognize the tool 8 by an image processing engine generated by machine learning.

[0076] The correction unit 147 corrects the difference data so as to exclude, from the difference between the reference line and the estimated boundary line, any difference occurring within a predetermined range including the position of the tool 8. For example, the correction unit 147 acquires the position of the tool 8 based on the tool recognition result by the tool recognition unit 146, calculates the above-mentioned predetermined range based on the acquired position of the tool 8, and excludes the difference occurring within the predetermined range from the difference data. For example, the correction unit 147 calculates a range of a predetermined shape and size centered on the position of the tool in the image as the above-mentioned predetermined range.

[0077] In a configuration in which the arithmetic device 100 further includes a correction unit 147, the determination unit 132 may determine whether or not dross removal is necessary based on the corrected difference data. The boundary line may move significantly due to the tool 8. For example, an opening may be formed by the tool in an area covered by dross, generating a new boundary line. It may be a wasteful process to determine that dross removal is necessary based on the difference between the reference line and the boundary line that occurs during the dross removal process. By excluding the difference that occurs within a specified range from the difference data, the above-mentioned wasteful process can be avoided.

[0078] Fig. 7 is a block diagram illustrating a hardware configuration of the arithmetic device 100. As shown in Fig. 7, the arithmetic device 100 has a circuit 190. The circuit 190 has a processor 191, a memory 192, a storage 193, a graphics circuit 194, a communication port 195, and a user interface 196.

[0079] The storage 193 is composed of one or more non-volatile memory devices such as a flash memory or a hard disk. The storage 193 stores a program for causing the calculation device 100 to execute the following operations: generating a displacement map; and estimating a boundary line based on an estimation model and new input data. For example, the storage 193 stores a program for causing the calculation device 100 to configure each of the above-mentioned functional blocks.

[0080] The memory 192 is composed of one or more volatile memory devices such as a random access memory. The memory 192 temporarily stores a program loaded from the storage 193. The processor 191 is composed of one or more arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The processor 191 executes the program loaded in the memory 192, thereby configuring each of the above-mentioned functional blocks in the arithmetic device 100. The calculation results by the processor 191 are temporarily stored in the memory 192.

[0081] The graphic circuit 194 controls the camera 40 in response to a request from the processor 191. The communication port 195 communicates (e.g., network communication) with the control device 20 in response to a request from the processor 191. The user interface 196 includes a display device and an input device. The display device displays information for the user. Examples of the display device include a liquid crystal display and an organic EL (Electro-Luminescence) display. The input device acquires input by the user. Examples of the input device include a keyboard and a mouse. The input device may be integrated with the display device as, for example, a touch panel.

[0082] The hardware configuration described above is merely an example and can be modified as appropriate. For example, the circuit 190 may be divided into multiple circuits that can communicate with each other. The circuit 190 may also be incorporated into the dross position estimation device 30.

[0083] [Dross location estimation procedure] Hereinafter, as an example of a dross position estimation method, a dross position estimation procedure executed by the calculation device 100 will be illustrated.

[0084] This procedure includes acquiring video data including multiple images captured in time series by camera 40; generating a displacement map representing the displacement vector of the subject within each of multiple areas in the field of view of camera 40 based on the video data; acquiring ground truth data indicating the boundary line between areas of the liquid surface covered by dross and areas not covered by dross, as viewed based on the video data; accumulating in a database learning records that correspond the input data including the displacement map to the ground truth data; generating an estimation model representing the relationship between the input data and the boundary line based on the multiple learning records accumulated in the database; generating a new displacement map based on the newly acquired video data; and estimating the boundary line based on the new input data including the newly generated displacement map and the estimation model.

[0085] As an example, the dross position estimation procedure includes a procedure for generating an estimation model, a procedure for generating a reference profile, and a procedure for determining whether dross removal is required. The procedure for generating the reference profile is performed after the procedure for generating the estimation model. The procedure for determining whether dross removal is required is performed after the procedure for generating the estimation model and the procedure for generating the reference profile. Below, the procedure for generating the estimation model, the procedure for generating the reference profile, and the procedure for determining whether dross removal is required are illustrated in order.

[0086] (Procedure for generating estimation model) As shown in Fig. 8, the arithmetic device 100 first executes steps S01, S02, S03, and S04. In step S01, the recording unit 111 stores video data including a plurality of images captured in time series by the camera 40 in the video data storage unit 112. In step S02, the displacement map generation unit 114 generates the displacement map based on the video data stored in the video data storage unit 112. In step S03, the brightness map generation unit 113 generates the brightness map based on the video data stored in the video data storage unit 112. In step S04, the input data generation unit 115 generates input data including the displacement map generated by the displacement map generation unit 114 and the brightness map generated by the brightness map generation unit 113.

[0087] Next, the arithmetic device 100 executes steps S05, S06, and S07. In step S05, the correct answer data acquisition unit 116 acquires the correct answer data. In step S06, the storage unit 117 generates a learning record in which the input data generated by the input data generation unit 115 and the correct answer data acquired by the correct answer data acquisition unit 116 are associated with each other, and stores the learning record in the database 118. In step S07, the model generation unit 119 checks whether the number of learning records stored in the database 118 exceeds a predetermined threshold value.

[0088] If it is determined in step S07 that the number of learning records does not exceed the threshold, the arithmetic device 100 returns the process to step S01. Thereafter, the acquisition of video data, the generation of input data, the acquisition of correct answer data, and the accumulation of learning records are repeated until the number of learning records exceeds the threshold.

[0089] If it is determined in step S07 that the number of learning records exceeds the threshold, the calculation device 100 executes step S08. In step S08, the model generation unit 119 generates the estimation model by machine learning based on the learning records accumulated in the database 118, and stores the estimation model in the model storage unit 121. This completes the procedure for generating the estimation model.

[0090] Note that step S03 may be executed before step S02. The execution period of step S02 and the execution period of step S03 may partially overlap. Also, performing step S05 after step S04 is merely an example, and step S05 may be executed at any timing after step S01 or before step S06.

[0091] (Steps for generating a reference profile) As shown in Fig. 9, the calculation device 100 executes steps S11, S12, S13, and S14 similar to steps S01, S02, S03, and S04. In step S11, the recording unit 111 stores video data in the video data storage unit 112. In step S12, the displacement map generation unit 114 generates the above-mentioned displacement map based on the video data stored in the video data storage unit 112. In step S13, the brightness map generation unit 113 generates the above-mentioned brightness map based on the video data stored in the video data storage unit 112. In step S14, the input data generation unit 115 generates input data including the displacement map generated by the displacement map generation unit 114 and the brightness map generated by the brightness map generation unit 113.

[0092] Next, the calculation device 100 executes steps S15, S16, and S17. In step S15, the estimation unit 122 estimates the boundary line based on the estimation model stored in the model storage unit 121 and the input data newly generated by the input data generation unit 115. In step S16, the operating condition acquisition unit 135 acquires the operating conditions of the plating apparatus 1 from the control device 20, etc. In step S17, the reference data accumulation unit 141 associates the operating conditions acquired by the operating condition acquisition unit 135 with the estimation result of the boundary line by the estimation unit 122 and stores them in the reference database 142.

[0093] Next, the arithmetic device 100 executes step S18. In step S18, the profile generating unit 143 checks whether or not the type and number of records that can generate a reference profile have been accumulated in the reference database 142. For example, the profile generating unit 143 checks whether or not the reference profile can be generated based on the number of records accumulated in the reference database 142 and the degree of dispersion of the operating conditions of the accumulated records. If it is determined in step S18 that the reference profile cannot be generated, the arithmetic device 100 returns the process to step S11. If it is determined in step S18 that the reference profile can be generated, the arithmetic device 100 executes step S19. In step S19, the profile generating unit 143 generates a reference profile based on the records accumulated in the reference database 142, and stores the reference profile in the profile storage unit 144.

[0094] This completes the procedure for generating the reference profile. Note that step S13 may be executed before step S12. The execution period of step S12 and the execution period of step S13 may partially overlap. Also, step S16 being executed after step S15 is merely an example. Step S16 may be executed at any timing from before step S11 to before step S17.

[0095] (Location Estimation Procedure) As shown in FIG. 10, the arithmetic device 100 executes steps S21, S22, S23, S24, S25, and S26 similar to steps S11, S12, S13, S14, S15, and S16. In step S21, the recording unit 111 stores video data in the video data storage unit 112. In step S22, the displacement map generation unit 114 generates the above-mentioned displacement map based on the video data stored in the video data storage unit 112. In step S23, the brightness map generation unit 113 generates the above-mentioned brightness map based on the video data stored in the video data storage unit 112. In step S24, the input data generation unit 115 generates input data including the displacement map generated by the displacement map generation unit 114 and the brightness map generated by the brightness map generation unit 113. In step S25, the estimation unit 122 estimates a boundary line based on the estimation model stored in the model storage unit 121 and the input data newly generated by the input data generation unit 115. In step S26, the operation condition acquisition unit 135 acquires the operation conditions of the plating apparatus 1 from the control device 20 or the like.

[0096] Next, the arithmetic device 100 executes steps S27 and S28. In step S27, the reference setting unit 134 sets a reference line based on the operating conditions acquired in step S26 and the reference profile stored in the profile storage unit 144. For example, the reference setting unit 134 identifies a reference line corresponding to the operating conditions acquired in step S26 from the reference profile. In step S28, the drawing unit 133 draws a map image of the liquid level viewed from above, including the boundary line estimated by the estimation unit 122 and the reference line set by the reference setting unit 134, on the user interface 196 (described later) or the like.

[0097] 11, the arithmetic device 100 then executes steps S31, S32, S33, and S34. In step S31, the difference data generating unit 131 generates difference data between the reference line set by the reference setting unit 134 and the boundary line estimated by the estimation unit 122. In step S32, the determination unit 132 calculates the displacement speed of the boundary line. For example, the determination unit 132 calculates the displacement speed of the boundary line based on the difference between the boundary line estimated in step S25 and the boundary line estimated in the previous step S25, and the elapsed time from the previous step S25 to step S25.

[0098] In step S33, the tool recognition unit 146 recognizes the position of the tool 8 based on the video data stored in the video data storage unit 112. In step S34, the correction unit 147 corrects the difference data generated in step S31 so as to exclude a difference occurring in a predetermined range including the position of the tool 8 from the difference between the reference line and the estimated boundary line.

[0099] Next, the calculation device 100 executes step S35. In step S35, the determination unit 132 checks whether or not at least any of the differences at multiple locations including the difference data exceeds a difference threshold value.

[0100] If it is determined in step S35 that all the differences are below the difference threshold, the calculation device 100 executes step S36. In step S36, the determination unit 132 checks whether the displacement speed of the boundary line exceeds the speed threshold.

[0101] If it is determined in step S36 that the displacement speed of the boundary line is below the speed threshold, the calculation device 100 returns the process to step S21 (see FIG. 10). If it is determined in step S35 that at least one of the differences at multiple locations exceeds the difference threshold, and if it is determined in step S36 that the displacement speed exceeds the speed threshold, the calculation device 100 executes step S37. In step S37, the determination unit 132 determines that dross removal is necessary, and notifies the determination result. After that, the calculation device 100 returns the process to step S21. The calculation device 100 repeats the above process at a predetermined cycle.

[0102] 〔summary〕 The above-described embodiment includes the following configurations. (1) In a continuous plating apparatus 1 in which a steel strip is continuously passed through a liquid plating material contained in a plating pot 2, the apparatus comprises: a camera 40 that photographs the liquid surface of the plating material in the plating pot 2; a displacement map generation unit 114 that generates a displacement map representing the displacement vector of the photographed object in each of a plurality of areas in the field of view of the camera 40 based on video data including a plurality of images photographed in time series by the camera 40; and a dross position estimation device 30 that estimates the boundary line based on input data including the displacement map, an estimation model generated by machine learning to represent the relationship between the boundary line between the area of ​​the liquid surface that is covered by dross and the area that is not covered by dross, and input data including the newly generated displacement map. In Patent Document 1, it is possible to detect whether or not dross is present at a specific position. However, even if there is no dross at a specific position, there is a possibility that dross has approached the steel strip at another position, so the need for dross removal cannot be detected with high reliability only by detection at a specific position. To detect the position of dross over a wide range, it is effective to use an image captured over a wide range, but when the brightness difference between the area covered with dross and the area not covered with dross is small, the boundary line cannot be detected with high reliability. According to the dross position estimation device 30 of the present disclosure, a displacement map representing the displacement vector of the photographed object within each of multiple areas in the field of view of the camera 40 is generated, and the boundary line is estimated based on input data including the displacement map, an estimation model generated by machine learning to represent the relationship with the boundary line, and new input data including the newly generated displacement map. Even when the brightness difference between the area covered with dross and the area not covered with dross is small, the difference in displacement vector appears clearly. Therefore, based on the estimation model generated by including the displacement map in the input data and the new input data including the newly generated displacement map, the boundary line in a wide range can be estimated with high reliability. Therefore, the dross position estimation device 30 is effective in detecting the necessity of removing dross with high reliability.

[0103] (2) The dross position estimation device 30 described in (1) further includes a judgment unit 132 that judges whether or not dross needs to be removed based on differential data representing the difference between a predetermined reference line and the estimated boundary line. Based on the difference between the boundary line estimated with high reliability and the reference line, it is possible to determine with high reliability whether or not dross needs to be removed.

[0104] (3) The dross position estimation device 30 described in (2) further includes a reference setting unit 134 that sets a reference line based on the estimation result of the boundary line, and the judgment unit 132 judges whether or not dross needs to be removed based on differential data representing the difference between the set reference line and the estimated boundary line. The estimated boundary line results can also be used to set the reference line, improving usability.

[0105] (4) A dross position estimation device 30 described in (2) or (3), further comprising a reference setting unit 134 that sets a reference line based on operating conditions including at least the feed speed of the steel strip, a reference profile that is predetermined to represent a relationship with the reference line, and newly acquired operating conditions, and a judgment unit 132 that judges whether or not dross needs to be removed based on the difference between the set reference line and the estimated boundary line. The boundary line may also change depending on the operating conditions. In response to this, by setting a reference line according to the operating conditions, it is possible to more reliably determine whether or not dross removal is required.

[0106] (5) The dross position estimation device 30 described in (4) further includes a profile generation unit 143 that generates a reference profile based on a plurality of records, each of which includes operating conditions including at least the feed speed of the steel strip and an estimated result of the boundary line, and the reference setting unit 134 sets a reference line based on the generated reference profile and newly acquired operating conditions. The results of estimating the boundary lines can also be used to generate a reference profile, improving usability.

[0107] (6) The dross position estimation device 30 described in any one of (1) to (5), further comprising a drawing unit 133 that draws a map image of the liquid surface including the boundary line viewed from above based on the estimated boundary line. By showing the boundary lines graphically, usability can be further improved.

[0108] (7) A dross position estimation device 30 described in any one of (1) to (6), further comprising a brightness map generation unit 113 that generates a brightness map representing the brightness for each of multiple areas based on video data, and the estimation unit 122 estimates the boundary line based on input data further including the brightness map, an estimation model generated by machine learning to represent a relationship with the boundary line, and new input data further including the brightness map. The boundary line can be estimated with greater reliability.

[0109] (8) The dross position estimation device 30 described in any one of (2) to (5) further includes a tool recognition unit 146 that recognizes the position of a tool (e.g., the above-mentioned tool 8) for work around the plating pot 2, such as for work to remove dross, based on video data, and a correction unit 147 that corrects difference data so as to exclude a difference between the reference line and the estimated boundary line that occurs within a predetermined range including the position of the tool, and the judgment unit 132 judges whether or not dross removal is necessary based on the corrected difference data. It is possible to determine with greater reliability whether or not dross needs to be removed.

[0110] (9) The dross position estimation device 30 according to any one of (2) to (5), wherein the determination unit 132 determines whether or not the dross needs to be removed based further on the changing speed of the boundary line. The need for dross removal can be determined with greater reliability.

[0111] (10) A dross position estimation device 30 described in any one of (1) to (9), further comprising a correct answer data acquisition unit 116 that acquires correct answer data indicating a boundary line visually recognized based on video data, a storage unit 117 that stores learning records in a database 118 that correspond to input data and correct answer data, and a model generation unit 119 that generates an estimation model by machine learning based on a plurality of learning records stored in the database 118. By further performing processes from accumulating learning records to generating an estimation model, the usability of the dross position estimation device 30 is further improved.

[0112] (11) A dross position estimation device 30 as described in (10), wherein the model generation unit 119 generates an estimation model by deep learning. The boundary line can be estimated with greater reliability.

[0113] (12) A method for estimating a dross position in a continuous plating apparatus 1 in which a steel strip is continuously passed through a liquid plating material contained in a plating pot 2, the method comprising: acquiring video data including a plurality of images captured in a time series by a camera 40 that captures an image of the liquid surface of the plating material in the plating pot 2; generating a displacement map representing a displacement vector of an object captured in each of a plurality of areas in the field of view of the camera 40 based on the video data; acquiring ground truth data indicating a boundary line between an area of ​​the liquid surface that is covered with dross and an area that is not covered with dross, as viewed based on the video data; accumulating in a database 118 learning records that correspond to input data including the displacement map and the ground truth data; generating an estimation model representing the relationship between the input data and the boundary line based on the plurality of learning records accumulated in the database 118; generating a new displacement map based on the newly acquired video data; and estimating the boundary line based on the new input data including the newly generated displacement map and the estimation model. [Explanation of symbols]

[0114] 1...plating device, 30...dross position estimation device, 40...camera, 113...brightness map generation unit, 114...displacement map generation unit, 116...correct data acquisition unit, 117...storage unit, 118...database, 119...model generation unit, 122...estimation unit, 132...judgment unit, 133...drawing unit, 134...reference setting unit, 143...profile generation unit, 146...tool recognition unit, 147...correction unit.

Claims

1. In a continuous plating apparatus in which a steel strip is continuously passed through a liquid plating material contained in a plating pot, a camera is provided for photographing the liquid surface of the plating material in the plating pot; a displacement map generating unit that generates a displacement map representing a displacement vector of a subject within a plurality of areas in a field of view of the camera based on video data including a plurality of images captured by the camera in time series; an estimation unit that estimates the boundary line based on an estimation model generated by machine learning to represent a relationship between input data including the displacement map, a boundary line between an area of ​​the liquid surface that is covered with dross and an area of ​​the liquid surface that is not covered with dross, and the input data including the newly generated displacement map; A dross position estimation device comprising:

2. The method further includes a determination unit that determines whether or not the dross needs to be removed based on difference data representing a difference between a predetermined reference line and the estimated boundary line. The dross position estimation device according to claim 1.

3. a reference setting unit that sets the reference line based on a result of estimating the boundary line, The determination unit determines whether or not the dross needs to be removed based on difference data representing a difference between the set reference line and the estimated boundary line. The dross position estimation device according to claim 2.

4. Further comprising a reference setting unit that sets the reference line based on a reference profile that is predetermined to represent a relationship between the operating conditions including at least the feed speed of the steel strip and the reference line, and the newly acquired operating conditions; The determination unit determines whether or not the dross needs to be removed based on a difference between the set reference line and the estimated boundary line. The dross position estimating device according to claim 2.

5. A profile generating unit generates the reference profile based on a plurality of records each including an operating condition including at least a feed speed of the steel strip and an estimated result of the boundary line, The reference setting unit sets the reference line based on the generated reference profile and the newly acquired operating conditions. The dross position estimating device according to claim 4.

6. Based on the estimated boundary line, A drawing unit that draws a map image of the liquid surface including the boundary line viewed from above. The dross position estimation device according to any one of claims 1 to 5.

7. a brightness map generating unit that generates a brightness map representing a brightness of each of the plurality of areas based on the moving image data, The estimation unit performs the following calculation based on the input data further including the luminance map, the estimation model generated by machine learning to represent a relationship with the boundary line, and new input data further including the luminance map: estimating the boundary line; The dross position estimation device according to any one of claims 1 to 5.

8. a tool recognition unit that recognizes the position of a tool for work around the plating pot based on the video data; a correction unit that corrects difference data so as to exclude a difference occurring within a predetermined range including a position of the tool, from a difference between the reference line and the estimated boundary line; Further comprising: The determination unit determines whether or not the dross needs to be removed based on the corrected difference data. The dross position estimation device according to any one of claims 2 to 5.

9. The determination unit determines whether or not the dross needs to be removed based on the change speed of the boundary line. The dross position estimation device according to any one of claims 2 to 5.

10. a correct answer data acquisition unit that acquires correct answer data indicating the boundary line visually recognized based on the video data; a storage unit that stores in a database a learning record in which the input data and the correct answer data are associated with each other; a model generation unit that generates the estimation model by machine learning based on a plurality of learning records stored in the database; Further comprising: The dross position estimation device according to any one of claims 1 to 5.

11. The model generation unit generates the estimation model by deep learning. The dross position estimation device according to claim 10.

12. In a continuous plating apparatus in which a steel strip is continuously passed through a liquid plating material contained in a plating pot, a camera is used to photograph the liquid surface of the plating material in the plating pot, and video data including a plurality of images photographed in time series is acquired; generating a displacement map representing a displacement vector of a subject within each of a plurality of areas in a field of view of the camera based on the video data; obtaining answer data indicating a boundary line between an area of ​​the liquid surface that is covered with dross and an area of ​​the liquid surface that is not covered with dross, the boundary line being visually recognized based on the video data; accumulating a learning record in a database in which input data including the displacement map is associated with the correct answer data; generating an estimation model representing a relationship between input data and the boundary line based on a plurality of learning records stored in the database; generating a new displacement map based on the newly acquired video data; estimating the boundary line based on new input data including the newly generated displacement map and an estimation model; A dross position estimation method comprising:

Images