Abnormity recognition method for metal plate feeding process based on visual detection

Abstract

The invention relates to the technical field of machine vision detection, and discloses a metal plate feeding process abnormity identification method based on vision detection. Collecting image frames on a conveying path, calculating a gray threshold segmentation board area by using a background and a standard board calibration image, and recording board missing when the board area is not segmented; extracting a geometric center in the segmented region and calculating a main direction angle, and obtaining a center displacement amount and a periodically converted direction angle drift amount between adjacent frames; selecting a stable period effective frame to construct a normal motion model, obtaining a mean value and a threshold value of two drifts, comparing an offset with the threshold value in real time, performing normal or abnormal judgment and abnormal typing on the frame, and generating a severity score; according to the method, records containing frame numbers, offsets, scores and types are generated for abnormal frames, an abnormal event sequence is formed according to the numbers, a three-state mark is set for each frame to shield invalid frames, and traceable feeding abnormity recognition under the changing working condition is achieved.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, specifically to a method for identifying anomalies in the metal sheet feeding process based on vision inspection. Background Technology

[0002] Automatic sheet metal feeding systems are widely used in discrete manufacturing production lines such as stamping, bending, and cutting. To prevent sheet metal from shifting, flipping, or jamming during conveying, positioning, and feeding, the production line is usually equipped with position detection and attitude detection devices.

[0003] In existing technologies, one type of solution mainly relies on contact detection elements such as mechanical limit switches and pneumatic detection rods to determine whether the sheet metal is in place or has experienced severe displacement based on whether it is triggered. Another type of solution uses a single industrial camera with simple threshold segmentation or edge extraction to detect the sheet metal contour and determine whether the sheet metal is within a preset area. Some systems also introduce pattern recognition or machine learning algorithms based on a large number of samples to classify and judge the sheet metal posture. However, existing technologies still have several shortcomings in anomaly detection during the metal sheet metal loading process. First, most detection solutions only perform static determination of the sheet metal position or posture at a single station, lacking continuous tracking of the geometric center and quantitative description of the main direction changes during the multi-station transfer process, making it difficult to detect chain anomalies between stations caused by the gradual accumulation of slight displacements. Second, existing image processing methods mostly remain at the level of simple region judgment or whether the edge enters a certain threshold box, lacking a mathematical model based on the overall geometric distribution characteristics of the sheet metal. They cannot stably describe the subtle changes in the position and posture of the sheet metal through quantitative features such as the center of gravity and second moment, resulting in insufficient sensitivity to small but trending drifts. Furthermore, existing technologies often directly set empirical thresholds or rules for anomaly detection. The sources of these thresholds lack unified statistical basis, making it difficult to form a self-consistent benchmark model based on the "normal movement behavior" of the production line itself. This easily leads to significant adjustment workloads between different batches and production lines. Moreover, under interference from operating conditions such as changes in lighting and surface reflection, false alarms and false negatives coexist. While some solutions based on complex learning models can achieve good recognition rates in specific environments, they often rely on large-scale labeled samples and high computational resources. The internal decision-making mechanisms of these models are opaque, hindering verifiable and traceable parameter adjustments in industrial settings.

[0004] Therefore, this case aims to propose an anomaly identification method for the metal sheet feeding process based on visual detection. First, a grayscale threshold is set by setting a background calibration image to quickly segment the sheet area and capture sheet missing anomalies. Then, a covariance matrix is ​​constructed based on second-order geometric moments, and the principal orientation angle is calculated to achieve a quantitative description of the sheet's posture and position. Next, the center displacement and orientation angle drift are calculated between two adjacent frames, and periodic calculations are used to eliminate misjudgments caused by angle jumps. Then, during the system's stable period, a normal motion model is automatically established using multi-frame samples, and the average value and fluctuation range are extracted to generate an adaptively correctable range threshold. For subsequent frames, the real-time offset is compared with the model range to quickly mark normal or abnormal. Abnormal frames are further classified into "center drift," "orientation drift," and "compound drift," and a severity score is constructed by comprehensively normalizing the offset to achieve multi-level graded alarms. Finally, all abnormal events are organized into a time series, and a complete event sequence report is output after skipping invalid frames. Summary of the Invention

[0005] This invention provides a method for identifying anomalies in the metal sheet feeding process based on visual inspection, which helps to solve the problems mentioned in the background art.

[0006] This invention provides the following technical solution: a method for anomaly identification in the metal sheet feeding process based on visual inspection, comprising:

[0007] Acquire running image frames along the transmission path, set a fixed threshold based on the background reference area and the board reference area in the calibration image, segment the board area according to the threshold, and generate a board missing abnormal record when it is not segmented.

[0008] Calculate the coordinates of the geometric center within each frame of the board area, construct the geometric moment covariance matrix based on the second-order geometric moments about the geometric center, and obtain the principal direction angle;

[0009] In two adjacent frames, the center displacement is obtained by the difference between the geometric centers of the two frames, and the direction angle drift is obtained by periodically calculating the difference between the main direction angles of the two frames.

[0010] A normal motion sample set is formed in consecutive valid frames. The average and fluctuation ranges are calculated based on the center displacement and directional angle drift of adjacent two frames to obtain the corrected center displacement range and the corrected directional angle range, which are used as reference parameters.

[0011] In subsequent frames, when the plate area is detected in two adjacent frames, the absolute value of the difference between the center displacement and the direction angle drift and the corresponding average value is taken to obtain the center displacement offset and the direction angle offset. The frame is divided into a normal frame or an abnormal frame based on the comparison between the two offsets and the corrected range.

[0012] For abnormal frames, center drift, direction drift or compound anomaly is determined based on whether the two offsets exceed the corresponding corrected range, and the severity score of the anomaly is generated and classified according to the normalized result of the two offsets relative to the corrected range.

[0013] For each abnormal frame, generate an abnormal record vector containing a frame number, two offsets, a severity score, and an abnormality type label, and form an abnormal event sequence set in order of frame number;

[0014] Set a status flag variable for each frame to distinguish between invalid, normal and abnormal, skip the feature and index calculation of invalid frames, and output the set of abnormal event sequences as the recognition result.

[0015] Optionally, the step of acquiring running image frames along the transmission path, setting a fixed threshold based on the background reference area and the board reference area in the calibration image, segmenting the board area according to the threshold, and generating a board missing anomaly record when no segmentation occurs, specifically includes:

[0016] Multiple image acquisition units are arranged along the conveying direction on the metal sheet conveying path. Each image acquisition unit is perpendicular to the running path of the sheet. Each image acquisition unit acquires a sequence of image frames of the metal sheet running process at a set acquisition frequency. Each frame image is a rectangular image. The projection outline of the metal sheet in each frame image is a rectangular outline with a pair of long sides and a pair of short sides, with the length of the long side being greater than the length of the short side.

[0017] For each frame of the image, obtain the pixel width in the horizontal direction and the pixel height in the vertical direction. Establish a rectangular coordinate system on the image plane, set the upper left corner of the image as the origin, the horizontal axis points to the right side of the image, and the vertical axis points to the bottom of the image. Associate the position of each pixel in each frame of the image with the corresponding gray value to form a data set of pixel coordinates and gray value.

[0018] During the system initialization phase, a calibration image containing only the background and a calibration image containing the complete board material in the standard position are acquired. In the calibration image, the pixel set of the background reference area and the pixel set of the board material reference area are selected respectively. The arithmetic mean of the gray values ​​of all pixels in each reference area is calculated to obtain the average gray value of the background and the average gray value of the board material. A fixed board material detection threshold is set based on the intermediate gray value of the two.

[0019] During normal operation, for each frame of image, the coordinates of pixels with gray values ​​greater than the fixed board detection threshold are added to the board region pixel set. The number of pixels in the board region pixel set is counted. When the number of pixels is zero, a board missing anomaly record is generated. The board missing anomaly record contains at least the frame number of the current frame and the missing type marker, and the subsequent processing of the current frame is terminated.

[0020] Optionally, the step of calculating the geometric center coordinates within each frame of the board material region, constructing the geometric moment covariance matrix based on the second-order geometric moments about the geometric center, and obtaining the principal direction angle specifically includes:

[0021] For each frame of image, the set of pixels of the board region obtained through the image acquisition and board region extraction stages is used to accumulate the horizontal and vertical coordinates of all pixels in the set of pixels of the board region, and then divide by the number of pixels in the set to obtain the geometric center coordinates of the board region on the horizontal and vertical axes.

[0022] Using the geometric center coordinates as a reference, calculate the difference between the horizontal coordinate and the geometric center horizontal coordinate, and the difference between the vertical coordinate and the geometric center vertical coordinate for each pixel in the board area, and obtain the horizontal and vertical offsets of each pixel.

[0023] For the horizontal offset of all pixels, the squared horizontal offset is averaged to obtain the second-order geometric moment in the horizontal direction about the geometric center; for the vertical offset of all pixels, the squared vertical offset is averaged to obtain the second-order geometric moment in the vertical direction about the geometric center; the product of the horizontal offset and the vertical offset of all pixels is averaged to obtain the mixed second-order geometric moment about the geometric center.

[0024] A 2x2 geometric moment covariance matrix is ​​constructed using the second-order geometric moments in the horizontal direction, the second-order geometric moments in the vertical direction, and the mixed second-order geometric moments as elements. Based on the characteristic relationship of the geometric moment covariance matrix, the principal direction angle of the long side of the board relative to the horizontal axis is calculated. When the second-order geometric moments in the horizontal direction and the second-order geometric moments in the vertical direction are not equal, the principal direction angle is calculated through the analytical relationship between the three. When the second-order geometric moments in the horizontal direction and the second-order geometric moments in the vertical direction are equal, the principal direction angle is set to the direction close to the diagonal from the upper left to the lower right or the direction from the lower left to the upper right according to the sign of the mixed second-order geometric moment.

[0025] Optionally, the step of obtaining the center displacement based on the geometric center difference between two adjacent image frames, and obtaining the orientation angle drift based on the principal orientation angle difference between the two frames after periodic calculation, specifically includes:

[0026] For any two adjacent frames, under the condition that the board area is detected in both the previous frame and the current frame, read the horizontal and vertical coordinates of the geometric center of the board area in the previous frame and the current frame, construct the Euclidean distance based on the difference between the horizontal and vertical coordinates of the geometric center of the two frames, and obtain the center displacement of the two adjacent frames.

[0027] For the same pair of adjacent frames, read the principal orientation angles of the previous frame and the current frame, and calculate the original orientation difference by subtracting the principal orientation angle of the previous frame from the principal orientation angle of the current frame.

[0028] The original direction difference is converted according to the periodic characteristics of the principal direction angle. When the original direction difference is greater than the upper limit of the preset positive symmetry angle, a complete half-cycle is subtracted from the original direction difference. When the original direction difference is less than the lower limit of the preset negative symmetry angle, a complete half-cycle is added to the original direction difference. The original direction difference remains unchanged when it is within the symmetry angle range, and the converted direction difference is obtained.

[0029] The absolute value of the calculated direction difference is taken to obtain the direction angle drift between two adjacent frames. The direction angle drift represents the rotation amplitude of the plate posture between the two frames.

[0030] Optionally, the step of forming a normal motion sample set in consecutive valid frames, and calculating the average and fluctuation range based on the center displacement and directional angle drift of adjacent frames to obtain the corrected center displacement range and corrected directional angle range as reference parameters, specifically includes:

[0031] In the initial stage of stable system operation, from the image frames in which the board area is continuously detected, a number of consecutive frames that detect the board area in every frame of the selected frame sequence starting from the starting frame are selected to form a normal moving sample set. The number of frames in the normal moving sample set is not less than the preset minimum number of frames.

[0032] If no consecutive frame sequence satisfies the condition that the plate area is detected in every frame and the number of frames reaches the preset minimum frame number, a normal movement sample set is not established, and no abnormal judgment and output calculation based on the center displacement and directional angle drift is performed.

[0033] Given a normal motion sample set, read the center displacement and directional drift for each pair of adjacent frames in the normal motion sample set. Take the arithmetic mean of the center displacement for all pairs of adjacent frames to obtain the average center displacement of the normal motion sample set. Take the arithmetic mean of the directional drift for all pairs of adjacent frames to obtain the average directional drift of the normal motion sample set.

[0034] In the normal moving sample set, the difference between the maximum and minimum values ​​of the center displacement of all two adjacent frames is calculated to obtain the center displacement fluctuation range. The difference between the maximum and minimum values ​​of the directional drift of all two adjacent frames is calculated to obtain the directional drift fluctuation range.

[0035] For the center displacement fluctuation range, when the center displacement fluctuation range is greater than zero, the center displacement fluctuation range is taken as the center displacement range; when the center displacement fluctuation range is equal to zero, the center displacement range is corrected to a preset unit value. For the directional angle drift fluctuation range, when the directional angle drift fluctuation range is greater than zero, the directional angle drift fluctuation range is taken as the directional angle range; when the directional angle drift fluctuation range is equal to zero, the directional angle range is corrected to a preset unit value. The average center displacement, the average directional angle drift, the corrected center displacement range, and the corrected directional angle range together constitute the reference parameters of the normal motion model of the plate.

[0036] Optionally, in subsequent frames, when a plate area is detected in two adjacent frames, the absolute values ​​of the differences between the center displacement and the orientation angle drift and their corresponding average values ​​are taken to obtain the center displacement offset and the orientation angle offset. The frames are then classified as normal frames or abnormal frames based on a comparison of these two offsets with the corrected range. Specifically, this includes:

[0037] After the normal motion model baseline is established, for each frame image with a sequence number greater than the last frame number of the normal motion sample set, under the condition that the plate area is detected in both the previous frame and the current frame, the center displacement and direction angle drift of the two adjacent frames are read, and the average center displacement and the average direction angle drift in the plate normal motion model baseline parameters are read.

[0038] The absolute value of the difference between the center displacement of two adjacent frames and the average center displacement is taken to obtain the center displacement offset of the current frame; the absolute value of the difference between the azimuth drift of two adjacent frames and the average azimuth drift is taken to obtain the azimuth offset of the current frame.

[0039] For each frame that simultaneously satisfies the condition that the board area was detected in both the previous frame and the current frame, anomaly detection is performed based on the comparison between the center displacement offset and the corrected center displacement range, as well as the comparison between the directional angle offset and the corrected directional angle range. If the center displacement offset is greater than the corrected center displacement range or the directional angle offset is greater than the corrected directional angle range, the current frame is marked as an abnormal frame. If the center displacement offset does not exceed the corrected center displacement range and the directional angle offset does not exceed the corrected directional angle range, the current frame is marked as a normal frame.

[0040] Optionally, for abnormal frames, the system determines center drift, orientation drift, or compound anomalies based on whether the two offsets exceed the corresponding corrected range, and generates an anomaly severity score and classifies it according to the normalized result of the two offsets relative to the corrected range. Specifically, this includes:

[0041] For each image frame marked as an abnormal frame during the real-time anomaly detection and judgment phase, the center displacement offset and orientation angle offset corresponding to the abnormal frame are read. A combined judgment is made based on whether the center displacement offset exceeds the corrected center displacement range and whether the orientation angle offset exceeds the corrected orientation angle range. When only the center displacement offset exceeds the corrected center displacement range and the orientation angle offset does not exceed the corrected orientation angle range, the abnormal frame is marked as a center drift anomaly. When only the orientation angle offset exceeds the corrected orientation angle range and the center displacement offset does not exceed the corrected center displacement range, the abnormal frame is marked as an orientation drift anomaly. When both the center displacement offset and the orientation angle offset exceed the corrected center displacement range, the abnormal frame is marked as a composite anomaly.

[0042] For each anomalous frame, an anomalous severity score is constructed by summing the ratio of the center displacement offset to the corrected center displacement range and the ratio of the direction angle offset to the corrected direction angle range with equal weights. The larger the anomalous severity score, the higher the anomalous severity of the corresponding anomalous frame.

[0043] Based on the relationship between the severity score and at least two incremental score thresholds, abnormal frames are classified into minor, moderate, and severe abnormality levels, and a corresponding abnormality level label is added to each abnormal frame.

[0044] Optionally, the step of generating an anomaly record vector for each anomaly frame, including a frame number, two offsets, a severity score, and an anomaly type label, and assembling an anomaly event sequence set in frame number order, specifically includes:

[0045] For each abnormal frame, an abnormal record vector is constructed, which includes the abnormal frame number, center displacement offset, orientation angle offset, abnormal severity score, and abnormal type label.

[0046] All abnormal record vectors are sorted in ascending order of frame number to form an abnormal event sequence set arranged in frame time order. Each abnormal record vector in the abnormal event sequence set corresponds to an abnormal event, and the number of abnormal record vectors in the abnormal event sequence set is equal to the total number of abnormal events.

[0047] The abnormal event sequence set is written sequentially to a preset data storage medium. Each time a new abnormal event is added, an append write operation is performed in the order of frame time to form an abnormal event sequence index stored in chronological order.

[0048] Optionally, the step of setting a status flag variable for each frame to distinguish between invalid, normal, and abnormal frames, skipping the feature and index calculations for invalid frames, and outputting the set of abnormal event sequences as the identification result specifically includes:

[0049] For all image frames, a status flag variable is constructed for each frame. The value of the status flag variable includes three types: invalid frame status, normal frame status, and abnormal frame status.

[0050] For each frame that is within the normal moving sample set and where the board area is detected, set the current frame's state flag variable to the normal frame state.

[0051] For frames with sequence numbers greater than the last frame number of the normal movement sample set, the following processing is performed sequentially: When no board area is detected, or when a board area is detected in the current frame but not in the previous frame, the status flag variable of the current frame is set to invalid frame status; when board areas are detected in both the current frame and the previous frame, based on the comparison results between the center displacement offset and the corrected center displacement range obtained in the real-time anomaly detection and judgment stage, and the comparison results between the direction angle offset and the corrected direction angle range, the status flag variable of the current frame is set to normal frame status when the center displacement offset does not exceed the corrected center displacement range and the direction angle offset does not exceed the corrected direction angle range; the status flag variable of the current frame is set to abnormal frame status when the center displacement offset exceeds the corrected center displacement range or the direction angle offset exceeds the corrected direction angle range.

[0052] For frames with invalid frame status, the calculation of center displacement, orientation angle drift, center displacement offset, orientation angle offset, and anomaly severity score is not performed, and the data corresponding to the current frame is not included in the anomaly event sequence set.

[0053] After the abnormal event sequence set is constructed, the abnormal event sequence set is output as the abnormal identification result of the metal sheet feeding process.

[0054] The present invention has the following beneficial effects:

[0055] 1. During system initialization, two types of calibration images are simultaneously acquired: a pure background and a standard board material. By calculating the intermediate grayscale threshold from the average grayscale values ​​of the background and board material reference areas, the segmentation threshold can distinguish between the board material and the background while also possessing adaptive capabilities to resist changes in light intensity and differences in board surface texture during production. Furthermore, if no board material area is detected, a board material missing anomaly record is generated immediately, and subsequent processes are terminated to avoid meaningless calculations and false alarms.

[0056] 2. For describing the position and orientation of rectangular panels, this scheme breaks away from the simple centroid calculation method. It constructs a two-dimensional covariance matrix based on second-order geometric moment parameters and directly calculates the angle (principal direction angle) between the long side of the panel and the image coordinate system through feature relationships. Compared with methods based on edge detection and then fitting straight lines, the geometric moment covariance matrix does not rely on low-level feature matching; the matrix eigenvalue relationships also avoid multiple iterative optimizations, resulting in high computational efficiency and stable accuracy.

[0057] 3. In calculating the displacement and angle changes between two frames, this scheme directly obtains the center displacement through Euclidean distance, while periodically recalculating the principal direction angle difference, eliminating the large misjudgment of differences caused by angle jumps. Compared to direct difference or complex processing based on phase expansion, the periodic recalculation logic is simple and efficient, and can also adapt to the calculation across the entire angle range.

[0058] 4. In the initial stage of system operation, this solution automatically selects a continuous set of valid frames. It calculates the average value and fluctuation range using the center displacement and directional angular drift of all adjacent frames, and applies a unit correction when the range is zero, generating an adaptive "corrected range" threshold combination as the motion model benchmark. This benchmark can automatically adjust with production line speed, sheet material type, and visual acquisition resolution, eliminating the need for manual threshold fine-tuning. Simultaneously, the zero-range correction avoids misjudgments when there is no vibration in the sample.

[0059] 5. In the real-time phase, this solution compares the average of the two offsets with the model baseline and the corrected range, and immediately classifies and labels them as "normal frames" or "abnormal frames" based on whether the offsets exceed the limits. This process requires no additional calculations, only absolute value and comparison operations, resulting in extremely low computational complexity, making it suitable for real-time deployment on high-speed industrial production lines.

[0060] 6. For frames marked as abnormal, this solution further categorizes them into three types: center drift abnormality, orientation drift abnormality, and composite abnormality. After normalizing the two offsets, an anomaly severity score is generated, which is then classified into three levels: mild, moderate, and severe based on the score and a fixed grade difference. This design balances type differentiation and level assessment, enabling rapid assistance in identifying the root cause of position or attitude deviations and providing risk level guidance for operational decisions.

[0061] 7. This scheme sets a status flag variable (invalid / normal / abnormal) for each frame and skips the calculation in invalid frames to ensure efficient use of computing resources; at the same time, it only includes abnormal events in valid frames into the event sequence to ensure that the results are concise and accurate. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the process of the present invention.

[0063] Figure 2 This is a schematic diagram of the rectangular coordinate system of the present invention.

[0064] In the diagram: 1 - origin, 2 - Shaft, 3- axis. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Example, refer to Figure 1 A method for anomaly identification in the metal sheet feeding process based on vision detection, comprising:

[0067] Acquire running image frames along the transmission path, set a fixed threshold based on the background reference area and the board reference area in the calibration image, segment the board area according to the threshold, and generate a board missing abnormal record when it is not segmented.

[0068] Calculate the coordinates of the geometric center within each frame of the board area, construct the geometric moment covariance matrix based on the second-order geometric moments about the geometric center, and obtain the principal direction angle;

[0069] In two adjacent frames, the center displacement is obtained by the difference between the geometric centers of the two frames, and the direction angle drift is obtained by periodically calculating the difference between the main direction angles of the two frames.

[0070] A normal motion sample set is formed in consecutive valid frames. The average and fluctuation ranges are calculated based on the center displacement and directional angle drift of adjacent two frames to obtain the corrected center displacement range and the corrected directional angle range, which are used as reference parameters.

[0071] In subsequent frames, when the plate area is detected in two adjacent frames, the absolute value of the difference between the center displacement and the direction angle drift and the corresponding average value is taken to obtain the center displacement offset and the direction angle offset. The frame is divided into a normal frame or an abnormal frame based on the comparison between the two offsets and the corrected range.

[0072] For abnormal frames, center drift, direction drift or compound anomaly is determined based on whether the two offsets exceed the corresponding corrected range, and the severity score of the anomaly is generated and classified according to the normalized result of the two offsets relative to the corrected range.

[0073] For each abnormal frame, generate an abnormal record vector containing a frame number, two offsets, a severity score, and an abnormality type label, and form an abnormal event sequence set in order of frame number;

[0074] Set a status flag variable for each frame to distinguish between invalid, normal and abnormal, skip the feature and index calculation of invalid frames, and output the set of abnormal event sequences as the recognition result.

[0075] First, by pre-collecting background and standard board images, an adaptive grayscale segmentation threshold is automatically generated, enabling reliable extraction of board areas and timely determination of board missingness even when lighting or material conditions change on the production line. Then, within each frame of board area, a covariance matrix is ​​constructed based on second-order geometric moments, and the principal orientation angle is calculated to quantitatively describe the board's center of gravity position and orientation. Next, the center displacement and orientation angle drift are calculated between adjacent frames, and a periodic adjustment strategy is used to eliminate abrupt errors caused by angle rotation. During the stable period of system operation, consecutive valid frames are automatically selected to construct a normal motion model including average drift and fluctuation range, generating an adaptively correctable range threshold. In real-time, the two drift values ​​are compared with the model threshold to quickly mark normal or abnormal frames. Abnormal frames are further subdivided into center drift, orientation drift, or composite drift, and the normalized scores of offset and range threshold are merged to generate multi-level severity scores. Finally, all abnormal events are organized into an event sequence output in chronological order, while each frame is set with an invalid—normal—abnormal three-state flag to ensure that invalid frames do not participate in subsequent analysis. Through this end-to-end, multi-dimensional, and adaptive design, the solution not only improves the accuracy and anti-interference capability of real-time detection, but also outputs hierarchical anomaly information, thus improving the problems of high false alarm rate, high false alarm rate, and high maintenance cost in traditional single alarm and experience threshold modes.

[0076] Reference Figure 2 The process of acquiring running image frames along the transmission path, setting a fixed threshold based on the background reference area and the board reference area in the calibration image, segmenting the board area according to the threshold, and generating a board missing anomaly record when no segmentation occurs, specifically includes:

[0077] Multiple image acquisition units are arranged along the conveying direction on the metal sheet conveying path. Each image acquisition unit is perpendicular to the running path of the sheet. Each image acquisition unit acquires a sequence of image frames of the metal sheet running process at a set acquisition frequency. Each frame image is a rectangular image. The projection outline of the metal sheet in each frame image is a rectangular outline with a pair of long sides and a pair of short sides, with the length of the long side being greater than the length of the short side.

[0078] For each frame of the image, obtain the pixel width in the horizontal direction and the pixel height in the vertical direction. Establish a rectangular coordinate system on the image plane, set the upper left corner of the image as the origin, the horizontal axis points to the right side of the image, and the vertical axis points to the bottom of the image. Associate the position of each pixel in each frame of the image with the corresponding gray value to form a data set of pixel coordinates and gray value.

[0079] During the system initialization phase, a calibration image containing only the background and a calibration image containing the complete board material in the standard position are acquired. In the calibration image, the pixel set of the background reference area and the pixel set of the board material reference area are selected respectively. The arithmetic mean of the gray values ​​of all pixels in each reference area is calculated to obtain the average gray value of the background and the average gray value of the board material. A fixed board material detection threshold is set based on the intermediate gray value of the two.

[0080] During normal operation, for each frame of image, the coordinates of pixels with gray values ​​greater than the fixed board detection threshold are added to the board region pixel set. The number of pixels in the board region pixel set is counted. When the number of pixels is zero, a board missing anomaly record is generated. The board missing anomaly record contains at least the frame number of the current frame and the missing type marker, and the subsequent processing of the current frame is terminated.

[0081] Set on the metal sheet conveying path There are one image acquisition unit, numbered as follows: Each acquisition unit views the running path of the board vertically from above and acquires image frames, denoted as follows: ;in, For the number The image acquisition unit acquired the first Frame image data;

[0082] The metal sheet is rectangular, comprising two sets of parallel sides, wherein the longer side is longer than the shorter side.

[0083] Obtain the horizontal pixel width of a single frame image, denoted as . ;

[0084] Obtain the vertical pixel height of a single frame image, denoted as . ;

[0085] Establish a rectangular coordinate system on the image plane ,origin Located in the top left corner of the image, The axis points to the right side of the image in the horizontal direction. The axis points vertically downwards in the image;

[0086] Image frames The grayscale value of each pixel in the image is set to... ;in, For the first Frame Image In the middle, the coordinates are The grayscale value of the pixel;

[0087] During the system initialization phase, a fixed threshold is calculated using a calibrated image. And based on this threshold, a pixel set for the board area is constructed, specifically as follows:

[0088] S101. Acquire a calibration image containing only the background, denoted as... Acquire calibration images containing complete plates in standard positions, denoted as... ;

[0089] S102. Select a set of background reference region pixels on the image plane. pixel set of the reference area of ​​the board ;

[0090] S103. Calculate the average grayscale value of the background and the board area based on the calibration image and the reference area: , ;in, Background reference area The average gray value; Reference area for the board material The average gray value; For set Number of pixels; For set Number of pixels; To calibrate the image in the background In the middle, the coordinates are The pixel grayscale value; To calibrate images on the board In the middle, the coordinates are The pixel grayscale value;

[0091] S104. Calculate the board detection threshold based on the background and the average gray level of the board: ;

[0092] S105, Construct the pixel set of the board area: ;in, In the first Frame Image All of the above satisfy The set of pixel coordinates; For set The number of elements in the middle;

[0093] when If no board material is detected in the frame image, a board material missing anomaly record is generated, and the record content includes the frame number. The missing type is marked as an abnormality in the board material, and the subsequent process of this frame is terminated.

[0094] The calculation of the geometric center coordinates within each frame of the board material area, the construction of the geometric moment covariance matrix based on the second-order geometric moments about the geometric center, and the determination of the principal direction angles specifically include:

[0095] For each frame of image, the set of pixels of the board region obtained through the image acquisition and board region extraction stages is used to accumulate the horizontal and vertical coordinates of all pixels in the set of pixels of the board region, and then divide by the number of pixels in the set to obtain the geometric center coordinates of the board region on the horizontal and vertical axes.

[0096] Using the geometric center coordinates as a reference, calculate the difference between the horizontal coordinate and the geometric center horizontal coordinate, and the difference between the vertical coordinate and the geometric center vertical coordinate for each pixel in the board area, and obtain the horizontal and vertical offsets of each pixel.

[0097] For the horizontal offset of all pixels, the squared horizontal offset is averaged to obtain the second-order geometric moment in the horizontal direction about the geometric center; for the vertical offset of all pixels, the squared vertical offset is averaged to obtain the second-order geometric moment in the vertical direction about the geometric center; the product of the horizontal offset and the vertical offset of all pixels is averaged to obtain the mixed second-order geometric moment about the geometric center.

[0098] A 2x2 geometric moment covariance matrix is ​​constructed using the second-order geometric moments in the horizontal direction, the second-order geometric moments in the vertical direction, and the mixed second-order geometric moments as elements. Based on the characteristic relationship of the geometric moment covariance matrix, the principal direction angle of the long side of the board relative to the horizontal axis is calculated. When the second-order geometric moments in the horizontal direction and the second-order geometric moments in the vertical direction are not equal, the principal direction angle is calculated through the analytical relationship between the three. When the second-order geometric moments in the horizontal direction and the second-order geometric moments in the vertical direction are equal, the principal direction angle is set to the direction close to the diagonal from the upper left to the lower right or the direction from the lower left to the upper right according to the sign of the mixed second-order geometric moment.

[0099] For each frame Extracted board area Calculate the coordinates of the geometric center of the region: , ;in, , The first Plate area in frame image The geometric center is at axis, Coordinates on the axis;

[0100] The normalized second-order geometric moments within the computation region are specifically calculated as follows: , , ; in, For the first The second moment of the plate region about the geometric center in the frame image is specifically: Second moment in the axial direction; For the first The second moment of the plate region about the geometric center in the frame image is specifically: Second moment in the axial direction; For the first The mixed second moment of the plate region about the geometric center in the frame image;

[0101] Construct the normalized covariance matrix: ;in, For the first Plate area in frame image Geometric moment covariance matrix;

[0102] Execute steps S201 and S202, according to the matrix Calculate the principal direction angle Specifically:

[0103] S201, when season ;in, For the first The principal orientation angle of the board in the frame image is the angle between the long side of the board and the plane. The angle between the positive directions of the axis;

[0104] S202, when season: .

[0105] The process of obtaining the center displacement based on the geometric center difference between two adjacent image frames, and obtaining the orientation angle drift based on the principal orientation angle difference between the two frames after periodic calculation, specifically includes:

[0106] For any two adjacent frames, under the condition that the board area is detected in both the previous frame and the current frame, read the horizontal and vertical coordinates of the geometric center of the board area in the previous frame and the current frame, construct the Euclidean distance based on the difference between the horizontal and vertical coordinates of the geometric center of the two frames, and obtain the center displacement of the two adjacent frames.

[0107] For the same pair of adjacent frames, read the principal orientation angles of the previous frame and the current frame, and calculate the original orientation difference by subtracting the principal orientation angle of the previous frame from the principal orientation angle of the current frame.

[0108] The original direction difference is converted according to the periodic characteristics of the principal direction angle. When the original direction difference is greater than the upper limit of the preset positive symmetry angle, a complete half-cycle is subtracted from the original direction difference. When the original direction difference is less than the lower limit of the preset negative symmetry angle, a complete half-cycle is added to the original direction difference. The original direction difference remains unchanged when it is within the symmetry angle range, and the converted direction difference is obtained.

[0109] The absolute value of the calculated direction difference is taken to obtain the direction angle drift between two adjacent frames. The direction angle drift represents the rotation amplitude of the plate posture between the two frames.

[0110] For satisfying and and Two consecutive frames of images and The center displacement is: ;in, For the first Frame to the The Euclidean displacement of the geometric center of the plate during the frame period;

[0111] Define the original orientation difference for the same pair of consecutive frames: ;in, For the first Frame and the The original difference in the principal orientation angle between frames;

[0112] Original difference of principal direction angle After periodic conversion, the difference in conversion direction is obtained as follows: ;in, The first one obtained after periodicity calculation of the angle. Frame direction difference;

[0113] The orientation angle drift is: ;in, For the first Frame and the The absolute drift of the principal direction angle between frames.

[0114] The process of forming a normal motion sample set in consecutive valid frames, averaging and calculating the fluctuation range based on the center displacement and directional angle drift of adjacent frames, yields the corrected center displacement range and corrected directional angle range, which serve as reference parameters. Specifically, this includes:

[0115] In the initial stage of stable system operation, from the image frames in which the board area is continuously detected, a number of consecutive frames that detect the board area in every frame of the selected frame sequence starting from the starting frame are selected to form a normal moving sample set. The number of frames in the normal moving sample set is not less than the preset minimum number of frames.

[0116] If no consecutive frame sequence satisfies the condition that the plate area is detected in every frame and the number of frames reaches the preset minimum frame number, a normal movement sample set is not established, and no abnormal judgment and output calculation based on the center displacement and directional angle drift is performed.

[0117] Given a normal motion sample set, read the center displacement and directional drift for each pair of adjacent frames in the normal motion sample set. Take the arithmetic mean of the center displacement for all pairs of adjacent frames to obtain the average center displacement of the normal motion sample set. Take the arithmetic mean of the directional drift for all pairs of adjacent frames to obtain the average directional drift of the normal motion sample set.

[0118] In the normal moving sample set, the difference between the maximum and minimum values ​​of the center displacement of all two adjacent frames is calculated to obtain the center displacement fluctuation range. The difference between the maximum and minimum values ​​of the directional drift of all two adjacent frames is calculated to obtain the directional drift fluctuation range.

[0119] For the center displacement fluctuation range, when the center displacement fluctuation range is greater than zero, the center displacement fluctuation range is taken as the center displacement range; when the center displacement fluctuation range is equal to zero, the center displacement range is corrected to a preset unit value. For the directional angle drift fluctuation range, when the directional angle drift fluctuation range is greater than zero, the directional angle drift fluctuation range is taken as the directional angle range; when the directional angle drift fluctuation range is equal to zero, the directional angle range is corrected to a preset unit value. The average center displacement, the average directional angle drift, the corrected center displacement range, and the corrected directional angle range together constitute the reference parameters of the normal motion model of the plate.

[0120] In the initial stage of stable system operation, after satisfying all All times Under the condition that, select number as to Image frames As a normal moving sample set; among which... The image frame number selected for inclusion in the normal motion sample set; This represents the number of frames contained in the normal moving sample set.

[0121] If there are no integers in the given set of image frames that satisfy the above conditions If no normal moving sample set is constructed, no anomaly detection and output calculation based on center shift and main direction angle drift will be performed.

[0122] Calculate the mean of the center displacement and the azimuth drift using a normal movement sample set: , ;in, This represents the average displacement of the geometric center of adjacent frames in the normal moving sample set. This represents the average of the directional angular drift between adjacent frames in the normal motion sample set. For the first normal sample set Frame and the Center displacement between frames; For the first normal sample set Frame and the The amount of directional angular drift between frames;

[0123] The range of fluctuations in displacement and orientation angle in normal samples is calculated as follows: , ;in, This represents the range of center displacement between adjacent frames in a normal sample. This represents the range of orientation angle drift between adjacent frames in a normal sample.

[0124] The corrected range is constructed as follows: , ;in, To The center displacement range after zero range correction; To The azimuth range after zero range correction.

[0125] In subsequent frames, when the plate area is detected in two adjacent frames, the absolute values ​​of the differences between the center displacement and the directional angle drift and their corresponding average values ​​are taken to obtain the center displacement offset and the directional angle offset. The frames are then classified as normal or abnormal frames based on a comparison of these two offsets with the corrected range. Specifically, this includes:

[0126] After the normal motion model baseline is established, for each frame image with a sequence number greater than the last frame number of the normal motion sample set, under the condition that the plate area is detected in both the previous frame and the current frame, the center displacement and direction angle drift of the two adjacent frames are read, and the average center displacement and the average direction angle drift in the plate normal motion model baseline parameters are read.

[0127] The absolute value of the difference between the center displacement of two adjacent frames and the average center displacement is taken to obtain the center displacement offset of the current frame; the absolute value of the difference between the azimuth drift of two adjacent frames and the average azimuth drift is taken to obtain the azimuth offset of the current frame.

[0128] For each frame that simultaneously satisfies the condition that the board area was detected in both the previous frame and the current frame, anomaly detection is performed based on the comparison between the center displacement offset and the corrected center displacement range, as well as the comparison between the directional angle offset and the corrected directional angle range. If the center displacement offset is greater than the corrected center displacement range or the directional angle offset is greater than the corrected directional angle range, the current frame is marked as an abnormal frame. If the center displacement offset does not exceed the corrected center displacement range and the directional angle offset does not exceed the corrected directional angle range, the current frame is marked as a normal frame.

[0129] For any satisfying and and frames Based on the calculation results of this frame and the previous frame, the real-time displacement offset and orientation angle offset are determined, specifically as follows: , ;in, For the first The center displacement offset between the frame and the normal model; For the first The angular offset between the frame and the normal model;

[0130] For any satisfying and and frames Anomaly detection is performed, specifically as follows:

[0131] S501, Constructing abnormal frame determination conditions: or ;

[0132] S502. When the abnormal frame determination condition is met, the frame is marked as an abnormal frame.

[0133] S503. When the abnormal frame determination condition is not met, the frame is marked as a normal frame.

[0134] For abnormal frames, center drift, orientation drift, or combined anomalies are determined based on whether the two offsets exceed the corresponding corrected range. An anomaly severity score is generated and graded according to the normalized result of the two offsets relative to the corrected range, specifically including:

[0135] For each image frame marked as an abnormal frame during the real-time anomaly detection and judgment phase, the center displacement offset and orientation angle offset corresponding to the abnormal frame are read. A combined judgment is made based on whether the center displacement offset exceeds the corrected center displacement range and whether the orientation angle offset exceeds the corrected orientation angle range. When only the center displacement offset exceeds the corrected center displacement range and the orientation angle offset does not exceed the corrected orientation angle range, the abnormal frame is marked as a center drift anomaly. When only the orientation angle offset exceeds the corrected orientation angle range and the center displacement offset does not exceed the corrected center displacement range, the abnormal frame is marked as an orientation drift anomaly. When both the center displacement offset and the orientation angle offset exceed the corrected center displacement range, the abnormal frame is marked as a composite anomaly.

[0136] For each anomalous frame, an anomalous severity score is constructed by summing the ratio of the center displacement offset to the corrected center displacement range and the ratio of the direction angle offset to the corrected direction angle range with equal weights. The larger the anomalous severity score, the higher the anomalous severity of the corresponding anomalous frame.

[0137] Based on the relationship between the severity score and at least two incremental score thresholds, abnormal frames are classified into minor, moderate, and severe abnormality levels, and a corresponding abnormality level label is added to each abnormal frame.

[0138] For frames that have been identified as abnormal Steps S601 to S603 determine the anomaly type based on the offset exceeding the limit, specifically as follows:

[0139] S601, if and only if and When the anomaly is identified, it is classified as a center drift anomaly.

[0140] S602, if and only if and When this occurs, the anomaly type is determined to be a direction drift anomaly;

[0141] S603, when simultaneously satisfying and When this happens, the anomaly type is determined to be a composite anomaly;

[0142] For each abnormal frame Construct an anomaly severity scoring function, specifically as follows: ;in, For the first Frame anomaly severity score;

[0143] Perform steps S604 to S606 to score according to the severity of the anomaly. Determine the anomaly level:

[0144] S604, when In this case, the abnormality is classified as a level one minor abnormality;

[0145] S605, when At that time, the abnormality was classified as a level two moderate abnormality;

[0146] S606, when At that time, the anomaly was classified as a level three severe anomaly.

[0147] The process of generating an anomaly record vector for each anomaly frame, including a frame number, two offsets, a severity score, and an anomaly type label, and assembling an anomaly event sequence set in frame number order, specifically includes:

[0148] For each abnormal frame, an abnormal record vector is constructed, which includes the abnormal frame number, center displacement offset, orientation angle offset, abnormal severity score, and abnormal type label.

[0149] All abnormal record vectors are sorted in ascending order of frame number to form an abnormal event sequence set arranged in frame time order. Each abnormal record vector in the abnormal event sequence set corresponds to an abnormal event, and the number of abnormal record vectors in the abnormal event sequence set is equal to the total number of abnormal events.

[0150] The abnormal event sequence set is written sequentially to a preset data storage medium. Each time a new abnormal event is added, an append write operation is performed in the order of frame time to form an abnormal event sequence index stored in chronological order.

[0151] For each frame that is judged as abnormal Construct the corresponding anomaly record vector: ;in, For the first The anomaly record vector corresponding to a frame consists of frame number, displacement offset, angle offset, severity score, and anomaly type label; For the first The frame's anomaly type is marked, including center drift anomaly, direction drift anomaly, and composite anomaly;

[0152] Arrange all anomaly record vectors in ascending order of frame number to construct the anomaly event sequence set as follows: ;in, It is the set of abnormal record vectors corresponding to all abnormal frames, that is, the set of abnormal event sequences; The first in the sequence of abnormal events The frame number corresponding to each abnormal record; This refers to the sequence number of the abnormal event in the abnormal event sequence; This represents the total number of abnormal events, i.e., the number of abnormal record vectors.

[0153] Set of abnormal event sequences The abnormal events are sequentially appended to the preset data storage medium. Each new abnormal event is appended sequentially to form a sequence of abnormal events arranged in chronological order.

[0154] The process of setting a status flag variable for each frame to distinguish between invalid, normal, and abnormal frames, skipping the feature and index calculations for invalid frames, and outputting the set of abnormal event sequences as the identification result specifically includes:

[0155] For all image frames, a status flag variable is constructed for each frame. The value of the status flag variable includes three types: invalid frame status, normal frame status, and abnormal frame status.

[0156] For each frame that is within the normal moving sample set and where the board area is detected, set the current frame's state flag variable to the normal frame state.

[0157] For frames with sequence numbers greater than the last frame number of the normal movement sample set, the following processing is performed sequentially: When no board area is detected, or when a board area is detected in the current frame but not in the previous frame, the status flag variable of the current frame is set to invalid frame status; when board areas are detected in both the current frame and the previous frame, based on the comparison results between the center displacement offset and the corrected center displacement range obtained in the real-time anomaly detection and judgment stage, and the comparison results between the direction angle offset and the corrected direction angle range, the status flag variable of the current frame is set to normal frame status when the center displacement offset does not exceed the corrected center displacement range and the direction angle offset does not exceed the corrected direction angle range; the status flag variable of the current frame is set to abnormal frame status when the center displacement offset exceeds the corrected center displacement range or the direction angle offset exceeds the corrected direction angle range.

[0158] For frames with invalid frame status, the calculation of center displacement, orientation angle drift, center displacement offset, orientation angle offset, and anomaly severity score is not performed, and the data corresponding to the current frame is not included in the anomaly event sequence set.

[0159] After the abnormal event sequence set is constructed, the abnormal event sequence set is output as the abnormal identification result of the metal sheet feeding process.

[0160] For each frame of image Construct state flag variables The range of values ​​is ;in, For the first The frame status flag variable has a value of 0 indicating an invalid frame, a value of 1 indicating a normal frame, and a value of 2 indicating an abnormal frame.

[0161] S8100, for satisfying and The frame, setting ;

[0162] S8200, for meeting Frames:

[0163] S8210, when At that time, keep ;

[0164] S8220, when and At that time, set ;

[0165] S8230, when and At that time, steps S8231 and S8232 are executed:

[0166] S8231, when satisfied and When, set ;

[0167] S8232, when satisfied or When, set ;

[0168] For status flags The frame is not executed. , , , as well as The calculation does not include this frame in the abnormal event sequence set. middle;

[0169] Set of abnormal event sequences Output.

[0170] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0171] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for anomaly identification in the metal sheet feeding process based on visual inspection, characterized in that, include: Acquire running image frames along the transmission path, set a fixed threshold based on the background reference area and the board reference area in the calibration image, segment the board area according to the threshold, and generate a board missing abnormal record when it is not segmented. Calculate the coordinates of the geometric center within each frame of the board area, construct the geometric moment covariance matrix based on the second-order geometric moments about the geometric center, and obtain the principal direction angle; In two adjacent frames, the center displacement is obtained by the difference between the geometric centers of the two frames, and the direction angle drift is obtained by periodically calculating the difference between the main direction angles of the two frames. A normal motion sample set is formed in consecutive valid frames. The average and fluctuation ranges are calculated based on the center displacement and directional angle drift of adjacent two frames to obtain the corrected center displacement range and the corrected directional angle range, which are used as reference parameters. In subsequent frames, when the plate area is detected in two adjacent frames, the absolute value of the difference between the center displacement and the direction angle drift and the corresponding average value is taken to obtain the center displacement offset and the direction angle offset. The frame is divided into a normal frame or an abnormal frame based on the comparison between the two offsets and the corrected range. For abnormal frames, center drift, direction drift or compound anomaly is determined based on whether the two offsets exceed the corresponding corrected range, and the severity score of the anomaly is generated and classified according to the normalized result of the two offsets relative to the corrected range. For each abnormal frame, generate an abnormal record vector containing a frame number, two offsets, a severity score, and an abnormality type label, and form an abnormal event sequence set in order of frame number; Set a status flag variable for each frame to distinguish between invalid, normal and abnormal, skip the feature and index calculation of invalid frames, and output the set of abnormal event sequences as the recognition result.

2. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 1, characterized in that, The process of acquiring running image frames along the transmission path, setting a fixed threshold based on the background reference region and the board reference region in the calibration image, segmenting the board region according to the threshold, and generating a board missing anomaly record when no segmentation occurs, specifically includes: Multiple image acquisition units are arranged along the conveying direction on the metal sheet conveying path. Each image acquisition unit is perpendicular to the running path of the sheet. Each image acquisition unit acquires a sequence of image frames of the metal sheet running process at a set acquisition frequency. Each frame image is a rectangular image. The projection outline of the metal sheet in each frame image is a rectangular outline with a pair of long sides and a pair of short sides, with the length of the long side being greater than the length of the short side. For each frame of the image, obtain the pixel width in the horizontal direction and the pixel height in the vertical direction. Establish a rectangular coordinate system on the image plane, set the upper left corner of the image as the origin, the horizontal axis points to the right side of the image, and the vertical axis points to the bottom of the image. Associate the position of each pixel in each frame of the image with the corresponding gray value to form a data set of pixel coordinates and gray value. During the system initialization phase, a calibration image containing only the background and a calibration image containing the complete board material in the standard position are acquired. In the calibration image, the pixel set of the background reference area and the pixel set of the board material reference area are selected respectively. The arithmetic mean of the gray values ​​of all pixels in each reference area is calculated to obtain the average gray value of the background and the average gray value of the board material. A fixed board material detection threshold is set based on the intermediate gray value of the two. During normal operation, for each frame of image, the coordinates of pixels with gray values ​​greater than the fixed board detection threshold are added to the board region pixel set. The number of pixels in the board region pixel set is counted. When the number of pixels is zero, a board missing anomaly record is generated. The board missing anomaly record contains at least the frame number of the current frame and the missing type marker, and the subsequent processing of the current frame is terminated.

3. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 2, characterized in that, The calculation of the geometric center coordinates within each frame of the board material area, the construction of the geometric moment covariance matrix based on the second-order geometric moments about the geometric center, and the determination of the principal direction angles specifically include: For each frame of image, the set of pixels of the board region obtained through the image acquisition and board region extraction stages is used to accumulate the horizontal and vertical coordinates of all pixels in the set of pixels of the board region, and then divide by the number of pixels in the set to obtain the geometric center coordinates of the board region on the horizontal and vertical axes. Using the geometric center coordinates as a reference, calculate the difference between the horizontal coordinate and the geometric center horizontal coordinate, and the difference between the vertical coordinate and the geometric center vertical coordinate for each pixel in the board area, and obtain the horizontal and vertical offsets of each pixel. For the horizontal offset of all pixels, the squared horizontal offset is averaged to obtain the second-order geometric moment in the horizontal direction about the geometric center; for the vertical offset of all pixels, the squared vertical offset is averaged to obtain the second-order geometric moment in the vertical direction about the geometric center; the product of the horizontal offset and the vertical offset of all pixels is averaged to obtain the mixed second-order geometric moment about the geometric center. A 2x2 geometric moment covariance matrix is ​​constructed using the second-order geometric moments in the horizontal direction, the second-order geometric moments in the vertical direction, and the mixed second-order geometric moments as elements. Based on the characteristic relationship of the geometric moment covariance matrix, the principal direction angle of the long side of the board relative to the horizontal axis is calculated. When the second-order geometric moments in the horizontal direction and the second-order geometric moments in the vertical direction are not equal, the principal direction angle is calculated through the analytical relationship between the three. When the second-order geometric moments in the horizontal direction and the second-order geometric moments in the vertical direction are equal, the principal direction angle is set to the direction close to the diagonal from the upper left to the lower right or the direction from the lower left to the upper right according to the sign of the mixed second-order geometric moment.

4. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 3, characterized in that, The process of obtaining the center displacement based on the geometric center difference between two adjacent image frames, and obtaining the orientation angle drift based on the principal orientation angle difference between the two frames after periodic calculation, specifically includes: For any two adjacent frames, under the condition that the board area is detected in both the previous frame and the current frame, read the horizontal and vertical coordinates of the geometric center of the board area in the previous frame and the current frame, construct the Euclidean distance based on the difference between the horizontal and vertical coordinates of the geometric center of the two frames, and obtain the center displacement of the two adjacent frames. For the same pair of adjacent frames, read the principal orientation angles of the previous frame and the current frame, and calculate the original orientation difference by subtracting the principal orientation angle of the previous frame from the principal orientation angle of the current frame. The original direction difference is converted according to the periodic characteristics of the principal direction angle. When the original direction difference is greater than the upper limit of the preset positive symmetry angle, a complete half-cycle is subtracted from the original direction difference. When the original direction difference is less than the lower limit of the preset negative symmetry angle, a complete half-cycle is added to the original direction difference. The original direction difference remains unchanged when it is within the symmetry angle range, and the converted direction difference is obtained. The absolute value of the calculated direction difference is taken to obtain the direction angle drift between two adjacent frames. The direction angle drift represents the rotation amplitude of the plate posture between the two frames.

5. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 4, characterized in that, The process of forming a normal motion sample set in consecutive valid frames, averaging and calculating the fluctuation range based on the center displacement and directional angle drift of adjacent frames, yields the corrected center displacement range and corrected directional angle range, which serve as reference parameters. Specifically, this includes: In the initial stage of stable system operation, from the image frames in which the board area is continuously detected, a number of consecutive frames that detect the board area in every frame of the selected frame sequence starting from the starting frame are selected to form a normal moving sample set. The number of frames in the normal moving sample set is not less than the preset minimum number of frames. If no consecutive frame sequence satisfies the condition that the plate area is detected in every frame and the number of frames reaches the preset minimum frame number, a normal movement sample set is not established, and no abnormal judgment and output calculation based on the center displacement and directional angle drift is performed. Given a normal motion sample set, read the center displacement and directional drift for each pair of adjacent frames in the normal motion sample set. Take the arithmetic mean of the center displacement for all pairs of adjacent frames to obtain the average center displacement of the normal motion sample set. Take the arithmetic mean of the directional drift for all pairs of adjacent frames to obtain the average directional drift of the normal motion sample set. In the normal moving sample set, the difference between the maximum and minimum values ​​of the center displacement of all two adjacent frames is calculated to obtain the center displacement fluctuation range. The difference between the maximum and minimum values ​​of the directional drift of all two adjacent frames is calculated to obtain the directional drift fluctuation range. For the center displacement fluctuation range, when the center displacement fluctuation range is greater than zero, the center displacement fluctuation range is taken as the center displacement range; when the center displacement fluctuation range is equal to zero, the center displacement range is corrected to a preset unit value. For the directional angle drift fluctuation range, when the directional angle drift fluctuation range is greater than zero, the directional angle drift fluctuation range is taken as the directional angle range; when the directional angle drift fluctuation range is equal to zero, the directional angle range is corrected to a preset unit value. The average center displacement, the average directional angle drift, the corrected center displacement range, and the corrected directional angle range together constitute the reference parameters of the normal motion model of the plate.

6. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 5, characterized in that, In subsequent frames, when the plate area is detected in two adjacent frames, the absolute values ​​of the differences between the center displacement and the directional angle drift and their corresponding average values ​​are taken to obtain the center displacement offset and the directional angle offset. The frames are then classified as normal or abnormal frames based on a comparison of these two offsets with the corrected range. Specifically, this includes: After the normal motion model baseline is established, for each frame image with a sequence number greater than the last frame number of the normal motion sample set, under the condition that the plate area is detected in both the previous frame and the current frame, the center displacement and direction angle drift of the two adjacent frames are read, and the average center displacement and the average direction angle drift in the plate normal motion model baseline parameters are read. The absolute value of the difference between the center displacement of two adjacent frames and the average center displacement is taken to obtain the center displacement offset of the current frame; the absolute value of the difference between the azimuth drift of two adjacent frames and the average azimuth drift is taken to obtain the azimuth offset of the current frame. For each frame that simultaneously satisfies the condition that the board area was detected in both the previous frame and the current frame, anomaly detection is performed based on the comparison between the center displacement offset and the corrected center displacement range, as well as the comparison between the directional angle offset and the corrected directional angle range. If the center displacement offset is greater than the corrected center displacement range or the directional angle offset is greater than the corrected directional angle range, the current frame is marked as an abnormal frame. If the center displacement offset does not exceed the corrected center displacement range and the directional angle offset does not exceed the corrected directional angle range, the current frame is marked as a normal frame.

7. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 6, characterized in that, For abnormal frames, center drift, orientation drift, or combined anomalies are determined based on whether the two offsets exceed the corresponding corrected range. An anomaly severity score is generated and graded according to the normalized result of the two offsets relative to the corrected range, specifically including: For each image frame marked as an abnormal frame during the real-time anomaly detection and judgment phase, the center displacement offset and orientation angle offset corresponding to the abnormal frame are read. A combined judgment is made based on whether the center displacement offset exceeds the corrected center displacement range and whether the orientation angle offset exceeds the corrected orientation angle range. When only the center displacement offset exceeds the corrected center displacement range and the orientation angle offset does not exceed the corrected orientation angle range, the abnormal frame is marked as a center drift anomaly. When only the orientation angle offset exceeds the corrected orientation angle range and the center displacement offset does not exceed the corrected center displacement range, the abnormal frame is marked as an orientation drift anomaly. When both the center displacement offset and the orientation angle offset exceed the corrected center displacement range, the abnormal frame is marked as a composite anomaly. For each anomalous frame, an anomalous severity score is constructed by summing the ratio of the center displacement offset to the corrected center displacement range and the ratio of the direction angle offset to the corrected direction angle range with equal weights. The larger the anomalous severity score, the higher the anomalous severity of the corresponding anomalous frame. Based on the relationship between the severity score and at least two incremental score thresholds, abnormal frames are classified into minor, moderate, and severe abnormality levels, and a corresponding abnormality level label is added to each abnormal frame.

8. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 7, characterized in that, The process of generating an anomaly record vector for each anomaly frame, including a frame number, two offsets, a severity score, and an anomaly type label, and assembling an anomaly event sequence set in frame number order, specifically includes: For each abnormal frame, an abnormal record vector is constructed, which includes the abnormal frame number, center displacement offset, orientation angle offset, abnormal severity score, and abnormal type label. All abnormal record vectors are sorted in ascending order of frame number to form an abnormal event sequence set arranged in frame time order. Each abnormal record vector in the abnormal event sequence set corresponds to an abnormal event, and the number of abnormal record vectors in the abnormal event sequence set is equal to the total number of abnormal events. The abnormal event sequence set is written sequentially to a preset data storage medium. Each time a new abnormal event is added, an append write operation is performed in the order of frame time to form an abnormal event sequence index stored in chronological order.

9. The method for anomaly identification in the metal sheet feeding process based on visual inspection according to claim 8, characterized in that, The process of setting a status flag variable for each frame to distinguish between invalid, normal, and abnormal frames, skipping the feature and index calculations for invalid frames, and outputting the set of abnormal event sequences as the identification result specifically includes: For all image frames, a status flag variable is constructed for each frame. The value of the status flag variable includes three types: invalid frame status, normal frame status, and abnormal frame status. For each frame that is within the normal moving sample set and where the board area is detected, set the current frame's state flag variable to the normal frame state. For frames with sequence numbers greater than the last frame number of the normal movement sample set, the following processing is performed sequentially: When no board area is detected, or when a board area is detected in the current frame but not in the previous frame, the status flag variable of the current frame is set to invalid frame status; when board areas are detected in both the current frame and the previous frame, based on the comparison results between the center displacement offset and the corrected center displacement range obtained in the real-time anomaly detection and judgment stage, and the comparison results between the direction angle offset and the corrected direction angle range, the status flag variable of the current frame is set to normal frame status when the center displacement offset does not exceed the corrected center displacement range and the direction angle offset does not exceed the corrected direction angle range; the status flag variable of the current frame is set to abnormal frame status when the center displacement offset exceeds the corrected center displacement range or the direction angle offset exceeds the corrected direction angle range. For frames with invalid frame status, the calculation of center displacement, orientation angle drift, center displacement offset, orientation angle offset, and anomaly severity score is not performed, and the data corresponding to the current frame is not included in the anomaly event sequence set. After the abnormal event sequence set is constructed, the abnormal event sequence set is output as the abnormal identification result of the metal sheet feeding process.

Images