Intermediate billet camber monitoring method and device

Abstract

The invention provides an intermediate billet camber monitoring method and device, and the method comprises the steps: obtaining a current time sequence image and a historical time sequence image obtained by monitoring an intermediate billet, carrying out the preprocessing of the current time sequence image, obtaining a preprocessed image, inputting the preprocessed image into a semantic segmentation network model, obtaining a segmentation mask image of the intermediate billet, and obtaining a camber camber of the intermediate billet. Determining a detection result of the segmented mask images according to a comparison result of the sum of pixel values in the segmented mask images and the sum of preset pixel values, and splicing the segmented mask images according to the detection result to obtain a spliced mask image, based on the pixel point coordinates of the head section image, the pixel point coordinates of the middle section image and the pixel point coordinates of the tail section image, determining a monitoring result of the intermediate billet; the device can automatically monitor the camber defect of the intermediate billet, and compared with a manual observation mode, the monitoring efficiency and precision of the camber defect of the intermediate billet are improved, the operation stability is ensured, and the labor intensity of operators is reduced.

Description

Technical Field

[0001] This application relates to the field of steel production technology, and in particular to a method and device for monitoring the sickle bend of intermediate billets. Background Technology

[0002] Hot rolling is a high-efficiency, continuous steel rolling process, mainly used to produce hot-rolled strips, profiles, etc. During the rolling process, the steel billet passes through multiple rolling mill stands in sequence at a high temperature (usually 1100℃~1250℃), gradually thinning and lengthening until the desired finished size is formed.

[0003] In the roughing process of hot strip milling, camber is a relatively common quality defect in intermediate slab shape. This defect has a significant impact on subsequent hot-rolled production, causing fluctuations in the strip's wedge shape and overall centerline shift. In thin-gauge rolling, it easily generates inter-stand waviness, affecting product quality and yield. Currently, the identification and detection of camber mainly relies on manual observation, requiring an operator to constantly monitor the intermediate slabs in the control room for camber. This method is not only time-consuming and labor-intensive, but also lacks guaranteed accuracy.

[0004] Therefore, there is an urgent need for an automatic method to monitor the camber defects of intermediate billets, so as to reduce the labor intensity of operators, improve monitoring efficiency and accuracy, and ensure operational stability. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, this application provides a method and apparatus for monitoring the camber of intermediate billets, so as to solve the technical problems of low efficiency and accuracy in monitoring the camber of intermediate billets and the time-consuming and labor-intensive monitoring process.

[0006] According to one aspect of the embodiments of this application, a method for detecting the sickle bend of an intermediate billet is provided. The method includes: acquiring a current time-series image and a historical time-series image obtained by monitoring the intermediate billet; preprocessing the current time-series image to obtain a preprocessed image; the preprocessing method includes: distortion correction and perspective transformation; inputting the preprocessed image into a semantic segmentation network model to obtain a segmentation mask image of the intermediate billet; the semantic segmentation network model is trained on a preset semantic segmentation network model based on the historical time-series image; determining the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the sum of preset pixel values; the detection result includes: the intermediate billet does not exist, the intermediate billet exists and belongs to the head of the intermediate billet. The intermediate blank includes a middle section, a tail section, and a head section. The head section is a first preset length segment of the intermediate blank. The middle section is a second preset length segment of the intermediate blank. The tail section is a third preset length segment of the intermediate blank. The sum of the first, second, and third preset lengths is equal to the length of the intermediate blank. According to the detection results, the segmented mask image is stitched together to obtain a stitched mask image. The stitched mask image includes the head section image, the middle section image, and the tail section image of the intermediate blank. Based on the pixel coordinates of the head section image, the pixel coordinates of the middle section image, and the pixel coordinates of the tail section image, the monitoring result of the intermediate blank is determined.

[0007] In one embodiment of this application, if the segmentation mask image includes a first segmentation mask image and a second segmentation mask image, the process of determining the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values ​​includes: recording the comparison result of the sum of pixel values ​​in the first segmentation mask image with the preset sum of pixel values ​​as a first comparison result; and recording the comparison result of the sum of pixel values ​​in the second segmentation mask image with the preset sum of pixel values ​​as a second comparison result; the first segmentation mask image is the result of the first image captured. The first field of view is obtained by preprocessing and segmenting the temporal images captured by the camera with the second field of view; the second segmentation mask image is obtained by preprocessing and segmenting the temporal images captured by the camera with the second field of view; the first field of view is smaller than the second field of view; if the first comparison result is consistent with the second comparison result, then the first comparison result or the second comparison result is taken as the final comparison result; if the first comparison result is inconsistent with the second comparison result, then the second comparison result is taken as the final comparison result; the detection result is determined according to the final comparison result.

[0008] In one embodiment of this application, if the first segmentation mask image includes multiple closed contour mask images and the sum of the preset pixel values ​​includes the sum of a first preset pixel value and the sum of a second preset pixel value, then the process of comparing the sum of pixel values ​​in the first segmentation mask image with the sum of the preset pixel values ​​includes: taking the closed contour mask image with the largest area among the multiple closed contour mask images as the target mask image; comparing the area of ​​the target mask image with a preset area threshold to obtain a third comparison result; the third comparison result includes the target mask image having an area less than or equal to the preset area threshold, or the target mask image having an area greater than the preset area threshold. Threshold; if the third comparison result indicates that the area of ​​the target mask image is greater than the preset area threshold, then a first preset region of interest and a second preset region of interest in the target mask image are obtained, and the sum of pixel values ​​in the first preset region of interest and the sum of pixel values ​​in the second preset region of interest are calculated. The comparison results of the sum of pixel values ​​in the first preset region of interest and the second preset region of interest, as well as the comparison results of the second preset region of interest and the first preset region of interest, are then established. The comparison results of the sum of pixel values ​​in the first preset region of interest (PGI) and the sum of pixel values ​​in the second preset region of interest (PGI) are combined to obtain a fourth comparison result; the first preset PGI is smaller than the second preset PGI; in the fourth comparison result, the sum of pixel values ​​in the first preset PGI is less than the sum of pixel values ​​in the first preset PGI and the sum of pixel values ​​in the second preset PGI is greater than the sum of pixel values ​​in the second preset PGI, and the sum of pixel values ​​in the second preset PGI is less than the sum of pixel values ​​in the first preset PGI. Furthermore, the sum of pixel values ​​in the second preset region of interest is less than the sum of pixel values ​​in the first preset region of interest, and the sum of pixel values ​​in the first preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, and the sum of pixel values ​​in the second preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, which are taken as the comparison result of the presence of the intermediate blank; and, the comparison result of the presence of the intermediate blank is deleted from the fourth comparison result to obtain the comparison result of the absence of the intermediate blank, and the comparison result of the absence of the intermediate blank, the comparison result of the presence of the intermediate blank, and the area of ​​the target mask image in the third comparison result is less than or equal to the preset area threshold, which are taken as the first comparison result.

[0009] In one embodiment of this application, the process of determining the detection result based on the final comparison result includes: if the final comparison result is that the area of ​​the target mask image is less than or equal to the preset area threshold, or if the final comparison result is that the intermediate blank does not exist, then the detection result is determined to be that the intermediate blank does not exist; if the final comparison result is that the sum of pixel values ​​in the first preset region of interest is less than the sum of the first preset pixel values ​​and the sum of pixel values ​​in the second preset region of interest is greater than the sum of the second preset pixel values, then the detection result is determined to be that the intermediate blank exists and belongs to the head segment of the intermediate blank; if the final comparison result is that the sum of pixel values ​​in the first preset region of interest is greater than the sum of the second preset pixel values, then the detection result is determined to be that the intermediate blank exists and belongs to the head segment of the intermediate blank; if the final comparison result is that the sum of pixel values ​​in the first preset region of interest is greater than the sum of the second preset pixel values... If the sum of the second preset pixel values ​​and the sum of the pixel values ​​in the second preset region of interest are less than the sum of the first preset pixel values, or if the final comparison result is that the sum of the pixel values ​​in the first preset region of interest is less than the sum of the first preset pixel values ​​and the sum of the pixel values ​​in the second preset region of interest is less than the sum of the first preset pixel values, then the detection result is determined to be that the intermediate blank exists and belongs to the tail segment of the intermediate blank; if the final comparison result is that the sum of the pixel values ​​in the first preset region of interest is greater than the sum of the second preset pixel values ​​and the sum of the pixel values ​​in the second preset region of interest is greater than the sum of the second preset pixel values, then the detection result is determined to be that the intermediate blank exists and belongs to the middle segment of the intermediate blank.

[0010] In one embodiment of this application, if the detection result includes a current detection result and a previous detection result, then the process of stitching the segmentation mask image according to the detection result includes: if the current detection result indicates that the intermediate blank exists and belongs to the head segment of the intermediate blank, or if the previous detection result indicates that the intermediate blank does not exist and the current detection result indicates that the intermediate blank exists and belongs to the head segment of the intermediate blank, then the intermediate blank is captured to obtain a subsequent time-series image of the intermediate blank, and the current time-series image and the subsequent time-series image are stitched together; the current detection result is the current time-series image... The detection result is obtained by performing a detection; the previous detection result is the detection result obtained by detecting the previous time series image; if the current detection result is that the intermediate blank exists and belongs to the middle segment of the intermediate blank, then the current time series image and the previous time series image are stitched together; if the current detection result is that the intermediate blank exists and belongs to the tail segment of the intermediate blank, then image stitching is stopped, and the stitched mask image is obtained; or, if the previous detection result is that the intermediate blank exists and belongs to the middle segment of the intermediate blank, and the current detection result is that the intermediate blank does not exist, then image stitching is stopped, and the stitched mask image is obtained.

[0011] In one embodiment of this application, the process of determining the monitoring result based on the pixel coordinates of the head segment image, the pixel coordinates of the middle segment image, and the pixel coordinates of the tail segment image includes: determining the sickle curve value of the head region based on the pixel coordinates of the head segment image; determining the sickle curve value of the middle region based on the pixel coordinates of the middle segment image; and determining the sickle curve value of the tail region based on the pixel coordinates of the tail segment image; amplifying the sickle curve values ​​of the head region, the middle region, and the tail region respectively, and using the amplified values ​​of the head region sickle curve, the middle region sickle curve, and the tail region sickle curve as the monitoring result.

[0012] In one embodiment of this application, the process of determining the sickle curve value of the head region based on the pixel coordinates of the head segment image includes: scanning the pixel coordinates of the head segment image in a preset direction to obtain a set of centerline points of the head region; and taking the difference between the height value of the first center point in the set of centerline points of the head region and the height value of the last center point in the set of centerline points of the head region as the sickle curve value of the head region.

[0013] In one embodiment of this application, the process of magnifying the sickle curve value of the head region includes: obtaining the sickle curve magnification ratio; the sickle curve magnification ratio is determined based on a preset magnification ratio and the reduction ratio of the intermediate blank in the current time series image; multiplying the sickle curve value of the head region by the sickle curve magnification ratio to obtain the magnified sickle curve value of the head region.

[0014] In one embodiment of this application, the process of training a preset semantic segmentation network model based on the historical time-series image to obtain the semantic segmentation network model includes: denoising and filling missing values ​​in the historical time-series image to obtain a processed time-series image; labeling the contours of intermediate blanks in the processed time-series image to obtain a sample dataset; and training the preset semantic segmentation network model using the sample dataset to obtain the semantic segmentation network model.

[0015] According to one aspect of the embodiments of this application, a device for monitoring the sickle bend of an intermediate billet is provided, comprising: a data acquisition module for acquiring a current time-series image and a historical time-series image obtained by monitoring the intermediate billet; an image processing module for preprocessing the current time-series image to obtain a preprocessed image; the preprocessing method includes: distortion correction and perspective transformation; an image segmentation module for inputting the preprocessed image into a semantic segmentation network model to obtain a segmentation mask image of the intermediate billet; the semantic segmentation network model is trained on a preset semantic segmentation network model based on the historical time-series image; and an image detection module for determining a detection result of the segmentation mask image based on a comparison result of the sum of pixel values ​​in the segmentation mask image with the sum of preset pixel values; the detection result includes: the intermediate billet does not exist, the intermediate billet exists and belongs to the specified range, etc. The intermediate blank comprises a head segment, a middle segment containing the intermediate blank, and a tail segment containing the intermediate blank; the head segment is a first preset length segment of the intermediate blank; the middle segment is a second preset length segment of the intermediate blank; and the tail segment is a third preset length segment of the intermediate blank; the sum of the first preset length, the second preset length, and the third preset length is equal to the length of the intermediate blank; an image stitching module is used to stitch the segmented mask image according to the detection result to obtain a stitched mask image; the stitched mask image includes the head segment image, the middle segment image, and the tail segment image of the intermediate blank; and a result determination module is used to determine the monitoring result of the intermediate blank based on the pixel coordinates of the head segment image, the pixel coordinates of the middle segment image, and the pixel coordinates of the tail segment image.

[0016] The beneficial effects of this application are as follows: This application acquires current and historical time-series images obtained from monitoring intermediate billets, preprocesses the current time-series image to obtain a preprocessed image, inputs the preprocessed image into a semantic segmentation network model to obtain a segmentation mask image of the intermediate billet, determines the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values, stitches the segmentation mask image according to the detection result to obtain a stitched mask image, and determines the monitoring result of the intermediate billet based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image. The above process can realize automatic monitoring of the sickle-shaped defects of intermediate billets. Compared with manual observation, it improves the monitoring efficiency and accuracy of sickle-shaped defects of intermediate billets, ensures the stability of operation, and reduces the labor intensity of operators.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0019] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present application of a method for monitoring the sickle bend of an intermediate billet;

[0021] Figure 3 This is a flowchart illustrating a method for monitoring the sickle bend of an intermediate billet, as shown in another exemplary embodiment of this application;

[0022] Figure 4 This is a block diagram illustrating a segmentation mask system as shown in an exemplary embodiment of this application;

[0023] Figure 5 This is a schematic diagram illustrating a sickle-shaped defect in an exemplary embodiment of this application;

[0024] Figure 6 This is a block diagram illustrating an intermediate billet sickle bending monitoring device according to an exemplary embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the structure of a computer system for an electronic device, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0027] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0028] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0029] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0030] Reference Figure 1 As shown, the system architecture may include an acquisition device 101 and an electronic device 102. The electronic device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, or a neural network computer. Those skilled in the art can use the electronic device 102 to acquire current and historical time-series images obtained from monitoring the intermediate blank. The current time-series image is preprocessed to obtain a preprocessed image. This preprocessed image is then input into a semantic segmentation network model to obtain a segmentation mask image for the intermediate blank. Based on the comparison between the sum of pixel values ​​in the segmentation mask image and a preset sum of pixel values, the detection result of the segmentation mask image is determined. According to the detection result, the segmentation mask images are stitched together to obtain a stitched mask image. Based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image, the monitoring result of the intermediate blank is determined. The acquisition device 101 is used to acquire current time-series images and historical time-series images of intermediate billets. In this embodiment, the acquisition device 101 uses an industrial camera or the like to acquire images of intermediate billets on the conveyor rollers during the roughing process and provides them to the electronic device 102 for processing.

[0031] Indicatively, after acquiring the current time-series image and historical time-series image acquired by the acquisition device 101, the electronic device 102 preprocesses the current time-series image to obtain a preprocessed image. The preprocessed image is then input into the semantic segmentation network model to obtain a segmentation mask image of the intermediate billet. Based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values, the detection result of the segmentation mask image is determined. According to the detection result, the segmentation mask image is stitched together to obtain a stitched mask image. Based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image, the monitoring result of the intermediate billet is determined. The above process can automatically monitor the sickle-shaped defects of the intermediate billet. Compared with the manual observation method, it improves the monitoring efficiency and accuracy of the sickle-shaped defects of the intermediate billet, ensures the stability of operation, and reduces the labor intensity of operators.

[0032] It should be noted that the intermediate billet camber monitoring method provided in this application embodiment is generally executed by electronic device 102, and correspondingly, the intermediate billet camber monitoring device is generally installed in electronic device 102.

[0033] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0034] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present application of a method for monitoring the camber of an intermediate billet. This method can be executed by a computational processing device, which may be... Figure 1 The electronic device 102 shown is illustrated. (Refer to...) Figure 2 As shown, the intermediate billet camber monitoring method includes at least steps S210 to S260, which are described in detail below:

[0035] In step S210, the current time-series image and historical time-series image obtained from monitoring the intermediate billet are acquired. In one embodiment of this application, the current time-series image and historical time-series image are acquired by an industrial camera, and the resolution, working distance and shooting angle of the industrial camera are preset. At the same time, the intermediate billet conveyor roller area in the acquired image is set as the acquisition area.

[0036] In step S220, the current time-series image is preprocessed to obtain a preprocessed image. In one embodiment of this application, the preprocessing method includes distortion correction and perspective transformation. The distortion correction process for the current time-series image includes: calibrating the industrial camera using the Zhang Zhengyou calibration method, solving for the intrinsic parameters and distortion coefficients (including radial and tangential distortion) of the industrial camera, and correcting the distortion of the current time-series image using the distortion coefficients and the intrinsic parameters of the industrial camera to obtain a corrected image. The perspective transformation process for the corrected image includes: pre-selecting a non-standard viewpoint image and a standard viewpoint image; determining the perspective transformation matrix based on the correspondence between the four corner points of the roller conveyor in the non-standard viewpoint image and the four corner points of the roller conveyor in the standard viewpoint image; transforming the corrected image into a standard viewpoint image (e.g., a standard top-down view image) using the perspective transformation matrix, and using the standard viewpoint image as the preprocessed image. A standard viewpoint image refers to an image acquired under predefined, uniform viewing angles and conditions.

[0037] In step S230, the preprocessed image is input into the semantic segmentation network model to obtain the segmentation mask image of the intermediate blank. In one embodiment of this application, the semantic segmentation network model is trained on a preset semantic segmentation network model based on historical time-series images. The preset semantic segmentation network model can be a lightweight neural network model or a multi-scale feature fusion model, etc.

[0038] In step S240, the detection result of the segmentation mask image is determined based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values. In one embodiment of this application, the detection result includes: no intermediate blank, intermediate blank present and belonging to the head segment of the intermediate blank, intermediate blank present and belonging to the middle segment of the intermediate blank, and intermediate blank present and belonging to the tail segment of the intermediate blank. The head segment is a first preset length segment of the intermediate blank; the middle segment is a second preset length segment of the intermediate blank; the tail segment is a third preset length segment of the intermediate blank; the sum of the first preset length, the second preset length, and the third preset length is equal to the length of the intermediate blank. The segmentation mask image includes a first segmentation mask image and a second segmentation mask image. The process of determining the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values ​​includes: recording the comparison result of the sum of pixel values ​​in the first segmentation mask image with the preset sum of pixel values ​​as the first comparison result; and recording the comparison result of the sum of pixel values ​​in the second segmentation mask image with the preset sum of pixel values ​​as the second comparison result; if the first comparison result and the second comparison result are consistent, then the first comparison result or the second comparison result is used as the final comparison result; if the first comparison result and the second comparison result are inconsistent, then the second comparison result is used as the final comparison result; and the detection result is determined based on the final comparison result. By combining the first comparison result and the second comparison result to determine the final comparison result, the accuracy of the final comparison result determination is improved, thereby improving the accuracy of the detection result determination.

[0039] In step S250, the segmented mask images are stitched together according to the detection results to obtain a stitched mask image. In one embodiment of this application, the stitched mask image includes a head segment image, a middle segment image, and a tail segment image of the intermediate blank, and the stitched mask image is an image containing the complete intermediate blank.

[0040] In step S260, the monitoring result of the intermediate billet is determined based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image. In one embodiment of this application, the monitoring result of the intermediate billet can be automatically determined based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image, realizing automatic monitoring of the camber defect of the intermediate billet. Compared with manual observation, this improves the monitoring efficiency and accuracy of the camber defect of the intermediate billet, ensures operational stability, and reduces the labor intensity of operators.

[0041] In one embodiment of this application, if the segmentation mask image includes a first segmentation mask image and a second segmentation mask image, the process of determining the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values ​​includes:

[0042] The comparison result of the sum of pixel values ​​in the first segmentation mask image with the preset sum of pixel values ​​is recorded as the first comparison result; and the comparison result of the sum of pixel values ​​in the second segmentation mask image with the preset sum of pixel values ​​is recorded as the second comparison result. In one embodiment of this application, the first segmentation mask image is obtained by preprocessing and segmenting a time-series image acquired by a camera with a first field of view; the second segmentation mask image is obtained by preprocessing and segmenting a time-series image acquired by a camera with a second field of view; the first field of view is smaller than the second field of view; the camera with the first field of view is a first industrial camera, and the camera with the second field of view is a second industrial camera; the first and second industrial cameras acquire time-series images of the intermediate billet in parallel; the first industrial camera acquires the first time-series image, and the first industrial camera acquires the second time-series image; after preprocessing and segmenting the first time-series image, the first segmentation mask image is obtained; after preprocessing and segmenting the second time-series image, the second segmentation mask image is obtained; the resolution, working distance, and shooting angle of the first industrial camera are consistent with those of the second industrial camera; and the first field of view is half the size of the second field of view.

[0043] If the first comparison result matches the second comparison result, then either the first comparison result or the second comparison result is taken as the final comparison result. In one embodiment of this application, the final comparison result is determined by combining the first comparison result and the second comparison result, thereby improving the accuracy of the final comparison result.

[0044] If the first comparison result and the second comparison result are inconsistent, the second comparison result shall be taken as the final comparison result. In one embodiment of this application, if the first comparison result and the second comparison result are inconsistent, the second comparison result shall prevail. The second comparison result is determined based on the sum of pixel values ​​in the second segmentation mask image and the sum of preset pixel values. Since the second segmentation mask image has a larger field of view and more pixels, the second comparison result is more accurate than the first comparison result, thereby improving the accuracy of the final comparison result.

[0045] The detection result is determined based on the final comparison result. In one embodiment of this application, the accuracy of the detection result also improves as the accuracy of the final comparison result improves.

[0046] In one embodiment of this application, if the first segmentation mask image includes multiple closed contour mask images and the sum of preset pixel values ​​includes the sum of a first preset pixel value and the sum of a second preset pixel value, then the process of comparing the sum of pixel values ​​in the first segmentation mask image with the sum of preset pixel values ​​includes:

[0047] The closed contour mask image with the largest area among multiple closed contour mask images is used as the target mask image. In one embodiment of this application, the closed contour mask image with the largest area among multiple closed contour mask images has more pixels, thereby obtaining a more accurate comparison result when comparing the sum of pixel values ​​in the first segmentation mask image with the sum of preset pixel values.

[0048] The area of ​​the target mask image is compared with a preset area threshold to obtain a third comparison result. In one embodiment of this application, the third comparison result includes whether the area of ​​the target mask image is less than or equal to the preset area threshold or whether the area of ​​the target mask image is greater than the preset area threshold; the preset area threshold is set according to the actual situation. If the third comparison result is that the area of ​​the target mask image is less than or equal to the preset area threshold, it is determined that there is no intermediate blank in the segmented mask image; if the third comparison result is that the area of ​​the target mask image is greater than the preset area threshold, it is also necessary to determine whether the comparison result meets the requirement of the presence of an intermediate blank.

[0049] If the third comparison result indicates that the area of ​​the target mask image is greater than a preset area threshold, then a first preset region of interest (ROI) and a second preset ROI in the target mask image are obtained. The sum of pixel values ​​in the first preset ROI and the sum of pixel values ​​in the second preset ROI are calculated. The comparison results of the sum of pixel values ​​in the first preset ROI and the sum of pixel values ​​in the second preset ROI and the sum of pixel values ​​in the second preset ROI are combined to obtain a fourth comparison result. In one embodiment of this application, the first preset ROI is smaller than the second preset ROI; the first preset ROI is the smallest region range determined by the first shooting field of view, and the second preset ROI is the largest region range determined by the first shooting field of view.

[0050] In another embodiment of this application, taking the sum of pixel values ​​in the first preset region of interest as an example, the calculation formula for the sum of pixel values ​​in the first preset region of interest is as follows:

[0051]

[0052] Where mask_sum represents the sum of pixel values ​​in the first preset region of interest, and max_mask i,j Let (i,j) represent the first preset region of interest image in the target mask image, (i,j) represent the pixel in the i-th row and j-th column on the plane where the first preset region of interest image is located, w represents the maximum number of rows, i.e. the maximum value in the width dimension, and h represents the maximum number of columns, i.e. the maximum value in the length dimension.

[0053] In the fourth comparison result, if the sum of pixel values ​​in the first preset region of interest is less than the sum of the first preset pixel values ​​and the sum of pixel values ​​in the second preset region of interest is greater than the sum of the second preset pixel values, and vice versa, then the presence of an intermediate blank is considered a comparison result. In one embodiment of this application, the fourth comparison result includes not only the comparison result indicating the presence of an intermediate blank but also the comparison result indicating the absence of an intermediate blank. When the fourth comparison result indicates the presence of an intermediate blank, it is determined that an intermediate blank exists; when the fourth comparison result indicates the absence of an intermediate blank, it is determined that no intermediate blank exists. The sum of the first preset pixel values ​​and the sum of the second preset pixel values ​​are determined based on actual conditions.

[0054] Furthermore, the comparison results containing intermediate blanks are deleted from the fourth comparison results to obtain comparison results without intermediate blanks. The comparison results without intermediate blanks, the comparison results containing intermediate blanks, and the comparison results in the third comparison results where the area of ​​the target mask image is less than or equal to a preset area threshold are taken as the first comparison results. In one embodiment of this application, in the first comparison results, except for the comparison results containing intermediate blanks, the detection results corresponding to the other comparison results are all for the absence of intermediate blanks.

[0055] In another embodiment of this application, the second segmentation mask image includes multiple second closed contour mask images, and the sum of preset pixel values ​​also includes a third preset pixel value sum and a fourth preset pixel value sum. The process of comparing the sum of pixel values ​​in the second segmentation mask image with the preset pixel value sum includes: taking the closed contour mask image with the largest area among the multiple second closed contour mask images as the second target mask image; comparing the area of ​​the second target mask image with a preset area threshold to obtain a fifth comparison result; the fifth comparison result includes: the area of ​​the second target mask image is less than or equal to the preset area threshold, the second... If the area of ​​the target mask image is greater than a preset area threshold, and the fifth comparison result indicates that the area of ​​the second target mask image is greater than the preset area threshold, then the third preset region of interest and the fourth preset region of interest in the second target mask image are obtained. The sum of pixel values ​​in the third preset region of interest and the sum of pixel values ​​in the fourth preset region of interest are calculated. The comparison results of the sum of pixel values ​​in the third preset region of interest and the fourth preset region of interest are then compared. The comparison results are combined with the comparison results of the sum of pixel values ​​in the fourth preset region of interest and the sum of pixel values ​​in the fourth preset region of interest to obtain the sixth comparison result; the third preset region of interest is smaller than the fourth preset region of interest; in the sixth comparison result, the sum of pixel values ​​in the third preset region of interest is less than the sum of pixel values ​​in the third preset region of interest and the sum of pixel values ​​in the fourth preset region of interest is greater than the sum of pixel values ​​in the fourth preset region of interest, and the sum of pixel values ​​in the fourth preset region of interest is less than the sum of pixel values ​​in the third preset region of interest. The sum of preset pixel values ​​and the sum of pixel values ​​in the fourth preset region of interest are less than the sum of the third preset pixel values; the sum of pixel values ​​in the third preset region of interest is greater than the sum of the fourth preset pixel values, and the sum of pixel values ​​in the fourth preset region of interest is greater than the sum of the fourth preset pixel values. These are considered comparison results indicating the presence of an intermediate blank. Furthermore, the comparison results indicating the presence of an intermediate blank are deleted from the sixth comparison results to obtain comparison results indicating the absence of an intermediate blank. The area of ​​the second target mask image in the comparison results indicating the absence of an intermediate blank, the comparison results indicating the presence of an intermediate blank, and the fifth comparison results is considered as the second comparison results, where the area is less than or equal to a preset area threshold. In the second comparison results, except for the comparison results indicating the presence of an intermediate blank, all other comparison results correspond to the detection result indicating the absence of an intermediate blank.

[0056] In one embodiment of this application, the process of determining the detection result based on the final comparison result includes:

[0057] If the final comparison result is that the area of ​​the target mask image is less than or equal to a preset area threshold, or if the final comparison result is that there is no intermediate blank, then the detection result is determined to be that there is no intermediate blank. In one embodiment of this application, if the final comparison result is determined by the first comparison result, and the final comparison result is that the area of ​​the target mask image is less than or equal to a preset area threshold, or if there is no intermediate blank, then the detection result is determined to be that there is no intermediate blank.

[0058] If the final comparison result shows that the sum of pixel values ​​in the first preset region of interest is less than the sum of pixel values ​​in the first preset region of interest and the sum of pixel values ​​in the second preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, then the detection result is determined to be the presence of an intermediate blank and belonging to the head segment of the intermediate blank. In one embodiment of this application, if the final comparison result is determined by the first comparison result, and the final comparison result shows that the sum of pixel values ​​in the first preset region of interest is less than the sum of pixel values ​​in the first preset region of interest and the sum of pixel values ​​in the second preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, then the detection result is determined to be the presence of an intermediate blank and belonging to the head segment of the intermediate blank.

[0059] If the final comparison result is that the sum of pixel values ​​in the first preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest and the sum of pixel values ​​in the second preset region of interest is less than the sum of pixel values ​​in the first preset region of interest, or if the final comparison result is that the sum of pixel values ​​in the first preset region of interest is less than the sum of pixel values ​​in the first preset region of interest and the sum of pixel values ​​in the second preset region of interest is less than the sum of pixel values ​​in the first preset region of interest, then the detection result is determined to be the presence of an intermediate blank and belonging to the tail segment of the intermediate blank. In one embodiment of this application, if the final comparison result is determined by the first comparison result, and the final comparison result is that the sum of pixel values ​​in the first preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest and the sum of pixel values ​​in the second preset region of interest is less than the sum of pixel values ​​in the first preset region of interest, or if the final comparison result is that the sum of pixel values ​​in the first preset region of interest is less than the sum of pixel values ​​in the first preset region of interest and the sum of pixel values ​​in the second preset region of interest is less than the sum of pixel values ​​in the first preset region of interest, then the detection result is determined to be the presence of an intermediate blank and belonging to the tail segment of the intermediate blank.

[0060] If the final comparison result shows that the sum of pixel values ​​in the first preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, and the sum of pixel values ​​in the second preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, then the detection result is determined to be the presence of an intermediate blank and belonging to the middle segment of the intermediate blank. In one embodiment of this application, if the final comparison result is determined by the first comparison result, and the final comparison result shows that the sum of pixel values ​​in the first preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, and the sum of pixel values ​​in the second preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, then the detection result is determined to be the presence of an intermediate blank and belonging to the middle segment of the intermediate blank.

[0061] In one embodiment of this application, if the detection result includes: the current detection result and the previous detection result, then the process of stitching the segmentation mask image according to the detection result includes:

[0062] If the current detection result indicates the presence of an intermediate blank and it belongs to the head segment of the intermediate blank, or if the previous detection result indicates the absence of an intermediate blank and the current detection result indicates the presence of an intermediate blank and it belongs to the head segment of the intermediate blank, then the intermediate blank is captured to obtain a subsequent time-series image of the intermediate blank, and the current time-series image and the subsequent time-series image are stitched together. In one embodiment of this application, the process of capturing the intermediate blank continues until the detection result of a consecutive preset number of frames indicates the absence of an intermediate blank. The process of stitching the segmentation mask image according to the detection result is the process of stitching the first segmentation mask image according to the detection result. The current detection result is the detection result obtained by detecting the current time-series image; the previous detection result is the detection result obtained by detecting the previous time-series image. The formula for stitching the current time-series image and the subsequent time-series image is as follows:

[0063]

[0064] Among them, mask final1 This represents the image obtained by stitching the current time series image and the next time series image. `stitch(·,·)` represents the stitching function, and `mask` represents the image. k Mask represents the current time series image. k+1 This represents the next time series image, where n represents the number of time series images and k represents the sequence number of the time series image.

[0065] In another embodiment of this application, the previous time sequence image, the current time sequence image, and the next time sequence image are all single-frame images. Before image stitching, the image overlap area needs to be set according to the conveying speed of the intermediate billet on the conveying roller. During the image stitching process, the image overlap area in adjacent two frames is pixel-stacked according to the length of the image overlap area.

[0066] If the current detection result indicates the presence of an intermediate blank and that it belongs to the middle segment of the intermediate blank, then the current time-series image and the previous time-series image are stitched together. In one embodiment of this application, the formula for stitching the current time-series image and the previous time-series image is as follows:

[0067]

[0068] Among them, mask final2 This represents the image obtained by stitching the current time series image and the previous time series image. `stitch(·,·)` represents the stitching function, and `mask` represents the image. k Mask represents the current time series image. k-1This represents the previous time series image, where n represents the number of time series images and k represents the sequence number of the time series image.

[0069] If the current detection result indicates the presence of an intermediate blank and that it belongs to the tail segment of the intermediate blank, then image stitching is stopped, and a stitched mask image is obtained; or, if the previous detection result indicates the presence of an intermediate blank and that it belongs to the middle segment of the intermediate blank, and the current detection result indicates the absence of an intermediate blank, then image stitching is stopped, and a stitched mask image is obtained. In one embodiment of this application, on the one hand, during the determination of the detection result, the final comparison result is jointly determined by the first comparison result and the second comparison result, thereby achieving compensation of the first comparison result by the second comparison result, improving the accuracy of the detection result, which is beneficial for eliminating jagged edges and noise interference in the stitched mask image, improving the quality of the stitched mask image, and thus improving the monitoring accuracy; on the other hand, the stitched mask image is obtained by stitching together the first segmented mask image. Since the first segmented mask image has fewer pixels, it is beneficial for improving the image processing speed, thereby improving the monitoring efficiency.

[0070] In one embodiment of this application, the process of determining the monitoring result based on the pixel coordinates of the head segment image, the pixel coordinates of the middle segment image, and the pixel coordinates of the tail segment image includes:

[0071] The following methods are used to determine the sickle curve value of the head region based on the pixel coordinates of the head segment image; the sickle curve value of the middle region based on the pixel coordinates of the middle segment image; and the sickle curve value of the tail region based on the pixel coordinates of the tail segment image. In one embodiment of this application, the process of determining the sickle curve value of the head region based on the pixel coordinates of the head segment image includes: scanning the pixel coordinates of the head segment image in a preset direction to obtain a set of centerline points of the head region; and using the difference between the height value of the first center point in the set of centerline points of the head region and the height value of the last center point in the set of centerline points of the head region as the sickle curve value of the head region. The process of determining the sickle curve value of the middle region based on the pixel coordinates of the middle segment image is the same as the process of determining the sickle curve value of the head region based on the pixel coordinates of the head segment image. The process of determining the sickle curve value of the tail region based on the pixel coordinates of the tail segment image is the same as the process of determining the sickle curve value of the head region based on the pixel coordinates of the head segment image.

[0072] The sickle curve values ​​of the head region, middle region, and tail region are magnified respectively, and the magnified values ​​of the head region sickle curve, middle region sickle curve, and tail region sickle curve are used as monitoring results. In one embodiment of this application, the process of magnifying the sickle curve value of the head region includes: obtaining the sickle curve magnification ratio; the sickle curve magnification ratio is determined based on a preset magnification ratio and the reduction ratio of the intermediate billet in the current time-series image; multiplying the head region sickle curve value by the sickle curve magnification ratio to obtain the head region sickle curve magnification value. The process of magnifying the sickle curve value of the middle region is the same as the process of magnifying the head region sickle curve value, and the process of magnifying the sickle curve value of the tail region is the same as the process of magnifying the head region sickle curve value.

[0073] In one embodiment of this application, the process of determining the sickle curve value of the head region based on the pixel coordinates of the head segment image includes:

[0074] The pixel coordinates of the head segment image are scanned according to a preset direction to obtain the centerline point set of the head region. In one embodiment of this application, the preset direction is the width direction of the head segment image, which is perpendicular to the conveying direction of the intermediate billet on the conveying roller. The process of scanning the pixel coordinates of the head segment image according to the preset direction includes: setting a fixed width value in the width direction; determining the area to be scanned for each time based on the length of the head segment image and the fixed width value; obtaining a center point after each scan, until the head region scanning is completed, and obtaining the centerline point set of the head region. The process of scanning the pixel coordinates of the head segment image according to the preset direction to obtain the centerline point set of the head region can be completed by a preset scanning method, which can be a scanning method based on a computer vision library or a coordinate extraction method based on deep learning, etc.

[0075] The difference between the height of the first center point of the head region centerline point set and the height of the last center point of the head region centerline point set is taken as the sickle curve value of the head region. In one embodiment of this application, the process of determining the sickle curve value of the head region based on the pixel coordinates of the head segment image is more efficient and accurate than manual observation, and is more conducive to ensuring the stability of the operation.

[0076] In one embodiment of this application, the process of amplifying the scythe curve value of the head region includes:

[0077] Obtain the magnification ratio of the sickle bend. In one embodiment of this application, the magnification ratio of the sickle bend is determined based on a preset magnification ratio and the reduction ratio of the intermediate blank in the current time-series image. The calculation formula for the magnification ratio of the sickle bend is as follows:

[0078]

[0079] Where scale represents the magnification ratio of the sickle shape, m represents the preset magnification ratio, and L is a priori parameter calculated based on the actual image imaging size. real L represents the length of the intermediate billet. pixel This indicates the length of the intermediate blank in the current time series image. This represents the ratio of the length of the intermediate billet to its length in the current time series image, which is the reciprocal of the reduction ratio of the intermediate billet in the current time series image.

[0080] The head region sickle curvature value is obtained by multiplying it by the sickle curvature magnification ratio. In one embodiment of this application, the calculation formula for the head region sickle curvature magnification value is as follows:

[0081] camber_value_real=camber_value*scale Formula (5)

[0082] Wherein, camber_value_real represents the magnification value of the sickle curve in the head region, camber_value represents the sickle curve value in the head region, and scale represents the magnification ratio of the sickle curve.

[0083] In one embodiment of this application, the process of training a preset semantic segmentation network model based on historical time-series images to obtain the semantic segmentation network model includes:

[0084] Denoising and filling missing values ​​in historical time-series images yield a processed time-series image. In one embodiment of this application, denoising and filling missing values ​​in historical time-series images helps reduce noise and interference in the processed time-series image, thereby improving its quality.

[0085] The contours of intermediate blanks in the processed time-series images are annotated to obtain a sample dataset. In one embodiment of this application, the annotation tool for the contours of intermediate blanks in the processed time-series images can be selected as needed, and no specific limitation is made here.

[0086] A semantic segmentation network model is obtained by training a pre-defined semantic segmentation network model using a sample dataset. In one embodiment of this application, training the pre-defined semantic segmentation network model using a sample dataset improves the accuracy of the semantic segmentation network model in segmenting the outline of the intermediate billet.

[0087] Figure 3 This is a flowchart illustrating another exemplary embodiment of the present application of a method for monitoring the camber of an intermediate billet, as shown below. Figure 3As shown, the process of the intermediate billet sickle bend monitoring method includes: (1) continuously acquiring time-series images of the intermediate billet through an industrial camera; (2) cleaning and annotating the outline of the intermediate billet in the time-series images to obtain a sample dataset; (3) training a preset semantic segmentation network model using the sample dataset to obtain a semantic segmentation network model; (4) acquiring time-series images of the intermediate billet in parallel with a first industrial camera and a second industrial camera. The first industrial camera acquires a first time-series image and a second time-series image. The resolution, working distance, and shooting angle of the first industrial camera are consistent with those of the second industrial camera. The shooting field of view of the first industrial camera is half that of the shooting field of view of the second industrial camera; (5) preprocessing the first time-series image to obtain a first preprocessed image; preprocessing the second time-series image to obtain a second preprocessed image; (6) inputting the first preprocessed image into the semantic segmentation network. (7) Based on the comparison results of the sum of pixel values ​​in the first segmentation mask image and the sum of preset pixel values, and based on the comparison results of the sum of pixel values ​​in the second segmentation mask image and the sum of preset pixel values, the detection result of the segmentation mask image is determined; (8) According to the detection result, the first segmentation mask image is spliced ​​to obtain the spliced ​​mask image; (9) Based on the pixel coordinates of the head segment image, the sickle curve value of the head region is determined; based on the pixel coordinates of the middle segment image, the sickle curve value of the middle region is determined; and based on the pixel coordinates of the tail segment image, the sickle curve value of the tail region is determined; (10) The sickle curve values ​​of the head region, the middle region, and the tail region are magnified respectively, and the magnified values ​​of the sickle curve values ​​of the head region, the middle region, and the tail region are used as the monitoring result.

[0088] Figure 4 This is a block diagram illustrating a segmentation mask system in an exemplary embodiment of this application. Figure 4 The segmentation mask system includes: an acquisition module, a training module, and a mask extraction module. The acquisition module is used to acquire time-series images of intermediate blanks, clean the time-series images of intermediate blanks, and annotate the contours of intermediate blanks to obtain a sample dataset. The training module is used to train a preset semantic segmentation network model using the sample dataset to obtain a semantic segmentation network model. The mask extraction module is used to extract the contour images of intermediate blanks from the first preprocessed image using the semantic segmentation network model to obtain a first segmentation mask image; and to extract the contour images of intermediate blanks from the second preprocessed image using the semantic segmentation network model to obtain a second segmentation mask image.

[0089] Figure 5 This is a schematic diagram illustrating a sickle-shaped defect in an exemplary embodiment of this application, as shown below. Figure 5As shown, the white area in the figure represents the splicing mask image of the intermediate billet. The splicing mask image includes the head segment image, the middle segment image, and the tail segment image. Based on the pixel coordinates of the head segment image, the head sickle curve is obtained. Based on the pixel coordinates of the middle segment image, the middle sickle curve value is determined. Based on the pixel coordinates of the tail segment image, the tail sickle curve value is determined.

[0090] This application acquires current and historical time-series images of intermediate billets through monitoring. The current time-series image is preprocessed to obtain a preprocessed image, which is then input into a semantic segmentation network model to obtain a segmentation mask image of the intermediate billet. Based on the comparison between the sum of pixel values ​​in the segmentation mask image and a preset sum of pixel values, the detection result of the segmentation mask image is determined. According to the detection result, the segmentation mask images are stitched together to obtain a stitched mask image. Based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image, the monitoring result of the intermediate billet is determined. This process enables automatic monitoring of camber defects in intermediate billets. Compared to manual observation, this improves the monitoring efficiency and accuracy of camber defects in intermediate billets, ensures operational stability, and reduces the labor intensity of operators.

[0091] The following describes an embodiment of the apparatus described in this application, which can be used to execute the intermediate billet camber monitoring method described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the intermediate billet camber monitoring method described above in this application.

[0092] Figure 6 This is a block diagram illustrating an intermediate billet camber monitoring device according to an exemplary embodiment of this application. The device can be applied to... Figure 1 The implementation environment shown is specifically configured in electronic device 102. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0093] like Figure 6 As shown, the exemplary intermediate billet camber monitoring device 600 includes:

[0094] The data acquisition module 601 is used to acquire the current time-series image and historical time-series image obtained by monitoring the intermediate billet.

[0095] The image processing module 602 is used to preprocess the current time series image to obtain a preprocessed image.

[0096] The image segmentation module 603 is used to input the preprocessed image into the semantic segmentation network model to obtain the segmentation mask image of the intermediate blank.

[0097] The image detection module 604 is used to determine the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values.

[0098] The image stitching module 605 is used to stitch the segmented mask image according to the detection results to obtain a stitched mask image.

[0099] The result determination module 606 is used to determine the monitoring results of the intermediate billet based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image.

[0100] In one embodiment of this application, the current time series image and the historical time series image are acquired by an industrial camera, and the resolution, working distance and shooting angle of the industrial camera are preset. At the same time, the intermediate billet conveying roller area in the acquired image is set as the acquisition area.

[0101] In one embodiment of this application, the preprocessing method includes distortion correction and perspective transformation. The distortion correction process for the current time-series image includes: calibrating the industrial camera using the Zhang Zhengyou calibration method, solving for the intrinsic parameters and distortion coefficients (including radial and tangential distortion) of the industrial camera, and then correcting the distortion of the current time-series image using the distortion coefficients and the intrinsic parameters of the industrial camera to obtain the corrected image. The perspective transformation process for the corrected image includes: pre-selecting a non-standard viewpoint image and a standard viewpoint image; determining the perspective transformation matrix based on the correspondence between the four corner points of the roller conveyor in the non-standard viewpoint image and the four corner points of the roller conveyor in the standard viewpoint image; transforming the corrected image into a standard viewpoint image (e.g., a standard top-down view image) using the perspective transformation matrix, and using the standard viewpoint image as the preprocessed image. A standard viewpoint image refers to an image acquired under predefined, uniform viewing angles and conditions.

[0102] In one embodiment of this application, the semantic segmentation network model is obtained by training a preset semantic segmentation network model based on historical time-series images. The preset semantic segmentation network model can be a lightweight neural network model or a multi-scale feature fusion model, etc.

[0103] In one embodiment of this application, the detection results include: no intermediate billet, intermediate billet present and belonging to the head segment of the intermediate billet, intermediate billet present and belonging to the middle segment of the intermediate billet, and intermediate billet present and belonging to the tail segment of the intermediate billet. The head segment is a first preset length segment of the intermediate billet; the middle segment is a second preset length segment of the intermediate billet; and the tail segment is a third preset length segment of the intermediate billet. The sum of the first preset length, the second preset length, and the third preset length is equal to the length of the intermediate billet. The segmentation mask image includes a first segmentation mask image and a second segmentation mask image. The process of determining the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values ​​includes: recording the comparison result of the sum of pixel values ​​in the first segmentation mask image with the preset sum of pixel values ​​as the first comparison result; and recording the comparison result of the sum of pixel values ​​in the second segmentation mask image with the preset sum of pixel values ​​as the second comparison result; if the first comparison result and the second comparison result are consistent, then the first comparison result or the second comparison result is used as the final comparison result; if the first comparison result and the second comparison result are inconsistent, then the second comparison result is used as the final comparison result; and the detection result is determined based on the final comparison result. By combining the first comparison result and the second comparison result to determine the final comparison result, the accuracy of the final comparison result determination is improved, thereby improving the accuracy of the detection result determination.

[0104] In one embodiment of this application, the stitched mask image includes a head segment image, a middle segment image, and a tail segment image of the intermediate blank, and the stitched mask image is an image containing the complete intermediate blank.

[0105] In one embodiment of this application, the monitoring results of the intermediate billet can be automatically determined based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image, thereby realizing automatic monitoring of the camber defect of the intermediate billet. Compared with the manual observation method, this improves the monitoring efficiency and accuracy of the camber defect of the intermediate billet, ensures the stability of the operation, and reduces the labor intensity of the operators.

[0106] It should be noted that the intermediate billet camber monitoring device and the intermediate billet camber monitoring method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the intermediate billet camber monitoring device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0107] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the intermediate billet sickle bending monitoring method provided in the above embodiments.

[0108] Figure 7 This is a schematic diagram illustrating the structure of a computer system for an electronic device, as shown in an exemplary embodiment of this application. It should be noted that... Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0109] like Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 702 or programs loaded from storage portion 708 into Random Access Memory (RAM) 703. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0110] The following components are connected to I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 710 as needed so that computer programs read from it can be installed into storage section 708 as needed.

[0111] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs various functions defined in the system of this application.

[0112] Another aspect of this application provides a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a computer's processor, cause the computer to perform the intermediate billet camber monitoring method provided in the above embodiments. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0113] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0114] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for monitoring the camber of an intermediate billet, characterized in that, The method includes: Acquire the current time-series image and historical time-series image obtained from monitoring the intermediate billet; The current time-series image is preprocessed to obtain a preprocessed image; the preprocessing methods include distortion correction and perspective transformation. The preprocessed image is input into the semantic segmentation network model to obtain the segmentation mask image of the intermediate blank; the semantic segmentation network model is trained based on the historical time series image. The detection result of the segmentation mask image is determined based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the sum of preset pixel values; the detection result includes: the intermediate blank does not exist, the intermediate blank exists and belongs to the head segment of the intermediate blank, the intermediate blank exists and belongs to the middle segment of the intermediate blank, and the intermediate blank exists and belongs to the tail segment of the intermediate blank; the head segment is the first preset length segment of the intermediate blank; the middle segment is the second preset length segment of the intermediate blank; the tail segment is the third preset length segment of the intermediate blank; the sum of the first preset length, the second preset length, and the third preset length is equal to the length of the intermediate blank; Based on the detection results, the segmented mask images are stitched together to obtain a stitched mask image; the stitched mask image includes the head segment image, middle segment image, and tail segment image of the intermediate blank; The monitoring results of the intermediate blank are determined based on the pixel coordinates of the head segment image, the pixel coordinates of the middle segment image, and the pixel coordinates of the tail segment image.

2. The method for monitoring the camber of intermediate billets according to claim 1, characterized in that, If the segmentation mask image includes a first segmentation mask image and a second segmentation mask image, then the process of determining the detection result of the segmentation mask image based on the comparison result of the sum of pixel values ​​in the segmentation mask image with the preset sum of pixel values ​​includes: The comparison result of the sum of pixel values ​​in the first segmentation mask image with the preset sum of pixel values ​​is recorded as the first comparison result; and the comparison result of the sum of pixel values ​​in the second segmentation mask image with the preset sum of pixel values ​​is recorded as the second comparison result; the first segmentation mask image is obtained by preprocessing and segmenting a time-series image captured by a camera with a first field of view; the second segmentation mask image is obtained by preprocessing and segmenting a time-series image captured by a camera with a second field of view; the first field of view is smaller than the second field of view; If the first comparison result is consistent with the second comparison result, then the first comparison result or the second comparison result shall be taken as the final comparison result; If the first comparison result is inconsistent with the second comparison result, the second comparison result shall be taken as the final comparison result; The detection result is determined based on the final comparison result.

3. The method for monitoring the camber of intermediate billets according to claim 2, characterized in that, If the first segmentation mask image includes multiple closed contour mask images and the sum of the preset pixel values ​​includes the sum of a first preset pixel value and the sum of a second preset pixel value, then the process of comparing the sum of pixel values ​​in the first segmentation mask image with the sum of the preset pixel values ​​includes: The closed contour mask image with the largest area among the multiple closed contour mask images is used as the target mask image; The area of ​​the target mask image is compared with a preset area threshold to obtain a third comparison result; the third comparison result includes the area of ​​the target mask image being less than or equal to the preset area threshold, and the area of ​​the target mask image being greater than the preset area threshold. If the third comparison result indicates that the area of ​​the target mask image is greater than the preset area threshold, then a first preset region of interest and a second preset region of interest in the target mask image are obtained, and the sum of pixel values ​​in the first preset region of interest and the sum of pixel values ​​in the second preset region of interest are calculated. The comparison results of the sum of pixel values ​​in the first preset region of interest with the sum of the first preset pixel values, the sum of pixel values ​​in the first preset region of interest with the sum of the second preset pixel values, the sum of pixel values ​​in the second preset region of interest with the sum of the first preset pixel values, and the sum of pixel values ​​in the second preset region of interest with the sum of the second preset pixel values ​​are combined to obtain a fourth comparison result; the first preset region of interest is smaller than the second preset region of interest. The following are considered comparison results in the fourth comparison result: the sum of pixel values ​​in the first preset region of interest is less than the sum of the first preset pixel values ​​and the sum of pixel values ​​in the second preset region of interest is greater than the sum of the second preset pixel values; the sum of pixel values ​​in the first preset region of interest is greater than the sum of the second preset pixel values ​​and the sum of pixel values ​​in the second preset region of interest is less than the sum of the first preset pixel values; the sum of pixel values ​​in the first preset region of interest is less than the sum of the first preset pixel values ​​and the sum of pixel values ​​in the second preset region of interest is less than the sum of the first preset pixel values; the sum of pixel values ​​in the first preset region of interest is greater than the sum of the second preset pixel values ​​and the sum of pixel values ​​in the second preset region of interest is greater than the sum of the second preset pixel values. Furthermore, the comparison results containing the intermediate blank are deleted from the fourth comparison results to obtain the comparison results without the intermediate blank, and the comparison results without the intermediate blank, the comparison results containing the intermediate blank, and the area of ​​the target mask image in the third comparison results being less than or equal to the preset area threshold are taken as the first comparison results.

4. The method for monitoring the camber of intermediate billets according to claim 3, characterized in that, The process of determining the detection result based on the final comparison result includes: If the final comparison result is that the area of ​​the target mask image is less than or equal to the preset area threshold, or if the final comparison result is that the intermediate blank does not exist, then the detection result is determined to be that the intermediate blank does not exist. If the final comparison result is that the sum of pixel values ​​in the first preset region of interest is less than the sum of the first preset pixel values ​​and the sum of pixel values ​​in the second preset region of interest is greater than the sum of the second preset pixel values, then the detection result is determined to be that the intermediate blank exists and belongs to the head segment of the intermediate blank; If the final comparison result is that the sum of pixel values ​​in the first preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest and the sum of pixel values ​​in the second preset region of interest is less than the sum of pixel values ​​in the first preset region of interest, or if the final comparison result is that the sum of pixel values ​​in the first preset region of interest is less than the sum of pixel values ​​in the first preset region of interest and the sum of pixel values ​​in the second preset region of interest is less than the sum of pixel values ​​in the first preset region of interest, then the detection result is determined to be that the intermediate blank exists and belongs to the tail segment of the intermediate blank; If the final comparison result is that the sum of pixel values ​​in the first preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, and the sum of pixel values ​​in the second preset region of interest is greater than the sum of pixel values ​​in the second preset region of interest, then the detection result is determined to be that the intermediate blank exists and belongs to the middle segment of the intermediate blank.

5. The method for monitoring the camber of an intermediate billet according to any one of claims 1-4, characterized in that, If the detection result includes the current detection result and the previous detection result, then the process of stitching the segmentation mask image according to the detection result includes: If the current detection result indicates that the intermediate blank exists and belongs to the head segment of the intermediate blank, or if the previous detection result indicates that the intermediate blank does not exist and the current detection result indicates that the intermediate blank exists and belongs to the head segment of the intermediate blank, then the intermediate blank is captured to obtain a subsequent time-series image of the intermediate blank, and the current time-series image and the subsequent time-series image are stitched together; the current detection result is the detection result obtained by detecting the current time-series image; the previous detection result is the detection result obtained by detecting the previous time-series image; If the current detection result indicates that the intermediate blank exists and belongs to the middle segment of the intermediate blank, then the current time series image and the previous time series image are stitched together. If the current detection result indicates that the intermediate blank exists and belongs to the tail segment of the intermediate blank, then image stitching is stopped to obtain the stitched mask image; or, if the previous detection result indicates that the intermediate blank exists and belongs to the middle segment of the intermediate blank, and the current detection result indicates that the intermediate blank does not exist, then image stitching is stopped to obtain the stitched mask image.

6. The method for monitoring the camber of an intermediate billet according to any one of claims 1-4, characterized in that, The process of determining the monitoring result based on the pixel coordinates of the head segment image, the middle segment image, and the tail segment image includes: Based on the pixel coordinates of the head segment image, the sickle curve value of the head region is determined; based on the pixel coordinates of the middle segment image, the sickle curve value of the middle region is determined; and based on the pixel coordinates of the tail segment image, the sickle curve value of the tail region is determined. The sickle curve values ​​of the head region, the middle region, and the tail region are magnified respectively, and the magnified values ​​of the sickle curves of the head region, the middle region, and the tail region are used as the monitoring results.

7. The method for monitoring the camber of intermediate billets according to claim 6, characterized in that, The process of determining the sickle curve value of the head region based on the pixel coordinates of the head segment image includes: The pixel coordinates of the head segment image are scanned according to a preset direction to obtain the center line point set of the head region. The difference between the height of the first center point in the set of center line points of the head region and the height of the last center point in the set of center line points of the head region is taken as the sickle curve value of the head region.

8. The method for monitoring the camber of intermediate billets according to claim 6, characterized in that, The process of amplifying the scythe curvature value of the head region includes: Obtain the magnification ratio of the sickle curve; the magnification ratio of the sickle curve is determined based on a preset magnification ratio and the reduction ratio of the intermediate blank in the current time series image; Multiply the sickle curve value of the head region by the sickle curve magnification ratio to obtain the sickle curve magnification value of the head region.

9. The method for monitoring the camber of an intermediate billet according to any one of claims 1-4, characterized in that, The process of training a preset semantic segmentation network model based on the historical time-series images to obtain the semantic segmentation network model includes: The historical time series image is denoised and missing values ​​are filled to obtain the processed time series image; The contours of intermediate blanks in the processed time-series images are labeled to obtain the sample dataset; The semantic segmentation network model is obtained by training the preset semantic segmentation network model using the sample dataset.

10. A device for monitoring the sickle-shaped bend of an intermediate billet, characterized in that, include: The data acquisition module is used to acquire the current time-series image and historical time-series image obtained from monitoring the intermediate billet; The image processing module is used to preprocess the current time-series image to obtain a preprocessed image; Preprocessing methods include: distortion correction and perspective transformation; The image segmentation module is used to input the preprocessed image into the semantic segmentation network model to obtain the segmentation mask image of the intermediate blank; the semantic segmentation network model is trained on the preset semantic segmentation network model based on the historical time series image; An image detection module is used to determine the detection result of the segmentation mask image based on a comparison between the sum of pixel values ​​in the segmentation mask image and a preset sum of pixel values. The detection result includes: the absence of the intermediate blank, the presence of the intermediate blank and belonging to the head segment of the intermediate blank, the presence of the intermediate blank and belonging to the middle segment of the intermediate blank, and the presence of the intermediate blank and belonging to the tail segment of the intermediate blank. The head segment is a first preset length segment of the intermediate blank; the middle segment is a second preset length segment of the intermediate blank; and the tail segment is a third preset length segment of the intermediate blank. The sum of the first preset length, the second preset length, and the third preset length is equal to the length of the intermediate blank. An image stitching module is used to stitch the segmented mask image according to the detection results to obtain a stitched mask image; the stitched mask image includes the head segment image, middle segment image, and tail segment image of the intermediate blank; The result determination module is used to determine the monitoring result of the intermediate blank based on the pixel coordinates of the head segment image, the pixel coordinates of the middle segment image, and the pixel coordinates of the tail segment image.

Images