A mold state visual monitoring method and system based on multi-scale template matching and deep learning cooperation and a medium

Abstract

The application provides a mold state visual monitoring method and system based on multi-scale template matching and deep learning cooperation and a medium, the method comprising: generating multi-angle and multi-scale templates based on a three-dimensional CAD model of a mold; collecting template images or mold images, and using a lightweight deep learning target detection network to perform coarse positioning on the collected template images or mold images in combination with a hierarchical detection mechanism guided by coarse positioning; pre-processing and enhancing the positioned image area; post-processing and false detection filtering of the pre-processed and enhanced image to locate the defect and foreign matter attachment area of the template or mold; and linking the detection result with a programmable logic controller to realize semi-automatic cyclic monitoring and alarm. Through the fusion of multi-template dynamic management, deep learning coarse positioning, multi-scale pyramid matching, adaptive image enhancement and refined post-processing, the application realizes millisecond-level and high-reliability detection of the mold state.

Description

Technical Field

[0001] This invention relates to the field of mold monitoring technology, and in particular to a method, system and medium for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration. Background Technology

[0002] Various processing equipment (such as injection molding machines, stamping machines, and die-casting machines) have achieved high-speed and high-precision production. However, this has also placed higher demands on mold monitoring and condition detection. Typical production lines have a single-machine cycle time of 0.5 to 2 seconds per piece, posing a significant challenge to the real-time performance, accuracy, and stability of detection algorithms. Furthermore, in semi-automated production modes, the system needs to be closely linked to the switching on and off of the processing equipment—each time the machine starts up, the vision monitoring system immediately performs a complete image acquisition and analysis, making the entire process exhibit an integrated "machine-inspection" cyclical operation characteristic, which places higher demands on the algorithm's response latency and fault recovery capabilities.

[0003] Existing visual inspection systems mainly rely on the following traditional algorithms: 1. Based on pixel-level template matching: Using a single or small number of templates, sliding window matching is performed on the image to be tested using metrics such as SSD (Sum of Squared Differences) and NCC (Normalized Cross-Correlation). The advantages are that it is simple to implement and easy to understand; the disadvantages are that it is sensitive to changes in scale, rotation and lighting, and may cause positioning errors due to small displacements (0.5° to 2°) or scaling (0.5% to 2%) of the mold.

[0004] 2. Local matching based on key point descriptors: Using algorithms such as SIFT, SURF, and ORB to extract and match feature points can, to some extent, cope with changes in viewpoint and scale. However, in industrial settings, mold surfaces are flat and lack texture, and matching points are sparse and easily affected by light and noise, resulting in a mismatch rate of over 5%.

[0005] 3. Detection methods based on global features or contours: Comparison is performed by extracting edges, contours, shape context, etc. It is suitable for molds with clear outlines, but it is not sensitive enough to details such as surface engravings, small grooves, and reflective areas, and often misses minor defects.

[0006] 4. Matching based on grayscale / color histograms: Calculate the similarity of grayscale and color distribution in the whole or a local area; It is not very robust to changes in lighting and material, and is particularly prone to failure on highly reflective metal surfaces (such as aluminum alloys and chrome-plated steel).

[0007] Furthermore, most existing systems lack complete image preprocessing and post-processing modules: Preprocessing: usually only includes simple noise reduction or gamma correction, which is difficult to effectively deal with splashes, dust and complex lighting in the production environment; Post-processing: It often focuses on thresholding the output results and lacks secondary filtering based on connected components and geometric features, resulting in a false positive and false negative rate that often exceeds 3%.

[0008] Although verification is often conducted on-site at injection molding machines during the testing and development phase, real-world application environments may involve various mold operating conditions (such as stamping, die casting, precision milling, etc.). Therefore, a general-purpose, lightweight, stable, and robust traditional visual monitoring solution is needed. Summary of the Invention

[0009] The main objective of this invention is to provide a method, system, and medium for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration. The aim is to achieve millisecond-level, highly reliable detection of mold status by integrating multi-template dynamic management, deep learning coarse localization, multi-scale pyramid matching, adaptive image enhancement, and refined post-processing.

[0010] To achieve the above objectives, this invention proposes a visual monitoring method for mold state based on multi-scale template matching and deep learning collaboration. The method includes the following steps: Step S10: Based on the 3D CAD model of the mold, generate a template with multiple angles and scales through rotation and scaling transformations; Step S20: Acquire template images or mold images, and use a lightweight deep learning object detection network, combined with a coarse localization-guided hierarchical detection mechanism, to coarsely localize the acquired template images or mold images; when the confidence level is greater than or equal to the first preset threshold, execute the pyramid matching process, output the coordinates of the region of interest (ROI), and use a pyramid-style multi-scale template matching algorithm within the ROI, using normalized cross-correlation for matching, and improving localization accuracy through sub-pixel interpolation; Step S30: Preprocess and enhance the located image region; Step S40: Perform post-processing and false detection filtering on the pre-processed and enhanced images to locate the defects and foreign object attachment areas of the template or mold; Step S50: Link the detection results with the programmable logic controller to achieve semi-automatic cyclic monitoring and alarm.

[0011] A further technical solution of the present invention is that the lightweight deep learning object detection network is a YOLOv8 model with ShuffleNetV2 as the backbone network.

[0012] A further technical solution of the present invention is that step S20 further includes: When the confidence level of consecutive first preset frame images is greater than or equal to the first preset threshold, execute the high confidence mode, lock the ROI mode, close the pyramid matching process, and directly execute step S30. When the confidence level is less than the first preset threshold and greater than or equal to the second preset threshold, the low confidence mode is executed and the process is switched to the pyramid matching process. When the confidence level is less than the second preset threshold, the programmable logic controller is triggered to alarm and prompt manual re-inspection.

[0013] A further technical solution of the present invention is that the pyramid matching process in step S20 includes the following steps: Step S201, the original image A three-layer pyramid is generated by downsampling, with the resolution of each layer decreasing sequentially. It is the original image. This is an image of the second layer of the pyramid. It is the third layer of the pyramid image. Downsample means that the image is downsampled, which is to calculate the mean or interpolation of each pixel region to obtain a low-resolution image. Step S202: Calculate the similarity between the template and the image region based on normalized cross-correlation. The calculation formula is as follows: It represents the coordinates of the current pixel in the image, indicating the position of the current local image patch; It is the pyramid level The image above The pixel value in the coordinates is , represents the matching region when the template is aligned with the image block; It's a template. The pixel value in the image is located at the position of the pixel value in the image. ; and These are images respectively and templates Mean: This represents the number of pixels in the image patch; the denominator includes the standard deviation of the template and the image patch, ensuring that the calculation result is not affected by changes in image brightness, so that the final NCC value is within the acceptable range. between; Indicates the location Similarity between the image patch and the template; The steps for calculating the similarity between the template and the image region based on normalized cross-correlation include: By using quadratic interpolation to perform sub-pixel-level positioning of the NCC peak, the positioning accuracy is improved to two decimal places. This allows for the identification of the maximum NCC value. Then, quadratic interpolation is used to finely locate the peak and calculate a more precise peak position. : It is the peak position of NCC, which indicates the initially estimated matching position of the image patch with the template; The sub-pixel precise matching position is calculated through quadratic interpolation, improving the accuracy to two decimal places. and They are about and NCC value in direction The first derivative at that point, i.e., the rate of change of the NCC value; and They are about and NCC value in direction The second derivative at that point is the curvature of the NCC value.

[0014] A further technical solution of the present invention is that step S10 includes: Step S101, using 3D CAD model parameters The rotation transformation matrix R(θ) is used to generate view templates of the mold at various angles. ; Among them, 3D CAD model parameters The coordinate parameters represent the coordinates of the 3D CAD model, describing the geometry of the mold; the rotation transformation matrix R(θ) represents the rotation about a certain axis. This represents the rotation angle of the model in space. This refers to the different viewing angles of the mold, ranging from 0° to 345°, increasing by 15° each time; Render represents the rendering operation, which renders the 3D model. Through rotation transformation matrix Generate 2D view templates from different angles ; Step S102, apply scaling transformation Generate size variants and scale the transformation matrix. Used to scale the rotated view template to generate different size variations of the mold; It is a scaling factor, representing different sizes of the mold view template: It is a view template generated by rotation. It is a scaling transformation matrix. This is the final template after scaling. Step S103: Combining historical matching accuracy, call frequency, and ambient lighting adaptability, update the template weights using a weighted function. Weight This reflects the applicability and priority of each template; the weight update formula is as follows: It's a template. The weight indicates the priority of calling the template; It represents the historical matching precision, indicating the template. Accuracy in past matching processes; It refers to the call frequency, indicating the template. Frequency of use; It is the ambient light adaptability, representing the template. Adaptability to different lighting conditions; These are weighting coefficients, which respectively control the impact of historical matching accuracy, call frequency, and ambient lighting adaptability on the final weight. Step S104: Update and optimize the template based on the self-evolution mechanism, according to the current features. and old features Weighted update template feature vector The updated formula is: It is the updated template feature vector, representing the new feature representation of the template; It is the old feature vector of the template, representing the features of the template before the update; It is the template feature vector obtained based on the new detection data; It is the learning rate, which controls the degree of fusion between new and old features.

[0015] A further technical solution of the present invention is that step S30 includes: Step S301: Perform Gaussian filtering, bilateral filtering to preserve edges, adaptive thresholding and illumination compensation, reflection suppression, and CLAHE local contrast enhancement on the mold image. The step of performing Gaussian filtering on the mold image in step S301 includes: Image smoothing is achieved through a weighted averaging method, where the value of each pixel is weighted and averaged with the values ​​of its neighboring pixels according to a Gaussian distribution. The formula used is as follows: in, It is the original image; It is a Gaussian filter kernel, representing a two-dimensional Gaussian distribution with a standard deviation of . This controls the width of the filter; Indicates the convolution operation; in: It is the standard deviation of the Gaussian kernel, which controls the degree of image blur. These are the coordinates of each pixel in the image; s is the exponential part of the Gaussian distribution, and its weight gradually decreases as the distance from the center point increases; The step of performing bilateral filtering on the mold image to preserve edges in step S301 includes: weighted averaging based on spatial distance and pixel value similarity. The image after bilateral filtering is located at... Pixel value at that location, These are the coordinates of the current pixel; Neighborhood pixels Pixel values; It represents the neighborhood range of the current pixel; It is a similarity weight function based on pixel value differences, used to measure the similarity between the neighboring pixel values ​​and the current pixel value; It is a similarity weight function based on spatial distance, used to measure the spatial distance between neighboring pixels and the current pixel; It is the normalization coefficient, used to ensure that the sum of the weighted values ​​is 1; Pixel value difference weights in bilateral filtering The calculation formula is: in, It controls the impact of pixel value differences on the weighting. Spatial distance weights of bilateral filtering The calculation formula is: in, It controls the impact of spatial distance on weighting; In step S301, the step of adaptive thresholding and illumination compensation for the mold image includes: dynamically adjusting the threshold according to the local statistical characteristics of the image to process image data under different illumination conditions, eliminate the influence of background illumination, enhance the contrast between the target and the background, and improve the accuracy of image processing. Specifically, this includes: Given the location of each pixel in the image Through local statistics and To calculate the dynamic threshold of this pixel : In position Adaptive threshold at the location; It is a location The local mean of the surrounding area reflects the average brightness of the image in that region; local mean ,in Therefore The surrounding area centered on, It is the value of each pixel in the neighborhood. It is the number of pixels in the neighborhood; It is a location The local standard deviation of a neighborhood represents the dispersion of pixel values ​​within that neighborhood; local standard deviation ; These are empirical parameters that control the influence of local standard deviations on the threshold. It is a constant parameter used to normalize the standard deviation, eliminate the influence of background illumination changes, and make the threshold more stable. This represents a gain based on local standard deviation, which enhances the contrast of prominent target areas in an image and reduces background noise interference. In step S301, the step of suppressing reflection in the mold image includes: A spectral filtering model is used to analyze reflective areas, and high-reflectivity artifacts are eliminated based on saturation and brightness thresholds to enhance the realistic representation of surface details of metal molds. In step S301, the step of performing CLAHE local contrast enhancement on the mold image includes: First, the image is divided into multiple small blocks, and then histogram equalization is performed on each small block. The gain of the histogram is limited by CLAHE. Histogram equalization is calculated by dividing the local region into blocks to limit contrast gain and avoid excessive enhancement. The formula is as follows: in, It is the original input image; It is the th in the image histogram The count value of each gray level indicates the number of gray levels in the image. The number of pixels; It is the contrast limiting threshold; It refers to grayscale levels; It is the gray level, representing the gray value of each pixel in the image; This involves limiting the count value for each gray level to ensure that histogram equalization does not lead to over-enhancement.

[0016] A further technical solution of the present invention is that step S40 includes: performing morphological processing and Canny edge extraction on the image, filtering out false detections based on geometric features, removing abnormal areas, and locating the defects and foreign object attachment areas of the template or mold through connected component analysis.

[0017] A further technical solution of the present invention is that the step of performing morphological processing on the image in step S40 includes: Detecting images from binarized images The closing operation merges neighboring regions and fills small holes. The formula for the closing operation is: It is a structural element that eliminates small holes and merges adjacent areas; It is the original binary image; It is a structural element used to control the shape and size of expansion and corrosion. Shapes used in expansion and corrosion operations; It is an expansion operation; It is a corrosion operation; The step S40, which involves Canny edge extraction of the image, includes: extracting edge contours in the image by calculating the image gradient to aid in geometric feature analysis. The gradient calculation uses the Sobel operator. Horizontal gradient ; in, It is the original image. and It is the convolution kernel of the Sobel operator in the horizontal and vertical directions; The step S40, which filters false positives in the image based on geometric features, includes: analyzing the shape features of candidate regions and determining whether the target region meets the requirements according to set rules. The shape features include area, aspect ratio, and convex hull defect rate. The formula used to calculate the aspect ratio of the candidate region is: in, The aspect ratio of the candidate region represents the ratio of the region's width to its height. It is the width of the candidate region. It is the height of the candidate region; Calculate the convex hull defect rate The formula used is: in, It is the convex hull area of ​​the candidate region. This is the actual area of ​​the candidate region; Perimeter-area ratio This is the ratio of the perimeter to the area of ​​the candidate region, used to determine the shape of the candidate region; The elimination rule in step S40 is: if If the candidate region is not found to be an abnormal region, it should be removed. In step S40, the formula used to locate the defective and foreign object attachment areas of the template or mold through connectivity component analysis is as follows: in, It is the first A connected component represents a candidate region in the image; Indicates candidate region Contour differences; It is used to determine whether the contour difference exceeds the set abnormality threshold.

[0018] To achieve the above objectives, the present invention also provides a mold monitoring system based on deep learning and multi-scale template matching. The system includes a memory, a processor, and a mold monitoring program based on deep learning and multi-scale template matching stored on the processor. The mold monitoring program based on deep learning and multi-scale template matching is executed by the processor to perform the steps of the method described above.

[0019] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a mold monitoring program based on deep learning and multi-scale template matching, wherein the mold monitoring program based on deep learning and multi-scale template matching is executed by a processor to perform the steps of the method described above.

[0020] The beneficial effects of the present invention, which is a method, system, and medium for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration, are as follows: 1. High robustness: Multi-angle, multi-scale templates and CLAHE reinforcement ensure stable matching under different lighting and vibration conditions; 2. Collaborative Detection Architecture: This architecture combines a lightweight object detection network with traditional template matching, using the former's output to dynamically constrain the latter's search space. 3. Intelligent computation allocation: Lightweight YOLOv8 excludes non-critical image regions from computation, reducing template matching time to 1 / 4 of the original method; 4. Failure-safe design: The dynamic switching mechanism seamlessly reverts to traditional methods when the deep learning model fails, ensuring continuous system operation; 5. Real-time performance: Pyramid coarse matching and asynchronous preprocessing are executed in parallel, with a typical detection latency of ≤200ms; 6. Lightweight: Based on lightweight libraries such as OpenCV, it can run on embedded PCs without a GPU; 7. Scalability: The template set can be dynamically updated, supporting quick switching between multiple mold types; 8. Intelligent linkage: It interfaces with the equipment PLC to support semi-automatic production cycles and automatic alarm for failures. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a preferred embodiment of the visual monitoring method for mold state based on multi-scale template matching and deep learning collaboration of the present invention. Figure 2 This is a schematic diagram of the overall process of the visual monitoring method for mold status based on multi-scale template matching and deep learning collaboration of the present invention. Figure 3 This is a hardware architecture diagram of the mold monitoring system based on deep learning and multi-scale template matching of the present invention.

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0024] This invention proposes a visual monitoring method for mold status based on multi-scale template matching and deep learning collaboration. It is a highly robust visual detection method suitable for monitoring the status of molds in various processing equipment. It integrates multi-template dynamic management, deep learning coarse localization, multi-scale pyramid matching, adaptive image enhancement and refined post-processing, and a semi-automatic linkage mechanism, which can achieve millisecond-level response and stable detection in high-speed and complex environments.

[0025] like Figure 1 and Figure 2 As shown, a preferred embodiment of the mold state visual monitoring method based on multi-scale template matching and deep learning collaboration of the present invention includes the following steps: Step S10: Based on the 3D CAD model of the mold, generate a template with multiple angles and scales through rotation and scaling transformations.

[0026] In this embodiment, a multi-angle, multi-size, and different-angle view template is pre-constructed. After the image is pre-processed, the multi-angle, multi-size template is input, and image matching is performed through a pyramid algorithm to facilitate subsequent judgment of whether the mold has defects. In addition, by performing pyramid coarse matching and asynchronous preprocessing in parallel, the detection latency time is greatly reduced compared with the existing technology, with a detection latency of ≤200ms.

[0027] Step S20: Acquire template images or mold images, and use a lightweight deep learning object detection network, combined with a coarse localization-guided hierarchical detection mechanism, to coarsely localize the acquired template images or mold images; when the confidence level is greater than or equal to the first preset threshold, execute the pyramid matching process, output the coordinates of the region of interest (ROI), and use a pyramid-style multi-scale template matching algorithm within the ROI, using normalized cross-correlation for matching, and improving the localization accuracy through sub-pixel interpolation.

[0028] In this embodiment, an industrial camera can be used to capture template images or mold images. It should be noted that the template in this embodiment is used to manufacture molds; therefore, by inspecting the template image, it is also possible to detect whether there are defects in the processed mold.

[0029] The lightweight deep learning object detection network is a YOLOv8 model with ShuffleNetV2 as the backbone network.

[0030] In this embodiment, the lightweight deep learning object detection network uses ShuffleNetV2 to replace the original Darknet backbone. It reduces the number of parameters by 70% through group convolution and channel shuffle, while retaining the ability to extract features at multiple scales.

[0031] In this embodiment, during the rapid coarse localization process based on the hierarchical detection mechanism guided by coarse localization, the YOLOv8 model detects the bounding box of the mold with low resolution input. When the confidence level is >0.7, it outputs the ROI coordinates and performs multi-scale pyramid template matching within the ROI area of ​​the original image. The search range is reduced to 15%-20% of the entire image, thereby achieving rapid coarse localization.

[0032] In this embodiment, an adaptive decision logic control system with dynamic switching and failure protection mechanisms is used to control the pyramid matching process.

[0033] Specifically, step S20 further includes: When the confidence level of consecutive first preset frame images is greater than or equal to the first preset threshold, execute the high confidence mode, lock the ROI mode, close the pyramid matching process, and directly execute step S30.

[0034] Preferably, in this embodiment, when the confidence level of YOLOv8 is ≥0.7 for 5 consecutive frames, the ROI mode is locked and full-image pyramid matching is turned off.

[0035] When the confidence level is less than the first preset threshold and greater than or equal to the second preset threshold, the low confidence mode is executed and the process is switched to the pyramid matching process.

[0036] Preferably, in this embodiment, when the confidence level is <0.7 (e.g., strong reflection causes feature blurring), the process automatically switches to the full-image pyramid matching process.

[0037] When the confidence level is less than the second preset threshold, the abnormal circuit breaker mode is executed, triggering an alarm in the programmable logic controller and prompting manual re-inspection.

[0038] Preferably, in this embodiment, a PLC alarm is triggered when the confidence level is <0.5 for 10 consecutive frames, prompting manual re-inspection.

[0039] Step S30: Preprocess and enhance the located image region.

[0040] Specifically, this embodiment performs Gaussian filtering, bilateral filtering to preserve edges, adaptive thresholding and illumination compensation, reflection suppression, and CLAHE local contrast enhancement on the located image.

[0041] Step S40: Perform post-processing and false detection filtering on the pre-processed and enhanced image to locate the defects and foreign object attachment areas of the template or mold.

[0042] Specifically, in this embodiment, morphological processing and Canny edge extraction are performed on the image, and false detection filtering is performed based on geometric features to remove abnormal areas. Through connected component analysis, the defects and foreign object attachment areas of the template or mold are located.

[0043] Step S50: Link the detection results with the programmable logic controller to achieve semi-automatic cyclic monitoring and alarm.

[0044] The system listens for the start signal of the processing equipment via the PLC interface, triggers a detection cycle, executes semi-automatic linkage and cyclic monitoring, and triggers an alarm mechanism based on continuous low-confidence results. Details are as follows: A) Closed-loop control interface: This invention monitors the equipment start signal through an industrial PLC interface and uses event-driven triggering detection. .

[0045] B) Data Synchronization and Storage The test results are written to local and cloud databases in real time, supporting big data analysis and traceability.

[0046] C) Anomaly Alarm Mechanism: continuous The second-lowest confidence level triggers manual review; alarm logic: .

[0047] It is the first The value of an event or condition may be a measurement result, a threshold detection value, an anomaly detection value, etc. This is the threshold for triggering an alarm, indicating that when an event or condition value is less than this threshold, it is considered an abnormal situation or a situation that requires attention. It is an indicator function used to determine conditions. Does it satisfy less than In that case. If The value is 1 if it is true, otherwise it is 0. This represents the total number of events or conditions, used to cumulatively check all events. It is the cumulative threshold for alarm triggering, indicating when the condition is met. The number of events reached or exceeded The system will trigger an alarm at this time. It is an indicator function, which only applies when the condition is met. An event is considered an abnormal event only when it is satisfied. The values ​​of each abnormal event are summed up until a predetermined threshold is reached. .

[0048] This alarm mechanism is typically used to detect multiple anomalies occurring over a period of time (such as multiple data points exceeding a threshold). For example, in scenarios such as image analysis, sensor data monitoring, and fault detection, this method can be used to discover potential problems in advance.

[0049] D) Multi-mold switching scheduling In the case of multiple molds on the production line, this invention is based on task queue scheduling template loading, which can realize automated and rapid mold changeover and maximize production efficiency.

[0050] Furthermore, in this embodiment, the pyramid matching process in step S20 includes the following steps: It is the original image. The original image As the first layer of the pyramid, its resolution is... ; This is the second-layer pyramid image, obtained by downsampling the first-layer image. The resolution is half that of the original image, with a resolution of [missing information]. ; This is the third layer of the pyramid image. We then downsample the second layer image to obtain the third layer image. The resolution is one-quarter of the original image, and the resolution is... Downsampling refers to the image downsampling process, which involves averaging or interpolating each pixel region to obtain a lower-resolution image. Downsampling generates layers of an image pyramid by reducing the image's resolution. Common downsampling methods include downsampling through mean pooling or other image processing techniques.

[0051] Pyramid image construction is used to represent the input image of the object to be detected at multiple scales, so that feature extraction and region analysis can be performed at different resolutions.

[0052] For each entity to be detected, first obtain the corresponding input image to be detected, denoted as the original image. The original image This represents the initial input image when performing the detection process on all images to be detected, which is used for subsequent unified multi-scale processing and analysis.

[0053] Step S202: Calculate the similarity between the template and the image region based on normalized cross-correlation.

[0054] The top-level fast matching employs Normalized Cross-Correlation (NCC), a commonly used image matching metric used to calculate the similarity between an image patch (template) and an image region. NCC calculates their similarity by comparing the differences in grayscale values ​​between the image region and the template region, and is frequently used for feature matching and template matching.

[0055] The calculation formula is: It represents the coordinates of the current pixel in the image, indicating the position of the current local image patch; It is the pyramid level The image above The pixel value in the coordinates is , represents the matching region when the template is aligned with the image block; It's a template. The pixel value in the image is located at the position of the pixel value in the image. ; and These are images respectively and templates Mean: This represents the number of pixels in the image patch; the denominator includes the standard deviation of the template and the image patch, ensuring that the calculation result is not affected by changes in image brightness, so that the final NCC value is within the acceptable range. The similarity is between 1 and 1, with the closer the similarity being to 1, the higher the similarity. Indicates the location The similarity between the image patch and the template is such that a value closer to 1 indicates a better match.

[0056] The normalization properties of NCC make it insensitive to changes in image brightness and contrast, enabling it to find the optimal match under different lighting conditions. By calculating the correlation between local image regions and the template, the degree of matching between the image regions and the template can be determined.

[0057] In image matching, NCC (Normalized Cross-Correlation) is commonly used to measure the matching degree between a template and an image. To improve accuracy during the matching process, especially in locating peaks, this invention utilizes quadratic interpolation (such as parabolic fitting) to locate NCC peaks at the sub-pixel level. This method can improve matching accuracy to two decimal places, achieving sub-pixel precision.

[0058] Specifically, in this embodiment, the step of calculating the similarity between the template and the image region based on normalized cross-correlation includes: At the bottom layer, quadratic interpolation (such as parabolic fitting) is used to perform sub-pixel-level positioning of the NCC peak, improving the positional accuracy to two decimal places. This allows for the identification of the maximum NCC value. Then, quadratic interpolation is used to finely locate the peak and calculate a more precise peak position. : It is the peak position of NCC, which indicates the initially estimated matching position of the image patch with the template; The sub-pixel precise matching position is calculated through quadratic interpolation, improving the accuracy to two decimal places. and They are about and NCC value in direction The first derivative at that point, i.e., the rate of change of the NCC value; and They are about and NCC value in direction The second derivative at that point is the curvature of the NCC value.

[0059] It should be noted that this embodiment can combine OpenMP parallelism and SIMD instruction sets (such as AVX2) to accelerate the template matching process and control the detection latency to below 200ms to meet the real-time requirements of industry.

[0060] Further, in this embodiment, step S10 includes: Step S101, using 3D CAD model parameters The rotation transformation matrix R(θ) is used to generate view templates of the mold at various angles. The generating formula is: Among them, 3D CAD model parameters The coordinate parameters represent the coordinates of the 3D CAD model, describing the geometry of the mold; the rotation transformation matrix R(θ) represents the rotation about a certain axis. This represents the rotation angle of the model in space. This refers to the different viewing angles of the mold, ranging from 0° to 345°, increasing by 15° each time; Render represents the rendering operation, which renders the 3D model. Through rotation transformation matrix Generate 2D view templates from different angles ; Step S102, apply scaling transformation Generate size variants and scale the transformation matrix. Used to scale the rotated view template to generate different size variations of the mold; , is the scaling factor, representing different sizes of the mold view template, for example, Indicates no scaling. Indicates reduction, Indicates magnification: It is a view template generated by rotation. It is a scaling transformation matrix. This is the final template after scaling.

[0061] Step S103: Combining historical matching accuracy, call frequency, and ambient lighting adaptability, update the template weights using a weighted function. Weight This reflects the applicability and priority of each template.

[0062] To dynamically adjust template priority, this embodiment uses a weighted function to update the template weights, taking into account factors such as historical matching accuracy, call frequency, and ambient lighting adaptability. Weights This reflects the applicability and priority of each template, prioritizing the use of templates with higher weights to improve matching efficiency and accuracy.

[0063] Weight The updated formula is: It's a template. The weight indicates the priority of calling the template; It represents the historical matching precision, indicating the template. The accuracy in past matching processes is usually calculated as the template's matching accuracy on historical data; It refers to the call frequency, indicating the template. The frequency of use and the frequency of calls are usually the most commonly used templates, which may be applicable to more scenarios; It is the ambient light adaptability, representing the template. Adaptability to different lighting conditions: Templates with high adaptability to ambient light can maintain good matching performance under different lighting conditions; These are weighting coefficients, which control the impact of historical matching accuracy, call frequency, and ambient lighting adaptability on the final weight. Their sum is 1 to ensure the total weight does not exceed a reasonable range. By adjusting the weighting coefficients... The system can flexibly adjust the priority of templates according to the needs of different scenarios. For example, in low-light conditions, greater emphasis may be placed on adaptability to ambient light. Under normal lighting conditions, greater emphasis is placed on historical matching accuracy. .

[0064] Step S104: Update and optimize the template based on the self-evolution mechanism.

[0065] In the intelligent mold monitoring system, the template library is continuously updated and optimized through a self-evolution mechanism. Low-weight templates are replaced and updated during idle detection periods, utilizing an online incremental learning algorithm to adapt to new environmental changes and mold wear. This mechanism ensures that the template library gradually adapts to new working conditions over time, improving matching accuracy and system reliability. The template replacement and update mechanism is as follows: (1) Low-weight template replacement: When it is detected that some templates have low weights, these templates are considered no longer suitable for the current working environment. Template replacement will be performed during idle periods to replace them with more suitable templates.

[0066] (2) Online incremental learning: Using the incremental learning algorithm, the feature vector of the template will be updated according to the new data. Incremental learning can adjust itself according to the changes in the current environment without completely relying on a large amount of historical data.

[0067] Online incremental learning algorithms are used to optimize template features on each update. This step specifically includes: Based on the current characteristics and old features Weighted update template feature vector The updated formula is: It is the updated template feature vector, representing the new feature representation of the template; It is the old feature vector of the template, representing the features of the template before the update; It is the template feature vector obtained based on the new detection data; It refers to the learning rate, which controls the degree to which new and old features are integrated. A larger learning rate... It tends to use features from the current data more, while a smaller learning rate will rely more on older features.

[0068] Learning rate The learning rate controls the degree to which new features influence template updates. A higher learning rate means that features in the current new environment will influence template updates more quickly, while a lower learning rate means the system is more conservative and updates more slowly. Choosing an appropriate learning rate is crucial for adaptability.

[0069] Through incremental learning, the template library can self-adjust based on changes in the environment or mold wear, maintaining sensitivity to new data. This mechanism is particularly suitable for dynamically changing production environments.

[0070] Furthermore, in this embodiment, step S30 specifically includes: Step S301: Perform Gaussian filtering, bilateral filtering to preserve edges, adaptive thresholding and illumination compensation, reflection suppression, and CLAHE local contrast enhancement on the mold image.

[0071] This embodiment uses dual-channel noise reduction filtering for the image, that is, Gaussian filtering is applied to the image and then bilateral filtering is applied to preserve the edges.

[0072] For the acquired images First, Gaussian filtering is performed. Gaussian filtering is a common noise reduction method used to smooth images. Its core idea is to smooth the image using a weighted average method, so that the value of each pixel is weighted and averaged with the values ​​of its neighboring pixels according to a Gaussian distribution, thereby achieving the effect of noise reduction.

[0073] The core idea of ​​bilateral filtering is a weighted average based on spatial distance and pixel value similarity.

[0074] The step of performing Gaussian filtering on the mold image in step S301 specifically includes: Image smoothing is achieved through a weighted averaging method, where the value of each pixel is weighted and averaged with the values ​​of its neighboring pixels according to a Gaussian distribution. The formula used is as follows: in, It is the original image; It is a Gaussian filter kernel, representing a two-dimensional Gaussian distribution with a standard deviation of . This controls the width of the filter; Indicates the convolution operation; It is the standard deviation of the Gaussian kernel, which controls the degree of image blur. A smaller standard deviation results in a smaller standard deviation. The value will retain more details, while the larger one The values ​​will undergo more smoothing processing; These are the coordinates of each pixel in the image; s is the exponential part of the Gaussian distribution, and its weight gradually decreases as the distance from the center point increases; The step of performing bilateral filtering on the mold image to preserve edges in step S301 includes: weighted averaging based on spatial distance and pixel value similarity. The image after bilateral filtering is located at... Pixel value at that location, These are the coordinates of the current pixel; Neighborhood pixels Pixel values; It represents the neighborhood range of the current pixel; It is a similarity weight function based on pixel value differences, used to measure the similarity between the neighboring pixel values ​​and the current pixel value; It is a similarity weight function based on spatial distance, used to measure the spatial distance between neighboring pixels and the current pixel; It is the normalization coefficient, used to ensure that the sum of the weighted values ​​is 1.

[0075] Pixel value difference weights in bilateral filtering The calculation formula is: in, It controls the impact of pixel value differences on weighting, and has a relatively large impact. This value implies tolerance for larger differences in pixel values; Spatial distance weights of bilateral filtering The calculation formula is: in, It controls the impact of spatial distance on weighting, which is relatively large. The value means that pixels that are farther away from the neighborhood will also have a larger weight.

[0076] Compared to existing linear filtering methods, bilateral filtering preserves edge information better because it considers both pixel values ​​and spatial distance. Furthermore, bilateral filtering effectively removes noise, especially in smooth areas of the image, while maintaining sharpness at edges.

[0077] In image processing, adaptive thresholding is used to dynamically adjust the threshold based on local statistical characteristics of the image (such as local mean and standard deviation) to better handle image data under different lighting conditions. In this way, the system can eliminate the influence of background illumination, enhance the contrast between the target and the background, and thus improve the accuracy of image processing.

[0078] In step S301, the step of adaptive thresholding and illumination compensation for the mold image includes: dynamically adjusting the threshold according to the local statistical characteristics of the image to process image data under different illumination conditions, eliminate the influence of background illumination, enhance the contrast between the target and the background, and improve the accuracy of image processing. Specifically, this includes: Given the location of each pixel in the image Through local statistics and To calculate the dynamic threshold of this pixel : In position Adaptive threshold at the location; It is a location The local mean of a nearby region is typically calculated as the average value of pixels within that neighborhood, reflecting the average brightness of the image in that area; local mean ,in Therefore The surrounding area centered on, It is the value of each pixel in the neighborhood. It is the number of pixels in the neighborhood; It is a location The local standard deviation of a nearby region represents the dispersion of pixel values ​​within that neighborhood; a larger standard deviation indicates greater brightness variation in that region. ; These are empirical parameters that control the influence of local standard deviations on the threshold; larger ones... This value makes the standard deviation more sensitive to adjustments in the threshold; It is a constant parameter, usually taken as a value related to the image brightness range, used to normalize the standard deviation, eliminate the influence of background illumination changes, and make the threshold more stable; This represents a gain based on the local standard deviation, which enhances the contrast of prominent target areas in an image and reduces background noise interference.

[0079] This embodiment eliminates background interference and enhances target contrast through adaptive thresholding. Specifically, this embodiment dynamically calculates the local threshold for each pixel, adapting to different lighting conditions and avoiding background interference in global thresholding. When the standard deviation of a certain region is large, the threshold increases, thereby enhancing the contrast between that region and the background, which helps extract edge and target information. This method is particularly effective in scenes with large lighting variations because it can adaptively adjust the threshold according to the characteristics of local regions, adapting to different environmental conditions.

[0080] In this embodiment, step S301, the step of suppressing reflection in the mold image, includes: A spectral filtering model is used to analyze reflective areas, and high-reflectivity artifacts are eliminated based on saturation and brightness thresholds to enhance the realistic representation of surface details of metal molds.

[0081] CLAHE (Contrast Limited Adaptive Histogram Equalization) is a technique commonly used in image processing, particularly suitable for improving low-contrast images. It improves the local contrast of an image by performing histogram equalization on local regions while limiting contrast gain to avoid over-enhancement.

[0082] The basic principle of CLAHE is that it first divides the image into multiple small patches (often called "grids"), and then performs histogram equalization on each patch. To avoid over-enhancement, CLAHE limits the gain of the histogram. In this way, CLAHE can improve local details of the image while maintaining global contrast.

[0083] In this embodiment, step S301, which involves performing CLAHE local contrast enhancement on the mold image, includes: First, the image is divided into multiple small blocks. Then, histogram equalization is performed on each block, and the gain of the histogram is limited by CLAHE.

[0084] Histogram equalization is calculated by dividing the local region into blocks to limit contrast gain and avoid excessive enhancement. The formula is as follows: in, It is the original input image; It is the th in the image histogram The count value of each gray level indicates the number of gray levels in the image. The number of pixels; This is the Contrast Limit, which limits the maximum contrast gain of each gray level during local histogram equalization to avoid excessive local contrast enhancement. An empirical value is usually chosen to adjust this parameter to control the intensity of image enhancement. It refers to the number of gray levels; typically, an image has 256 gray levels (i.e., ...). (), representing the range of pixel values ​​in the image from 0 to 255; It is the gray level, representing the gray value of each pixel in the image; This involves limiting the count value for each gray level to ensure that histogram equalization does not lead to over-enhancement.

[0085] It should be noted that image parameters can be dynamically adjusted, specifically by using PLC feedback of environmental parameters. Adjust the filter radius, threshold parameters, etc., to adapt to the dynamic industrial production environment.

[0086] Furthermore, in this embodiment, the step of performing morphological processing on the image in step S40 includes: Detecting images from binarized images Perform closing operations to merge neighboring regions and fill small holes.

[0087] Morphological processing is used for shape analysis in binarized images, often for removing small noise points and filling small holes. Closing is a combination of dilation and erosion operations, primarily used to merge neighboring regions and fill small holes. For binarized detection images... Perform the closing operation (corrosion after expansion). The formula for the closing operation is: It is a structural element that eliminates small holes and merges adjacent areas; It is the original binary image; It is a structural element used to control the shape and size of expansion and corrosion. Shapes used in expansion and corrosion operations; It is an expansion operation; It is a corrosion operation; The step S40, which involves Canny edge extraction of the defect region, includes: extracting edge contours in the image by calculating the image gradient to aid in geometric feature analysis; the gradient calculation uses the Sobel operator. Horizontal gradient ; in, It is the original image. and It is the convolution kernel of the Sobel operator in the horizontal and vertical directions.

[0088] The step S40 of filtering false detections of defective regions based on geometric features includes: analyzing the shape features of candidate regions and determining whether the target region meets the requirements according to the set rules. The shape features include area, aspect ratio, and convex hull defect rate, etc.

[0089] Geometric feature consistency discrimination is used to analyze the shape features of the entity under test within a predetermined detection area and compare them with the features of a pre-established good product template to determine whether the area meets the geometric constraints of a good product. During system initialization, several standard good product samples are selected, and their geometric features are extracted within the same detection area to construct a reference geometric feature template or feature statistical range.

[0090] The formula used to calculate the aspect ratio of the candidate region is: in, The aspect ratio of the candidate region represents the ratio of the region's width to its height. It is used to determine the shape characteristics of the region and is typically used to distinguish between long strips and square or circular regions. It is the width of the candidate region. It is the height of the candidate region.

[0091] Calculate the convex hull defect rate The formula used is: The convex hull defect rate reflects the difference between the boundary of a region and its convex hull. It is typically used to detect irregularly shaped regions. In the calculation formula, It is the convex hull area of ​​the candidate region. It is the actual area of ​​the candidate region.

[0092] Perimeter-area ratio The ratio of the perimeter to the area of ​​the candidate region is used to determine the shape of the candidate region and measure the "smoothness" of the region. A higher ratio may mean that the boundary is irregular.

[0093] In this embodiment, the aspect ratio is used to distinguish between slender and approximately regular shaped regions; the convex hull defect rate reflects the degree of deviation between the candidate region boundary and its convex hull, and the larger the value, the more irregular the region shape; the perimeter-area ratio is used to evaluate the complexity of the region boundary, and a higher ratio usually corresponds to regions with unsmooth boundaries or complex shapes.

[0094] The elimination rule in step S40 is: if If the candidate region is not found to be an abnormal region, it should be removed.

[0095] To achieve geometric consistency screening, the system determines the threshold based on the statistical distribution of the convex hull defect rate in the good product samples. During the detection process, when the convex hull defect rate of the candidate region satisfies: When the geometry of the candidate region does not meet the geometric constraints of the good product template, it is determined that the region is filtered out in subsequent detection processes and will not participate in further feature analysis or consistency judgment; when If the candidate region is deemed to meet the geometric characteristics requirements for good products, it will proceed to the next testing process.

[0096] Furthermore, in this embodiment, through connectivity component analysis, the defects and foreign object attachment areas of the mold can be located, and a threshold can be set. To determine the degree of anomaly in the region.

[0097] In step S40, the formula used to locate the defective and foreign object attachment areas of the template or mold through connectivity component analysis is as follows: in, It is the first A connected component represents a candidate region in the image; Indicates candidate region The contour difference is usually measured by calculating the difference between the boundary of the region and the template or reference contour, which can also be understood as: candidate region The result of measuring the difference between the actual contour and the good template contour; It is used to determine whether the contour difference exceeds the set abnormality threshold.

[0098] After completing the connectivity component analysis, the system analyzes each device. The contour features are calculated and their differences are measured against a pre-established good product template contour. This embodiment can locate defects and foreign object attachment areas in the mold. By setting a threshold... Determine the degree of abnormality in the area. Locate mold defects and foreign object attachments, and set threshold values. Determine the degree of abnormality: .

[0099] When the contour difference satisfies: When, then determine the candidate region. If the condition does not meet the constraints of the good product template, it is marked as an NG region; otherwise, the region is determined to be 0. K is the allowable deviation threshold determined based on statistics of good product samples.

[0100] In this embodiment, after locating the defective area of ​​the mold, the post-processing results can be superimposed onto the original image for visualization output, which facilitates manual review and improves the efficiency of quality traceability.

[0101] The following describes the performance metrics and system architecture of the mold state visual monitoring method based on multi-scale template matching and deep learning collaboration of this invention: A) Detection delay: Detection latency refers to the time between image acquisition and result output. To ensure the real-time performance of the system, detection latency needs to be controlled within a certain range. Total detection latency It includes three parts: Image preprocessing time typically includes operations such as denoising, filtering, and enhancement.

[0102] Image matching time refers to the time required to match the image with the template.

[0103] Post-processing time includes operations such as result evaluation, screening, and output.

[0104] B) Positioning accuracy: Spatial positioning error: Positioning accuracy refers to the error between the estimated target location and the actual location after image matching. In practical applications, high positioning accuracy is required to provide accurate target location.

[0105] In the spatial positioning error formula, It is spatial positioning error, representing the error between the estimated position and the actual position, and the unit is pixels; It is the estimated target location; It is the actual target location.

[0106] C) Robustness indicators: Under conditions of ±20% illumination variation and ±5% dimensional deformation, the matching success rate is no less than 95%.

[0107] D) System module architecture: The beneficial effects of the mold state visual monitoring method based on multi-scale template matching and deep learning collaboration in this invention are: 1. High robustness: Multi-angle, multi-scale templates and CLAHE reinforcement ensure stable matching under different lighting and vibration conditions; 2. Collaborative Detection Architecture: This architecture combines a lightweight object detection network with traditional template matching, using the former's output to dynamically constrain the latter's search space. 3. Intelligent computation allocation: Lightweight YOLOv8 excludes non-critical image regions from computation, reducing template matching time to 1 / 4 of the original method; 4. Failure-safe design: The dynamic switching mechanism seamlessly reverts to traditional methods when the deep learning model fails, ensuring continuous system operation; 5. Real-time performance: Pyramid coarse matching and asynchronous preprocessing are executed in parallel, with a typical detection latency of ≤200ms; 6. Lightweight: Based on lightweight libraries such as OpenCV, it can run on embedded PCs without a GPU; 7. Scalability: The template set can be dynamically updated, supporting quick switching between multiple mold types; 8. Intelligent linkage: It interfaces with the equipment PLC to support semi-automatic production cycles and automatic alarm for failures.

[0108] To achieve the above objectives, this invention also proposes a mold state visual monitoring system based on multi-scale template matching and deep learning collaboration, such as... Figure 3 As shown, the system includes a processor 1001, a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002, and a mold state visual monitoring program based on multi-scale template matching and deep learning collaboration stored on the processor. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0109] Those skilled in the art will understand that Figure 3The system structure shown does not constitute a limitation on the system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0110] like Figure 3 As shown, the memory 1005, which serves as a computer storage medium, may include an operating device, a network communication module, a user interface module, and a mold monitoring program based on deep learning and multi-scale template matching.

[0111] exist Figure 3 In the system shown, the network interface 1004 is mainly used to connect to the network server and communicate with the network server; the user interface 1003 is mainly used to interact with the user terminal and receive user input instructions; and the processor 1001 can be used to call the mold state visual monitoring program based on multi-scale template matching and deep learning collaboration stored in the memory 1005.

[0112] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a mold state visual monitoring program based on multi-scale template matching and deep learning collaboration. When the processor runs the mold state visual monitoring program based on multi-scale template matching and deep learning collaboration, it executes the steps of the method described above, which will not be repeated here.

[0113] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural changes made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration, characterized in that, The method includes the following steps: Step S10: Based on the 3D CAD model of the mold, generate a template with multiple angles and scales through rotation and scaling transformations; Step S20: Acquire template images or mold images, and use a lightweight deep learning object detection network, combined with a coarse localization-guided hierarchical detection mechanism, to coarsely localize the acquired template images or mold images; when the confidence level is greater than or equal to the first preset threshold, execute the pyramid matching process, output the coordinates of the region of interest (ROI), and use a pyramid-style multi-scale template matching algorithm within the ROI, using normalized cross-correlation for matching, and improving localization accuracy through sub-pixel interpolation; Step S30: Preprocess and enhance the located image region; Step S40: Perform post-processing and false detection filtering on the pre-processed and enhanced images to locate the defects and foreign object attachment areas of the template or mold; Step S50: Link the detection results with the programmable logic controller to achieve semi-automatic cyclic monitoring and alarm.

2. The method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration as described in claim 1, characterized in that, The lightweight deep learning object detection network is a YOLOv8 model with ShuffleNetV2 as the backbone network.

3. The method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration as described in claim 2, characterized in that, Step S20 further includes: When the confidence level of consecutive first preset frame images is greater than or equal to the first preset threshold, execute the high confidence mode, lock the ROI mode, close the pyramid matching process, and directly execute step S30. When the confidence level is less than the first preset threshold and greater than or equal to the second preset threshold, the low confidence mode is executed and the process is switched to the pyramid matching process. When the confidence level is less than the second preset threshold, the programmable logic controller is triggered to alarm and prompt manual re-inspection.

4. The method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration as described in claim 2, characterized in that, The pyramid matching process in step S20 includes the following steps: Step S201, the original image A three-layer pyramid is generated by downsampling, with the resolution of each layer decreasing sequentially. It is the original image. This is an image of the second layer of the pyramid. It is the third layer of the pyramid image. Downsample means that the image is downsampled, which is to calculate the mean or interpolation of each pixel region to obtain a low-resolution image. Step S202: Calculate the similarity between the template and the image region based on normalized cross-correlation. The calculation formula is as follows: It represents the coordinates of the current pixel in the image, indicating the position of the current local image patch; It is the pyramid level The image above The pixel value in the image has coordinates of... , represents the matching region when the template is aligned with the image block; It's a template. The pixel value in the image is located at the position of the pixel value in the image. ; and These are images respectively and templates Mean: This represents the number of pixels in the image patch; the denominator includes the standard deviation of the template and the image patch, ensuring that the calculation result is not affected by changes in image brightness, so that the final NCC value is within the acceptable range. between; Indicates the location Similarity between the image patch and the template; The steps for calculating the similarity between the template and the image region based on normalized cross-correlation include: By using quadratic interpolation to perform sub-pixel-level positioning of the NCC peak, the positioning accuracy is improved to two decimal places, thus finding the position of the maximum NCC value. Then, quadratic interpolation is used to finely locate the peak and calculate a more precise peak position. : It is the peak position of NCC, indicating the initially estimated matching position of the image patch with the template; The sub-pixel precise matching position is calculated through quadratic interpolation, improving the accuracy to two decimal places. and They are about and NCC value in direction The first derivative at that point, i.e., the rate of change of the NCC value; and They are about and NCC value in direction The second derivative at that point is the curvature of the NCC value.

5. The method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration as described in claim 4, characterized in that, Step S10 includes: Step S101, using 3D CAD model parameters The rotation transformation matrix R(θ) is used to generate view templates of the mold at various angles. ; Among them, 3D CAD model parameters The coordinate parameters represent the coordinates of the 3D CAD model, describing the geometry of the mold; the rotation transformation matrix R(θ) represents the rotation about a certain axis. This represents the rotation angle of the model in space. This refers to the different viewing angles of the mold, ranging from 0° to 345°, increasing by 15° each time; Render represents the rendering operation, which renders the 3D model. Through rotation transformation matrix Generate 2D view templates from different angles ; Step S102, apply scaling transformation Generate size variants and scale the transformation matrix. Used to scale the rotated view template to generate different size variations of the mold; It is a scaling factor, representing different sizes of the mold view template: It is a view template generated by rotation. It is a scaling transformation matrix. This is the final template after scaling. Step S103: Combining historical matching accuracy, call frequency, and ambient lighting adaptability, update the template weights using a weighted function. Weight This reflects the applicability and priority of each template; the weight update formula is as follows: It's a template. The weight indicates the priority of calling the template; It represents the historical matching precision, indicating the template. Accuracy in past matching processes; It refers to the call frequency, indicating the template. Frequency of use; It is the ambient light adaptability, representing the template. Adaptability to different lighting conditions; These are weighting coefficients, which respectively control the impact of historical matching accuracy, call frequency, and ambient lighting adaptability on the final weight. Step S104: Update and optimize the template based on the self-evolution mechanism, according to the current features. and old features Weighted update template feature vector The updated formula is: It is the updated template feature vector, representing the new feature representation of the template; It is the old feature vector of the template, representing the features of the template before the update; It is the template feature vector obtained based on the new detection data; It is the learning rate, which controls the degree of fusion between new and old features.

6. The method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration as described in claim 5, characterized in that, Step S30 includes: Step S301: Perform Gaussian filtering, bilateral filtering to preserve edges, adaptive thresholding and illumination compensation, reflection suppression, and CLAHE local contrast enhancement on the mold image. The step of performing Gaussian filtering on the mold image in step S301 includes: Image smoothing is achieved through a weighted averaging method, where the value of each pixel is weighted and averaged with the values ​​of its neighboring pixels according to a Gaussian distribution. The formula used is as follows: in, It is the original image; It is a Gaussian filter kernel, representing a two-dimensional Gaussian distribution with a standard deviation of . This controls the width of the filter; Indicates the convolution operation; in: It is the standard deviation of the Gaussian kernel, which controls the degree of image blur. These are the coordinates of each pixel in the image; s is the exponential part of the Gaussian distribution, and its weight gradually decreases as the distance from the center point increases; The step of performing bilateral filtering on the mold image to preserve edges in step S301 includes: weighted averaging based on spatial distance and pixel value similarity. The image after bilateral filtering is located at... Pixel value at that location, These are the coordinates of the current pixel; Neighborhood pixels Pixel values; It represents the neighborhood range of the current pixel; It is a similarity weight function based on pixel value differences, used to measure the similarity between the neighboring pixel values ​​and the current pixel value; It is a similarity weight function based on spatial distance, used to measure the spatial distance between neighboring pixels and the current pixel; It is the normalization coefficient, used to ensure that the sum of the weighted values ​​is 1; Pixel value difference weights in bilateral filtering The calculation formula is: in, It controls the impact of pixel value differences on the weighting. Spatial distance weights of bilateral filtering The calculation formula is: in, It controls the impact of spatial distance on weighting; In step S301, the step of adaptive thresholding and illumination compensation for the mold image includes: dynamically adjusting the threshold according to the local statistical characteristics of the image to process image data under different illumination conditions, eliminate the influence of background illumination, enhance the contrast between the target and the background, and improve the accuracy of image processing. Specifically, this includes: Given the location of each pixel in the image Through local statistics and To calculate the dynamic threshold of this pixel : In position Adaptive threshold at the location; It is a location The local mean of the surrounding area reflects the average brightness of the image in that region; local mean ,in Therefore The surrounding area centered on, It is the value of each pixel in the neighborhood. It is the number of pixels in the neighborhood; It is a location The local standard deviation of a neighborhood represents the dispersion of pixel values ​​within that neighborhood; local standard deviation ; These are empirical parameters that control the influence of local standard deviations on the threshold. It is a constant parameter used to normalize the standard deviation, eliminate the influence of background illumination changes, and make the threshold more stable. This represents a gain based on local standard deviation, which enhances the contrast of prominent target areas in an image and reduces background noise interference. In step S301, the step of suppressing reflection in the mold image includes: A spectral filtering model is used to analyze reflective areas, and high-reflectivity artifacts are eliminated based on saturation and brightness thresholds to enhance the realistic representation of surface details of metal molds. In step S301, the step of performing CLAHE local contrast enhancement on the mold image includes: First, the image is divided into multiple small blocks, and then histogram equalization is performed on each small block. The gain of the histogram is limited by CLAHE. Histogram equalization is calculated by dividing the local region into blocks to limit contrast gain and avoid excessive enhancement. The formula is as follows: in, It is the original input image; It is the th in the image histogram The count value of each gray level indicates the number of gray levels in the image. The number of pixels; It is the contrast limiting threshold; It refers to grayscale levels; It is the gray level, representing the gray value of each pixel in the image; This involves limiting the count value for each gray level to ensure that histogram equalization does not lead to over-enhancement.

7. The method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration as described in claim 1, characterized in that, Step S40 includes: performing morphological processing and Canny edge extraction on the image, filtering out false positives based on geometric features, removing abnormal areas, and locating defects and foreign object attachment areas of the template or mold through connected component analysis.

8. The method for visual monitoring of mold status based on multi-scale template matching and deep learning collaboration as described in claim 7, characterized in that, The morphological processing step in step S40 includes: Detecting images from binarized images The closing operation merges neighboring regions and fills small holes. The formula for the closing operation is: It is a structural element that eliminates small holes and merges adjacent areas; It is the original binary image; It is a structural element used to control the shape and size of expansion and corrosion. Shapes used in expansion and corrosion operations; It is an expansion operation; It is a corrosion operation; The step S40, which involves Canny edge extraction of the image, includes: extracting edge contours in the image by calculating the image gradient to aid in geometric feature analysis. The gradient calculation uses the Sobel operator. Horizontal gradient ; in, It is the original image. and It is the convolution kernel of the Sobel operator in the horizontal and vertical directions; The step S40, which filters false positives in the image based on geometric features, includes: analyzing the shape features of candidate regions and determining whether the target region meets the requirements according to set rules. The shape features include area, aspect ratio, and convex hull defect rate. The formula used to calculate the aspect ratio of the candidate region is: in, The aspect ratio of the candidate region represents the ratio of the region's width to its height. It is the width of the candidate region. It is the height of the candidate region; Calculate the convex hull defect rate The formula used is: in, It is the convex hull area of ​​the candidate region. This is the actual area of ​​the candidate region; Perimeter-area ratio This is the ratio of the perimeter to the area of ​​the candidate region, used to determine the shape of the candidate region; The elimination rule in step S40 is: if If the candidate region is not found to be an abnormal region, it should be removed. In step S40, the formula used to locate the defective and foreign object attachment areas of the template or mold through connectivity component analysis is as follows: in, It is the first A connected component represents a candidate region in the image; Indicates candidate region Contour differences; It is used to determine whether the contour difference exceeds the set abnormality threshold.

9. A mold monitoring system based on deep learning and multi-scale template matching, characterized in that, The system includes a memory, a processor, and a mold monitoring program based on deep learning and multi-scale template matching stored on the processor, wherein the mold monitoring program based on deep learning and multi-scale template matching is executed by the processor to perform the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a mold monitoring program based on deep learning and multi-scale template matching, which, when run by a processor, performs the steps of the method as described in any one of claims 1 to 8.

Images