Fireplace surface defect detection method and system driven by machine vision

Abstract

The invention provides a fireplace surface defect detection method and system driven by machine vision, and relates to the technical field of defect detection, and the method comprises the steps: collecting a historical missing detection sample set corresponding to a preset defect detection model; establishing a defect sample to be reinforced; performing shadow effect enhancement analysis on the to-be-enhanced defect sample, establishing a light source arrangement parameter of which the enhancement distinction degree is greater than a preset distinction degree, and obtaining a corresponding shadow enhancement distinguishing sample; optimizing the preset defect detection model; and after shadow effect enhancement processing is carried out through an arranged light source, fireplace images are collected and input into the optimized preset defect detection model for defect detection. According to the method and the device, the technical problem that in the prior art, due to the fact that image enhancement is generally carried out in a unified mode but discrimination analysis is lacked, pertinence is insufficient, and then defect detection stability is affected can be solved, and the defect detection accuracy is improved by enhancing analysis and optimizing the defect detection model through the shadow effect. The technical effect of improving the robustness and accuracy of the defect detection model is achieved.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to a machine vision-driven fireplace surface defect detection method and system. Background Art

[0002] With the continuous development of the manufacturing industry and the advancement of technology, the demand for various types of surface defect detection has gradually increased, especially in the surface detection of some precision equipment and high-value items. Early detection of defects is crucial to ensure product quality and extend service life. In this process, traditional manual detection methods are often inefficient and greatly affected by the operator's experience and subjective factors, making it difficult to ensure the accuracy and consistency of detection.

[0003] In order to improve detection efficiency and accuracy, more and more companies are beginning to use a combination of machine vision and intelligent algorithms to achieve automated surface defect detection. Machine vision technology uses high-resolution cameras to capture surface images of objects and identifies surface defects through image processing and analysis technology. This method can monitor in real time during the production process and quickly locate the defect position, greatly improving the detection speed and reducing labor costs. However, existing machine vision systems are still prone to missed detection when facing some small surface defects. The commonly used method in existing technologies is to uniformly enhance the image, but the lack of differentiation analysis leads to insufficient targeting.

[0004] In summary, the prior art has a technical problem that image enhancement is usually performed uniformly but lacks analysis of the degree of distinction, resulting in insufficient targeting, which in turn affects the stability of defect detection. Summary of the invention

[0005] The purpose of this application is to provide a machine vision-driven fireplace surface defect detection method and system to solve the technical problem in the prior art that image enhancement is usually performed uniformly but lacks analysis of the degree of distinction, resulting in insufficient targeting, which in turn affects the stability of defect detection.

[0006] In view of the above problems, the present application provides a machine vision-driven fireplace surface defect detection method and system.

[0007] In a first aspect, the present application provides a machine vision-driven fireplace surface defect detection method, which is implemented by a machine vision-driven fireplace surface defect detection system, wherein the machine vision-driven fireplace surface defect detection method includes: determining a preset defect type of the fireplace to be detected, and collecting a historical missed detection sample set corresponding to a preset defect detection model with the preset defect type as a constraint; performing sample construction based on the historical missed detection sample set to establish defect samples to be enhanced; performing shadow effect enhancement analysis on the defect samples to be enhanced, establishing light source arrangement parameters with an enhanced distinction greater than a preset distinction, and obtaining corresponding shadow enhancement distinction samples; optimizing the preset defect detection model with the shadow enhancement distinction samples; performing light source setting based on the light source arrangement parameters, and when performing defect detection on the fireplace to be detected, collecting a fireplace image after performing shadow effect enhancement processing with the set light source and inputting the optimized preset defect detection model for defect detection.

[0008] In a second aspect, the present application further provides a machine vision-driven fireplace surface defect detection system, which is used to execute the machine vision-driven fireplace surface defect detection method as described in the first aspect, wherein the machine vision-driven fireplace surface defect detection system includes: a missed detection sample acquisition module, which is used to determine the preset defect type of the fireplace to be detected, and collect a historical missed detection sample set corresponding to a preset defect detection model with the preset defect type as a constraint; a sample construction module, which is used to perform sample construction based on the historical missed detection sample set, and establish defect samples to be enhanced; a shadow effect enhancement analysis module, which is used to perform shadow effect enhancement analysis on the defect samples to be enhanced, establish light source arrangement parameters with an enhanced distinction greater than the preset distinction, and obtain corresponding shadow enhancement distinction samples; a model optimization module, which is used to optimize the preset defect detection model with the shadow enhancement distinction samples; a defect detection module, which is used to set the light source based on the light source arrangement parameters, and when performing defect detection on the fireplace to be detected, collect the fireplace image after performing shadow effect enhancement processing through the set light source and input it into the optimized preset defect detection model for defect detection.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages: Determine the preset defect type of the fireplace to be detected, and collect the historical missed sample set corresponding to the preset defect detection model with the preset defect type as a constraint; construct samples based on the historical missed sample set to establish defect samples to be enhanced; perform shadow effect enhancement analysis on the defect samples to be enhanced, establish light source arrangement parameters with enhanced distinction greater than the preset distinction, and obtain corresponding shadow enhancement distinction samples; optimize the preset defect detection model with the shadow enhancement distinction samples; set the light source based on the light source arrangement parameters, and when performing defect detection on the fireplace to be detected, perform shadow effect enhancement processing on the set light source, and then collect the fireplace image and input it into the optimized preset defect detection model for defect detection. Through shadow effect enhancement analysis, optimize the light source arrangement, and then optimize the defect detection model, so as to achieve the technical effect of improving the robustness and accuracy of the defect detection model.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0012] Figure 1 This is a flow chart of the machine vision driven fireplace surface defect detection method of this application.

[0013] Figure 2 This is a schematic diagram of the structure of the machine vision-driven fireplace surface defect detection system of this application.

[0014] Explanation of the reference numerals: missed detection sample acquisition module 11, sample construction module 12, shadow effect enhancement analysis module 13, model optimization module 14, defect detection module 15. DETAILED DESCRIPTION

[0015] This application provides a machine vision-driven fireplace surface defect detection method and system, which solves the technical problem that the conventional technology generally performs image enhancement uniformly but lacks analysis of the degree of differentiation, resulting in insufficient targeting and thus affecting the stability of defect detection. Through shadow effect enhancement analysis, the light source arrangement is optimized, and then the defect detection model is optimized, thereby achieving the technical effect of improving the robustness and accuracy of the defect detection model.

[0016] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0017] For example, please refer to the attached Figure 1 The present application provides a machine vision driven fireplace surface defect detection method, wherein the machine vision driven fireplace surface defect detection method is applied to a machine vision driven fireplace surface defect detection system, and the machine vision driven fireplace surface defect detection method specifically includes the following steps: Step 1: Determine the preset defect type of the fireplace to be detected, and collect a historical missed detection sample set corresponding to the preset defect detection model based on the preset defect type as a constraint.

[0018] Specifically, the preset defect types of the fireplace to be inspected may include cracks, bubbles, peeling, uneven surfaces, etc., which are usually defined by industry standards or expert experience. These preset defect types provide the basis for subsequent inspections. After determining the preset defect types, the next key step is to collect the corresponding historical missed sample sets. The historical missed sample sets refer to defect image data that have not been correctly identified or missed in previous inspections. The role of the historical missed sample sets is crucial. They can provide the model with potential blind spots, that is, those defect types that may be ignored by traditional inspection methods. These samples usually contain defect images with high difficulty and complex backgrounds.

[0019] In order to collect historical missed detection sample sets, missed detection samples can be screened out from the known defect image library, or retrospective analysis can be performed through the error output of the actual detection system. For example, in the past, suppose a convolutional neural network model was used for fireplace surface detection, but during the detection process, the model failed to identify certain tiny cracks or shadow areas. These unrecognized image samples constitute the historical missed detection sample set. Through these samples, the limitations of the model in dealing with certain types of defects can be revealed, such as the difficulty in identifying cracks in complex texture backgrounds, or the unclear distinction between shadows and defective areas under weak lighting conditions. By systematically collecting and organizing missed detection samples, data support can be provided for subsequent model improvements and the existing preset defect detection model can be optimized. Among them, the preset defect detection model is the defect detection model currently in use, such as the convolutional neural network model.

[0020] Step 2: construct samples based on the historical missed detection sample set to establish defect samples to be strengthened.

[0021] Specifically, the process of sample construction based on historical missed detection sample sets aims to strengthen the defect recognition ability of the detection model in a targeted manner by analyzing the types of defects missed in previous detections. First, it is necessary to extract a sample set of missed defect images from the historical missed detection sample set, which contains defect images that have not been recognized or misjudged in previous detection processes. These missed detection samples often have a certain degree of complexity. For example, due to factors such as cluttered backgrounds or blurred defect features, the standard defect detection model cannot accurately identify them. Next, the missed defect sample set is used to construct defect samples to be strengthened. The key to this process is to extract those representative and difficult-to-detect defect features through in-depth analysis of missed detection samples, thereby providing a basis for subsequent model optimization, helping to optimize the existing defect detection model, and enabling the model to better identify defects that were previously undetectable in practical applications.

[0022] Step 3: Perform shadow effect enhancement analysis on the defect sample to be enhanced, establish light source arrangement parameters with enhanced distinction greater than a preset distinction, and obtain corresponding shadow enhancement distinction samples.

[0023] Specifically, the shadow effect enhancement analysis is performed on the defect samples to be enhanced, aiming to enhance the distinction between defects and background by simulating or actually adjusting the arrangement and illumination mode of the light source. Generally, lighting conditions have an important influence on the appearance of surface defects. Different light source positions, intensities, and angles may cause changes in the shadow effect, thereby affecting the visibility and clarity of the defects in the image. Therefore, in defect detection, by enhancing the shadow effect of the defect samples to be enhanced, the visibility of the defects can be effectively improved, helping the detection model to more accurately identify these difficult-to-detect defects.

[0024] By adjusting the angle, intensity and illumination range of the light source, we try to enhance the distinction between the defect and the background. Specifically, the shadow effect is enhanced by simulating different light source arrangement parameters, such as adjusting the tilt angle or brightness of the light source to make the shadow area of ​​the defect more prominent and the shadow effect of the background softer or blurred to enhance the recognizability of the defect. In order to quantify the impact of the shadow effect on the distinction, the analysis step usually relies on a preset distinction, that is, the standard of shadow distinction effect preset by the model or engineer during design. If the distinction of the shadow effect generated by adjusting the light source parameters is greater than the preset distinction, it means that the shadow enhancement can effectively improve the contrast between the defect area and the background, helping the model to distinguish these defects. Discrimination is usually evaluated by parameters such as gray value difference, edge sharpness or contrast. For example, in the detection of cracks on the surface of a fireplace, the cracks may be more obvious under a certain light source arrangement under different light sources, and the boundary between the shadow area and the crack itself is clearer, making it easier to be recognized by the detection model.

[0025] Once the light source arrangement parameters that can effectively improve the distinction between defects and background are determined, it is necessary to use these parameters to generate shadow enhancement distinction samples. These samples are new images collected by setting the optimized light source parameters on the original defect samples to be enhanced after the shadow effect is enhanced. These new image samples contain defect areas under strong shadow effects. Their characteristics are that the defects are more prominent, the interference factors of the background are suppressed, and the enhancement of the shadow makes the grayscale difference between the defect and the background greater, thereby increasing the difficulty of defect recognition and detection accuracy.

[0026] Step 4: Optimizing the preset defect detection model by using the shadow to enhance the sample differentiation.

[0027] Specifically, optimizing the preset defect detection model with shadow-enhanced distinguishing samples is to apply samples that have been optimized for light sources and enhanced for shadow effects to the defect detection model to improve the accuracy and robustness of the model. Specifically. First, the generation of shadow-enhanced distinguishing samples provides defect images that contain significantly enhanced shadow effects. These samples increase the prominent effect of shadows on defect areas by comparing the grayscale difference between the background and the defect, making the defects more visible in the image. These enhanced samples will be used to optimize the original defect detection model, especially when the boundary between the defect area and the background is more blurred, so that the detection model can perform better under complex lighting and background conditions. When optimizing the preset defect detection model, the shadow-enhanced distinguishing samples will be used as training data to improve the original model in an incremental learning manner. Incremental learning refers to adding new training samples to the existing model to improve the detection capability of the model, so as to better adapt to the defect images enhanced by shadows.

[0028] Step 5: Setting the light source based on the light source arrangement parameters, when performing defect detection on the fireplace to be detected, collecting the fireplace image after performing shadow effect enhancement processing through the set light source and inputting the optimized preset defect detection model for defect detection.

[0029] Specifically, light source arrangement parameters usually include information such as the position, angle, illumination intensity, and distance of the light source. These settings can produce the best shadow effect by adjusting the light source, thereby maximizing the effect of defect visualization. For example, assuming that through analysis and training, the optimized light source arrangement parameters may recommend setting the light source at a 45-degree angle to the fireplace surface, adjusting the light source intensity to medium, and keeping a distance of 30 cm from the fireplace surface. These settings can effectively enhance the shadow effect of defects such as cracks or surface peeling, making the grayscale difference between the defect and the background more obvious. At this time, the fireplace image will be able to show the defect more clearly with the help of the shadow effect when shooting.

[0030] When defect detection is performed on a fireplace to be inspected, the light source will first be set based on the preset light source arrangement parameters. The adjusted light source conditions will enhance the shadow effect, making the defects more prominent in the image. Then, the image will be taken using the set light source, and the collected image will be input into the optimized defect detection model for detection. During this process, the optimized detection model can identify the defect features under the enhanced shadow effect, accurately distinguish the defect area from the background area, and make corresponding defect judgments.

[0031] For example, in an experiment, assuming there is a thin crack on the surface of the fireplace, the crack is difficult to be identified by the detection model under traditional lighting conditions. However, after the shadow effect is enhanced and the light source arrangement parameters are optimized, a 45-degree light source is set, the shadow of the crack is enhanced, and the background interference is minimized. Using the optimized defect detection model, the crack can be successfully detected in the image after the light source setting and shadow enhancement.

[0032] In summary, by setting the light source arrangement parameters and enhancing the shadow effect, the defect detection model can be significantly optimized and the model's recognition accuracy and robustness for defects can be improved.

[0033] Further, step 2 of this application includes: A missed-detection defect image sample set is extracted according to the historical missed-detection sample set; and the defect samples to be enhanced are generated by using the missed-detection defect image sample set.

[0034] Specifically, the process of extracting the missed defect image sample set based on the historical missed detection sample set is mainly aimed at identifying those images that were not accurately detected or misjudged as non-defective from the past detection data, which serves as the basis for the subsequent optimization of the detection model. These missed defect images usually have some special features, such as low contrast, blur, occlusion or complex background, which make them ignored or misclassified in traditional detection algorithms. In order to accurately extract these missed samples, it is first necessary to carefully analyze the historical missed detection sample set and mark the defect areas that were not identified in the original detection process. This step often relies on the combination of manual annotation and computer-aided detection tools. Through multiple iterations and manual confirmation, it is ensured that the extracted missed defect samples can cover various detection failures. The missed defect image sample set is used to generate defect samples to be strengthened, and subsequent analysis is conducted on how to strengthen these samples to improve detection accuracy.

[0035] Further, step three of this application includes: A historical light source illumination parameter set is obtained, and a shadow effect enhancement analysis is performed on the defect sample to be enhanced based on a shadow enhancement generator to generate a defect enhancement sample set; the degree of distinction between defects and backgrounds in the defect enhancement sample set is analyzed to determine the enhancement distinction, and then the historical light source illumination parameters whose enhancement distinction is greater than the preset distinction are determined to generate the light source arrangement parameters; based on the light source arrangement parameters, corresponding defect enhancement samples are extracted and defect areas and background areas are labeled and grayscale processed to determine the shadow enhancement distinction samples.

[0036] Specifically, first, the historical light source illumination parameter set is obtained in order to understand the impact of different light source configurations on the defect image performance in previous experiments. The historical light source illumination parameter set contains information such as illumination angle, intensity, and distance under different light source conditions. By analyzing these historical data, it is possible to identify which light source configurations effectively enhanced the contrast between defects and backgrounds in previous experiments, which provides a reference for the subsequent enhancement of shadow effects. Usually, the historical light source illumination parameter set will include a variety of different light source settings, such as direct light source, diffuse light source, side light source, etc. Each configuration may produce different effects on different surfaces or defect types.

[0037] Next, based on the historical light source illumination parameter set, the shadow enhancement generator is used to analyze the shadow effect of the defect samples to be enhanced. The shadow enhancement generator is usually a deep learning system based on a generative adversarial network or a similar model, which is specifically used to simulate the impact of lighting changes on defect images. In this analysis process, the generator simulates the change of shadow effects by changing the illumination mode of the light source to generate a defect enhancement sample set. If the defect area in the defect enhancement sample set shows blurred edges or unclear features, this is because the illumination angle or intensity of the light source has not been fully optimized, thus affecting the performance of the defect.

[0038] Next, analyzing the distinction between defects and background in these defect enhancement sample sets is a key optimization step. By performing image processing on the blurred samples and marking the defect area and background area, the grayscale difference, edge clarity or contrast between the defect and the background can be calculated. The enhanced distinction is determined by comparing these features, with the goal of finding a light source configuration that maximizes the contrast between the defect area and the background area. Once the light source arrangement parameters that can produce a greater degree of distinction are determined, these parameters can be used to extract the corresponding defect enhancement samples, and further image annotation and grayscale processing can be performed, that is, precise defect area and background area annotation are performed to obtain shadow enhancement differentiation samples, ensuring that the boundaries and background areas of each defect can be accurately identified. Through grayscale processing, the grayscale difference between the defect area and the background is further emphasized, and the performance of the defect is optimized, making it more prominent and easier to detect.

[0039] Finally, the shadow-enhanced differentiation samples generated through these steps have the following characteristics: the contrast between the defect area and the background is significantly enhanced, and the shadow effect is clearly visible and more prominent in the image. These samples will be used as optimized training data to further train and improve the defect detection model so that it can maintain high accuracy and robustness. For example, 500 defect enhancement samples were used. After the shadow effect analysis, the light source arrangement was optimized and it was found that the differentiation under a certain setting reached 70%. Through this optimization, the generated shadow enhancement differentiation samples performed better than the unoptimized samples in subsequent model training.

[0040] Furthermore, the present application also includes the following steps: The defect enhancement sample set is annotated with defect areas and background areas to generate an annotated defect enhancement sample set; after grayscale processing is performed on the annotated defect enhancement sample set, the grayscale difference between the defect area and the background area is compared to generate the enhancement distinction.

[0041] Specifically, labeling the defect area and background area of ​​the defect enhancement sample set is a key step to provide accurate labels for subsequent model training and discrimination evaluation. In this process, it is first necessary to manually or semi-automatically label the defect sample image after the shadow effect is enhanced, and clearly indicate the specific location of the defect area and the background area in the image. Labeling is usually completed through image segmentation technology, using marking tools such as rectangles, circles, and polygons to separate the defect area in the image from the surrounding background area. For some more complex defects, more detailed pixel-level labeling may be required to ensure that the outline of the defect area is clear and accurate. After the labeling of the defect area and the background area is completed, a labeled defect enhancement sample set is generated. These sample sets provide key data for subsequent analysis, allowing the model to clearly distinguish which areas are defects and which are backgrounds, and then improve the accuracy of defect detection through subsequent processing. Next, grayscale processing is performed on these labeled defect enhancement sample sets in order to enhance the distinguishability between the defect area and the background area. Grayscale processing usually includes converting the color information of the image into a grayscale image and analyzing it based on the brightness difference between the defect area and the background area. Specifically, each pixel value in the image can be converted into a grayscale value (usually a value between 0 and 255), and then the grayscale difference between the defect area and the background area can be calculated. Common grayscale processing methods include local contrast enhancement and histogram equalization, which can make the defect area more prominent while retaining the defect features, while the background area is relatively weak or blurred.

[0042] After grayscale processing of the annotated sample set, the next step is to compare the grayscale difference between the defect area and the background area. The purpose is to quantify the distinction between the defect area and the background area, that is, the degree of grayscale difference between the two. Specifically, the difference between the average grayscale value of the defect area and the average grayscale value of the background area can be calculated first, and then divided by the standard deviation of the grayscale value of the background area to obtain the enhanced distinction. If the enhanced distinction between the defect area and the background area is large, it means that the shadow effect is enhanced better and can more effectively separate the defect from the background. On the contrary, if the enhanced distinction is small, it means that the current light source configuration or enhancement method needs to be further optimized.

[0043] Through this series of grayscale processing and discrimination calculation, a sample set with enhanced discrimination higher than the preset standard can be generated, which will play an important role in subsequent model training. After such processing, the model can not only identify defect areas more accurately, but also maintain high detection accuracy under complex lighting and background conditions.

[0044] Furthermore, the present application also includes the following steps: Taking the fireplace features of the fireplace to be inspected and the preset defect types as constraints, the original image sample set, the light source illumination feature sample set and the shadow image generation sample set are collected; the sample combination in the original image sample set and the light source illumination feature sample set is used as the training input, and the corresponding samples in the shadow image generation sample set are used as the output supervision truth value, and the shadow enhancement generator is constructed based on generative adversarial network training.

[0045] Specifically, in the process of building a shadow enhancement generator, it is first necessary to clarify the fireplace features and preset defect types of the fireplace to be detected, which provides clear constraints for data collection and generator training. Fireplace features include information such as the texture, material, and shape of the fireplace surface, while the preset defect types refer to the types of defects that may exist on the fireplace surface, such as cracks, peeling, and depressions. Through the constraints of these features and defect types, it can be ensured that the collected image samples can cover various defect manifestations that may appear on the fireplace surface, and provide real and effective data for subsequent shadow effect enhancement. Therefore, the original image sample set, the light source illumination feature sample set, and the shadow image generation sample set are collected. The original image sample set refers to the fireplace surface image collected under natural light or standard light source. These images reflect the actual texture and defects of the fireplace surface. The light source illumination feature sample set includes the illumination features under different light source conditions, such as the angle, intensity, and distance of the light source. By simulating a variety of light source configurations, a variety of samples can be generated. These samples are helpful for analyzing the impact of lighting changes on the performance of defects. The shadow image generation sample set is a sample generated by simulating the shadow effect of the fireplace image under different light source conditions. These samples can help train the generator to simulate and enhance the shadow effect.

[0046] The original image sample set and the sample set of the light source illumination feature sample set are used as the training input, and the corresponding samples in the shadow image generation sample set are used as the output supervision truth value. Each sample in the shadow image generation sample set is a shadow-processed image generated under different light source conditions. As the output truth value of the generator, a shadow enhancement generator is constructed based on the generative adversarial network training. The generative adversarial network is a deep learning model. Through adversarial training, the generator is trained to generate images similar to real samples. In this scenario, the generator continuously adjusts the generated shadow enhancement image to generate samples close to the real shadow image. At the same time, the role of the discriminator is to judge whether the generated shadow image is similar to the real shadow image, thereby providing feedback signals to the generator. As the training progresses, the generator gradually learns how to generate realistic shadow enhancement images based on the original image and light source characteristics. During the training process, the generator and the discriminator are continuously optimized, and finally achieve the generation of high-quality shadow effects through generative adversarial methods. These shadow enhancement images can highlight the defect area under different light source conditions, reduce background interference, and enhance the visibility and detectability of defects.

[0047] Experimental example: Assume that in a fireplace crack detection task, 1,000 original images were collected, and corresponding light source illumination feature samples were generated with up to 5 different light source configurations. The sample set was generated through the shadow image, and the crack image and its shadow effect under different light source configurations were simulated. Finally, the generative adversarial network was trained based on these data, and the generator successfully learned to generate corresponding shadow-enhanced images according to different light source conditions. After 1,000 training iterations, the generator was able to generate images that were very close to the real shadow images, which ultimately greatly improved the crack detection accuracy of the model under complex lighting conditions. Through this process, the shadow enhancement generator can not only effectively simulate and enhance the shadow effects of defect images under variable lighting conditions, but also provide clearer and distinguishable defect samples for subsequent detection models.

[0048] Further, step 4 of this application includes: The shadow enhanced distinction samples are used as incremental learning samples; the network structure of the preset defect detection model is expanded to generate a new convolutional layer; the new convolutional layer is trained with the incremental learning samples, and the preset defect detection model is optimized with the new convolutional layer that has converged in training.

[0049] Specifically, incremental learning samples refer to samples generated by optimizing lighting and shadow effects. They enhance the distinction between defect areas and backgrounds in images through enhanced shadow effects. These samples can help the model learn how to effectively identify defect features and distinguish defects from backgrounds under different light source conditions. Therefore, using shadow-enhanced distinction samples as incremental learning samples is to continuously expand the existing training data set during the training process, thereby improving the generalization ability of the model. The introduction of new convolutional layers is to meet the challenges brought by new samples. Traditional convolutional neural networks extract features through multiple convolutional layers. However, with the diversification and complexity of samples, existing convolutional layers may not be able to effectively capture defect features under enhanced shadows. Therefore, the expansion of the network structure can be achieved by adding new convolutional layers, especially to adapt to new features brought by shadow enhancement.

[0050] The new convolutional layer will be an independent module and will be located behind the original convolutional block of the preset defect detection model. This design ensures that the introduction of the new structure will not interfere with the function of the existing convolutional block, but will enhance the model's ability to identify complex defects through independent modules. The new convolutional layer will be trained specifically for incremental learning samples to extract key features under enhanced shadow conditions. The next step is to train the new convolutional layer with incremental learning samples. For example, in shadow-enhanced samples, defects such as cracks or dents often show stronger local contrast or structural features due to the enhanced shadow effect. The new convolutional layer is specially trained to learn how to recognize these features under enhanced shadow conditions. The incremental learning method usually does not affect the learning ability of the original model, but in a lightweight way, it optimizes the new convolutional layer to adapt to the new training data. During the training process, the weights of the original convolutional layer remain unchanged, and the weights of the new convolutional layer are gradually adjusted until the model's feature extraction ability for the incremental samples is optimal.

[0051] After the training converges, the newly added convolutional layer will optimize the preset defect detection model. As the training process deepens, the newly added convolutional layer can not only extract the detailed features under the enhanced shadow, but also locate and classify the defect area more accurately in the entire network. For example, in the crack detection task, the image enhanced by the shadow effect may produce stronger local feature changes due to the interference of the shadow. The traditional convolutional layer may find it difficult to capture these changes. The newly added convolutional layer can make up for this deficiency by learning specific shadow enhancement features, enhancing the model's ability to identify defects, thereby improving the detection accuracy while ensuring the adaptability and robustness of the model.

[0052] In summary, the machine vision-driven fireplace surface defect detection method provided in this application has the following technical effects: Determine the preset defect type of the fireplace to be detected, and collect the historical missed sample set corresponding to the preset defect detection model with the preset defect type as a constraint; construct samples based on the historical missed sample set to establish defect samples to be enhanced; perform shadow effect enhancement analysis on the defect samples to be enhanced, establish light source arrangement parameters with enhanced distinction greater than the preset distinction, and obtain corresponding shadow enhancement distinction samples; optimize the preset defect detection model with the shadow enhancement distinction samples; set the light source based on the light source arrangement parameters, and when performing defect detection on the fireplace to be detected, perform shadow effect enhancement processing on the set light source, and then collect the fireplace image and input it into the optimized preset defect detection model for defect detection. Through shadow effect enhancement analysis, optimize the light source arrangement, and then optimize the defect detection model, thereby improving the robustness and accuracy of the defect detection model.

[0053] Embodiment 2: Based on the same inventive concept as the machine vision driven fireplace surface defect detection method in the aforementioned embodiment, the present application also provides a machine vision driven fireplace surface defect detection system, please refer to the attached Figure 2 , the machine vision driven fireplace surface defect detection system comprises: The missed detection sample acquisition module 11 is used to determine the preset defect type of the fireplace to be detected, and collect a historical missed detection sample set corresponding to a preset defect detection model based on the preset defect type.

[0054] The sample construction module 12 is used to construct samples based on the historical missed detection sample set to establish defect samples to be strengthened.

[0055] The shadow effect enhancement analysis module 13 is used to perform shadow effect enhancement analysis on the defect sample to be enhanced, establish light source arrangement parameters with enhanced distinction greater than a preset distinction, and obtain corresponding shadow enhancement distinction samples.

[0056] The model optimization module 14 is used to optimize the preset defect detection model by using the shadow enhanced distinguishing samples.

[0057] The defect detection module 15 is used to set the light source based on the light source arrangement parameters. When the defect detection is performed on the fireplace to be detected, the image of the fireplace is collected after the shadow effect is enhanced by the set light source and then input into the optimized preset defect detection model for defect detection.

[0058] Furthermore, the missed detection sample acquisition module 12 in the machine vision driven fireplace surface defect detection system is also used for: A missed-detection defect image sample set is extracted according to the historical missed-detection sample set; and the defect samples to be enhanced are generated by using the missed-detection defect image sample set.

[0059] Furthermore, the shadow effect enhancement analysis module 13 in the machine vision driven fireplace surface defect detection system is also used for: A historical light source illumination parameter set is obtained, and a shadow effect enhancement analysis is performed on the defect sample to be enhanced based on a shadow enhancement generator to generate a defect enhancement sample set; the degree of distinction between defects and backgrounds in the defect enhancement sample set is analyzed to determine the enhancement distinction, and then the historical light source illumination parameters whose enhancement distinction is greater than the preset distinction are determined to generate the light source arrangement parameters; based on the light source arrangement parameters, corresponding defect enhancement samples are extracted and defect areas and background areas are labeled and grayscale processed to determine the shadow enhancement distinction samples.

[0060] Furthermore, the shadow effect enhancement analysis module 13 in the machine vision driven fireplace surface defect detection system is also used for: The defect enhancement sample set is annotated with defect areas and background areas to generate an annotated defect enhancement sample set; after grayscale processing is performed on the annotated defect enhancement sample set, the grayscale difference between the defect area and the background area is compared to generate the enhancement distinction.

[0061] Furthermore, the shadow effect enhancement analysis module 13 in the machine vision driven fireplace surface defect detection system is also used for: Taking the fireplace features of the fireplace to be inspected and the preset defect types as constraints, the original image sample set, the light source illumination feature sample set and the shadow image generation sample set are collected; the sample combination in the original image sample set and the light source illumination feature sample set is used as the training input, and the corresponding samples in the shadow image generation sample set are used as the output supervision truth value, and the shadow enhancement generator is constructed based on generative adversarial network training.

[0062] Furthermore, the model optimization module 14 in the machine vision driven fireplace surface defect detection system is also used for: The shadow enhanced distinction samples are used as incremental learning samples; the network structure of the preset defect detection model is expanded to generate a new convolutional layer; the new convolutional layer is trained with the incremental learning samples, and the preset defect detection model is optimized with the new convolutional layer that has converged in training.

[0063] Among them, the newly added convolution layer is an independent module, which is located behind the original convolution block of the preset defect detection model.

[0064] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. Figure 1The machine vision driven fireplace surface defect detection method and specific examples in the first embodiment are also applicable to the machine vision driven fireplace surface defect detection system of the present embodiment. Through the above detailed description of the machine vision driven fireplace surface defect detection method, those skilled in the art can clearly understand the machine vision driven fireplace surface defect detection system of the present embodiment, so for the sake of brevity of the specification, it will not be described in detail here. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0065] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0066] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. A machine vision-driven fireplace surface defect detection method, characterized in that: include: Determine a preset defect type of the fireplace to be detected, and collect a historical missed detection sample set corresponding to a preset defect detection model with the preset defect type as a constraint; Constructing samples based on the historical missed detection sample set to establish defect samples to be strengthened; Performing shadow effect enhancement analysis on the defect sample to be enhanced, establishing light source arrangement parameters with an enhanced distinction greater than a preset distinction, and obtaining corresponding shadow enhancement distinction samples; Optimizing the preset defect detection model by using the shadow to enhance the distinguishing samples; The light source is set based on the light source arrangement parameters. When the defect detection is performed on the fireplace to be detected, the image of the fireplace is collected after the shadow effect is enhanced by the set light source and then input into the optimized preset defect detection model for defect detection.

2. The machine vision driven fireplace surface defect detection method according to claim 1, characterized in that: Based on the historical missed sample set, sample construction is performed to establish defect samples to be strengthened, including: Extracting a missed-detection defect image sample set according to the historical missed-detection sample set; The defect samples to be enhanced are generated by using the missed-detection defect image sample set.

3. The machine vision driven fireplace surface defect detection method according to claim 1, characterized in that: Performing shadow effect enhancement analysis on the defect sample to be enhanced, establishing light source arrangement parameters with enhanced distinction greater than a preset distinction, and obtaining corresponding shadow enhancement distinction samples, including: Acquire a historical light source illumination parameter set, perform shadow effect enhancement analysis on the defect sample to be enhanced based on a shadow enhancement generator, and generate a defect enhancement sample set; Analyze the degree of distinction between defects and background in the defect enhancement sample set, determine the enhanced distinction, and then determine the historical light source illumination parameter with the enhanced distinction greater than the preset distinction, and generate the light source arrangement parameter; Based on the light source arrangement parameters, corresponding defect enhancement samples are extracted, and defect areas and background areas are labeled and grayscale processed to determine the shadow enhancement distinction samples.

4. The machine vision driven fireplace surface defect detection method according to claim 3, characterized in that: Analyzing the distinction between the defect and the background in the defect enhancement sample set to determine the enhancement distinction includes: Annotating defect areas and background areas of the defect enhancement sample set to generate an annotated defect enhancement sample set; After grayscale processing is performed on the labeled defect enhancement sample set, the grayscale difference between the defect area and the background area is compared to generate the enhancement distinction.

5. The machine vision driven fireplace surface defect detection method according to claim 3, characterized in that: The construction steps of the shadow enhancement generator include: Taking the fireplace features of the fireplace to be inspected and the preset defect type as constraints, collecting an original image sample set, a light source illumination feature sample set, and a shadow image generation sample set; The shadow enhancement generator is constructed based on generative adversarial network training, using the sample combination of the original image sample set and the light source illumination feature sample set as training input, and the corresponding samples in the shadow image generation sample set as output supervision truth values.

6. The machine vision driven fireplace surface defect detection method according to claim 1, characterized in that: Optimizing the preset defect detection model by using the shadow to enhance the distinction of samples includes: Using the shadow enhanced distinguishing samples as incremental learning samples; Expanding the network structure of the preset defect detection model to generate a new convolutional layer; The newly added convolutional layer is trained with the incremental learning samples, so as to optimize the preset defect detection model with the newly added convolutional layer that has been trained to converge.

7. The machine vision driven fireplace surface defect detection method according to claim 6, characterized in that: The newly added convolution layer is an independent module and is located behind the original convolution block of the preset defect detection model.

8. A machine vision-driven fireplace surface defect detection system, characterized in that: The method for detecting surface defects of a fireplace driven by machine vision according to any one of claims 1 to 7 comprises: A missed detection sample acquisition module is used to determine a preset defect type of the fireplace to be detected, and collect a historical missed detection sample set corresponding to a preset defect detection model based on the preset defect type as a constraint; A sample construction module, used to construct samples based on the historical missed detection sample set to establish defect samples to be strengthened; A shadow effect enhancement analysis module, used to perform shadow effect enhancement analysis on the defect sample to be enhanced, establish light source arrangement parameters with an enhanced distinction greater than a preset distinction, and obtain corresponding shadow enhancement distinction samples; A model optimization module, used for optimizing the preset defect detection model by using the shadow enhanced distinguishing samples; The defect detection module is used to set the light source based on the light source arrangement parameters. When the defect detection is performed on the fireplace to be detected, the image of the fireplace is collected after the shadow effect is enhanced by the set light source and then input into the optimized preset defect detection model for defect detection.

Images