A method and device for optimizing training set feedback of deep learning film evaluation model

Abstract

This application belongs to the field of quality inspection and specifically discloses a method and device for optimizing the feedback of a training set for a deep learning film evaluation model. The method includes: scaling, cropping, and enhancing a flaw detection image of a casting to obtain an enhanced sub-image; determining the path of a marker file for the flaw detection image, combining the marker file path with the image cropping method, and utilizing a coordinate transformation algorithm to obtain a sub-image marker file for the cropped sub-image; screening the enhanced sub-image according to the marker file, and inputting the screened enhanced sub-image as a training set into a neural network model for training to obtain a trained detection model; utilizing the detection model to obtain a model marker file corresponding to a validation set, and inspecting and modifying the model marker file according to established verification marking software to obtain a modified sub-image; and inputting the modified sub-image as an optimized training set into a neural network model for training. This application can improve the accuracy of model detection and recognition.

Description

Technical Field

[0001] The present application belongs to the field of quality inspection, and more specifically, relates to a method and device for optimizing feedback of a training set of a deep learning film evaluation model. Background Art

[0002] Defect detection (segmentation and positioning - classification and rating) driven by deep learning target detection technology is a cutting-edge technology and development trend for intelligent flaw assessment of defects such as loose holes, inclusions, and cracks in light alloy castings. Although deep learning technology has made certain progress in X-ray image analysis, it still faces many challenges in practical application. First, the labeled samples for training are manually labeled by researchers, and their accuracy is easily affected by the researchers' subjective consciousness and film assessment experience, resulting in large fluctuations in the reliability of model training. Second, the resolution of casting flaw detection images is generally very large, resulting in high computational complexity for training, and the network structure has difficulty effectively learning the training set features with a small number of samples. Third, deep learning models are easily affected by changes in data distribution, resulting in performance degradation during actual verification. Finally, deep learning models lack a feedback mechanism for the actual detection process, making it difficult to meet the high-precision detection requirements in complex industrial scenarios.

[0003] In response to the above problems, how to improve the model's ability to recognize complex defects in flaw detection images, improve recognition accuracy, and meet the industrial production's needs for efficient, accurate, and intelligent detection is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In response to the defects of the existing technology, the purpose of this application is to provide a deep learning film evaluation model training set feedback optimization method and device, aiming to solve the problems in the existing technology that the model has weak recognition ability and insufficient recognition accuracy, and cannot meet industrial production needs.

[0005] To achieve the above objectives, in a first aspect, the present application provides a method for optimizing the training set feedback of a deep learning film evaluation model, comprising:

[0006] scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image;

[0007] Determine the path of the mark file of the flaw detection image, combine the mark file path with the image cropping method, and use a coordinate transformation algorithm to obtain a sub-image mark file of the cropped sub-image;

[0008] Screening the enhanced sub-images according to the sub-image labeling file, and inputting the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model;

[0009] The detection model is used to obtain the model tag file corresponding to the verification set, and the model tag file is detected and modified according to the established verification tag software to obtain the modified sub-image;

[0010] The modified sub-image is input into the neural network model as an optimized training set for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

[0011] Optionally, the flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including:

[0012] Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image;

[0013] Reducing the length and width of the flaw detection image to a maximum integer multiple of the target resolution, determining the image reduction ratio, and obtaining a scaled image;

[0014] The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

[0015] Optionally, performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes:

[0016] Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram;

[0017] Accumulating the number of pixels from the minimum grayscale level to the current grayscale level according to the grayscale histogram to obtain a cumulative distribution function;

[0018] The cumulative distribution function is normalized and mapped to obtain enhanced pixel values ​​of each gray level, an enhanced sub-image is determined according to the enhanced pixel values ​​of the gray level, and the enhanced sub-image is stored in an enhanced image folder.

[0019] Optionally, the method for determining the marker file includes:

[0020] Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder;

[0021] Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image;

[0022] Performing multiple and remainder value processing on the coordinates of the reduced marker frame to determine the number of sub-images covered by the reduced marker frame in the x-direction and the y-direction;

[0023] Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marker box in the cropped sub-image;

[0024] Determining the length and width of the reduced marker frame according to a position determination condition, generating sub-image marker frame coordinates of a target sub-image, and creating a sub-image marker file of the target sub-image based on the sub-image marker frame coordinates, wherein the target sub-image is any image containing a marker frame in the cropped sub-image;

[0025] After traversing all inner and outer loops, all generated sub-image markup files are stored in the sub-image XML folder.

[0026] Optionally, it also includes:

[0027] When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes;

[0028] The original marking file of the target sub-image is determined, creation of the sub-image marking file is canceled, and coordinate information of the sub-image marking frame coordinates is added to the original marking file.

[0029] Optionally, the training method of the detection model includes:

[0030] Traverse the subgraph XML folder to obtain all the file names to be filtered;

[0031] Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder;

[0032] The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target set value and the training is completed, thereby obtaining a trained detection model.

[0033] Optionally, the model optimization process includes:

[0034] Inputting the verification set into the detection model to obtain a model labeling file after detection;

[0035] The model marking file is detected according to the established verification marking software. If a detection error occurs, the marking frame of the model marking file is adjusted to obtain an adjusted marking file;

[0036] The verification set images are stored in the image folder of the flaw detection images, and the adjusted markup files are stored in the XML folder;

[0037] When the file storage in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are performed again until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

[0038] In a second aspect, the present application also provides a deep learning film evaluation model training set feedback optimization device, comprising:

[0039] A cropping and enhancement module, configured to scale and crop the flaw detection image of the casting to obtain a cropped sub-image, and perform image enhancement on the cropped sub-image to obtain an enhanced sub-image;

[0040] a marking module, configured to determine a marking file path of the flaw detection image, combine the marking file path with an image cropping method, and use a coordinate transformation algorithm to obtain a sub-image marking file of the cropped sub-image;

[0041] A training module, configured to screen the enhanced sub-images according to the sub-image labeling file, and input the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model;

[0042] A modification module is used to obtain a model tag file corresponding to the verification set using the detection model, and detect and modify the model tag file according to the established verification tag software to obtain a modified sub-image;

[0043] The optimization module is used to input the modified sub-image as an optimized training set into the neural network model for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

[0044] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0046] In a fifth aspect, the present application provides a computer program product, which, when executed on a processor, enables the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0047] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0048] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the existing technologies:

[0049] (1) This application constructs a marker coordinate transformation algorithm to ensure that the marker box can accurately reflect the actual position and size in the sub-image, making subsequent detection and training more accurate. The marker coordinate transformation ensures that the accuracy of the marker information can be maintained after cropping, so that the neural network model can fully learn the characteristics of the training set to improve the model accuracy, thereby improving the model recognition ability and recognition accuracy, thereby meeting the needs of industrial production for efficient, accurate and intelligent detection.

[0050] (2) This application constructs a feedback optimization method for the film evaluation model, which can utilize multiple iterations of training to more comprehensively evaluate the performance of the model, further reduce fluctuations in model training, and improve stability. Furthermore, feedback optimization can not only improve the model's ability to train new data, but also enhance its robustness when processing unseen samples. The optimized model can more effectively cope with various changes that may occur in the production environment, achieving the model's generalization and adaptability.

[0051] (3) This application integrates the generation of training files, model training and optimization feedback to ensure that the model maintains high accuracy during training and use, significantly improving the model training efficiency and prediction accuracy. The final detection model can make more reliable and accurate predictions in casting flaw detection, reducing the probability of false detection or missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is one of the flow charts of the deep learning film evaluation model training set feedback optimization method provided in the embodiment of the present application;

[0053] Figure 2 This is the second flow chart of the deep learning film evaluation model training set feedback optimization method provided in the embodiment of the present application;

[0054] Figure 3 This is the third flow chart of the deep learning film evaluation model training set feedback optimization method provided in the embodiment of the present application;

[0055] Figure 4 Schematic diagram of the structure of the deep learning film evaluation model training set feedback optimization device provided in an embodiment of the present application;

[0056] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0059] The terms "first" and "second" in this specification and claims are used to distinguish different objects rather than to describe a specific order of objects. For example, "first response message" and "second response message" are used to distinguish different response messages rather than to describe a specific order of response messages.

[0060] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0061] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more, for example, multiple processing units means two or more processing units, etc.; multiple elements means two or more elements, etc.

[0062] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0063] Reference Figure 1 This application provides a method for optimizing the training set feedback of a deep learning film evaluation model, including:

[0064] S101. Scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image;

[0065] S102. Determine the path of the mark file of the inspection image, combine the mark file path with the image cropping method, and use the coordinate transformation algorithm to obtain the sub-image mark file of the cropped sub-image;

[0066] S103. Filtering the enhanced sub-images according to the sub-image tag file, and inputting the filtered enhanced sub-images as a training set into a neural network model for training to obtain a trained detection model;

[0067] S104. Using the detection model to obtain the model tag file corresponding to the validation set, the model tag file is detected and modified according to the established verification tag software to obtain a modified sub-image;

[0068] S105. Input the modified sub-image as an optimized training set into the neural network model for training until there are no errors in the model tag file detection, thereby obtaining an optimized detection model.

[0069] First, the input casting flaw detection image is scaled by S101 to adapt it to the needs of subsequent processing (such as uniform size). The file path of the flaw detection image is . The reduced image is then cropped using an image cropping algorithm. In this embodiment, the image can be cropped into a sub-image with a resolution of 416*416. Furthermore, the cropped sub-image is enhanced to improve the image quality and visibility. Common enhancement techniques include histogram equalization, contrast enhancement, noise reduction, etc. Image enhancement helps improve the feature recognition ability of the deep learning model.

[0070] Secondly, coordinate transformation and marker file generation are performed through S102. First, ensure that there is a marker file path corresponding to the initial image. The file usually stores the position and category information of the marker box in XML, JSON and other formats. In this embodiment, the marker file path is an XML folder, which is recorded as .

[0071] According to the image cropping method, a coordinate transformation algorithm is applied to map the original marker frame coordinates to the cropped sub-image coordinates, thereby ensuring that the cropped sub-image also has valid and accurate marker frame information.

[0072] Furthermore, image screening and model training are performed in S103. The enhanced sub-images are screened using the converted marker box information. Only sub-images containing features within the relevant marker box are included in the training set. The screened enhanced sub-images are used as the training set and input into the neural network model for training.

[0073] S104 then performs model verification and tag file correction. After model training is complete, the model's predicted tag file is obtained based on the validation set data. The tag file contains the model-processed detection results. Specialized verification and tagging software is used to verify and modify the model tag file, thereby identifying errors that were not corrected during model training and inference, and making necessary adjustments to the tag boxes.

[0074] In some implementations, it is necessary to manually or semi-automatically check the accuracy of the marked boxes to ensure that they conform to the actual defect characteristics.

[0075] Finally, S105 performs optimization training and model adjustments. The verified and modified sub-images are incorporated into a new training set, which contains higher-quality labeled data and more accurate feature representations. The optimized training set is then fed into the neural network model, and the model is trained again until it no longer encounters errors when detecting labeled files. This feedback loop helps continuously improve the model's accuracy and robustness.

[0076] Optionally, the flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including:

[0077] Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image;

[0078] Reducing the length and width of the flaw detection image to a maximum integer multiple of the target resolution, determining the image reduction ratio, and obtaining a scaled image;

[0079] The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

[0080] Specifically, in this embodiment, the path to the flaw detection image file is first entered through the graphical user interface. This path contains a folder containing all the flaw detection images to be processed. The process then iterates through all images in the folder and reads their length and width. For each flaw detection image, the reduction ratio between its length and width and the target resolution is calculated, and the integer portion of the ratio is used as the scaling factor. In this embodiment, the target resolution is preferably 416*416, but this can be adjusted in practice and is not specifically limited here.

[0081] By applying the image cropping algorithm, a series of cropped sub-images with a resolution of 416*416 are obtained. Each cropped sub-image is named as follows: the original image name ,in Indicates that the sub-image is located in the first Row and List.

[0082] Optionally, performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes:

[0083] Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram;

[0084] Accumulating the number of pixels from the minimum grayscale level to the current grayscale level according to the grayscale histogram to obtain a cumulative distribution function;

[0085] The cumulative distribution function is normalized and mapped to obtain enhanced pixel values ​​of each gray level, an enhanced sub-image is determined according to the enhanced pixel values ​​of the gray level, and the enhanced sub-image is stored in an enhanced image folder.

[0086] Specifically, after generating the cropped sub-image, calculate each gray level in the sub-image The number of pixels, get the histogram , , Is the total number of gray levels. According to the histogram, the number of pixels from the minimum gray level to the current gray level is accumulated to obtain the cumulative distribution function , and then normalize and map them to get the pixel value of each gray level , as shown in the following formula (1). Create a folder and an enhanced image folder to store the enhanced sub-images. The path is: .

[0087]

[0088] in, is the minimum value of the cumulative distribution function, is the maximum value of the cumulative distribution function, is the normalized cumulative distribution function value, is the cumulative distribution function, is the total number of gray levels, is the grayscale level, is the enhanced gray level.

[0089] Optionally, the method for determining the marker file includes:

[0090] Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder;

[0091] Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image;

[0092] Performing multiple and remainder value processing on the coordinates of the reduced marker frame to determine the number of sub-images covered by the reduced marker frame in the x-direction and the y-direction;

[0093] Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marker box in the cropped sub-image;

[0094] Determining the length and width of the reduced marker frame according to a position determination condition, generating sub-image marker frame coordinates of a target sub-image, and creating a sub-image marker file of the target sub-image based on the sub-image marker frame coordinates, wherein the target sub-image is any image containing a marker frame in the cropped sub-image;

[0095] When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes;

[0096] The original marking file of the target sub-image is determined, creation of the sub-image marking file is canceled, and coordinate information of the sub-image marking frame coordinates is added to the original marking file.

[0097] After traversing all inner and outer loops, all generated sub-image markup files are stored in the sub-image XML folder.

[0098] Specifically, this embodiment is a specific process of determining a marked file.

[0099] 1. Enter the image tag file path, i.e., the XML folder, through the graphical user interface, recorded as , traverse all the marker files in the XML folder, record the coordinates of the upper left corner, upper right corner, lower left corner, and lower right corner of the marker box in each file, and then multiply them by the ratio of the length and width of the image calculated in the previous step to get the four coordinates of the marker box after the image is reduced ( .

[0100] 2. After the original image is cut into sub-images, each sub-image needs to be equipped with a new marker file, and the division of the marker box should be considered. Take the multiples and remainders of 416 for the four coordinates obtained in the previous step to get .

[0101] in, The integer part after dividing by 416, that is, the left boundary of the x coordinate, The remainder of 416, that is, the offset of the left boundary;

[0102] is the integer part after dividing by 416, that is, the right boundary of the x coordinate, The remainder of 416, that is, the offset of the right boundary;

[0103] is the integer part after dividing by 416, that is, the upper boundary of the y coordinate, The remainder of 416, i.e., the offset of the upper boundary;

[0104] is the integer part after dividing by 416, that is, the lower boundary of the y coordinate, It is the remainder of 416, that is, the offset of the lower boundary.

[0105] remember for and The difference is the length of the marker box in the x direction, for and The difference between the length of the marker box in the y direction is obtained by and To determine the mark box Direction and The number of subgraphs covered in the direction.

[0106] 3. Create an outer loop and an inner loop. The outer loop counter Starting from 0, to End, step size is 1; inner loop counter Starting from 0, to End, the step size is 1. When the inner loop ends, the outer loop count increases and continues to loop; when the outer loop enters the next round, the inner loop counter is increased. Return to zero and start the inner loop.

[0107] 4. Make the following judgments in the inner loop:

[0108] if ,but , ;

[0109] Otherwise if ,but , ;

[0110] Otherwise if ,but , ;

[0111] otherwise , .

[0112] After making the above judgments, continue to make the next set of judgments:

[0113] if ,but , ;

[0114] Otherwise if ,but , ;

[0115] Otherwise if ,but , ;

[0116] otherwise , .

[0117] Based on the above two sets of judgments, we can get a set of , this set of numbers represents the four coordinates of the marked box in a sub-image.

[0118] Specifically, the steps of this application are as follows:

[0119] x-coordinate judgment:

[0120] if If it is equal to 0, it means that the marker box does not cross the sub-image boundary in the x direction. Assign to ,Will Assign to .

[0121] if Not equal to 0, continue with the following judgment:

[0122] If the outer loop counter m is equal to 0, it means that the current is the first subgraph of the marked box in the x direction. Assign to , assign 416 to .

[0123] If the outer loop counter m is equal to , indicating that the current subgraph is the last one in the x direction of the marker box, and 0 is assigned to ,Will Assign to .

[0124] If the value of the outer loop counter m is between 0 and It means that the current sub-image is the middle sub-image of the marker box in the x direction. At this time, 0 is assigned to , assign 416 to .

[0125] Y coordinate judgment:

[0126] if If it is equal to 0, it means that the marker box does not cross the sub-image boundary in the y direction. Assign to ,Will Assign to .

[0127] if Not equal to 0, continue with the following judgment:

[0128] If the inner loop counter n is equal to 0, it means that the current is the first sub-graph of the marked box in the y direction. Assign to , assign 416 to .

[0129] If the inner loop counter n is equal to , indicating that the current subgraph is the last one in the y direction of the marker box, and 0 is assigned to ,Will Assign to .

[0130] If the value of the inner loop counter n is between 0 and It means that the current sub-image is the middle sub-image of the marker box in the y direction. At this time, 0 is assigned to , assign 416 to .

[0131] Through the above two sets of judgments, based on the above two sets of judgments, we can get a set of , this set of numbers represents the four coordinates of the marked box in a sub-image.

[0132] 5. Create a marker file for the sub-image based on the obtained coordinates. The name of the marker file should match the sub-image name. If a marker file with the same name already exists when creating a sub-image marker file, it means that the sub-image has multiple marker boxes. In this case, you do not need to create a new marker file. Instead, continue to add information based on the existing marker file.

[0133] 6. After traversing all loops, store all generated marker files in a folder, the subgraph XML folder, the path is: .

[0134] The embodiment of the present application can adapt to images of different sizes by scaling the coordinates of the marker box, making image processing more flexible. When the original image is cropped into multiple sub-images, a corresponding marker box can be generated for each sub-image, ensuring the accuracy of the marking information. The inner and outer loops can correctly handle the situation where the marker box spans multiple sub-images, avoiding the loss of marking information. A new marker file is generated for each sub-image, facilitating subsequent image analysis and model training. By accurately dividing and marking sub-images, it helps to improve the quality of the dataset.

[0135] Reference Figure 2 , Figure 2 It is the specific generation process of the sub-image, including:

[0136] Get the image folder and XML folder;

[0137] Original image and XML markup;

[0138] Image cropping and enhancement;

[0139] Linked modification of annotation files;

[0140] Get the marker subgraph folder and subgraph XML folder.

[0141] Optionally, the training method of the detection model includes:

[0142] Traverse the subgraph XML folder to obtain all the file names to be filtered;

[0143] Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder;

[0144] The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target set value and the training is completed, thereby obtaining a trained detection model.

[0145] In this embodiment, traverse the folder path , get all the file names of the folder, and then find the folder path according to the file name All files in , all sub-graphs with the same name as the marked file are stored in a new folder, the path is Then set the training image path in model training to , marking the path as , after 200 rounds of iterations, the trained detection model is obtained.

[0146] The number of model iterations can be adjusted according to actual needs and is not specifically limited in this application.

[0147] Optionally, the model optimization process includes:

[0148] Inputting the verification set into the detection model to obtain a model labeling file after detection;

[0149] The model marking file is detected according to the established verification marking software. If a detection error occurs, the marking frame of the model marking file is adjusted to obtain an adjusted marking file;

[0150] The verification set images are stored in the image folder of the flaw detection images, and the adjusted markup files are stored in the XML folder;

[0151] When the file storage in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are performed again until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

[0152] In this embodiment, the validation set images are input into the detection model and the detected marked files are output. The detection results are viewed based on the self-created verification and marking software. If a detection error occurs, the marking box can be adjusted to the correct position in the software and the image can be saved in the , save the adjusted markup file in In. When After a certain number of files are stored in the image processing unit, the steps of image scaling, image cropping, image enhancement and model training in the above embodiment are automatically performed to complete the feedback optimization process to obtain the optimized detection model.

[0153] Reference Figure 3 , Figure 3 It is a flowchart of model training;

[0154] Film evaluation model testing;

[0155] Detect images and XML tags;

[0156] Correct XML markup;

[0157] Save into image folder and XML folder;

[0158] The training set is automatically generated;

[0159] Save to the sub-image XML folder and mark the sub-image folder.

[0160] Reference Figure 4 , the present application also provides a deep learning film evaluation model training set feedback optimization device, comprising:

[0161] The cropping and enhancement module 410 is configured to scale and crop the flaw detection image of the casting to obtain a cropped sub-image, and to enhance the cropped sub-image to obtain an enhanced sub-image.

[0162] The marking module 420 is used to determine the marking file path of the flaw detection image, combine the marking file path with the image cropping method, and use a coordinate transformation algorithm to obtain a sub-image marking file of the cropped sub-image;

[0163] A training module 430 is configured to screen the enhanced sub-images according to the sub-image labeling file, and input the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model;

[0164] A modification module 440 is configured to obtain a model tag file corresponding to the validation set using the detection model, detect and modify the model tag file using the established verification tag software, and obtain a modified sub-image;

[0165] The optimization module 450 is used to input the modified sub-image as an optimized training set into the neural network model for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

[0166] Optionally, the flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including:

[0167] Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image;

[0168] Reducing the length and width of the flaw detection image to a maximum integer multiple of the target resolution, determining the image reduction ratio, and obtaining a scaled image;

[0169] The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

[0170] Optionally, performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes:

[0171] Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram;

[0172] Accumulating the number of pixels from the minimum grayscale level to the current grayscale level according to the grayscale histogram to obtain a cumulative distribution function;

[0173] The cumulative distribution function is normalized and mapped to obtain enhanced pixel values ​​of each gray level, an enhanced sub-image is determined according to the enhanced pixel values ​​of the gray level, and the enhanced sub-image is stored in an enhanced image folder.

[0174] Optionally, the method for determining the marker file includes:

[0175] Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder;

[0176] Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image;

[0177] Performing multiple and remainder value processing on the coordinates of the reduced marker frame to determine the number of sub-images covered by the reduced marker frame in the x-direction and the y-direction;

[0178] Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marker box in the cropped sub-image;

[0179] Determining the length and width of the reduced marker frame according to a position determination condition, generating sub-image marker frame coordinates of a target sub-image, and creating a sub-image marker file of the target sub-image based on the sub-image marker frame coordinates, wherein the target sub-image is any image containing a marker frame in the cropped sub-image;

[0180] After traversing all inner and outer loops, all generated sub-image markup files are stored in the sub-image XML folder.

[0181] Optionally, a file cancellation creation module is further included, for:

[0182] When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes;

[0183] The original marking file of the target sub-image is determined, creation of the sub-image marking file is canceled, and coordinate information of the sub-image marking frame coordinates is added to the original marking file.

[0184] Optionally, the training method of the detection model includes:

[0185] Traverse the subgraph XML folder to obtain all the file names to be filtered;

[0186] Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder;

[0187] The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target set value and the training is completed, thereby obtaining a trained detection model.

[0188] Optionally, the model optimization process includes:

[0189] Inputting the verification set into the detection model to obtain a model labeling file after detection;

[0190] The model marking file is detected according to the established verification marking software. If a detection error occurs, the marking frame of the model marking file is adjusted to obtain an adjusted marking file;

[0191] The verification set images are stored in the image folder of the flaw detection images, and the adjusted markup files are stored in the XML folder;

[0192] When the file storage in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are performed again until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

[0193] It is understandable that the detailed functional implementation of each of the above units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0194] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method and will not be repeated here.

[0195] Based on the methods in the above embodiments, an embodiment of the present application provides an electronic device, which may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the methods in the above embodiments.

[0196] In addition, the logic instructions in the aforementioned memory 530 can be implemented in the form of a software functional unit and, when sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0197] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0198] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0199] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0200] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC.

[0201] The above embodiments can be implemented in whole or in part using software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions. When loaded and executed on a computer, the computer program instructions fully or partially produce the processes or functions described in the embodiments of this application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)).

[0202] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

[0203] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for optimizing the training set feedback of a deep learning film evaluation model, characterized in that: include: scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image; Determine the path of the mark file of the flaw detection image, combine the mark file path with the image cropping method, and use a coordinate transformation algorithm to obtain a sub-image mark file of the cropped sub-image; Screening the enhanced sub-images according to the sub-image labeling file, and inputting the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model; The detection model is used to obtain the model tag file corresponding to the verification set, and the model tag file is detected and modified according to the established verification tag software to obtain the modified sub-image; Inputting the modified sub-image as an optimized training set into a neural network model for training until no errors are detected in the model tag file, thereby obtaining an optimized detection model; The method for determining the marker file includes: Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder; Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image; Performing multiple and remainder value processing on the coordinates of the reduced marker frame to determine the number of sub-images covered by the reduced marker frame in the x-direction and the y-direction; Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marker box in the cropped sub-image; Determining the length and width of the reduced marker frame according to a position determination condition, generating sub-image marker frame coordinates of a target sub-image, and creating a sub-image marker file of the target sub-image based on the sub-image marker frame coordinates, wherein the target sub-image is any image containing a marker frame in the cropped sub-image; After traversing all inner and outer loops, all generated sub-image markup files are stored in the sub-image XML folder.

2. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: The flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including: Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image; Reducing the length and width of the flaw detection image to a maximum integer multiple of the target resolution, determining the image reduction ratio, and obtaining a scaled image; The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

3. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: Performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes: Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram; Accumulating the number of pixels from the minimum grayscale level to the current grayscale level according to the grayscale histogram to obtain a cumulative distribution function; The cumulative distribution function is normalized and mapped to obtain enhanced pixel values ​​of each gray level, an enhanced sub-image is determined according to the enhanced pixel values ​​of the gray level, and the enhanced sub-image is stored in an enhanced image folder.

4. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: Also includes: When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes; The original marking file of the target sub-image is determined, creation of the sub-image marking file is canceled, and coordinate information of the sub-image marking frame coordinates is added to the original marking file.

5. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: The training method of the detection model includes: Traverse the subgraph XML folder to obtain all the file names to be filtered; Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder; The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target set value and the training is completed, thereby obtaining a trained detection model.

6. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: The model optimization process includes: Inputting the verification set into the detection model to obtain a model labeling file after detection; The model marking file is detected according to the established verification marking software. If a detection error occurs, the marking frame of the model marking file is adjusted to obtain an adjusted marking file; The verification set images are stored in the image folder of the flaw detection images, and the adjusted markup files are stored in the XML folder; When the file storage in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are performed again until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

7. A deep learning film evaluation model training set feedback optimization device, characterized in that: include: A cropping and enhancement module, configured to scale and crop the flaw detection image of the casting to obtain a cropped sub-image, and perform image enhancement on the cropped sub-image to obtain an enhanced sub-image; a marking module, configured to determine a marking file path of the flaw detection image, combine the marking file path with an image cropping method, and use a coordinate transformation algorithm to obtain a sub-image marking file of the cropped sub-image; A training module, configured to screen the enhanced sub-images according to the sub-image labeling file, and input the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model; A modification module is used to obtain a model tag file corresponding to the verification set using the detection model, and detect and modify the model tag file according to the established verification tag software to obtain a modified sub-image; An optimization module is used to input the modified sub-image as an optimized training set into a neural network model for training until no errors are found in the model tag file detection, thereby obtaining an optimized detection model; The method for determining the marker file includes: Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder; Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image; Performing multiple and remainder value processing on the coordinates of the reduced marker frame to determine the number of sub-images covered by the reduced marker frame in the x-direction and the y-direction; Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marker box in the cropped sub-image; Determining the length and width of the reduced marker frame according to a position determination condition, generating sub-image marker frame coordinates of a target sub-image, and creating a sub-image marker file of the target sub-image based on the sub-image marker frame coordinates, wherein the target sub-image is any image containing a marker frame in the cropped sub-image; After traversing all inner and outer loops, all generated sub-image markup files are stored in the sub-image XML folder.

8. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed on a processor, the processor is caused to execute the method according to any one of claims 1 to 6.

Images