Deep learning film evaluation model training set feedback optimization method and device

Abstract

The invention belongs to the field of quality detection, and particularly discloses a deep learning film evaluation model training set feedback optimization method and device, and the method comprises the steps: carrying out the zooming, cutting and enhancement of a flaw detection image of a casting, and obtaining an enhanced sub-image; determining a mark file path of the flaw detection image, combining the mark file path with an image cutting mode, and utilizing a coordinate transformation algorithm to obtain a sub-image mark file of the cut sub-image; screening the enhanced sub-images according to the tag file, and inputting the screened enhanced sub-images as a training set into a neural network model for training to obtain a trained detection model; acquiring a model mark file corresponding to the verification set by using the detection model, and detecting and modifying the model mark file according to the created checking mark software to obtain a modified sub-image; and inputting the modified sub-images as an optimized training set into the neural network model for training. According to the invention, the accuracy of model detection and identification can be improved.

Description

Technical Field

[0001] The present application belongs to the field of quality inspection, and more specifically, to a method and device for optimizing feedback of a training set of a deep learning film evaluation model. Background Art

[0002] Defect detection (segmentation and positioning - classification and rating) driven by deep learning target detection technology is the cutting-edge technology and development trend of intelligent flaw detection and evaluation of defects such as loose holes, inclusions, and cracks in light alloy castings. Although deep learning technology has made certain progress in X-ray image analysis, it still faces many challenges in practical applications. First, the labeled samples for training are manually labeled by researchers, and their accuracy is easily affected by the subjective consciousness and film evaluation experience of researchers, resulting in large fluctuations in the reliability of model training; secondly, the resolution of casting flaw detection images is generally large, resulting in high computational complexity of training, and it is difficult for the network structure to effectively learn the training set features with a small number of samples; thirdly, the deep learning model is easily affected by changes in data distribution, resulting in performance degradation during actual verification; finally, the deep learning model lacks a feedback mechanism for the actual detection process, and it is difficult to meet the high-precision detection requirements in complex industrial scenarios.

[0003] In response to the above problems, how to improve the model's ability to recognize complex defects in flaw detection images, improve recognition accuracy, and meet the needs of industrial production for efficient, precise, and intelligent detection is a technical problem that urgently needs to be solved. Summary of the invention

[0004] In view of the defects of the prior art, the purpose of this application is to provide a method and device for optimizing the feedback of a deep learning film evaluation model training set, aiming to solve the problems in the prior art that the model has weak recognition ability and insufficient recognition accuracy and cannot meet the needs of industrial production.

[0005] To achieve the above objectives, in a first aspect, the present application provides a method for optimizing the training set feedback of a deep learning film evaluation model, comprising: Scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image; Determine the path of the mark file of the flaw detection image, combine the mark file path with the image cropping method, and use the coordinate transformation algorithm to obtain the sub-image mark file of the cropped sub-image; The enhanced sub-images are screened according to the sub-image labeling file, and the screened enhanced sub-images are input into the neural network model as training sets for training to obtain a trained detection model; The detection model is used to obtain the model labeling file corresponding to the verification set, and the model labeling file is detected and modified according to the created verification labeling software to obtain the modified sub-image; The modified sub-image is input as an optimized training set into a neural network model for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

[0006] Optionally, the flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including: Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image; The length and width of the flaw detection image are reduced to a maximum integer multiple of the target resolution, and the reduction ratio of the image is determined to obtain a scaled image; The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

[0007] Optionally, performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes: Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram; According to the grayscale histogram, the number of pixels from the minimum grayscale level to the current grayscale level is accumulated to obtain a cumulative distribution function; The cumulative distribution function is normalized and mapped to obtain the pixel value of each gray level after enhancement, an enhanced sub-image is determined according to the pixel value of the enhanced gray level, and the enhanced sub-image is stored in an enhanced image folder.

[0008] Optionally, the method for determining the marker file includes: Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder; Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image; Performing multiple and remainder value processing on the coordinates of the reduced marking frame to determine the number of sub-images covered by the reduced marking frame in the x direction and the y direction; Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marking box in the cropped sub-image; The length and width of the reduced mark frame are judged according to the position judgment condition, the sub-image mark frame coordinates of the target sub-image are generated, and a sub-image mark file of the target sub-image is created based on the sub-image mark frame coordinates, wherein the target sub-image is any image containing the mark frame in the cropped sub-image; After all inner and outer loops are traversed, all generated sub-image tag files are stored in the sub-image XML folder.

[0009] Optionally, it also includes: When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes; The original marking file of the target sub-image is determined, the creation of the sub-image marking file is canceled, and the coordinate information of the sub-image marking frame coordinates is added to the original marking file.

[0010] Optionally, the training method of the detection model includes: Traverse the sub-graph XML folder to obtain all the file names to be filtered; Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder; The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target setting value and the training is completed, thereby obtaining a trained detection model.

[0011] Optionally, the model optimization process includes: Inputting the verification set into the detection model to obtain a model labeling file after detection; The model marking file is detected according to the established verification marking software. If a detection error occurs, the position of the marking frame of the model marking file is adjusted to obtain an adjusted marking file; The verification set images are stored in the image folder of the flaw detection images, and the adjusted marker files are stored in the XML folder; When the number of files stored in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are repeated until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

[0012] In a second aspect, the present application also provides a deep learning film evaluation model training set feedback optimization device, comprising: A cropping and enhancement module, used for scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image; A marking module, used for determining a marking file path of the flaw detection image, combining the marking file path with an image cropping method, and using a coordinate transformation algorithm to obtain a sub-image marking file of the cropped sub-image; A training module, used to screen the enhanced sub-images according to the sub-image labeling file, and input the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model; A modification module is used to obtain a model tag file corresponding to the verification set by using the detection model, and detect and modify the model tag file according to the created verification tag software to obtain a modified sub-image; The optimization module is used to input the modified sub-image as an optimized training set into the neural network model for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

[0013] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0015] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0016] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0017] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art: (1) This application constructs a marker coordinate transformation algorithm to ensure that the marker box can accurately reflect the actual position and size in the sub-image, making subsequent detection and training more accurate. The marker coordinate transformation ensures that the accuracy of the marker information can be maintained after cropping, so that the neural network model can fully learn the characteristics of the training set to improve the model accuracy, thereby improving the model recognition ability and recognition accuracy, thereby meeting the needs of industrial production for efficient, accurate and intelligent detection.

[0018] (2) This application constructs a feedback optimization method for the film evaluation model, which can use multiple iterations of training to more comprehensively evaluate the performance of the model, further reduce fluctuations in model training, and improve stability. In addition, feedback optimization can not only improve the model's ability to train new data, but also enhance its robustness when processing unseen samples. The optimized model can more effectively cope with various changes that may occur in the production environment, achieving the generalization and adaptability of the model.

[0019] (3) This application integrates the generation of training files, model training and optimization feedback to ensure that the model maintains high accuracy during training and use, significantly improving the model training efficiency and prediction accuracy. The final detection model can make more reliable and accurate predictions in casting flaw detection, reducing the probability of false detection or missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is one of the flow charts of the deep learning film evaluation model training set feedback optimization method provided in the embodiment of the present application; Figure 2 This is the second flow chart of the deep learning film evaluation model training set feedback optimization method provided in the embodiment of the present application; Figure 3 This is the third flow chart of the deep learning film evaluation model training set feedback optimization method provided in the embodiment of the present application; Figure 4 It is a structural diagram of a deep learning film evaluation model training set feedback optimization device provided in an embodiment of the present application; Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0023] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0024] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0025] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0026] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0027] Reference Figure 1 The present application provides a method for optimizing the training set feedback of a deep learning film evaluation model, including: S101. Scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image; S102. Determine the path of the mark file of the flaw detection image, combine the mark file path with the image cropping method, and use the coordinate transformation algorithm to obtain the sub-image mark file of the cropped sub-image; S103. Screening the enhanced sub-image according to the sub-image labeling file, and inputting the screened enhanced sub-image as a training set into the neural network model for training to obtain a trained detection model; S104. Using the detection model to obtain the model tag file corresponding to the verification set, the model tag file is detected and modified according to the created verification tag software to obtain a modified sub-image; S105. Input the modified sub-image as an optimized training set into the neural network model for training until there are no errors in the model tag file detection, thereby obtaining an optimized detection model.

[0028] First, the input casting flaw detection image is scaled by S101 to adapt it to the needs of subsequent processing (such as uniform size). The file path of the flaw detection image is . Then, the reduced image is cropped using an image cropping algorithm. In this embodiment, the image can be cropped into a sub-image with a resolution of 416*416. Furthermore, the cropped sub-image is enhanced to improve the quality and visibility of the image. Common enhancement techniques include histogram equalization, contrast enhancement, noise reduction, etc. Image enhancement helps to improve the feature recognition ability of the deep learning model.

[0029] Secondly, coordinate transformation and marker file generation are performed through S102. First, ensure that there is a marker file path corresponding to the initial image. The file usually stores the position and category information of the marker box in XML, JSON and other formats. The marker file path in this embodiment is an XML folder, recorded as .

[0030] According to the image cropping method, a coordinate transformation algorithm is applied to map the original marker frame coordinates to the cropped sub-image coordinates, thereby ensuring that the cropped sub-image also has valid and accurate marker frame information.

[0031] Further, image screening and model training are performed through S103. The enhanced sub-images are screened using the converted marking frame information. Only when the sub-image contains the features in the relevant marking frame, it can be included in the training set. The screened enhanced sub-images are used as the training set and input into the neural network model for training.

[0032] The model is tested and the marking file is corrected through S104. After the model training is completed, the marking file predicted by the model is obtained according to the validation set data. The marking file contains the detection results after the model processing. Specialized verification marking software is used to detect and modify the model marking file, so as to find errors that the model cannot correct during training and reasoning, and make necessary adjustments to the marking box.

[0033] In some implementations, it is necessary to manually or semi-automatically check the accuracy of the marked boxes to ensure that they conform to the actual defect characteristics.

[0034] Finally, S105 is used for optimization training and model adjustment. The verified and modified sub-images are included in the new training set, which contains higher quality labeled data and more accurate feature expressions. The optimized training set is input into the neural network model, and the model is trained again until no errors occur in the model labeled file detection. The feedback loop helps to continuously improve the accuracy and robustness of the model.

[0035] Optionally, the flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including: Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image; The length and width of the flaw detection image are reduced to a maximum integer multiple of the target resolution, and the reduction ratio of the image is determined to obtain a scaled image; The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

[0036] Specifically, in this embodiment, the path of the flaw detection image file is first input through the graphical user interface. All the flaw detection images to be processed should be stored in the folder under this path. All the pictures in the folder are traversed to read the length and width of the pictures. For each flaw detection image, the reduction ratio between its length and width and the target resolution is calculated, and the integer part of the ratio is taken as the scaling ratio. The target resolution in this embodiment is preferably 416*416, which can be adjusted according to the actual implementation process, and is not specifically limited here.

[0037] By applying the image cropping algorithm, a series of cropped sub-images with a resolution of 416*416 are obtained. Each cropped sub-image is named as follows: Original image name ,in Indicates that the sub-image is located in the first Row and List.

[0038] Optionally, performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes: Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram; According to the grayscale histogram, the number of pixels from the minimum grayscale level to the current grayscale level is accumulated to obtain a cumulative distribution function; The cumulative distribution function is normalized and mapped to obtain the pixel value of each gray level after enhancement, an enhanced sub-image is determined according to the pixel value of the enhanced gray level, and the enhanced sub-image is stored in an enhanced image folder.

[0039] Specifically, after generating the cropped sub-image, calculate each gray level in the sub-image The number of pixels, get the histogram , , is the total number of gray levels. According to the histogram, the number of pixels from the minimum gray level to the current gray level is accumulated to obtain the cumulative distribution function , and then normalize and map them to get the pixel value of each gray level , as shown in the following formula (1). Create a folder and an enhanced image folder to store the enhanced sub-images. The path is: .

[0040]

[0041] in, is the minimum value of the cumulative distribution function, is the maximum value of the cumulative distribution function, is the normalized cumulative distribution function value, is the cumulative distribution function, is the total number of gray levels, is the gray level, is the enhanced gray level.

[0042] Optionally, the method for determining the marker file includes: Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder; Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image; Performing multiple and remainder value processing on the coordinates of the reduced marking frame to determine the number of sub-images covered by the reduced marking frame in the x direction and the y direction; Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marking box in the cropped sub-image; The length and width of the reduced mark frame are judged according to the position judgment condition, the sub-image mark frame coordinates of the target sub-image are generated, and a sub-image mark file of the target sub-image is created based on the sub-image mark frame coordinates, wherein the target sub-image is any image containing the mark frame in the cropped sub-image; When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes; The original marking file of the target sub-image is determined, the creation of the sub-image marking file is canceled, and the coordinate information of the sub-image marking frame coordinates is added to the original marking file.

[0043] After all inner and outer loops are traversed, all generated sub-image tag files are stored in the sub-image XML folder.

[0044] Specifically, this embodiment is a specific process of determining a marked file.

[0045] 1. Enter the image tag file path, i.e., XML folder, through the graphical user interface, recorded as , traverse all the marker files in the XML folder, record the coordinates of the upper left corner, upper right corner, lower left corner, and lower right corner of the marker box in each file, and then multiply them by the ratio of the image length and width calculated in the previous step to get the four coordinates of the marker box after the image is reduced ( .

[0046] 2. After the original image is cut into sub-images, each sub-image needs to be equipped with a new marker file, and the division of the marker box should be considered. Take the multiples and remainders of 416 for the four coordinates obtained in the previous step, and get .

[0047] in, is the integer part after dividing by 416, that is, the left boundary of the x coordinate, The remainder of 416, that is, the offset of the left boundary; is the integer part after dividing by 416, that is, the right boundary of the x-coordinate. The remainder of 416, that is, the offset of the right boundary; is the integer part after dividing by 416, that is, the upper boundary of the y coordinate, The remainder of 416, i.e., the offset of the upper boundary; is the integer part after dividing by 416, that is, the lower boundary of the y coordinate, It is the remainder of 416, that is, the offset of the lower boundary.

[0048] remember for and The difference is the length of the marked box in the x direction. for and The difference between the length of the marked box in the y direction is obtained by and To determine the marking box Direction and The number of subgraphs covered in the direction.

[0049] 3. Create an outer loop and an inner loop. Outer loop counter Starting from 0, to End, step size is 1; inner loop counter Starting from 0, to End, the step length is 1. When the inner loop ends, the outer loop count increases and continues to loop; when the outer loop enters the next round, the inner loop counter is increased. Return to zero and start the inner loop.

[0050] 4. Make the following judgments in the inner loop: if ,but , ; Otherwise if ,but , ; Otherwise if ,but , ; otherwise , .

[0051] After making the above judgments, continue to make the next set of judgments: if ,but , ; Otherwise if ,but , ; Otherwise if ,but , ; otherwise , .

[0052] Based on the above two sets of judgments, we can get a set of , this set of numbers represents the four coordinates of the marked box in a sub-image.

[0053] Specifically, the steps of this application are as follows: x coordinate judgment: if If it is equal to 0, it means that the marker box does not cross the sub-image boundary in the x direction. Assign to ,Will Assign to .

[0054] if Not equal to 0, continue with the following judgment: If the outer loop counter m is equal to 0, it means that the current subgraph is the first subgraph of the marked box in the x direction. Assign to , assign 416 to .

[0055] If the outer loop counter m is equal to , indicating that the current sub-image is the last sub-image of the marker box in the x direction, and 0 is assigned to ,Will Assign to .

[0056] If the value of the outer loop counter m is between 0 and It means that the current sub-image is the middle sub-image of the marker box in the x direction. At this time, 0 is assigned to , assign 416 to .

[0057] Y coordinate judgment: if If it is equal to 0, it means that the marker box does not cross the sub-image boundary in the y direction. Assign to ,Will Assign to .

[0058] if Not equal to 0, continue with the following judgment: If the inner loop counter n is equal to 0, it means that the current sub-image of the marked box in the y direction is Assign to , assign 416 to .

[0059] If the inner loop counter n is equal to , indicating that the current sub-image is the last sub-image of the marked box in the y direction, and 0 is assigned to ,Will Assign to .

[0060] If the value of the inner loop counter n is between 0 and It means that the current sub-image is the middle sub-image of the marked box in the y direction. At this time, 0 is assigned to , assign 416 to .

[0061] Through the above two sets of judgments, based on the above two sets of judgments, we can get a set of , this set of numbers represents the four coordinates of the marked box in a sub-image.

[0062] 5. Create a sub-image marker file based on the obtained coordinates. The name of the marker file should be consistent with the sub-image name. When creating a sub-image marker file, if the name of the marker file already exists, it means that the sub-image has multiple marker boxes. In this case, you do not need to create a new marker file, but continue to add information based on the original marker file.

[0063] 6. After traversing all loops, store all generated marker files in a folder, namely the sub-graph XML folder, the path is: .

[0064] The embodiment of the present application can adapt to images of different sizes by scaling the coordinates of the marking frame, making image processing more flexible. When the original image is cropped into multiple sub-images, a corresponding marking frame can be generated for each sub-image, ensuring the accuracy of the marking information. The inner and outer loops can correctly handle the situation where the marking frame spans multiple sub-images, avoiding the loss of marking information, and generating a new marking file for each sub-image, which is convenient for subsequent image analysis and model training. By accurately dividing and marking sub-images, it helps to improve the quality of the data set.

[0065] Reference Figure 2 , Figure 2 It is the specific generation process of the sub-image, including: Get the image folder and XML folder; Original image with XML markup; Image cropping and enhancement; Linked modification of annotation files; Get the marker subgraph folder and subgraph XML folder.

[0066] Optionally, the training method of the detection model includes: Traverse the sub-graph XML folder to obtain all the file names to be filtered; Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder; The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target setting value and the training is completed, thereby obtaining a trained detection model.

[0067] In this embodiment, the folder path is traversed , get all the file names of the folder, and then find the folder path according to the file name All files in , store all sub-images with the same name as the marked file in a new folder, the path is Then set the training image path in model training as , marking the path as , after 200 iterations, the trained detection model is obtained.

[0068] Among them, the number of model iterations can be adjusted according to actual needs, and this application does not make specific limitations.

[0069] Optionally, the model optimization process includes: Inputting the verification set into the detection model to obtain a model labeling file after detection; The model marking file is detected according to the established verification marking software. If a detection error occurs, the position of the marking frame of the model marking file is adjusted to obtain an adjusted marking file; The verification set images are stored in the image folder of the flaw detection images, and the adjusted marker files are stored in the XML folder; When the number of files stored in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are repeated until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

[0070] In this embodiment, the validation set images are input into the detection model and the detected marked files are output. The detection results are checked based on the self-created verification and marking software. If a detection error occurs, the marking box can be adjusted to the correct position in the software and the image can be saved in , save the adjusted marker file in In. When After a certain number of files are stored in the image processing unit, the steps of image scaling, image cropping, image enhancement and model training in the above-mentioned embodiment are automatically performed to complete the feedback optimization process to obtain the optimized detection model.

[0071] Reference Figure 3 , Figure 3 It is a flowchart of model training; Film review model testing; Detect images and XML tags; Corrected XML tags; Save into image folder and XML folder; The training set is automatically generated; Save in sub-image XML folder and mark sub-image folder.

[0072] Reference Figure 4 The present application also provides a deep learning film evaluation model training set feedback optimization device, comprising: The cropping and enhancement module 410 is used to scale and crop the flaw detection image of the casting to obtain a cropped sub-image, and to enhance the cropped sub-image to obtain an enhanced sub-image; The marking module 420 is used to determine the marking file path of the flaw detection image, combine the marking file path with the image cropping method, and use the coordinate transformation algorithm to obtain the sub-image marking file of the cropped sub-image; A training module 430 is used to screen the enhanced sub-images according to the sub-image labeling file, and input the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model; A modification module 440 is used to obtain a model tag file corresponding to the verification set using the detection model, and detect and modify the model tag file according to the created verification tag software to obtain a modified sub-image; The optimization module 450 is used to input the modified sub-image as an optimized training set into the neural network model for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

[0073] Optionally, the flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including: Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image; The length and width of the flaw detection image are reduced to a maximum integer multiple of the target resolution, and the reduction ratio of the image is determined to obtain a scaled image; The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

[0074] Optionally, performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes: Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram; According to the grayscale histogram, the number of pixels from the minimum grayscale level to the current grayscale level is accumulated to obtain a cumulative distribution function; The cumulative distribution function is normalized and mapped to obtain the pixel value of each gray level after enhancement, an enhanced sub-image is determined according to the pixel value of the enhanced gray level, and the enhanced sub-image is stored in an enhanced image folder.

[0075] Optionally, the method for determining the marker file includes: Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder; Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image; Performing multiple and remainder value processing on the coordinates of the reduced marking frame to determine the number of sub-images covered by the reduced marking frame in the x direction and the y direction; Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marking box in the cropped sub-image; The length and width of the reduced mark frame are judged according to the position judgment condition, the sub-image mark frame coordinates of the target sub-image are generated, and a sub-image mark file of the target sub-image is created based on the sub-image mark frame coordinates, wherein the target sub-image is any image containing the mark frame in the cropped sub-image; After all inner and outer loops are traversed, all generated sub-image tag files are stored in the sub-image XML folder.

[0076] Optionally, a file uncreation module is further included, for: When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes; The original marking file of the target sub-image is determined, the creation of the sub-image marking file is canceled, and the coordinate information of the sub-image marking frame coordinates is added to the original marking file.

[0077] Optionally, the training method of the detection model includes: Traverse the sub-graph XML folder to obtain all the file names to be filtered; Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder; The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target setting value and the training is completed, thereby obtaining a trained detection model.

[0078] Optionally, the model optimization process includes: Inputting the verification set into the detection model to obtain a model labeling file after detection; The model marking file is detected according to the established verification marking software. If a detection error occurs, the position of the marking frame of the model marking file is adjusted to obtain an adjusted marking file; The verification set images are stored in the image folder of the flaw detection images, and the adjusted marker files are stored in the XML folder; When the number of files stored in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are repeated until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

[0079] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0080] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0081] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, which may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the method in the above embodiment.

[0082] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0083] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0084] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0085] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0086] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0087] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0088] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0089] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A deep learning film review model training set feedback optimization method, characterized in that: include: Scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image; Determine the path of the mark file of the flaw detection image, combine the mark file path with the image cropping method, and use the coordinate transformation algorithm to obtain the sub-image mark file of the cropped sub-image; The enhanced sub-images are screened according to the sub-image labeling file, and the screened enhanced sub-images are input into the neural network model as training sets for training to obtain a trained detection model; The detection model is used to obtain the model labeling file corresponding to the verification set, and the model labeling file is detected and modified according to the created verification labeling software to obtain the modified sub-image; The modified sub-image is input as an optimized training set into a neural network model for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

2. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: The flaw detection image of the casting is scaled and cropped to obtain a cropped sub-image, including: Determine an image folder corresponding to an image path of a flaw detection image, traverse all flaw detection images in the image folder, and determine the length and width of the flaw detection image; The length and width of the flaw detection image are reduced to a maximum integer multiple of the target resolution, and the reduction ratio of the image is determined to obtain a scaled image; The zoomed image is cropped using an image cropping algorithm to obtain a cropped sub-image that meets the target resolution.

3. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: Performing image enhancement on the cropped sub-image to obtain an enhanced sub-image includes: Calculate the number of pixels at each gray level in the cropped sub-image to obtain a grayscale histogram; According to the grayscale histogram, the number of pixels from the minimum grayscale level to the current grayscale level is accumulated to obtain a cumulative distribution function; The cumulative distribution function is normalized and mapped to obtain the pixel value of each gray level after enhancement, an enhanced sub-image is determined according to the pixel value of the enhanced gray level, and the enhanced sub-image is stored in an enhanced image folder.

4. The deep learning film evaluation model training set feedback optimization method according to claim 3, characterized in that: The method for determining the marker file comprises: Determine an XML folder of a tag file path, and traverse all XML tag files in the XML folder; Extract the upper left corner coordinates, upper right corner coordinates, lower left corner coordinates and lower right corner coordinates of the mark box of each XML mark file, and obtain the reduced mark box coordinates according to the scaling ratio of the length and width of the flaw detection image; Performing multiple and remainder value processing on the coordinates of the reduced marking frame to determine the number of sub-images covered by the reduced marking frame in the x direction and the y direction; Create an outer loop and an inner loop, and use the outer loop and the memory loop to traverse all possible positions of the reduced marking box in the cropped sub-image; The length and width of the reduced mark frame are judged according to the position judgment condition, the sub-image mark frame coordinates of the target sub-image are generated, and a sub-image mark file of the target sub-image is created based on the sub-image mark frame coordinates, wherein the target sub-image is any image containing the mark frame in the cropped sub-image; After all inner and outer loops are traversed, all generated sub-image tag files are stored in the sub-image XML folder.

5. The deep learning film evaluation model training set feedback optimization method according to claim 4, characterized in that: Also includes: When creating the sub-image mark file, if the file name of the sub-image mark file already exists, determining that the target sub-image has multiple mark boxes; The original marking file of the target sub-image is determined, the creation of the sub-image marking file is canceled, and the coordinate information of the sub-image marking frame coordinates is added to the original marking file.

6. The deep learning film evaluation model training set feedback optimization method according to claim 4, characterized in that: The training method of the detection model includes: Traverse the sub-graph XML folder to obtain all the file names to be filtered; Filter all files in the enhanced image folder based on the file name to be filtered, and store the enhanced sub-images that overlap with the file name to be filtered into a new folder; The enhanced sub-images in the new folder are used as training sets, and the sub-image labeling files in the sub-image XML folder are used as sample labels. The neural network model is iteratively trained until the number of training iterations reaches the target setting value and the training is completed, thereby obtaining a trained detection model.

7. The deep learning film evaluation model training set feedback optimization method according to claim 1, characterized in that: The model optimization process includes: Inputting the verification set into the detection model to obtain a model labeling file after detection; The model marking file is detected according to the established verification marking software. If a detection error occurs, the position of the marking frame of the model marking file is adjusted to obtain an adjusted marking file; The verification set images are stored in the image folder of the flaw detection images, and the adjusted marker files are stored in the XML folder; When the number of files stored in the image folder reaches a preset number, the image scaling, image cropping, image enhancement and model training processes are repeated until there are no detection errors in the model labeling file, thereby obtaining an optimized detection model.

8. A deep learning film evaluation model training set feedback optimization device, characterized in that: include: A cropping and enhancement module, used for scaling and cropping the flaw detection image of the casting to obtain a cropped sub-image, and performing image enhancement on the cropped sub-image to obtain an enhanced sub-image; A marking module, used for determining a marking file path of the flaw detection image, combining the marking file path with an image cropping method, and using a coordinate transformation algorithm to obtain a sub-image marking file of the cropped sub-image; A training module, used to screen the enhanced sub-images according to the sub-image labeling file, and input the screened enhanced sub-images as training sets into a neural network model for training to obtain a trained detection model; A modification module is used to obtain a model tag file corresponding to the verification set by using the detection model, and detect and modify the model tag file according to the created verification tag software to obtain a modified sub-image; The optimization module is used to input the modified sub-image as an optimized training set into the neural network model for training until there is no error in the model tag file detection, thereby obtaining an optimized detection model.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images