Two-stage intelligent analysis method and system for small sample of casting defects

Abstract

This invention provides a two-stage intelligent analysis method and system for small-sample casting defects. The method includes generating morphological text anchors based on image visual features and generating process cause text anchors based on historical process data; mapping class condition reference anchors to a language embedding space; extracting real visual features and unfreezing low-rank fitting parameters in a large language model to obtain an optimal detection model; performing similarity retrieval between the visual features to be detected and knowledge entries based on the optimal detection model to obtain retrieval results and calculating consistency; if consistent, outputting structured results including defect category, evidence description, and confidence level. This invention effectively solves the problems of insufficient samples of similar defects and easy confusion of similar defects under small-sample conditions.

Description

Technical Field

[0001] This invention belongs to the technical field of machine vision inspection, specifically relating to a two-stage intelligent analysis method and system for small samples of casting defects. Background Technology

[0002] With the development of industrial automation and intelligent manufacturing, the requirements for product quality in the industrial production process are constantly increasing. However, during industrial production, due to factors such as processing, design, and equipment failure, defects and damage often appear on the surface of the manufactured products, leading to increased production costs, waste of resources, and in severe cases, even harm to the personal safety of users. Therefore, it is necessary to conduct defect inspection in a timely manner after product production to confirm whether there are foreign objects, defects, or flaws on the surface of components or products.

[0003] Currently, the main defect detection methods include traditional machine vision-based surface defect detection and deep learning-based target detection methods. Traditional methods typically employ conventional image processing algorithms or manually designed features combined with classifiers. However, these methods suffer from low detection efficiency, poor universality, and difficulty in handling challenging industrial inspection scenarios. While deep learning-based methods have improved detection accuracy to some extent, they still face challenges such as difficult data acquisition and limited defect data, resulting in slow detection progress and low accuracy. This severely impacts the assembly speed of large assembly components and the safety performance of high-end equipment.

[0004] Especially in the field of casting defect detection, due to the complexity of the casting production process, the diversity of defect types, and their uneven distribution, traditional visual feature-based detection methods struggle to effectively identify and distinguish these defects. Furthermore, existing multimodal models are prone to class mismatch and insufficient evidence under small sample conditions, making it difficult to accurately identify and locate casting defects.

[0005] Current surface defect detection technologies face multiple challenges: traditional machine vision methods are inefficient and lack versatility; deep learning methods are limited by small sample sizes, data acquisition is difficult, and uneven defect distribution leads to low detection accuracy and slow speed; especially in casting defect detection, multimodal models are prone to class mismatch and insufficient evidence, inaccurate identification of small targets in complex backgrounds, and insufficient semantic information fusion. Existing methods lack efficient and intelligent detection means for small sample scenarios, making it difficult to meet the high-precision and high-efficiency requirements of industrial production. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a two-stage intelligent analysis method and system for small samples of casting defects, thereby overcoming the shortcomings of the prior art.

[0007] In a first aspect, the present invention provides a two-stage intelligent analysis method for small samples of casting defects, the method comprising: Extract image visual features from the pre-training dataset, generate morphology text anchors based on the image visual features, and generate process cause text anchors based on historical process data. Based on the visual features of the images, the morphological text anchors, and the process cause text anchors of similar real images, a global rollback prototype is performed, and weighted fusion normalization is performed to obtain class conditional reference anchors, and the class conditional reference anchors are mapped to the language embedding space. Freeze the main parameters of the visual encoder and the large language model, and update the projection layer parameters; The real casting defect image is input into the frozen visual encoder to extract real visual features. The real visual features are then mapped to the language embedding space through the projection layer. The low-rank adaptation parameters in the large language model are unfrozen, and the target loss is generated to obtain the optimal detection model. Extract the visual features to be detected from the image of the casting to be detected, and perform similarity retrieval between the visual features to be detected and knowledge entries based on the optimal detection model to obtain retrieval results. Calculate the consistency between the retrieval results, the defect category, and the category to which the selected evidence entry belongs. If the retrieval results, the defect category, and the category to which the selected evidence entry belongs are consistent, output the structured results of the defect category, evidence description, and confidence level.

[0008] Compared with existing technologies, the beneficial effects of this invention are as follows: by constructing class-conditional reference anchors and performing two-stage semantic alignment training, the problem of insufficient samples of the same type of defect and easy confusion of similar defects under small sample conditions is effectively solved. The first stage introduces confusion class interval constraints and global backtracking prototypes to improve training stability and sample utilization efficiency. The second stage combines low-rank adaptation and closed vocabulary constraints to enhance semantic fusion and category discrimination capabilities. In the reasoning stage, through evidence retrieval and consistency arbitration, defect conclusions, morphological and process evidence are output, which significantly improves detection accuracy, interpretability and engineering reliability.

[0009] Furthermore, the steps of extracting image visual features from the pre-training dataset, generating morphology text anchors based on the image visual features, and generating process cause text anchors based on historical process data include: A predetermined number of sample images are extracted from the pre-trained dataset, and the visual features of the sample images are extracted using a visual encoder. Based on the visual features of the image, shape text anchors are generated, and based on process experience data and historical data, process cause text anchors are generated.

[0010] Furthermore, the normalization expression is: ; In the formula, Indicates the first Class condition reference anchor point for class defects This represents the normalization operator. , , These represent three different non-negative weight coefficients. Indicates the first Visual prototypes of class defects Indicates the first Embedding of text anchor points for class defects Indicates the first Text anchor embedding of process causes of defects Indicates the first A regression reference prototype for class defects.

[0011] Furthermore, the steps of freezing the main parameters of the visual encoder and the large language model, and updating the projection layer parameters include: Initialize the parameters of the frozen visual encoder and the large language model, and freeze the main parameters of the frozen visual encoder and the large language model; Update the projection layer parameters from vision to language embedding space, and map the class conditional reference anchor point to the language embedding space through the projection layer; Calculate the category margin constraint of the pre-defined confusion defect pair, calculate the modeling loss of the large language model, and perform gradient descent operation. The expression for calculating the pre-defined confusion is: ; This indicates a confusion-type suppression defect. Indicates the current correct category index. This represents the set of defects that are easily confused with type c. This indicates a presupposed set of easily confused defects. Represents cosine similarity. Indicates the first Embedded representation of class defect reference anchor points after projection layer mapping. Indicates the first Class standard category text representation, Indicates the first Class standard category text representation, Indicates the preset interval; The calculation expression for the modeling loss of the large language model is as follows: ; In the formula, This represents the loss in the first stage of language modeling. This refers to language modeling. Indicates the total length of the target sequence. Indicates the time index in the target sequence. This represents the conditional probability distribution given by a large language model. Represents parameters of a large language model. Indicates the first One target token, Indicates at time The previously generated token sequence.

[0012] Furthermore, after the step of unfreezing the low-rank adaptation parameters in the large language model, the method further includes: The projection layer and the low-rank adaptation parameters are jointly updated; Calculate the loss of the generated target under the constraint of a closed defect vocabulary, and perform gradient descent operation.

[0013] Furthermore, after the step of calculating the consistency of the search results, defect categories, and the category to which the selected evidence item belongs, the method further includes: If the search results, the defect category, and the category of the selected evidence item are inconsistent, arbitration output will be triggered based on the difference in similarity, a preset threshold, or a conflict rule. Output the priority review results, along with a structured result including defect conclusions, evidence descriptions, confidence levels, and review flags.

[0014] Secondly, the present invention also provides a two-stage intelligent analysis system for small samples of casting defects, the system comprising: The extraction and generation module is used to extract image visual features from the pre-training dataset, generate morphology text anchors based on the image visual features, and generate process cause text anchors based on historical process data. The fusion mapping module is used to perform global backtracking prototype based on the image visual features, the morphological text anchor points and the process cause text anchor points of the same real image, and to perform weighted fusion normalization to obtain class conditional reference anchor points, and to map the class conditional reference anchor points to the language embedding space. The freeze-update module is used to freeze the main parameters of the visual encoder and the large language model, and update the projection layer parameters. The input extraction module is used to input real casting defect images into the frozen visual encoder to extract real visual features, map the real visual features to the language embedding space through the projection layer, unfreeze the low-rank adaptation parameters in the large language model, and generate target loss to obtain the optimal detection model. The retrieval and calculation module is used to extract the visual features to be detected from the image of the casting to be detected, perform similarity retrieval between the visual features to be detected and knowledge entries based on the optimal detection model to obtain retrieval results, and calculate the consistency of the retrieval results, defect categories and the categories to which the selected evidence entries belong. If the retrieval results, defect categories and the categories to which the selected evidence entries belong are consistent, the structured results of defect category, evidence description and confidence level are output.

[0015] Furthermore, the extraction and generation module includes: An extraction unit is used to extract a preset number of sample images from the pre-trained dataset and extract the image visual features from the sample images through a visual encoder. The generation unit is used to generate morphological text anchors based on the visual features of the image, and to generate process cause text anchors based on process experience data and historical data.

[0016] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the above-mentioned two-stage intelligent analysis method for small samples of casting defects.

[0017] Fourthly, the present invention also provides a storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the above-mentioned two-stage intelligent analysis method for small samples of casting defects. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the two-stage intelligent analysis method for small samples of casting defects in the first embodiment of the present invention. Figure 2 This is a structural block diagram of the two-stage intelligent analysis system for small samples of casting defects in the second embodiment of the present invention; Figure 3This is a schematic diagram of the structure of the electronic device in the third embodiment of the present invention.

[0020] Explanation of key component symbols: 10. Extraction and Generation Module; 20. Fusion Mapping Module; 30. Freeze and Update Module; 40. Input Extraction Module; 50. Retrieval and Calculation Module; 60. Bus; 61. Processor; 62. Memory; 63. Communication interface.

[0021] The embodiments of the present invention will be further described below with reference to the accompanying drawings. Detailed Implementation

[0022] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0023] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0025] Example 1 Please see Figure 1 The figure shows a two-stage intelligent analysis method for small samples of casting defects in the first embodiment of the present invention, the method comprising steps S1 to S5: S1, extract image visual features from the pre-training dataset, generate morphology text anchors based on the image visual features, and generate process cause text anchors based on historical process data. Specifically, step S1 includes steps S11 to S12: S11, Extract a preset number of sample images from the pre-trained dataset, and extract the image visual features from the sample images using a visual encoder; It should be noted that the preset defect categories include porosity, sand holes, shrinkage cavities, shrinkage porosity, cracks, and cold shuts. For each category, a small training set is constructed using 1-shot, 3-shot, or 5-shot methods. A closed defect vocabulary knowledge base is established, containing defect names, morphological descriptions, process causes, and alternative treatment suggestions. Let the total number of preset defect categories be... For the first Class defect, number The training image set for class defects is denoted as First, the visual features of each training sample are extracted using a visual encoder, expressed as: ; in, Indicates the first The training image set of class defects, Indicates the first Class 1 One training sample image, Indicates the first Number of defect samples Represents sample image Visual features This represents the visual encoder. Additionally, a visual prototype for this category is calculated based on features from similar real-world images: ; in, Indicates the first The visual prototype of a class of defects, that is, the mean representation of the visual features of samples of that class. Indicates the first Class 1 Visual features extracted from each sample image by a visual encoder Indicates the first Number of defect samples Indicates the sample index; It is worth noting that in this embodiment, the visual encoder is Chinese-CLIP ViT-L / 14.

[0026] S12, generate morphological text anchor points based on the image visual features, and generate process cause text anchor points based on process experience data and historical data. It should be noted that, according to the first Descriptive text of class defects Text on the origin of the process The text anchors for morphology and process origin are constructed respectively, with the following expressions: ; In the formula, Indicates the first Embedding of text anchor points for class defects This indicates a text encoder.

[0027] S2, based on the image visual features, the morphological text anchor points, and the process cause text anchor points of similar real images, perform global backtracking prototype and perform weighted fusion normalization to obtain class conditional reference anchor points, and map the class conditional reference anchor points to the language embedding space. It should be noted that, in order to supplement visual reference information when the number of similar samples is insufficient, a global fallback prototype is constructed from the visual prototypes of all categories, and the expression is: ; In the formula, This indicates a global rollback to the original prototype. This indicates the total number of preset defect categories. Indicates category index, Indicates the first Visual prototypes of class defects, when the first The number of class samples is lower than a preset threshold In this case, a global fallback prototype is used as a supplement, and the expression is: ; In the formula, Indicates the first A fallback reference prototype for class defects. Indicates the preset sample threshold; when the first sample... When the number of samples of a class is not less than the threshold, the visual prototype of that class is used. When the first When the number of class samples is less than the threshold, a global rollback prototype is used. By weighted and fused visual prototypes, shape text anchors, process origin text anchors, and fallback prototypes, class condition reference anchors are obtained: In this embodiment, the normalization expression is: ; In the formula, Indicates the first Class condition reference anchor point for class defects This represents the normalization operator. , , Let each represent a different non-negative weight coefficient, and satisfy the following conditions: , Indicates the first Visual prototypes of class defects Indicates the first Embedding of text anchor points for class defects Indicates the first Text anchor embedding of process causes of defects Indicates the first A fallback reference prototype for class defects; among which, , Represents a vector The result after L2 normalization Representing vectors The 2-norm.

[0028] S3, freeze the main parameters of the visual encoder and the large language model, and update the projection layer parameters; Specifically, step S3 includes steps S31 to S33: S31, initialize the parameters of the frozen visual encoder and the large language model, and freeze the main parameters of the frozen visual encoder and the large language model; S32, update the projection layer parameters from vision to language embedding space, and map the class conditional reference anchor point to the language embedding space through the projection layer; It should be noted that the first stage freezes the visual encoder. With large language models The main parameters are updated, only the projection layer parameters are updated. Mapping class conditional reference anchors to the language embedding space is expressed as follows: ; In the formula, Indicates the first The language embedding space representation of class reference anchors after projection layer mapping. Represents the projection layer mapping function. Indicates the projection layer parameters. Indicates the first Class condition reference anchor point for class defects , These represent two different weight matrices. , These represent two different bias terms. This represents a nonlinear activation function, preferably GELU or ReLU; in this embodiment, it is GELU. This is used as conditional embedding input to a large language model. Corresponding defect description text is generated. .

[0029] S33, calculate the category interval constraint of the preset confusion defect pair, calculate the modeling loss of the large language model, and perform gradient descent operation; It should be noted that, in order to suppress erroneous proximity between highly similar defect categories, a category spacing constraint is introduced for the predefined set of easily confused defects, assuming... For the first Class standard category text representation, The cosine similarity is expressed as: ; In the formula, Represent two distinct vectors with vector Cosine similarity between them; This represents the transpose of a vector. , This represents any two vectors to be compared; In this embodiment, the calculation expression for the preset obfuscation is: ; This indicates a confusion-type suppression defect. Indicates the current correct category index. This represents the set of defects that are easily confused with type c. This indicates a presupposed set of easily confused defects. Represents cosine similarity. Indicates the first Embedded representation of class defect reference anchor points after projection layer mapping. Indicates the first Class standard category text representation, Indicates the first Class standard category text representation, The preset interval is represented; the expression for the total loss in the first stage is: ; In the formula, This represents the total loss in the first phase. This represents the language modeling loss in the first stage. This represents the weight coefficients. In the first stage, only the projection layer parameters are updated. The process expression is as follows: , Indicates the projection layer parameters. This represents the learning rate in the first stage. Represents the loss function. Indicates to The gradient; thus enabling the projection layer to preferentially learn the cross-modal interface mapping oriented towards casting defect confusion under small sample conditions; The calculation expression for the modeling loss of the large language model is as follows: ; In the formula, This represents the language modeling loss in the first stage. This refers to language modeling. Indicates the total length of the target sequence. Indicates the time index in the target sequence. This represents the conditional probability distribution given by a large language model. Represents parameters of a large language model. Indicates the first One target token, Indicates at time The previously generated token sequence.

[0030] It is worth noting that the main parameters of the visual encoder and the large language model are frozen, and only the projection layer is updated. Preferably, the projection layer is a two-layer multilayer perceptron, with a first-stage learning rate of 2e-4, 20-30 training epochs, and a batch size of 1-4. Class margin constraints are added to address confusion defects such as pores / sand holes, shrinkage cavities / loosening, and cracks / cold shuts.

[0031] S4, input the real casting defect image into the frozen visual encoder to extract real visual features, map the real visual features to the language embedding space through the projection layer, unfreeze the low-rank adaptation parameters in the large language model, and generate the target loss to obtain the optimal detection model. It should be noted that a small number of real casting defect images are input into the frozen visual encoder to extract real visual features, which are then mapped to the language embedding space. The expression is as follows: ; In the formula, Represents a real image Visual features extracted by the visual encoder This represents a visual encoder. Represents the projection layer mapping function. express The language embedding representation after projection, This represents the input image of a real casting defect; simultaneously, it unfreezes the low-rank fitting parameters in the large language model and determines the weight matrix to be fitted. To perform a low-rank update, the expression is: ; ; In the formula, This represents the weight matrix after adding the low-rank fit. This represents the low-rank fitting increment matrix. Represents a low-rank dimension. Indicates the scaling factor. , , Represents the real number field. Indicates the input feature dimension. This indicates the output feature dimension; to reduce the risk of semantic drift in industrial quality inspection scenarios, the second stage optimizes the selection of a closed defect vocabulary. If generative training is performed under constraints, then the expression for the constrained probability distribution of the t-th output token is: ; In the formula, This indicates that under the constraint of a closed defect vocabulary, the first... The conditional probability of each token Indicates time The target output token, Indicates time The previously generated token sequence, This represents the conditional embedding representation input to a large language model. Represents any candidate token in the closed defect vocabulary. This indicates that the large language model is at time 1 / 2. The output is logits, which represents the unnormalized prediction scores before the model outputs probabilities. Indicates a closed defect vocabulary, 1 Represents the characteristic function, Indicates a time index; In this embodiment, the expression for the generation loss in the second stage is: ; The expression for the total loss in the second stage is: ; In the formula, This represents the second-stage generation loss, used to constrain the model to generate defect analysis results within a closed defect vocabulary under the condition of a small number of real casting defect images. This indicates the total length of the target output sequence. This represents the conditional probability distribution of a large language model under the constraint of a closed defect vocabulary. Indicates time The target output token, Indicates at time The previously generated token sequence, This represents the total loss for the second phase. In the current minimal implementable version, the total loss for the second phase consists only of the generation loss, therefore we have: .

[0032] Specifically, step S4 further includes steps S41 to S42: S41, jointly update the projection layer and the low-rank adaptation parameters; S42, calculate the generation target loss under the constraint of closed defect vocabulary, and perform gradient descent operation; It should be noted that the projection layer parameters are updated jointly. and low-rank adaptation parameters The expression is: ; ; In the formula, Indicates the projection layer parameters. Indicates the low-rank adaptation parameters. This indicates the learning rate in the second stage. Indicates the total loss in the second phase. For projection layer parameters gradient, Indicates the total loss in the second phase. For projection layer parameters The gradient is then calculated. It's worth noting that the visual encoder is frozen, while the low-rank adaptation parameters in the large language model are unfrozen, and the projection layer and low-rank adaptation parameters are jointly updated. Preferably, the low-rank dimension r is 16 or 32, the second-stage learning rate is 5e-5 to 1e-4, and the number of training epochs is 10 to 20. The output target is limited to the format "defect type—evidence—recommendation".

[0033] S5, extract the visual features to be detected from the image of the casting to be detected, perform similarity retrieval between the visual features to be detected and the knowledge entries based on the optimal detection model to obtain the retrieval results, and calculate the consistency of the retrieval results, the defect category and the category to which the selected evidence entry belongs. If the retrieval results, the defect category and the category to which the selected evidence entry belongs are consistent, then output the structured results of the defect category, evidence description and confidence level. Furthermore, step S5 also includes steps S51 to S52: S51, if the search results, the defect category, and the category to which the selected evidence item belongs are inconsistent, then arbitration output is triggered based on the difference in similarity, a preset threshold, or a conflict rule; S52 outputs the review priority results, and outputs the structured results of defect conclusions, evidence descriptions, confidence levels, and review tags; It should be noted that the image of the casting to be inspected Extract the visual features to be detected, expressed as: ; Suppose that the first in the knowledge base The text embedding representation of each knowledge entry is as follows Then, the expression for the similarity score between the image to be examined and the knowledge item is: ; In the formula, Indicates the first The text embedding representation of each knowledge entry, Indicates a text encoder. Indicates the image to be inspected and the first Similarity score of knowledge items, The similarity function is preferably cosine similarity. Visual features representing the image of the casting to be inspected; The top-1 category retrieved is expressed as: , This represents the top-1 defect category retrieved; the defect category parsed from the generated text is denoted as... The category to which the evidence item belongs is The expression for defining the consistency indicator variable is: ; In the formula, This indicates a consistency indicator variable, meaning that when the retrieval category, generation category, and evidence category are consistent, =1, This indicates other cases that do not meet the above consistency conditions; let the top two similarity scores be respectively... and The expression for the similarity difference is: ; In the formula, This represents the similarity difference; the expression for outputting the confidence score is: ; In the formula, Indicates the confidence level. , , These represent the weights of top-1 search similarity, consistency indicator, and similarity difference, respectively. Indicates a consistency indicator variable; based on a preset arbitration threshold The final output result is expressed as: ; Thus, the reasoning results form a structured analysis outcome of "defective conclusion - evidence explanation - confidence level - verification mark".

[0034] It is worth noting that similarity retrieval is performed based on visual features and knowledge item embedding to obtain the top-1 category and corresponding evidence. Subsequently, the consistency relationship between the retrieved category, the generated category, and the category to which the evidence belongs is compared. When they are consistent, they are directly output. When they are inconsistent, arbitration is triggered based on the similarity difference and a review mark is given.

[0035] In summary, the two-stage intelligent analysis method for casting defects with small samples in the above embodiments of the present invention improves the ability to distinguish similar categories of casting defects under small sample conditions by constructing class-conditional reference anchors and training with confusion class constraints, and reduces the false alarm rate and false negative rate among easily confused defects such as porosity, sand holes, shrinkage cavities, shrinkage porosity, cracks, and cold shuts. By freezing the main parameters of the visual encoder and the large language model in the first stage, updating only the projection layer parameters, and introducing global backtracking prototypes and class interval constraints, the model can obtain a more stable cross-modal alignment process under insufficient sample conditions, improving the stability of small sample training and sample utilization efficiency. Through evidence retrieval and consistency arbitration output under a closed defect vocabulary, the system outputs corresponding morphological evidence, process cause evidence, and verification prompts while giving the defect category, thereby improving the interpretability, verifiability, and engineering reliability of the analysis results. Without changing the existing mainstream visual encoder and large language model infrastructure, it can be implemented by only adding class-conditional reference anchor construction, confusion class suppression constraints, low-rank adaptation semantic calibration, and a closed vocabulary consistency arbitration module, which is particularly suitable for industrial production environments with limited resources.

[0036] Example 2 Please see Figure 2 The figure shows a two-stage intelligent analysis system for small samples of casting defects according to the second embodiment of the present invention. The system includes: The extraction and generation module 10 is used to extract image visual features from the pre-training dataset, generate morphology text anchors based on the image visual features, and generate process cause text anchors based on historical process data. The fusion mapping module 20 is used to perform global backtracking to a prototype based on the image visual features, the morphological text anchor points, and the process origin text anchor points of similar real images, and to perform weighted fusion normalization to obtain class conditional reference anchor points, and to map the class conditional reference anchor points to the language embedding space, wherein the normalization expression is: ; In the formula, Indicates the first Class condition reference anchor point for class defects This represents the normalization operator. , , These represent three different non-negative weight coefficients. Indicates the first Visual prototypes of class defects Indicates the first Embedding of text anchor points for class defects Indicates the first Text anchor embedding of process causes of defects Indicates the first A fallback prototype for class defects; The freeze-update module 30 is used to freeze the main parameters of the visual encoder and the large language model, and update the projection layer parameters. The input extraction module 40 is used to input the real casting defect image into the frozen visual encoder to extract the real visual features, map the real visual features to the language embedding space through the projection layer, unfreeze the low-rank adaptation parameters in the large language model, and generate the target loss to obtain the optimal detection model. The retrieval and calculation module 50 is used to extract the visual features to be detected from the image of the casting to be detected, perform similarity retrieval between the visual features to be detected and knowledge entries based on the optimal detection model to obtain retrieval results, and calculate the consistency of the retrieval results, defect categories and the category to which the selected evidence entries belong. If the retrieval results, defect categories and the category to which the selected evidence entries belong are consistent, the structured results of defect category, evidence description and confidence level are output.

[0037] In some optional embodiments, the extraction and generation module 10 includes: An extraction unit is used to extract a preset number of sample images from the pre-trained dataset and extract the image visual features from the sample images through a visual encoder. The generation unit is used to generate morphological text anchors based on the visual features of the image, and to generate process cause text anchors based on process experience data and historical data.

[0038] In some alternative embodiments, the freeze update module 30 includes: An initialization unit is used to initialize the parameters of the frozen visual encoder and the large language model, and to freeze the main parameters of the frozen visual encoder and the large language model. The update mapping unit is used to update the projection layer parameters from vision to language embedding space, and to map the class conditional reference anchor point to the language embedding space through the projection layer; The computational unit is used to calculate the category margin constraint of the preset confusion defect pair, calculate the modeling loss of the large language model, and perform gradient descent operation, wherein the calculation expression for the preset confusion is: ; This indicates a confusion-type suppression defect. Indicates the current correct category index. This represents the set of defects that are easily confused with type c. This indicates a presupposed set of easily confused defects. Represents cosine similarity. Indicates the first Embedded representation of class defect reference anchor points after projection layer mapping. Indicates the first Class standard category text representation, Indicates the first Class standard category text representation, Indicates the preset interval; The calculation expression for the modeling loss of the large language model is as follows: ; In the formula, This represents the loss in the first stage of language modeling. This refers to language modeling. Indicates the total length of the target sequence. Indicates the time index in the target sequence. This represents the conditional probability distribution given by a large language model. Represents parameters of a large language model. Indicates the first One target token, Indicates at time The previously generated token sequence.

[0039] In some alternative embodiments, the input extraction module 40 includes: A joint update unit is used to jointly update the projection layer and the low-rank adaptation parameters; The computational execution unit is used to calculate the generation target loss under the constraint of a closed defect vocabulary and to perform gradient descent operations.

[0040] In some alternative embodiments, the retrieval calculation module 50 includes: The first output unit is used to determine whether the retrieval result, the defect category, and the category to which the selected evidence item belongs are inconsistent. If so, arbitration output is triggered based on the difference in similarity, a preset threshold, or a conflict rule. The second output unit is used to output the priority review results, as well as the structured results including defect conclusions, evidence descriptions, confidence levels, and review markers.

[0041] The functions or operation steps implemented by the above modules and units are largely the same as those in the above method embodiments, and will not be repeated here.

[0042] The two-stage intelligent analysis system for small samples of casting defects provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0043] Example 3 The third embodiment of the present invention also proposes an electronic device, please refer to [link / reference]. Figure 3 The image shows an electronic device according to a third embodiment of the present invention.

[0044] The electronic device may include a processor 61 and a memory 62 storing computer program instructions.

[0045] Specifically, the processor 61 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the present application.

[0046] The memory 62 may include a large-capacity storage device for data or instructions. For example, and not limitingly, the memory 62 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 62 may include removable or non-removable (or fixed) media. Where appropriate, the memory 62 may be internal or external to a data processing device. In a particular embodiment, the memory 62 is non-volatile memory. In a particular embodiment, the memory 62 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0047] The memory 62 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 61.

[0048] The processor 61 reads and executes the computer program instructions stored in the memory 62 to implement the two-stage intelligent analysis method for small samples of casting defects in Embodiment 1 described above.

[0049] In some embodiments, the electronic device may further include a communication interface 63 and a bus 60. For example, Figure 3 As shown, the processor 61, memory 62, and communication interface 63 are connected through bus 60 and complete communication with each other.

[0050] The communication interface 63 is used to enable communication between the various modules, devices, units, and / or equipment in this application. The communication interface 63 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0051] Bus 60 includes hardware, software, or both, that couples components of a device together. Bus 60 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 60 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 60 may include one or more buses. Although this application describes and illustrates a specific bus, this application considers any suitable bus or interconnection.

[0052] The electronic device can acquire a two-stage intelligent analysis system for small samples of casting defects and execute the two-stage intelligent analysis method for small samples of casting defects in this embodiment.

[0053] Furthermore, in conjunction with the two-stage intelligent analysis method for small samples of casting defects in Embodiment 1 above, this application can provide a storage medium for implementation. This storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement the two-stage intelligent analysis method for small samples of casting defects in Embodiment 1 above.

[0054] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A two-stage intelligent analysis method for small samples of casting defects, characterized in that, The method includes: Extract image visual features from the pre-training dataset, generate morphology text anchors based on the image visual features, and generate process cause text anchors based on historical process data. Based on the visual features of the images, the morphological text anchors, and the process cause text anchors of similar real images, a global rollback prototype is performed, and weighted fusion normalization is performed to obtain class conditional reference anchors, and the class conditional reference anchors are mapped to the language embedding space. Freeze the main parameters of the visual encoder and the large language model, and update the projection layer parameters; The real casting defect image is input into the frozen visual encoder to extract real visual features. The real visual features are then mapped to the language embedding space through the projection layer. The low-rank adaptation parameters in the large language model are unfrozen, and the target loss is generated to obtain the optimal detection model. Extract the visual features to be detected from the image of the casting to be detected, and perform similarity retrieval between the visual features to be detected and knowledge entries based on the optimal detection model to obtain retrieval results. Calculate the consistency between the retrieval results, the defect category, and the category to which the selected evidence entry belongs. If the retrieval results, the defect category, and the category to which the selected evidence entry belongs are consistent, output the structured results of the defect category, evidence description, and confidence level.

2. The two-stage intelligent analysis method for small samples of casting defects according to claim 1, characterized in that, The steps of extracting image visual features from the pre-training dataset, generating morphology text anchors based on the image visual features, and generating process cause text anchors based on historical process data include: A predetermined number of sample images are extracted from the pre-trained dataset, and the visual features of the sample images are extracted using a visual encoder. Based on the visual features of the image, shape text anchors are generated, and based on process experience data and historical data, process cause text anchors are generated.

3. The two-stage intelligent analysis method for small samples of casting defects according to claim 1, characterized in that, The normalization expression is: ； In the formula, Indicates the first Class condition reference anchor point for class defects This represents the normalization operator. , , These represent three different non-negative weight coefficients. Indicates the first Visual prototypes of class defects Indicates the first Embedding of text anchor points for class defects Indicates the first Text anchor embedding of process causes of defects Indicates the first A regression reference prototype for class defects.

4. The two-stage intelligent analysis method for small samples of casting defects according to claim 1, characterized in that, The steps of freezing the main parameters of the visual encoder and the large language model, and updating the projection layer parameters include: Initialize the parameters of the frozen visual encoder and the large language model, and freeze the main parameters of the frozen visual encoder and the large language model; Update the projection layer parameters from vision to language embedding space, and map the class conditional reference anchor point to the language embedding space through the projection layer; Calculate the category margin constraint of the pre-defined confusion defect pair, calculate the modeling loss of the large language model, and perform gradient descent operation. The expression for calculating the pre-defined confusion is: ； This indicates a confusion-type suppression defect. Indicates the current correct category index. This represents the set of defects that are easily confused with type c. This indicates a presupposed set of easily confused defects. Represents cosine similarity. Indicates the first Embedded representation of class defect reference anchor points after projection layer mapping. Indicates the first Class standard category text representation, Indicates the first Class standard category text representation, Indicates the preset interval; The calculation expression for the modeling loss of the large language model is as follows: ； In the formula, This represents the loss in the first stage of language modeling. This refers to language modeling. Indicates the total length of the target sequence. Indicates the time index in the target sequence. This represents the conditional probability distribution given by a large language model. Represents parameters of a large language model. Indicates the first One target token, Indicates at time The previously generated token sequence.

5. The two-stage intelligent analysis method for small samples of casting defects according to claim 1, characterized in that, After the step of unfreezing and refreezing the low-rank adaptation parameters in the large language model, the method further includes: The projection layer and the low-rank adaptation parameters are jointly updated; Calculate the target loss under the constraint of a closed defect vocabulary and perform gradient descent operation.

6. The two-stage intelligent analysis method for small samples of casting defects according to claim 1, characterized in that, After the step of calculating the consistency of the search results, defect categories, and the category to which the selected evidence item belongs, the method further includes: If the search results, the defect category, and the category of the selected evidence item are inconsistent, arbitration output will be triggered based on the difference in similarity, a preset threshold, or a conflict rule. Output the priority review results, along with a structured result including defect conclusions, evidence descriptions, confidence levels, and review flags.

7. A two-stage intelligent analysis system for small samples of casting defects, characterized in that, The system includes: The extraction and generation module is used to extract image visual features from the pre-training dataset, generate morphology text anchors based on the image visual features, and generate process cause text anchors based on historical process data. The fusion mapping module is used to perform global backtracking prototype based on the image visual features, the morphological text anchor points and the process cause text anchor points of the same real image, and to perform weighted fusion normalization to obtain class conditional reference anchor points, and to map the class conditional reference anchor points to the language embedding space. The freeze-update module is used to freeze the main parameters of the visual encoder and the large language model, and update the projection layer parameters. The input extraction module is used to input real casting defect images into the frozen visual encoder to extract real visual features, map the real visual features to the language embedding space through the projection layer, unfreeze the low-rank adaptation parameters in the large language model, and generate target loss to obtain the optimal detection model. The retrieval and calculation module is used to extract the visual features to be detected from the image of the casting to be detected, perform similarity retrieval between the visual features to be detected and knowledge entries based on the optimal detection model to obtain retrieval results, and calculate the consistency of the retrieval results, defect categories and the categories to which the selected evidence entries belong. If the retrieval results, defect categories and the categories to which the selected evidence entries belong are consistent, the structured results of defect category, evidence description and confidence level are output.

8. The two-stage intelligent analysis system for small samples of casting defects according to claim 7, characterized in that, The extraction and generation module includes: An extraction unit is used to extract a preset number of sample images from the pre-trained dataset and extract the image visual features from the sample images through a visual encoder. The generation unit is used to generate morphological text anchors based on the visual features of the image, and to generate process cause text anchors based on process experience data and historical data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the two-stage intelligent analysis method for small samples of casting defects as described in any one of claims 1 to 6.

10. A storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the two-stage intelligent analysis method for small samples of casting defects as described in any one of claims 1 to 6.

Images