Precision lead frame design drawing retrieval method and device

Abstract

The invention discloses a precision lead frame design drawing retrieval method and device, and the method comprises the steps: inputting a target precision lead frame design drawing needing to be retrieved into an asymmetric depth self-encoder feature extraction model, and obtaining the local features and global features of the target precision lead frame design drawing; fusing the local features and the global features of the target precision lead frame design drawing, and performing generalized mean pooling on an obtained fusion result to obtain a fusion feature vector; and based on the fusion feature vector, obtaining the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing. By adopting the scheme, the precision and speed of the precision lead frame design drawing retrieval can be improved.

Description

Technical Field

[0001] The present invention relates to the field of lead frame design services, and in particular to a method and device for retrieving a precision lead frame design drawing. Background Art

[0002] In the field of leadframe design, high-precision, high-density, and wide-width leadframe design has become a key development direction in recent years. The rapid development and in-depth application of artificial intelligence technology has led to a rapid increase in demand for the design and manufacturing of semiconductors, such as high-performance heterogeneous computing chips. This has brought significant opportunities and challenges to chip design and manufacturing processes, such as leadframe design. Although major leadframe design and manufacturing companies have accumulated a large number of leadframe design models, they are currently stored and managed in a file system, which makes it impossible to retrieve the design model content. Leadframe design engineers often need to spend a lot of time and effort to complete repetitive design content. This not only wastes valuable manpower and resources, but also makes it difficult to improve the efficiency of leadframe model design.

[0003] Leadframes are the fundamental material for semiconductor chip packaging and the chip carrier for integrated circuits. Using bonding materials such as copper, gold, and aluminum wires, they connect the chip's internal circuit terminals to external leads, making them a key structural component for connecting semiconductor chips to external wiring. Designing high-precision, high-density, and wide leadframes is a challenging task, requiring design engineers to possess extensive leadframe design experience to master the complex architecture, high element density, and high precision requirements. Furthermore, compared to previous low-density leadframe models, new high-density leadframes involve a greater number of elements, requiring designers to focus their efforts and spend several times more time to complete a high-quality high-density leadframe model. A high-precision leadframe design drawing retrieval method based on content features enables leadframe design work to adopt a new working model based on fine-tuning the most similar existing design model. This significantly reduces the manpower and material resources required for leadframe model design, as well as potential design errors, thereby enabling a cost-effective and efficient new high-precision leadframe design process.

[0004] There are two potential technical approaches for lead frame design retrieval based on deep learning: supervised and unsupervised. Supervised deep learning methods require a dataset with labeled information to train the model, but manually labeling lead frame design drawings is a very time-consuming task. Unsupervised deep learning methods do not require manual labeling of existing lead frame design drawings. Dataset collection can be completed directly through simple and efficient data deduplication and image sample organization. Furthermore, when the model needs to be updated on new lead frame design drawings generated subsequently, manual labeling is also unnecessary. In contrast, unsupervised deep learning methods are more suitable for implementing content-based lead frame design retrieval methods.

[0005] Existing unsupervised deep learning methods have slow retrieval speed and poor accuracy when retrieving lead frame design drawings. Summary of the Invention

[0006] An object of the embodiments of the present invention is to provide a method for retrieving a precision lead frame design drawing, which can improve the accuracy and speed of retrieving a precision lead frame design drawing.

[0007] In a first aspect, the present invention provides a method for retrieving a precision lead frame design drawing, comprising: inputting a target precision lead frame design drawing to be retrieved into an asymmetric deep autoencoder feature extraction model to obtain local features and global features of the target precision lead frame design drawing; fusing the local features and global features of the target precision lead frame design drawing, performing generalized mean pooling on the obtained fusion result, and obtaining a fusion feature vector; based on the fusion feature vector, obtaining the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing.

[0008] Optionally, the local features and global features of the target precision lead frame design drawing are fused to obtain the fusion result; the fusion result is generalized mean pooled to obtain a fusion feature vector, including: adaptively average pooling the local features to obtain pooled local features with the same size as the global features; and fusing the pooled local features with the global features to obtain the fusion feature vector.

[0009] Optionally, based on the fused feature vector, the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing is obtained, including: concatenating the fused feature vector and the global feature in series; performing generalized mean pooling on the concatenated result; and calculating the Euclidean distance between the concatenated result after generalized mean pooling and the feature vector of each precision lead frame design drawing in the resource library.

[0010] Optionally, the asymmetric deep autoencoder model is trained using the following steps: obtaining a precision lead frame design drawing for model training; denoising the precision lead frame design drawing for model training, scaling the denoised precision lead frame design drawing to obtain sample images, and all sample images constitute a data set; the data set includes a training set, a validation set, and a test set; based on the data set, a pre-established initial asymmetric deep autoencoder is trained to obtain the asymmetric deep autoencoder feature extraction model.

[0011] Optionally, the denoising process of the precision lead frame design drawing used for model training includes: using a multi-scale Harris corner detection algorithm to detect the area where the precision lead frame design content is located in the precision lead frame design drawing used for model training; cropping the area where the precision lead frame design content is located from the precision lead frame design drawing used for model training to obtain the sample image.

[0012] Optionally, the scaling processing of the sample images in the data set includes: adopting a convolution resampling method based on a Bicubic kernel to scale all images in the data set.

[0013] Optionally, the initial asymmetric deep autoencoder is an asymmetric autoencoder model based on SwinTransformer.

[0014] Optionally, the pre-established initial asymmetric deep autoencoder is trained based on the scaled data set, including: using a mask image modeling method and an unsupervised training method, taking the samples in the training set as input, and pre-training the initial asymmetric deep autoencoder model; adjusting the pre-trained asymmetric deep autoencoder based on the validation set and the test set to obtain the asymmetric deep autoencoder feature extraction model.

[0015] In the second aspect, the present invention also provides a precision lead frame design drawing retrieval device, including: a feature acquisition unit, used to input the target precision lead frame design drawing to be retrieved into an asymmetric deep autoencoder feature extraction model to obtain local features and global features of the target precision lead frame design drawing; a feature fusion unit, used to fuse the local features and global features of the target precision lead frame design drawing, and perform generalized mean pooling on the obtained fusion result to obtain a fusion feature vector; a similarity calculation unit, used to obtain the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing based on the fusion feature vector.

[0016] The embodiments of the present invention have the following beneficial effects:

[0017] The local and global features of the target precision lead frame design drawing are obtained through an asymmetric deep autoencoder feature extraction model. By fusing local and global features, the extracted deep features can represent both the overall design content information of the precision lead design drawing and the local detail information of its key areas, thereby improving the accuracy of precision lead design frame drawing retrieval.

[0018] Furthermore, a multi-scale Harris corner detection method is used to determine the image region containing the precision lead frame design content. By combining the Harris corner detection operator with a Gaussian scale-space representation, accurate corner detection with grayscale invariance, rotation invariance, and scale invariance is achieved, improving the accuracy of extracting the image region containing the precision lead frame design content.

[0019] Furthermore, an image mask modeling method was used to implement unsupervised pre-training of a deep autoencoder based on the Swin Transformer. Based on the pre-trained model, a feature extractor for extracting both local and global features of precision lead frame design drawings was fine-tuned. This enabled the unsupervised feature extractor to perform at the same level as a supervised extractor trained on a large sample dataset, meeting the feature extraction requirements for precision lead frame design retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a precision lead frame design drawing retrieval method according to an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the area determined from the original precision lead frame design is given;

[0022] Figure 3 A precision lead frame design with blank edge areas removed is given;

[0023] Figure 4 Given the Figure 3 A schematic diagram of corner point distribution obtained by performing corner point detection on the schematic diagram of the precision lead frame in FIG;

[0024] Figure 5 The precision lead frame design after denoising is given;

[0025] Figure 6 A schematic structural diagram of a deep autoencoder according to an embodiment of the present invention is provided;

[0026] Figure 7 A schematic diagram of an application of a mask image modeling method in an embodiment of the present invention is provided;

[0027] Figure 8A schematic diagram of the structure of a paired Swin Transformer block in an embodiment of the present invention is provided;

[0028] Figure 9 A schematic diagram of a feature fusion method in an embodiment of the present invention is provided;

[0029] Figure 10 A schematic diagram of a target precision lead frame design drawing to be queried in an embodiment of the present invention is given;

[0030] Figure 11 A schematic diagram of the output results retrieved based on the target precision lead frame design drawing is given;

[0031] Figure 12 A structural schematic diagram of a precision lead frame design drawing retrieval device in an embodiment of the present invention is given. DETAILED DESCRIPTION

[0032] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0033] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0034] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form.

[0035] The embodiment of the present invention provides a method for retrieving a precision lead frame design drawing, referring to Figure 1 , the following is a detailed description through specific steps.

[0036] In step 101 , a target precision lead frame design drawing to be retrieved is input into an asymmetric deep autoencoder feature extraction model to obtain local features and global features of the target precision lead frame design drawing.

[0037] In step 102 , the local features and global features of the target precision lead frame design drawing are fused, and the obtained fusion result is subjected to generalized mean pooling to obtain a fusion feature vector.

[0038] Step 103 : Based on the fused feature vector, obtain the similarity between each precision lead frame design drawing in the preset resource library and the target precision lead frame design drawing.

[0039] In an embodiment of the present invention, a precision lead frame design drawing for model training can be obtained in advance. After preprocessing the precision lead frame design drawing for model training, corresponding sample images can be obtained, and the sample images are used for model training.

[0040] In specific implementations, precision lead frame design drawings are usually generated from bitmap files of fixed size (e.g., A4 size) printed out by drawing software (e.g., AutoCAD), rather than vector graphics files. The precision lead frame components involved in the precision lead frame design drawings include component shapes and their size identification, layout schemes, design unit information, etc. However, different units may design similar precision lead frames, and the retrieval of precision lead frame design drawings should not be affected by the design unit information. In other words, the design unit information in the precision lead frame design drawings is redundant noise information and needs to be removed from the precision lead frame design drawings.

[0041] In an embodiment of the present invention, after obtaining the precision lead frame design drawing for model training, the obtained precision lead frame design drawing may be first subjected to denoising processing.

[0042] In a specific implementation, the minimum bounding rectangle (MBR) algorithm can be used to determine the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2) of the minimum bounding rectangle in the original precision lead frame design. In other words, the coordinate range of the original precision lead frame design is actually (x1-x2, y2-y1).

[0043] In some embodiments, if the detected minimum bounding rectangle is directly used as the non-blank area of ​​the precision lead frame design drawing, some edge pixels may be cropped, which may impair the accuracy of subsequent corner point detection.

[0044] To avoid the above problem, in this embodiment of the present invention, an offset is defined. The upper left corner coordinate x1 is shifted leftward and the coordinate y1 is shifted upward by offset pixels, respectively, to form the upper left corner coordinate (x1-offset, y1-offset) of the non-blank area containing the precision lead frame design content. Similarly, the lower right corner coordinate x2 is shifted rightward and the coordinate y2 is shifted downward by offset pixels, respectively, to form the lower right corner coordinate (x2+offset, y2+offset) of the non-blank area containing the precision lead frame design content.

[0045] Finally, based on the area determined by the coordinates (x1-offset, y1-offset) and the coordinates (x2+offset, y2+offset), the precision lead frame design drawing with the blank edge area removed is cropped from the input original precision lead frame design drawing.

[0046] Reference Figure 2 , a schematic diagram of the area determined from the original precision lead frame design drawing is given. Figure 2 In the , the area determined from the original precision lead frame design includes blank edges due to the added offset. Figure 3 , a precision lead frame design diagram with blank edge areas removed is given.

[0047] After obtaining a precision lead frame design drawing with blank edge areas removed, a multi-scale Harris corner detection algorithm is used to detect corner points. To improve the detection speed and accuracy of the area containing the precision lead frame design content, a corner area ratio threshold r is defined. The corner points in the upper left corner area formed by the coordinates (0,0) and the coordinates (w*r,h*r), and the corner points in the right area formed by the coordinates (ww*r,0) and the coordinates (w,h), are used as the basis for determining the content area of ​​the precision lead frame design drawing. The upper left corner coordinates of the precision lead frame design content area are determined based on the corner points in the upper left corner area. The lower right corner coordinates of the precision lead frame design content area are determined based on the offset ratio between the corner points in the right area and the customized lower right noise area. Based on the above upper left and lower right corner coordinates of the precision lead frame design content area, the noise-removed precision lead frame schematic is cropped.

[0048] In a specific implementation, the values ​​of the offset, the corner area ratio threshold r, the offset ratio y_offset, etc. can be set according to specific application requirements. The specific values ​​do not affect the protection scope of the embodiments of the present invention.

[0049] In some embodiments, the value of the offset may be 6, the value of the corner area ratio threshold r may be 0.1, and the value of the offset ratio y_offset may be 0.78.

[0050] In some other embodiments, the value of the offset may be 4, the value of the corner area ratio threshold r may be 0.15, the value of the offset ratio y_offset may be 0.75, and so on.

[0051] Reference Figure 4 , gives the Figure 3 The corner point distribution diagram obtained by corner point detection based on the precision lead frame schematic in . Figure 5, the precision lead frame design diagram after denoising is given.

[0052] The specific process of performing the denoising process based on the corner points is described below.

[0053] Step 1) In the precision lead frame design drawing with the blank edge area removed, find the corner point with the smallest x-axis and y-axis coordinate values ​​as the initial coordinates of the upper left corner (tx, ty);

[0054] Step 2) traverse the corner points ((ltx, lty) of the upper left corner area. If ltx>tx and lty>ty, then tx=ltx and ty=lty;

[0055] Step 3) Find the corner point with the largest x-axis and y-axis coordinate values ​​and obtain the initial coordinates of the lower right corner (bx, by);

[0056] Step 4) Find the corner point with a y-axis coordinate value of by; if such a corner point exists, use by as the initial value of the y-axis coordinate of the lower right corner; otherwise, use the width w of the precision lead frame design as the initial by value; at the same time, use the x-axis coordinate value of the corner point with a y-axis coordinate equal to by as the initial bx;

[0057] Step 5), set the corner point coordinate offset and offset ratio y_offset;

[0058] In some embodiments, the corner point coordinate offset is set to 6, and the offset ratio y_offset is set to 0.78.

[0059] Step 6) Find the corner point with the largest y-axis coordinate value among the corner points in the coordinate area (bx-offset,by) and the coordinate area (bx+offset,h*(1-y_offset)), and use its coordinate value as the updated lower right corner coordinate (bx,by);

[0060] In step 7), the precision lead frame design drawing after denoising is obtained based on the expanded upper left corner coordinates (tx-offset, ty-offset) and lower right corner coordinates (bx+offset, by+offset).

[0061] In a specific implementation, the precision lead frame design drawing after denoising may contain duplicates or incomplete design content areas. Therefore, after obtaining the precision lead frame design drawing after denoising, duplicate samples, samples with incomplete design content areas, and samples with sizes smaller than a predetermined value can be deleted. The precision lead frame design drawing obtained after screening is scaled to obtain sample images. The sample images constitute a data set. The obtained data set can be divided into a training set, a validation set, and a test set in a ratio of 6:2:2. By scaling the precision lead frame design drawing obtained after screening, the geometric distortion problem of the lines in the precision lead frame design drawing can be avoided.

[0062] In some embodiments, the predetermined value may be 384 × 384. In other embodiments, the predetermined value may be other values.

[0063] Image retrieval involves searching for the most similar candidate images in a pre-set image library based on an input query image. During image retrieval, image features can be extracted and converted into lower-dimensional feature vectors to reduce the computational complexity of image retrieval.

[0064] In an embodiment of the present invention, an asymmetric deep autoencoder feature extraction model can be established to obtain global features and local features of a precision lead frame design drawing.

[0065] In a specific implementation, the global features of a precision lead frame design drawing may include all content information of the precision lead frame design drawing, such as the position, outline, and texture of each object in the precision lead frame design drawing. Using these global features, it is possible to quickly find candidate images that are most similar to the object's outline, position, and texture.

[0066] The local features of the precision lead frame design can be used to characterize the content information of the local area of ​​the precision lead frame design. Through the local features, the local key details of the candidate images can be compared.

[0067] In other words, the most similar candidate images can be quickly screened out by using global features, and identical images can be screened out by comparing details using local features.

[0068] By fusing global features with local features and considering both the global information and key information of local areas of the precision lead frame design drawing, the retrieval accuracy of the precision lead frame design drawing can be improved.

[0069] In an embodiment of the present invention, in order to effectively extract local features and global features of a precision lead frame design drawing, a deep feature extraction model can be designed based on a deep autoencoder with an asymmetric structure and a masked image modeling (MIM) method.

[0070] In a specific implementation, the deep autoencoder consists of an encoder for extracting local and global features and a decoder for reconstructing the input image. The encoder includes four submodules: the first consists of a patch merging layer and two Swin Transformer blocks; the second consists of a patch merging layer and two Swin Transformer blocks; the third consists of a patch merging layer and 18 Swin Transformer blocks; and the fourth consists of a patch merging layer and two Swin Transformer blocks. The decoder consists of eight Swin Transformer blocks.

[0071] It is understandable that the number of Swin Transformer blocks in each of the above submodules may also be other values, such as 4, 6, or 8. The number of Swin Transformer blocks in the first submodule, the second submodule, and the fourth submodule may be equal, and the number of Swin Transformer blocks in the third submodule may be greater than that in the first submodule, the second submodule, and the fourth submodule.

[0072] Reference Figure 6 , a structural diagram of a deep autoencoder in an embodiment of the present invention is given. Figure 6 In the embodiment, the deep autoencoder includes an encoder 61 and a decoder 62. The encoder 61 includes a first submodule 611, a second submodule 612, a third submodule 613, and a fourth submodule 614. The input end of the first submodule 611 inputs the precision lead frame design drawing, and the output end of the first submodule 611 is coupled to the input end of the second submodule 612. The output end of the second submodule 612 outputs local features of the precision lead frame design drawing. The output end of the second submodule 612 is also coupled to the input end of the third submodule 613. The output end of the third submodule 613 is coupled to the input end of the fourth submodule 614. The output end of the fourth submodule 614 is coupled to the input end of the decoder 62, and the output end of the fourth submodule 614 outputs all features of the precision lead frame design drawing.

[0073] In specific implementations, the input mask image mixing method can be used to accelerate the model pre-training process.

[0074] In some embodiments, the input precision lead frame design drawing can be segmented into fixed-size image blocks without pixel duplication, and then some image blocks are randomly selected from different precision lead frame design drawings to form a mixed mask image as an input training sample.

[0075] Specifically, the two input precision lead frame design drawings can be segmented into image blocks of 32 × 32 with no pixel duplication. Then, two image blocks are randomly selected from each of the two input precision lead frame design drawings to form a mixed mask image.

[0076] Assume that the two input image blocks of precision lead frame design drawings are and A binary mask M is used to mask the two images, and the resulting mixed mask image is:

[0077]

[0078] In the above formula (1), ⊙ represents pixel-by-pixel multiplication, the binary mask M takes the value of {0,1}, and p represents the p-th image block. Represents the pth mask image block in the mixed mask image. M=1 represents the selected image block As the pth mask image block in the mixed mask image, M=0 means selecting the image block As the p-th mask image block in the mixed mask image.

[0079] The encoder outputs features of the two input precision lead frame designs. These features are decomposed according to the sequence of binary mask values, M, to obtain the features of each precision lead frame design. The decoder then reconstructs the two input precision lead frame designs. This allows the reconstruction of two complete precision lead frame designs simultaneously using input image features with a 50% masking rate.

[0080] Since the pre-trained model inputs two precision lead frame design drawings at the same time and reconstructs the two original input images respectively, according to the value sequence of the binary mask M, the loss function l is designed as shown in formula (2) pre :

[0081]

[0082] in, and Respectively represent the input image blocks and Reconstructed image.

[0083] like Figure 7, a schematic diagram of an application of a mask image modeling method in an embodiment of the present invention is given. Figure 7 In the example, both precision lead frame design drawings 71 and 72 are divided into four equal parts. The image blocks at the upper left corner and the lower right corner of precision lead frame design drawing 71 are selected, while the image blocks at the lower left corner and the upper right corner of precision lead frame design drawing 72 are selected. The selected image blocks are concatenated to form a mixed mask image 73. This mixed mask image 73 is input to an asymmetric deep autoencoder. Through the encoder and decoder, the precision lead frame design drawings 71 and 72 are reconstructed.

[0084] It is understood that in the above mask image modeling method, the number of input precision lead frame design drawings can also be other values, such as 3, 4, or more. When the number of input precision lead frame design drawings is greater than 2, a corresponding number of image blocks can be selected from each precision lead frame design drawing and spliced ​​into a mixed mask image.

[0085] In an embodiment of the present invention, in each of the four submodules of the encoder, a downsampling ratio of 2 is first performed through a Patch Merging layer. Assuming that the size of the input image is H×W, the number of channels is C, and the image patch size is 4×4, it can be divided into H / 4×W / 4 image patches. First, the pixels at the same position in each 2×2 image patch are spliced ​​in the depth direction to obtain H / 8×W / 8 feature maps with a channel number of 4C. Secondly, a linear transformation in the depth direction is performed through a LayerNorm normalization layer and a fully connected layer to obtain H / 8×W / 8 feature maps with a channel number of 2C as the output of the PatchMerging layer. In this way, the Patch Merging layer can reduce the width and height of the input feature map to 1 / 2 of the original, and increase the number of channels to twice the original.

[0086] In an embodiment of the present invention, the Swin Transformer blocks in the deep feature extraction model appear in pairs and are stacked. The difference between the two paired Swin Transformer blocks is that the first block uses a window-based multi-head self-attention module W-MSA to divide the input features into M×M windows, and uses a divide-and-conquer strategy to perform self-attention calculations in each independent window. Although this can reduce computational overhead, information cannot be transferred between windows. In contrast, the second paired Swin Transformer block uses a moving window-based multi-head self-attention module SW-MSA to solve the problem of information exchange between different windows. In common, each Swin Transformer block contains 2 LayerNorm normalization layers, 1 multi-layer perceptron MLP and 2 residual connections. Among them, the multi-layer perceptron MLP is composed of 1 Linear layer, 1 GELU activation layer, 1 Dropout layer, 1 Linear layer and 1 Dropout layer in sequence.

[0087] Reference Figure 8 , a schematic diagram of the structure of a pair of Swin Transformer blocks in an embodiment of the present invention is given. The input feature F of the first Swin Transformer block is input l-1 , the output of the first Swin Transformer block outputs the feature F l1 The output of the first Swin Transformer block is coupled to the input of the second Swin Transformer block, and the output of the second Swin Transformer block outputs the feature F l+1 .

[0088] In an embodiment of the present invention, after obtaining the precision lead frame design drawing for model training, the precision lead frame design drawing for model training can be transformed by data enhancement techniques such as overexposure, rotation, translation, scaling, and flipping to enhance the diversity of samples, enhance the generalization ability of the model, and prevent the occurrence of overfitting problems. Afterwards, the input training samples are scaled by convolution resampling based on the Bicubic kernel to obtain an image of size 384×384 as the input training sample. Then, the mask image modeling MIM method is used to pre-train the designed initial asymmetric deep autoencoder model on the ImageNet-1K dataset. Finally, the pre-trained model is fine-tuned to obtain a deep encoder model suitable for precision lead frame feature extraction.

[0089] In practice, pre-training of the asymmetric deep autoencoder uses 384×384 images and 32×32 image block partitioning with a masking rate of 75%. Furthermore, the window size of the SwinTransformer block in the first submodule and the SwinTransformer module in the second submodule is set to 16×16, the window size of the SwinTransformer block in the third submodule is set to 24×24, and the window size of the SwinTransformer block in the fourth submodule is set to 12×12. The settings of the remaining hyperparameters are shown in Table 1.

[0090] Table 1 Hyperparameter settings of pre-trained models

[0091] Hyperparameters Parameter value Optimizer AdamW Base learning rate 1.5×10-4 Weight delay threshold 0.05 Optimizer Kinetic Energy <![CDATA[β1=0.90,β2=0.95]]> Data augmentation methods Overexposure, random cropping, and convolution-based resampling scaling

[0092] Using the hyperparameter settings in Table 1 above, we trained 600 iterations on the large image dataset ImageNet-1K to obtain a pre-trained model of the asymmetric deep autoencoder.

[0093] Using the pre-trained asymmetric deep autoencoder model and the hyperparameter settings shown in Table 2, we trained it for 110 iterations on the precision lead frame design dataset to obtain a fine-tuned asymmetric deep autoencoder feature extraction model.

[0094] Table 2 Hyperparameter settings for feature extraction model fine-tuning

[0095] Hyperparameters Parameter value Optimizer AdamW Base learning rate 1.5×10-4 Weight delay threshold 0.05 Optimizer Kinetic Energy <![CDATA[β1=0.90,β2=0.999]]> Data augmentation methods Overexposure, random cropping, and convolution-based resampling scaling

[0096] For the input target precision lead frame design drawing, the image containing only the precision lead frame design content is first obtained through denoising operation, which is used as the input of the asymmetric deep autoencoder feature extraction model. Then, after the feature extraction operation is performed using the asymmetric deep autoencoder feature extraction model, the local features F of the target precision lead frame design drawing are obtained. l and the global feature F g , to represent the key pixel features of the target precision lead frame design drawing, such as corner points and numerical annotations, as well as the design contents such as the shape and outline of the target precision lead frame design drawing.

[0097] After extracting the local features F of the target precision lead frame design l and the global feature F g After that, the local feature F l and the global feature F g Perform fusion and obtain the fusion result F p . For the fusion result F p By performing generalized mean pooling, the fused feature vector can be obtained.

[0098] Reference Figure 9 , a schematic diagram of a feature fusion method in an embodiment of the present invention is given.

[0099] In the specific implementation, the local feature F l Perform adaptive average pooling. The adaptive average pooling layer can automatically calculate the size of the average pooling layer and the moving step size according to the size of the set output feature map and the size of the input feature map, and can realize adaptive cropping and dimensionality reduction of the input feature map. It is not only applicable to input feature maps of different sizes, but also can reduce the loss of key features during the feature dimensionality reduction process. To this end, adaptive average pooling is used to optimize the local feature F. l Perform dimensionality reduction processing and transform the local feature F l Converted to a feature map with the same dimension as the global feature Fg.

[0100] After that, calculate the local feature F after adaptive average pooling l In the global feature F g The projection in the direction, and then the local feature F l With the global feature F g to integrate.

[0101] In specific implementation, the fusion result F can be calculated using the following formula (3): p :

[0102]

[0103] By introducing the fusion result F p , it can extract local information from key areas of the target precision lead frame design and eliminate globally redundant feature components. This allows local and global features to mutually enhance each other, resulting in a more representative feature vector for the precision lead frame design.

[0104] The calculated fusion result F p and the global feature F of the input g Concatenate and concatenate to obtain the concatenated feature vector F c .

[0105] Finally, for the eigenvector F c Perform the generalized mean GeM pooling as shown in formula (4) to obtain the fusion feature vector f that accurately represents the input target precision lead frame design drawing GeM .

[0106]

[0107] Where f is the fusion feature vector F cIn the example, p is an exponential parameter with a value range of (0, ∞]. When p>1, the contrast of the pooled features can be enhanced, and the pooled features can be made to focus more on the significant features in the input feature map. Therefore, the parameter p in the present invention is set to 3 to retain more spatial structural features and key features of the input feature map in the feature vector after generalized mean GeM pooling, so as to meet the image retrieval task's requirements for feature representation of the target spatial position and its contour, local texture details, etc. in the image.

[0108] Therefore, the fusion feature vector f GeM It can represent both the global outline and the local geometric shape features of the target precision lead frame design drawing, thus contributing to the realization of an efficient single-step precision lead frame design drawing retrieval method.

[0109] In specific implementations, Euclidean distance is usually used to measure the distance between two vectors in a multidimensional space. Assume that the fused feature vector extracted from the target precision lead frame design drawing is q, and the feature vector extracted from the kth precision lead frame design drawing in the resource library is u. k , then the Euclidean distance between these two precision lead frame designs is dist(q,u k )for

[0110]

[0111] Formula (5) is used to calculate the Euclidean distance between the characteristic vector q of the target precision lead frame design and the characteristic vector of each precision lead frame design in the resource library. If the maximum value of the calculated Euclidean distance is dist_max, after dist(q,u k ) is processed, the similarity s(q,u) between the target precision lead frame design drawing and the k-th precision lead frame design drawing in the resource library can be calculated using formula (6): k ):

[0112]

[0113] Formula (6) is used to calculate the similarity between the input target precision lead frame design drawing and each precision lead frame design drawing in the resource library one by one, and the m precision lead frame design drawings with the largest similarity in the resource library are returned and output as the retrieval results.

[0114] In this embodiment of the present invention, pre-training and fine-tuning of the asymmetric deep autoencoder model were performed on an A100 GPU server. This server is composed of four AMD EPYC 7742 CPUs, eight NVIDIA A100 GPUs, 2TB of memory, and a 6TB SSD hard drive. The software environment configured on this server includes the Ubuntu 22.04.3 operating system, CUDA SDK 11.7, cuDNN 8.9.2, PyTorch 2.0.1, torchvision 0.15.2, torchaudio 2.0.2, and timm 0.6.13.

[0115] In this embodiment of the present invention, the asymmetric deep autoencoder model was pre-trained using the publicly available ImageNet-1K image dataset. The deep feature extraction model for precision lead frame design drawings was fine-tuned using a dataset of 5,890 samples collected by this embodiment of the present invention.

[0116] In the embodiment of the present invention, assuming that there are N samples in the test set, the mAP (mean of Average Precision) as shown in formula (7) can be used to evaluate the performance of the precision lead frame design drawing retrieval method.

[0117]

[0118] Among them, mAP represents the average precision of the retrieval results of all N input target frame lead design drawings. The larger its value, the higher the performance of the retrieval method, and vice versa. AP(n) represents the mean precision of the retrieval results of the n-th input target frame lead design drawing, that is, the coverage area in the precision-recall curve PR diagram. The larger the AP(n) value, the higher the precision of the n-th retrieval. R represents the number of retrieval results that match the n-th target frame lead design drawing. P(k) represents the k most similar precision lead frame design drawings in the retrieval results. rel(k) is an identification function. If the retrieval result belongs to one of the k design drawings returned, its value is 1, otherwise its value is 0.

[0119] In a specific implementation, by using the precise lead frame design drawing retrieval method provided in the embodiment of the present invention, the mAP value of the retrieval accuracy in the test data set can reach 87.6.

[0120] Reference Figure 10 , gives a schematic diagram of a target precision lead frame design drawing to be queried in an embodiment of the present invention. Figure 11 , a schematic diagram of the output results retrieved based on the target precision lead frame design drawing is given.

[0121] Figure 11In the example, three precision lead frame design drawings were retrieved based on the query precision lead frame design drawing. The precision lead frame design drawing on the far left has the highest similarity to the query precision lead frame design drawing, at 100%. The precision lead frame design drawing in the middle has the second highest similarity to the query precision lead frame design drawing, and the precision lead frame design drawing on the far right has the lowest similarity to the query precision lead frame design drawing.

[0122] Reference Figure 12 The present invention also provides a precision lead frame design drawing retrieval device 12, comprising: a feature acquisition unit 121, a feature fusion unit 122 and a similarity calculation unit 123, wherein:

[0123] The feature acquisition unit 121 is used to input the target precision lead frame design drawing to be retrieved into the asymmetric deep feature autoencoder extraction model to obtain local features and global features of the target precision lead frame design drawing;

[0124] A feature fusion unit 122 is configured to fuse local features and global features of the target precision lead frame design drawing, perform generalized mean pooling on the obtained fusion result, and obtain a fusion feature vector;

[0125] The similarity calculation unit 123 is configured to obtain the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing based on the fused feature vector.

[0126] In a specific implementation, the specific execution process of the feature acquisition unit 121, the feature fusion unit 122 and the similarity calculation unit 123 may correspond to steps 101 to 103, which will not be described in detail here.

[0127] An embodiment of the present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the precision lead frame design drawing retrieval method provided in any of the above embodiments are executed.

[0128] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A precision lead frame design drawing retrieval method, characterized in that: include: Inputting the target precision lead frame design drawing to be retrieved into the asymmetric deep autoencoder feature extraction model to obtain the local features and global features of the target precision lead frame design drawing; fusing the local features and global features of the target precision lead frame design drawing, and performing generalized mean pooling on the obtained fusion result to obtain a fusion feature vector; Based on the fused feature vector, the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing is obtained.

2. The precision lead frame design drawing retrieval method according to claim 1, characterized in that: The obtaining, based on the fused feature vector, the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing includes: Concatenating the fused feature vector and the global feature in series; Perform generalized mean pooling on the concatenated results; The Euclidean distance between the concatenated result after generalized mean pooling and the feature vector of each precision lead frame design drawing in the resource library is calculated.

3. The precision lead frame design drawing retrieval method according to claim 1 or 2, characterized in that: The fusing of the local features and the global features of the target precision lead frame design drawing to obtain a fused feature vector includes: Performing adaptive average pooling on the local features to obtain pooled local features with the same size as the global features; Fusing the pooled local features with the global features to obtain the fusion result; The fusion result is subjected to generalized mean pooling to obtain the fusion feature vector.

4. The precision lead frame design drawing retrieval method according to claim 1, wherein: The asymmetric deep autoencoder model is trained using the following steps: Obtain the precise lead frame design for model training; Denoising the precision lead frame design drawing used for model training, scaling the denoised precision lead frame design drawing to obtain sample images, all of which constitute a data set; the data set includes a training set, a validation set, and a test set; Based on the data set, a pre-established initial asymmetric deep autoencoder is trained to obtain the asymmetric deep autoencoder feature extraction model.

5. The precision lead frame design drawing retrieval method according to claim 4, characterized in that: The denoising process of the precision lead frame design drawing used for model training includes: Using a multi-scale Harris corner detection algorithm to detect the area where the lead frame design content is located in the precision lead frame design drawing used for model training; The area where the lead frame design content is located is cropped from the collected precision lead frame design drawings to obtain the sample image.

6. The precision lead frame design drawing retrieval method according to claim 4, characterized in that: The scaling process of the sample images in the data set includes: All images in the dataset are scaled using a convolution resampling method based on the Bicubic kernel.

7. The precision lead frame design drawing retrieval method according to claim 4, characterized in that: The initial asymmetric deep autoencoder is an asymmetric autoencoder model based on SwinTransformer.

8. The precision lead frame design drawing retrieval method according to claim 4, wherein: The method of training a pre-established initial asymmetric deep autoencoder based on the scaled dataset includes: Using a mask image modeling method and an unsupervised training method, the samples in the training set are used as input to perform model pre-training on the initial asymmetric deep autoencoder; The pre-trained asymmetric deep autoencoder is adjusted based on the validation set and the test set to obtain the asymmetric deep autoencoder feature extraction model.

9. A precision lead frame design drawing retrieval device, characterized in that: include: A feature acquisition unit is used to input the target precision lead frame design drawing to be retrieved into the asymmetric deep autoencoder feature extraction model to obtain local features and global features of the target precision lead frame design drawing; a feature fusion unit, configured to fuse local features and global features of the target precision lead frame design drawing, and perform generalized mean pooling on the obtained fusion result to obtain a fusion feature vector; A similarity calculation unit is used to obtain the similarity between each precision lead frame design drawing in a preset resource library and the target precision lead frame design drawing based on the fused feature vector.

Images