Point material image feature enhancement method, device, medium, program product and terminal

Abstract

The present application provides a method, device, medium, program product and terminal for enhancing the features of a material spotting image. By combining the Transformer model and the U‑net model to form a center restoration model, the original material spotting image is enhanced to solve the problem of image center offset caused by the X-laser of the material spotting machine. At the same time, a variety of preprocessing methods are used to improve the image quality input to the center restoration model to ensure that the model can accurately analyze the image detail features and the overall features of the material after training, effectively improving the accuracy of material spotting recognition. This method can better adapt to a variety of new materials, improve the accuracy of image enhancement, achieve more efficient and accurate material spotting operations, and solve the problems of low counting efficiency of patch component spotting machines, difficulty in counting some materials, and X-laser image distortion.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method, device, medium, program product and terminal for enhancing feature of a point material image. Background Art

[0002] With the rapid development of electronic equipment, integrated circuits are widely used in various devices, and electronic components have become an indispensable part of the circuit. In printed circuit boards (PCBs), the soldering process of electronic components and the number of components in the tray are crucial. The warehouse management of point materials has become particularly important in welding plants.

[0003] The chip component counting machine is a device used in the automatic chip production line, mainly used to accurately remove the chip components from the material tray and accurately place them on the printed circuit board. This equipment has the characteristics of high efficiency and high precision, which can greatly improve the production efficiency of the chip production line, reduce errors and fatigue in manual operation, and is suitable for the placement of chip components of various specifications and types.

[0004] However, existing SMD component counting machines have some major drawbacks. First, they are inefficient and require multiple counting steps, such as the installation, rotation, and recycling of trays, which makes the operation cumbersome. Second, some materials are difficult to count, especially as devices become more diverse and the material strips cannot be effectively inspected through process holes. In addition, X-ray imaging will show obvious distortion in non-center positions, and traditional image algorithms cannot correct the image to the vertical center position, which affects the accuracy. Summary of the invention

[0005] In view of the shortcomings of the prior art mentioned above, the purpose of the present application is to provide a method, device, medium, program product and terminal for enhancing the feature of a material counting image, so as to solve the problems of low counting efficiency, difficulty in counting some materials and distortion of X-laser images in a patch component counting machine.

[0006] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a method for enhancing the features of a point material image, comprising: obtaining an original point material image; performing preprocessing and blur detection operations on the original point material image to generate a point material image to be enhanced; inputting the point material image to be enhanced into a central restoration model to perform an image enhancement operation to generate an enhanced point material image.

[0007] In some embodiments of the first aspect of the present application, the process of inputting the point material image to be enhanced into the central restoration model to perform an image enhancement operation includes: performing a convolution operation on the point material image to be enhanced to obtain shallow features of the point material image to be enhanced; inputting the shallow features into a first encoder to generate a first encoding feature, inputting the first encoding feature into a second encoder to generate a second encoding feature, and inputting the second encoding feature into a third encoder to generate a third encoding feature; passing the third encoding feature through an intermediate conversion layer to generate a conversion feature, and inputting the conversion feature into a third decoder to generate a third decoding feature; inputting the third decoding feature into the second decoder to generate a second decoding feature, and inputting the second decoding feature into the first decoder to generate a first decoding feature; performing a convolution operation on the first decoding feature to generate an enhanced point material image.

[0008] In some embodiments of the first aspect of the present application, the central restoration model also includes: the first encoder, the second encoder and the third encoder respectively include a CCA module and a downsampling module; the downsampling module of the first encoder is connected to the CCA module of the second encoder, the downsampling module of the second encoder is connected to the CCA module of the third encoder, the downsampling module of the third encoder is connected to the intermediate conversion layer, the first decoder, the second decoder and the third decoder respectively include a CCA module and an upsampling module; the intermediate conversion layer is connected to the upsampling module of the third encoder, the CCA module of the third encoder is connected to the upsampling module of the second encoder, and the CCA module of the second encoder is connected to the upsampling module of the first encoder.

[0009] In some embodiments of the first aspect of the present application, the central restoration model also includes: the third decoder performs an upsampling operation on the input conversion features, and features the upsampling results with the features processed by the CCA module of the third encoder to generate third fusion features, the third fusion features are input into the third Transformer model, and the output results of the third Transformer model are input into the CCA module of the third decoder, and then transmitted from the CCA module of the third decoder to the upsampling module of the second decoder; the second decoder performs an upsampling operation on the input third decoding features, and features the upsampling results with the features processed by the CCA module of the second encoder to generate second fusion features. The first decoder performs an upsampling operation on the input second decoding feature, and concatenates the upsampling result with the feature processed by the CCA module of the first encoder to generate a first fused feature, and inputs the first fused feature into the first Transformer model, and inputs the output result of the first Transformer model into the CCA module of the first decoder, so that the CCA module of the first decoder outputs the first decoding feature.

[0010] In some embodiments of the first aspect of the present application, the process of preprocessing the original dot material image includes: performing a downsampling operation on the original dot material image, and performing a traversal operation in the original dot material image after the downsampling operation to locate and crop the area containing the dots and generate a dot material area image; performing a contrast adjustment operation, an image equalization operation and a median filtering operation on the dot material area image based on a preset contrast adjustment factor.

[0011] In some embodiments of the first aspect of the present application, the process of performing blur detection operation on the original point material image includes: calculating the edge sharpness value of the preprocessed original point material image through the difference product set four-neighborhood EAV algorithm; based on a preset edge sharpness threshold, screening from the preprocessed original point material image to obtain the point material image to be enhanced.

[0012] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a point material image feature enhancement device, including: an image acquisition module: used to acquire the original point material image; a preprocessing module: used to perform preprocessing and blur detection operations on the original point material image to generate a point material image to be enhanced; a center restoration module: used to input the point material image to be enhanced into a center restoration model to perform image enhancement operations to generate an enhanced point material image.

[0013] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the point material image feature enhancement method when executed by a processor.

[0014] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer implements the point material image feature enhancement method.

[0015] To achieve the above-mentioned purpose and other related purposes, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the point material image feature enhancement method.

[0016] As described above, the method, device, medium, program product and terminal for enhancing the feature of material counting images of the present application have the following beneficial effects: The present invention can more accurately analyze the image detail features and the overall material features through the central reduction model constructed based on the Transformer model and the U-net model, and effectively improve the accuracy of material counting. It can better adapt to a variety of new materials, and improve the accuracy of image enhancement through more detailed attention information analysis, and finally achieve more efficient and accurate material counting operations. Thereby solving the problems of low counting efficiency, difficulty in counting some materials and distortion of X-laser images existing in the existing patch component counting machines. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of an embodiment of a method for enhancing the feature of a point material image of the present application is shown.

[0018] Figure 2 A schematic diagram of the structure of the CCA module in an embodiment of the point material image feature enhancement method of the present application is shown.

[0019] Figure 3 A schematic diagram of the structure of a depth-separable convolution module in an embodiment of a method for enhancing feature of a point material image of the present application is shown.

[0020] Figure 4A structural schematic diagram of a center restoration model in an embodiment of a point material image feature enhancement method of the present application is shown.

[0021] Figure 5 A schematic diagram of the structure of an embodiment of a device for enhancing the feature of a point material image of the present application is shown.

[0022] Figure 6 A structural schematic diagram of an embodiment of a point material image feature enhancement terminal of the present application is shown. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:

[0025] <1> Counting machine: A machine used to automatically identify and locate specific points of material in an image, often used for automatic inspection and sorting tasks in industrial production.

[0026] <2> Transformer model: A neural network model based on the self-attention mechanism, which is widely used in fields such as natural language processing and has achieved good results in sequence modeling and conversion tasks.

[0027] <3> U-net model: A convolutional neural network structure for image segmentation, which is characterized by connecting the encoder and decoder and is used to handle tasks such as medical image segmentation.

[0028] <4> Difference-product set four-neighborhood EAV algorithm: an algorithm used for image processing and feature extraction, which achieves efficient processing of image information by calculating the difference, product and set operations between pixel values ​​and combining the four-neighborhood relationship.

[0029] <5> CCA module: A module based on correlation analysis, commonly used in image processing and pattern recognition to capture the correlation and co-occurrence patterns between image features.

[0030] To facilitate understanding of the embodiments of the present application, first Figure 1 Detailed description. Figure 1The flowchart of a method for enhancing the feature of a point material image in an embodiment of the present invention is shown. The method for enhancing the feature of a point material image in this embodiment mainly includes the following steps:

[0031] Step S11: Acquire the original point material image.

[0032] In one embodiment of the present invention, the dot material image is collected from the X-ray laser flat panel detector of the dot material counting machine. The X-ray laser flat panel detector is installed on the dot material discharging port or the discharging conveyor belt of the dot material counting machine to collect the internal image of the dot material after the dot material is counted. The dot material counting machine continuously transmits the dot material during normal production, and the X-ray laser flat panel detector performs X-ray irradiation and image collection on the dot material according to the synchronization frequency of the dot material counting machine speed. The size of the collected original dot material image is 3072*3072. The dot materials that can be detected include plastic particles and original electronic components.

[0033] Step S12: performing preprocessing and blur detection operations on the original point material image to generate a point material image to be enhanced.

[0034] In one embodiment of the present invention, the process of preprocessing the original dot material image includes: performing a downsampling operation on the original dot material image, and performing a traversal operation in the original dot material image after the downsampling operation to locate and crop the area containing the dots and generate a dot material area image; performing a contrast adjustment operation, an image equalization operation and a median filtering operation on the dot material area image based on a preset contrast adjustment factor.

[0035] In one embodiment of the present invention, the method of preprocessing the collected original point material image includes: positioning and cropping operation, contrast adjustment operation, image equalization operation and median filtering operation. Among them, the purpose of positioning and cropping the image is to remove redundant background, extract the main area, improve processing efficiency and locate the identification area. The purpose of the contrast adjustment operation is to enhance the difference between bright and dark areas, improve the degree of distinction between the target and the background and the degree of feature extraction. The purpose of the image equalization operation is to compensate for the uneven brightness distribution, enhance the details of the dark area, and comprehensively extract texture features including defects. The purpose of median filtering is to eliminate the influence of discrete noise and retain edge information, and improve the effect of recognition based on contours and textures.

[0036] Furthermore, the process of the positioning and cutting operation includes: performing a downsampling operation on the dot material area image to generate a downsampled image. Subsequently, a full traversal operation is performed in the downsampled image. Specifically, a 50*50 pixel pixel detection matrix is ​​used to perform detection while traversing. If the number of pixels in the black (low gray value) connected domain accounts for more than 2% of the total number of pixels in the 50*50 pixel block, the current traversal area is determined to be an area containing dot materials. A mask is generated based on the detection results, and the minimum circumscribed circle is determined based on the mask, and finally the downsampled image is cut according to the position of the circle.

[0037] In this embodiment, the process of performing contrast adjustment operation on the cropped dot material area image is shown in Formula 1.

[0038] Po = [Pi - 127.5 (1 - L)] * d + 127.5 (1 + L) (Formula 1)

[0039] Where Pi represents the input pixel value, and its value range is [0, 255]. Po represents the output pixel value, and its value range is also [0, 255]. L represents the brightness adjustment parameter, and its value range is [-1, 1]. L = 0 means no brightness adjustment, L>0 means increase the brightness, and L<0 means decrease the brightness. d represents the contrast adjustment parameter, and its value is arctan(k), and its value range is [1, 89]. Wherein, the value of k is shown in Formula 2:

[0040] k = tan(45+44c) / (180*PI) (Formula 2)

[0041] Where c is the contrast adjustment factor input by the user, and its value range is [0, 1]. c = 0 means no contrast adjustment, and c = 1 means maximum contrast improvement.

[0042] Furthermore, when performing the contrast adjustment operation, the brightness adjustment operation may not be performed, and L is set to 0 for calculation. In this case, the process of the contrast adjustment operation is shown in Formula 3.

[0043] Po=Pi*d+127.5*(1-d) (Formula 3)

[0044] The contrast of the cropped dot material area image is adjusted to enhance the detail information of the dot material, improve the accuracy of dot material image recognition in the subsequent process, improve the visual effect, and enhance the automatic processing capability of the dot material image, making the texture, shape, color and other details of the dot material clearer and easier to identify, thereby improving the accuracy of tasks such as defect detection and providing more reliable data for subsequent image analysis, recognition, and detection tasks.

[0045] In this embodiment, an image equalization operation is performed on the dot material area image after the contrast adjustment operation. A normalized equalization operation is performed on the histogram of the dot material area image, which increases the contrast of the image by extending the grayscale distribution of the image to the entire grayscale range, making the image details more obvious. The process is shown in Formula 4.

[0046]

[0047] Among them, SS K It represents the gray value corresponding to the pixel value in interval k after normalization and equalization operation. L represents the gray level of the image, that is, the total number of gray values ​​that may appear in the image. n represents the total number of all pixels in the image. j Represents the number of pixels with grayscale value j, and k represents the current accumulated grayscale level. Histogram equalization first counts the number of pixels at each grayscale level in the image to obtain a histogram. Then, the cumulative histogram calculates the sum of the number of pixels at each grayscale level and all previous grayscale levels. Next, the value of the cumulative histogram is divided by the total number of pixels for normalization. Finally, the normalized cumulative histogram is multiplied by (L-1) and rounded to generate a new grayscale value, thereby achieving histogram equalization of the image. Performing histogram equalization on the contrast-adjusted dot pattern can further enhance the contrast of the image, improve image quality, and improve the effect of subsequent image processing.

[0048] It should be noted that the reason for performing contrast adjustment operations and image equalization operations in the present application is that, since the X-ray source adopts a cone beam output, a distribution will be formed on the flat-panel detector in which the radiation amount in the central area is the highest and the radiation amount away from the center area gradually decreases. Three interactions will occur after X-rays irradiate the material: photoelectric effect, Compton scattering and Rayleigh scattering. The existence of the latter two scattering phenomena will cause image artifacts, thereby reducing the image contrast and affecting the accuracy of material point detection. In view of the characteristics of the cone beam, that is, more photons are received near the center and fewer photons are received away from the center, the generated image can be corrected for image equalization. The correction method of contrast adjustment operation and image equalization operation can enhance the contrast between foreground and background, thereby improving image quality.

[0049] In one embodiment of the present invention, the process of the median filtering operation includes: taking the pixel to be processed as the center, selecting an area of ​​a specified shape as a filter. Exemplarily, the execution shape includes a rectangle, a circle, a cross, etc. Then, the grayscale values ​​of all pixels in the neighborhood are sorted. Finally, the median of the sorted grayscale values ​​replaces the grayscale value of the central pixel. The median filtering operation can effectively remove noise in the image while retaining important features such as image edges. It should be noted that the present application itself does not limit the size and shape of the neighborhood of the median filtering operation, and in actual production, it can be optimized for different noise types and image features.

[0050] In one embodiment of the present invention, the process of performing blur detection operation on the original point material image includes: calculating the edge sharpness value of the preprocessed original point material image through the difference product set four-neighborhood EAV algorithm; based on a preset edge sharpness threshold, screening from the preprocessed original point material image to obtain the point material image to be enhanced.

[0051] In one embodiment of the present invention, the process of performing image blur detection on the preprocessed dot material area image includes: calculating edge sharpness using the difference product set four-neighborhood EAV algorithm, which uses the difference in grayscale value between a pixel and an adjacent pixel to determine the existence of an edge, and measures the edge strength according to the size of the difference. The edge strength of the entire image can be obtained by summing and averaging the edge strengths of all pixels. The blur detection process is shown in Formula 5.

[0052]

[0053] Among them, EAV represents the edge sharpness value, f(x,y) represents the grayscale value of the pixel in the image, (x,y) represents the coordinate point, and m and n represent the length and width of the target image. The calculated EAV value and evaluation results are shown in Table 1. Based on the evaluation results, subsequent image enhancement operations are performed on the point material area image with a higher degree of image blur.

[0054] Evaluation results EAV value Very clear >100 Clearer 80-100 Clarity 40-80 Vague <40

[0055] Table 1: EAV image quality evaluation table

[0056] Step S13: inputting the point material image to be enhanced into the central restoration model to perform an image enhancement operation to generate an enhanced point material image.

[0057] In one embodiment of the present invention, the process of inputting the point material image to be enhanced into the central restoration model to perform an image enhancement operation includes: performing a convolution operation on the point material image to be enhanced to obtain shallow features of the point material image to be enhanced; inputting the shallow features into a first encoder to generate a first encoding feature, inputting the first encoding feature into a second encoder to generate a second encoding feature, and inputting the second encoding feature into a third encoder to generate a third encoding feature; passing the third encoding feature through an intermediate conversion layer to generate a conversion feature, and inputting the conversion feature into a third decoder to generate a third decoding feature; inputting the third decoding feature into the second decoder to generate a second decoding feature, and inputting the second decoding feature into the first decoder to generate a first decoding feature; performing a convolution operation on the first decoding feature to generate an enhanced point material image.

[0058] In this embodiment, the central restoration model also includes: the first encoder, the second encoder and the third encoder respectively include a CCA module and a downsampling module; the downsampling module of the first encoder is connected to the CCA module of the second encoder, the downsampling module of the second encoder is connected to the CCA module of the third encoder, the downsampling module of the third encoder is connected to the intermediate conversion layer, the first decoder, the second decoder and the third decoder respectively include a CCA module and an upsampling module; the intermediate conversion layer is connected to the upsampling module of the third encoder, the CCA module of the third encoder is connected to the upsampling module of the second encoder, and the CCA module of the second encoder is connected to the upsampling module of the first encoder.

[0059] Furthermore, the central restoration model also includes: the third decoder performs an upsampling operation on the input conversion features, and performs feature splicing on the upsampling result and the features processed by the CCA module of the third encoder to generate a third fused feature, the third fused feature is input into the third Transformer model, and the output result of the third Transformer model is input into the CCA module of the third decoder, and then transmitted from the CCA module of the third decoder to the upsampling module of the second decoder; the second decoder performs an upsampling operation on the input third decoding features, and performs feature splicing on the upsampling result and the features processed by the CCA module of the second encoder to generate a second fused feature, and the second fused The combined features are input into the second Transformer model, and the output results of the second Transformer model are input into the CCA module of the second decoder, and then transmitted to the upsampling module of the first decoder by the CCA module of the second decoder; the first decoder performs an upsampling operation on the input second decoding features, and concatenates the upsampling results with the features processed by the CCA module of the first encoder to generate first fused features, and inputs the first fused features into the first Transformer model, and inputs the output results of the first Transformer model into the CCA module of the first decoder, so that the CCA module of the first decoder outputs the first decoding features.

[0060] In one embodiment of the present invention, the process of image enhancement of the screened point material area image includes: inputting the screened point material area image with low definition into the model as an input image, extracting the shallow features of the input image through two convolutional layers, and inputting the obtained shallow features into the first encoder. Each encoder includes multiple CCA modules and corresponding downsampling modules, and is provided with a corresponding layer of decoders, and each layer of decoders includes an upsampling layer and a corresponding CCA module.

[0061] like Figure 2 The structure diagram of the CCA module in one embodiment of the present invention is shown. The module consists of five parts, namely, depthwise separable convolution, channel attention module, spatial attention module, 1*1 convolution and addition operation. The input signal first passes through the depthwise separable convolution, then passes through the channel attention module and the spatial attention module in sequence, and finally passes through the 1*1 convolution and performs an addition operation with the input signal.

[0062] In one embodiment of the present invention, after the dot material area image is preprocessed, damage is determined for each pixel in the degraded image and restored. Pixel degradation in an image refers to the phenomenon that the image quality decreases or is distorted, resulting in the loss of pixel information or blurring. Specifically, feature extraction is performed through the depthwise separable convolution module in the CCA module. The structure of the depthwise separable convolution is as follows: Figure 3 As shown, H×W represents the size of the image and C represents the number of channels. Next is a 3×3 convolution operation, followed by point-by-point convolution processing, and finally I' with a feature map is obtained. The calculation process is shown in Formula 6 to Formula 8.

[0063] I'=dwconv(image) (Formula 6)

[0064] I”=SA(Conv(CA(I')))+image (Formula 7)

[0065] I”’=Conv(I”)+I” (Formula 8)

[0066] Among them, dwconv represents deep convolution, SA represents spatial attention, CA represents channel attention operation, and Conv represents convolution operation.

[0067] In this embodiment, for each encoder, the CCA module therein first uses deep convolution to enhance local information, and then uses channel self-attention and spatial attention to capture global information. Finally, through the corresponding downsampling module, the output features of the encoder at the current level are generated. The enhancement model in this application includes a first encoder, a second encoder, and a third encoder from shallow to deep layers. By gradually extracting deeper features, the first features, second features, and third features of different depths. The third feature is input into the third decoder after intermediate conversion by the CCA module, and is transmitted from bottom to top by the third decoder.

[0068] Furthermore, each layer of decoder performs an upsampling operation on the feature information of the input decoder, and splices the upsampling result with the information transmitted by the corresponding layer encoder in the channel dimension to generate the fusion feature of the current layer, which is then input into the Transformer module for processing. Wherein, the Transformer module includes layer normalization, multi-head self-attention and feedforward layer. Through the self-attention mechanism, the Transformer module can capture global information and invest more resources in areas that need to be focused on, while suppressing information in other areas to obtain more important information. Finally, the image features output by each layer of the Transformer module are input into the CCA module of the current layer decoder again to generate the image features output by the current decoding layer, the third decoder inputs the image features of the current layer as input features into the second decoder, the second decoder inputs the image features of the current layer as input features into the first decoder, and finally, the first decoder inputs the image features of the current layer into the third convolutional layer and the fourth convolutional layer to obtain an enhanced point material area image. The residual distortion of the X-ray image is subtracted from the original image to generate an enhanced point material area image.

[0069] Figure 4 The schematic diagram of the model of image enhancement in this embodiment is shown. The position of the imaged object is not displayed in the center of the image, but tends to appear in the point material image I∈R at the bottom of the image H×W domain, firstly, two 3×3 convolution operations are performed to extract shallow features F 0 ∈R H×W×C , where H represents the height of the image, W represents the width of the image, and C represents the number of channels of the image. 0 Input to the first encoder for operation. First, F 0 The input is sent to the CCA module, and the local information is enhanced by deep convolution through dwconv, and the global information is obtained through channel attention and spatial attention mechanisms. The features output by the CCA module are downsampled with a convolution kernel of 4*4 with a step size of 2 to generate the output size of the first encoder. In the same way, F 1 Input into the second encoder and the third encoder in sequence to obtain the size of the output of the third encoder as The image features of F 3 After being input into the CCA module of the intermediate conversion layer, it enters the first decoding layer, the second decoding layer, and the third decoding layer in sequence. The input information of the first decoding layer and the output information of the second encoding layer are spliced ​​in the channel dimension to generate features Then F ′ 1The input is processed into the module in the Transformer to generate F 1 ″, and F 1 ″ is input to the CCA module of the current decoding layer, and then F is generated in the same way 2 ″ and F 3 ″. Finally, the output F 3 After two convolutional layers, the residual distortion of the X-ray image is subtracted from the original image to output an image at the center, and finally the enhanced image is used to count the material quantity.

[0070] In one embodiment of the present invention, models that can also be used for image enhancement include but are not limited to: YOLO model, Unet model, VGG model and CNN model. Among them, the YOLO model is good at real-time monitoring, can quickly identify the state of the hopper, and improve production efficiency and safety. The Unet model can repair image details and improve image quality to facilitate manual identification of point material information. The VGG model can change the image style, such as converting black and white images into color images, further improving recognition efficiency. The CNN model is versatile and customizable. It can be customized according to specific needs to identify different types of point materials and detect the size of point materials, such as identifying screws of different models and detecting the length and diameter of screws.

[0071] In one embodiment of the present invention, the process of enhancing the features of a point material image includes: first, an X-ray is emitted through an X-laser ray source, so that the X-ray passes through the object to be photographed and a projected point material image appears on a flat-panel detector. Subsequently, the point material image is transmitted to a PC through a communication interface. Before the computer processes the acquired image, the point material image is first preprocessed, including denoising, edge enhancement and other processing. The collected image is encoded using a trained model, and then the image of the central area is corrected to ensure the accuracy of the image during further analysis. Subsequently, the point materials are separated and judged, and finally the point materials are counted according to the analysis results. Finally, the processed results are stored in a medium for future reference and analysis.

[0072] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first coding layer and the second coding layer are only used to distinguish different coding layers, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0073] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0074] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0075] Figure 5 is a schematic block diagram of a point material image feature enhancement device 500 provided in an embodiment of the present application. Figure 5 As shown, the device includes an image acquisition module 501, a preprocessing module 502 and a center restoration module 503.

[0076] Image acquisition module 501: used to acquire the original point material image.

[0077] The preprocessing module 502 is used to perform preprocessing and blur detection operations on the original point material image to generate a point material image to be enhanced.

[0078] The center restoration module 505 is used to input the point material image to be enhanced into the center restoration model to perform image enhancement operation to generate an enhanced point material image.

[0079] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0080] It should also be understood that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0081] Figure 6 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 6 As shown, the electronic terminal includes: at least one processor 601, a memory 602, at least one network interface 603 and a user interface 605. The various components in the device are coupled together through a bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 6 In the specification, various buses are labeled as bus systems.

[0082] The user interface 605 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0083] It is understood that the memory 602 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0084] The memory 602 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic terminal 600. Examples of these data include: any executable program used to operate on the electronic terminal 600, such as an operating system 6021 and an application 6022; the operating system 6021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 6022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The 66 method provided in the embodiment of the present invention can be included in the application 6022.

[0085] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 601 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 601 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0086] In an exemplary embodiment, the electronic terminal 600 may be one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned point material image feature enhancement method.

[0087] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, the computer executes Figures 1 to 4 A method for enhancing point material image features in any one of the embodiments shown.

[0088] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figures 1 to 4 A method for enhancing point material image features in any one of the embodiments shown.

[0089] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0090] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0092] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0094] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0095] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0096] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0098] In summary, the present application provides a method, device, medium, program product and terminal for enhancing the features of a material point image. By combining the Transformer model and the U-net model to form a center restoration model, the original material point image is enhanced to solve the problem of image center offset caused by the X-laser of the material point machine. At the same time, a variety of preprocessing methods are used to improve the image quality input to the center restoration model to ensure that the image detail features and the overall material features can be accurately analyzed after the model is trained, effectively improving the accuracy of material point recognition. This method can better adapt to a variety of new materials, improve the accuracy of image enhancement, achieve more efficient and accurate material point operations, and solve the problems of low counting efficiency of patch component counting machines, difficulty in counting some materials, and X-laser image distortion. Therefore, the present application effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0099] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by knowledgeable persons in the relevant technical field without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for enhancing the feature of a point material image, characterized in that: include: Get the original point material image; Performing preprocessing and fuzzy detection operations on the original dot material image to generate a dot material image to be enhanced; Inputting the to-be-enhanced dot material image into the central restoration model to perform an image enhancement operation to generate an enhanced dot material image, including: performing a convolution operation on the to-be-enhanced dot material image to obtain shallow features of the to-be-enhanced dot material image; inputting the shallow features into a first encoder to generate a first encoding feature, inputting the first encoding feature into a second encoder to generate a second encoding feature, and inputting the second encoding feature into a third encoder to generate a third encoding feature; passing the third encoding feature through an intermediate conversion layer to generate a conversion feature, and inputting the conversion feature into a third decoder to generate a third decoding feature; inputting the third decoding feature into a second decoder to generate a second decoding feature, and inputting the second decoding feature into a first decoder to generate a first decoding feature; performing a convolution operation on the first decoding feature to generate an enhanced dot material image; The central restoration model also includes: the first encoder, the second encoder and the third encoder respectively include a CCA module and a downsampling module; the downsampling module of the first encoder is connected to the CCA module of the second encoder, the downsampling module of the second encoder is connected to the CCA module of the third encoder, the downsampling module of the third encoder is connected to the intermediate conversion layer, the first decoder, the second decoder and the third decoder respectively include a CCA module and an upsampling module; the intermediate conversion layer is connected to the upsampling module of the third encoder, the CCA module of the third encoder is connected to the upsampling module of the second encoder, and the CCA module of the second encoder is connected to the upsampling module of the first encoder.

2. The method for enhancing the feature of a point material image according to claim 1, characterized in that: The central reduction model also includes: The third decoder performs an upsampling operation on the input conversion feature, and performs feature splicing on the upsampling result and the feature processed by the CCA module of the third encoder to generate a third fused feature, inputs the third fused feature into the third Transformer model, and inputs the output result of the third Transformer model into the CCA module of the third decoder, which is then transmitted from the CCA module of the third decoder to the upsampling module of the second decoder; The second decoder performs an upsampling operation on the input third decoding feature, and performs feature splicing on the upsampling result and the feature processed by the CCA module of the second encoder to generate a second fused feature, inputs the second fused feature into the second Transformer model, and inputs the output result of the second Transformer model into the CCA module of the second decoder, which is then transmitted from the CCA module of the second decoder to the upsampling module of the first decoder; The first decoder performs an upsampling operation on the input second decoding feature, and concatenates the upsampling result with the feature processed by the CCA module of the first encoder to generate a first fusion feature, inputs the first fusion feature into the first Transformer model, and inputs the output result of the first Transformer model into the CCA module of the first decoder, so that the CCA module of the first decoder outputs the first decoding feature.

3. The method for enhancing the feature of a point material image according to claim 1, characterized in that: The process of preprocessing the original point material image includes: Performing a downsampling operation on the original dot material image, and performing a traversal operation in the original dot material image after the downsampling operation is performed, so as to locate and cut the area containing the dot material, and generate a dot material area image; A contrast adjustment operation, an image equalization operation and a median filtering operation are performed on the dot material area image based on a preset contrast adjustment factor.

4. The method for enhancing the feature of a point material image according to claim 1, characterized in that: The process of performing blur detection operation on the original point material image includes: The edge sharpness value of the pre-processed original dot material image is calculated by using the difference product set four-neighborhood EAV algorithm; Based on a preset edge sharpness threshold, the pre-processed original dot material image is screened to obtain the dot material image to be enhanced.

5. A point material image feature enhancement device, characterized in that: include: Image acquisition module: used to obtain the original point material image; Preprocessing module: used for preprocessing and blur detection operation on the original point material image to generate the point material image to be enhanced; The central restoration module is used to input the point material image to be enhanced into the central restoration model to perform image enhancement operations to generate an enhanced point material image, including: performing a convolution operation on the point material image to be enhanced to obtain shallow features of the point material image to be enhanced; inputting the shallow features into a first encoder to generate a first encoding feature, inputting the first encoding feature into a second encoder to generate a second encoding feature, and inputting the second encoding feature into a third encoder to generate a third encoding feature; passing the third encoding feature through an intermediate conversion layer to generate a conversion feature, and inputting the conversion feature into a third decoder to generate a third decoding feature; inputting the third decoding feature into the second decoder to generate a second decoding feature, and inputting the second decoding feature into the first decoder to generate a first decoding feature; The features perform convolution operations to generate an enhanced dot material image; the center restoration model also includes: the first encoder, the second encoder and the third encoder respectively include a CCA module and a downsampling module; the downsampling module of the first encoder is connected to the CCA module of the second encoder, the downsampling module of the second encoder is connected to the CCA module of the third encoder, the downsampling module of the third encoder is connected to the intermediate conversion layer, the first decoder, the second decoder and the third decoder respectively include a CCA module and an upsampling module; the intermediate conversion layer is connected to the upsampling module of the third encoder, the CCA module of the third encoder is connected to the upsampling module of the second encoder, and the CCA module of the second encoder is connected to the upsampling module of the first encoder.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for enhancing the feature of a dot material image according to any one of claims 1 to 4 is implemented.

7. A computer program product, characterized in that The computer program product includes computer program codes, and when the computer program codes are executed on a computer, the computer is enabled to implement the method for enhancing the feature of a dot material image according to any one of claims 1 to 4.

8. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the dot material image feature enhancement method according to any one of claims 1 to 4.

Images