Die casting surface defect detection method, equipment, medium and product

Abstract

The invention discloses a die casting surface defect detection method and device, a medium and a product, and relates to the technical field of data processing.The method comprises the steps that a two-dimensional feature map and a three-dimensional feature map of the surface of a to-be-detected die casting are obtained, and the two-dimensional feature map and the three-dimensional feature map are used for reflecting the actual state of the surface of the to-be-detected die casting; the two-dimensional feature map and the three-dimensional feature map are sent to an image reconstruction model to output a two-dimensional reconstruction map and a three-dimensional reconstruction map, and the two-dimensional reconstruction map and the three-dimensional reconstruction map are corresponding maps when the surface of the to-be-detected die casting is in a qualified state; determining an abnormal value of the surface of the to-be-detected die casting based on the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map; when the abnormal value is greater than or equal to a preset value, determining that the surface of the to-be-detected die casting is unqualified; or when the abnormal value is smaller than the preset value, it is determined that the surface of the to-be-detected die casting is qualified. The die casting surface defect detection precision is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, equipment, medium and product for detecting surface defects of die castings. Background Art

[0002] At present, the requirements for the surface quality of parts in the industrial manufacturing field are increasing day by day, especially for complex-shaped parts such as aluminum die-castings, especially complex parts such as radar sensor housings used in advanced driver assistance systems (ADAS) in the automotive industry. Their die-casting surface defects, such as burrs, cold flow, and damage, not only affect the appearance of the product, but may also cause functional problems, such as short circuits of electronic components and poor assembly. Existing die-casting surface defect detection solutions include manual detection and machine vision detection: Manual inspection: Manual inspection of the die-casting surface of parts by vision and touch relies on the experience of inspectors in terms of accuracy, but it has problems such as low inspection efficiency, poor operation repeatability, and large subjective errors. As the pace of production accelerates, manual inspection is difficult to meet the needs of industrial-scale production.

[0003] Machine vision inspection: Currently, the most commonly used is the two-dimensional machine vision system: it captures the surface features of die-casting parts based on optical imaging, has the advantages of high resolution and fast response speed, and is suitable for detecting defects such as scratches and texture changes on the surface of die-casting parts. However, because it is based on plane imaging, it is difficult to accurately detect defects related to geometric changes or deep features, such as burrs and cold flows, so there is a problem of low detection accuracy. Summary of the invention

[0004] The purpose of this application is to provide a method, device, medium and product for detecting surface defects of die castings to solve the problems described in the background technology.

[0005] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a method for detecting surface defects of a die casting, comprising: Acquire a two-dimensional feature map and a three-dimensional feature map of the surface of the die casting to be inspected, wherein the two-dimensional feature map and the three-dimensional feature map are used to reflect the actual state of the surface of the die casting to be inspected; Sending the two-dimensional feature map and the three-dimensional feature map to an image reconstruction model to output a two-dimensional reconstruction map and a three-dimensional reconstruction map, wherein the two-dimensional reconstruction map and the three-dimensional reconstruction map are maps corresponding to when the surface of the die casting to be inspected is in a qualified state; Based on the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map, determining an abnormal value of the surface of the die casting to be inspected, wherein the abnormal value is used to reflect the difference between the actual state of the surface of the die casting to be inspected and the qualified state; When the abnormal value is greater than or equal to the predetermined value, it is determined that the surface of the die casting to be inspected is unqualified; or, when the abnormal value is less than the predetermined value, it is determined that the surface of the die casting to be inspected is qualified.

[0006] Optionally, the two-dimensional feature map and the three-dimensional feature map are generated by the following method: Collecting two-dimensional image data and three-dimensional point cloud data of the surface of the die casting to be inspected; Performing posture calibration on the two-dimensional image data by using a first predetermined algorithm to obtain the two-dimensional feature map; Aligning the three-dimensional point cloud data to a design model using a second predetermined algorithm, wherein the design model is a model pre-designed for the surface of the die casting to be inspected; The distance between the three-dimensional point cloud data and the surface of the design model is calculated to generate a displacement map, and the displacement map is determined as the three-dimensional feature map.

[0007] Optionally, determining the abnormal value of the surface of the die casting to be inspected based on the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map includes: Generate a two-dimensional defect heat map based on the two-dimensional feature map and the two-dimensional reconstruction map, each pixel of the two-dimensional defect heat map is assigned an anomaly score, and the anomaly score is used to reflect the difference between the two-dimensional feature map and the two-dimensional reconstruction map of the pixel; generating a 3D defect heat map based on the 3D feature map and the 3D reconstruction map, wherein each pixel of the 3D defect heat map is assigned an anomaly score, and the anomaly score is used to reflect the difference between the 3D feature map and the 3D reconstruction map of the pixel; The outlier is determined based on the two-dimensional defect heat map and the three-dimensional defect heat map.

[0008] Optionally, determining the abnormal value based on the two-dimensional defect heat map and the three-dimensional defect heat map includes: Summing the two-dimensional defect heat map and the three-dimensional defect heat map to obtain a final defect heat map; The abnormal scores of the pixels of the final defect heat map are summed, and the obtained sum is determined as the abnormal value.

[0009] Optionally, determining the abnormal value based on the two-dimensional defect heat map and the three-dimensional defect heat map includes: Calculate the sum of the anomaly scores of each pixel in the two-dimensional defect heat map, and record it as a first anomaly score sum; Calculate the sum of the anomaly scores of each pixel of the three-dimensional defect heat map, and record it as a second anomaly score sum; A sum of the first anomaly score sum and the second anomaly score sum is calculated, and the sum is determined as the anomaly value.

[0010] Optionally, the image reconstruction model is a denoising autoencoder or a fully convolutional autoencoder based on a residual network.

[0011] Optionally, the first predetermined algorithm is a generalized Hough transform, and the second predetermined algorithm is an iterative closest point algorithm.

[0012] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the methods described in the first aspect.

[0013] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the methods in the first aspect.

[0015] According to the specific embodiments provided in this application, this application discloses the following technical effects: The die casting surface defect detection method provided in the embodiment of the present application obtains a two-dimensional feature map and a three-dimensional feature map of the die casting surface to be detected; then reconstructs a two-dimensional reconstruction map and a three-dimensional reconstruction map of the die casting surface when it is qualified based on the two-dimensional feature map and the three-dimensional feature map; finally, the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map are used to jointly determine whether there are defects on the die casting surface. Compared with the prior art that only uses two-dimensional feature maps to detect die casting surface defects, the present application improves the comprehensiveness of die casting surface defect detection, thereby also improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is an application environment diagram of a die casting surface defect detection method in one embodiment of the present application; Figure 2 A schematic flow chart of a method for detecting surface defects of a die casting provided in one embodiment of the present application; Figure 3 A schematic diagram of a flow chart of a method for generating a two-dimensional feature map and a three-dimensional feature map provided in another embodiment of the present application; Figure 4 A flowchart of a method for determining an outlier provided in another embodiment of the present application; Figure 5 A flowchart of another method for determining an outlier provided in an embodiment of the present application; Figure 6 A flowchart of another method for determining an outlier provided in another embodiment of the present application; Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0020] The die casting surface defect detection method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the application environment includes a terminal and a server. Among them, the terminal communicates with the server through a network. The data storage system can store the data that the server needs to process. The data storage system can be set up separately, integrated on the server, or placed on the cloud or other servers. The terminal can send the two-dimensional image data and three-dimensional point cloud data to be processed to the server. After the server receives the two-dimensional image data and three-dimensional point cloud data to be processed, it can first store the data, and then retrieve it from the storage location when it needs to be processed. Alternatively, the processing task can be performed while storing. The server processes the two-dimensional image data and the three-dimensional point cloud data to obtain a two-dimensional feature map and a three-dimensional feature map, and sends the two-dimensional feature map and the three-dimensional feature map to the image reconstruction model, generates a two-dimensional reconstructed image and a three-dimensional reconstructed image through the image reconstruction model, and finally judges whether the die-casting surface of the part to be inspected is qualified through the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstructed image and the three-dimensional reconstructed image. The server can feedback the detected results to the terminal. In addition, in some embodiments, the die-casting surface defect detection method can also be implemented independently by a server or a terminal. For example, the terminal can directly perform the die-casting surface defect detection process on the two-dimensional image data and the three-dimensional point cloud data, or the server can obtain the two-dimensional image data and the three-dimensional point cloud data to be processed from the data storage system, and perform die-casting surface defect detection related operations on the two-dimensional image data and the three-dimensional point cloud data to be processed.

[0021] The terminals may be, but are not limited to, various desktop computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers, or a cloud server.

[0022] In an exemplary embodiment, Figure 2 As shown, a method for detecting surface defects of die castings is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 201 to 204: Step 201, obtaining a two-dimensional feature map and a three-dimensional feature map of the surface of the die casting to be inspected, wherein the two-dimensional feature map and the three-dimensional feature map are used to reflect the actual state of the surface of the die casting to be inspected.

[0023] The die casting surface to be inspected may be the die casting surface of a certain part, for example, an aluminum die casting in the field of automobile manufacturing.

[0024] The two-dimensional feature map and the three-dimensional feature map may be maps obtained by processing using a predetermined processing method. For details, please refer to the relevant content below.

[0025] Step 202, sending the two-dimensional feature map and the three-dimensional feature map to an image reconstruction model to output a two-dimensional reconstruction map and a three-dimensional reconstruction map, wherein the two-dimensional reconstruction map and the three-dimensional reconstruction map are maps corresponding to when the surface of the die casting to be inspected is in a qualified state.

[0026] The image reconstruction model is used to reconstruct the two-dimensional feature map and the three-dimensional feature map to obtain a qualified image of the die casting surface. The process of step 202 can be: sending the two-dimensional feature map and the three-dimensional feature map to the image reconstruction model; the image reconstruction model reconstructs the two-dimensional feature map to obtain a two-dimensional reconstruction map; the image reconstruction model reconstructs the three-dimensional feature map to obtain a three-dimensional reconstruction map.

[0027] Step 203, based on the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map, determine the abnormal value of the surface of the die casting to be inspected, and the abnormal value is used to reflect the difference between the actual state of the surface of the die casting to be inspected and the qualified state.

[0028] The larger the outlier value, the greater the difference and the more unqualified it is; the smaller the outlier value, the smaller the difference and the closer to qualified it is.

[0029] Step 204: when the abnormal value is greater than or equal to a predetermined value, determining that the surface of the die casting to be inspected is unqualified; or, when the abnormal value is less than the predetermined value, determining that the surface of the die casting to be inspected is qualified.

[0030] When the abnormal value is greater than or equal to the predetermined value, it means that the difference between the surface of the die casting to be tested and the surface of the die casting in the qualified state is large, so the surface of the die casting to be tested is unqualified. When the abnormal value is less than the predetermined value, it means that the difference between the surface of the die casting to be tested and the surface of the die casting in the qualified state is small, so the surface of the die casting to be tested is qualified.

[0031] Furthermore, the predetermined value may be an empirical value, or may be obtained through optimization of a validation set.

[0032] Furthermore, step 204 can also be expressed by the following formula:

[0033] in: s is an outlier; xi is the first i The pixel value of pixels, where the pixel value is the abnormal score; is the first i The pixel value of pixels, where the pixel value is the abnormal score; N is the total number of pixels.

[0034] If the abnormality score S ≥ the predetermined value, the part is judged as unqualified; otherwise, it is qualified, realizing automatic classification of parts.

[0035] Furthermore, the test results may be sent to a human-machine interface for confirmation by an operator. Furthermore, during the confirmation operation, only the surfaces of the die-castings that failed the test may be confirmed.

[0036] For the unqualified parts detected, the robot will sort them into specific areas to avoid confusion with qualified parts.

[0037] The die casting surface defect detection method provided in the embodiment of the present application obtains a two-dimensional feature map and a three-dimensional feature map of the die casting surface to be detected; then reconstructs a two-dimensional reconstruction map and a three-dimensional reconstruction map of the die casting surface when it is qualified based on the two-dimensional feature map and the three-dimensional feature map; finally, the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map are used to jointly determine whether there are defects on the die casting surface. Compared with the prior art that only uses two-dimensional feature maps to detect die casting surface defects, the present application improves the comprehensiveness of die casting surface defect detection, thereby also improving the detection accuracy.

[0038] By combining two-dimensional feature maps with three-dimensional feature maps, we can give full play to the advantages of two-dimensional feature maps in detail detection (such as edges, surface scratches, and texture changes), as well as the capabilities of three-dimensional feature maps in deep feature detection (such as burrs, geometric changes, cold flows, and damage defects). This enables the system to have both plane texture recognition and deep feature capture capabilities, achieve comprehensive coverage of defect types, make up for the shortcomings of single-dimensional detection, and improve detection accuracy and stability.

[0039] In addition, the present application simultaneously processes the two-dimensional feature map and the three-dimensional feature map through the image reconstruction model, so that the two-dimensional feature map and the three-dimensional feature map do not need to be processed separately, thereby improving the data processing speed and further improving the efficiency of die casting surface defect detection. Compared with the traditional three-dimensional detection system, the present application avoids the delay problem caused by excessive data volume.

[0040] In addition, compared with manual inspection, the inspection time of this application is shortened by about 20%, meeting the online inspection needs of industrial production lines.

[0041] After practical verification, the detection accuracy (true positive TP + true negative TN) of the method of fusion detection of two-dimensional feature maps is over 95%, while when only two-dimensional or three-dimensional is used for detection, the accuracy is only about 90%.

[0042] In addition, in actual industrial cases, especially in the detection of complex die-casting surface defects (such as cold flow) in aluminum die-castings, the detection accuracy of the present application is significantly enhanced compared to the detection accuracy of the prior art.

[0043] Alternatively, see Figure 3 In another exemplary embodiment of the present application, the two-dimensional feature map and the three-dimensional feature map are generated by the following steps 2011 to 2014: Step 2011, collecting two-dimensional image data and three-dimensional point cloud data of the surface of the die casting to be inspected.

[0044] The two-dimensional image data can be obtained by taking a two-dimensional camera. Furthermore, the two-dimensional camera is installed on a rotating platform, and the two-dimensional camera is rotated by the rotating platform, so that the surface of the die-casting to be inspected can be photographed from different angles to obtain two-dimensional image data at different positions and angles.

[0045] Among them, the three-dimensional point cloud data is collected by a three-dimensional laser scanner. Furthermore, the three-dimensional laser scanner is installed on a rotating platform, and the three-dimensional laser scanner is driven to move by the rotating platform to collect three-dimensional point cloud data at different positions, directions and angles.

[0046] Of course, when collecting point cloud data, the 3D laser scanner can remain stationary, and the part can be installed on a rotating platform. The rotating platform can be used to drive the part to rotate to adjust the part's posture, ensuring that all areas of the die-casting surface to be inspected are covered to avoid shadow areas affecting the scanning integrity.

[0047] The performance of the 3D laser scanner can be selected according to the needs. For example, the sampling rate of the 3D laser scanner is 1.7kHz, generating point cloud data of about 50 million points, the Z-axis resolution is 3µm, and the X-axis resolution is 13-17µm.

[0048] In addition, when collecting 2D image data and 3D point cloud data, a light source (such as a backlight system) can be set to optimize lighting conditions to ensure edge clarity and defect visibility of the image of the surface of the die casting to be inspected.

[0049] In addition, when collecting two-dimensional image data and three-dimensional point cloud data through a rotating platform, an incremental encoder can be used to record the position of the rotating platform to ensure the alignment of the two-dimensional image data and the three-dimensional point cloud data.

[0050] Step 2012: Perform posture calibration on the two-dimensional image data using a first predetermined algorithm to obtain the two-dimensional feature map.

[0051] Step 2013: align the three-dimensional point cloud data to a design model using a second predetermined algorithm, wherein the design model is a model pre-designed for the surface of the die casting to be inspected.

[0052] Step 2014: calculate the distance between the three-dimensional point cloud data and the surface of the design model, generate a displacement map, and determine the displacement map as the three-dimensional feature map.

[0053] Furthermore, the first predetermined algorithm includes Generalized Hough Transform (English full name: Generalized Hough Transform, English abbreviation: GHT).

[0054] The second predetermined algorithm includes: an iterative closest point algorithm (English full name: Iterative Closest Point, English abbreviation: ICP).

[0055] Among them, the process of using general Hough transform to calibrate the posture of a two-dimensional image is as follows: First, the Canny operator is used to perform edge detection on the two-dimensional image to identify the surface of the die casting to be detected from the two-dimensional image and obtain a two-dimensional image after edge detection; Second, the posture of the two-dimensional image after edge detection is calibrated by a template matching method. Here, the posture can be a position, direction, etc., so that it is aligned with a reference image, wherein the reference image is a pre-determined image of the die-casting surface of a part of the same model as the die-casting surface of the part to be detected, and the posture of the reference image is an image that is easier to detect whether the die-casting surface is qualified; Third, the calibrated two-dimensional image is cropped according to the edge detection result, and the grayscale value of the cropped two-dimensional image is normalized to the range of [−1, 1]; Fourth, the grayscale value of the background area of ​​the cropped two-dimensional image is set to 0 to reduce irrelevant interference, thereby obtaining a two-dimensional feature map.

[0056] The pre-designed design model may be a CAD (full name: Autodesk Computer Aided Design) model. Furthermore, a three-dimensional feature map may be generated by a CAD distance method. Specifically, the distance between the surface of the die-casting to be inspected represented by the three-dimensional point cloud data and the surface of the die-casting of the CAD model is calculated to generate a displacement map, each pixel in the displacement map is assigned a calculated distance value, and then the displacement map is normalized to the range of [−1, 1], and then the position map is converted into a grayscale map as a three-dimensional feature map.

[0057] In addition to using the CAD distance method, the die-casting surface gradient method can also be used to generate a three-dimensional feature map: the three-dimensional point cloud data is regarded as a normal vector, the angle between the normal vector and the CAD model is calculated, and the die-casting surface gradient map is generated to highlight the high curvature area. The die-casting surface gradient map is then converted into a grayscale image and normalized to the range of [−1, 1].

[0058] After preprocessing the 2D image data and 3D point cloud data, the alignment and uniformity of the data can be improved, providing high-quality input for the image reconstruction model. In addition, preprocessing reduces the complexity of the 2D image data and 3D point cloud data, and reduces the dependence on the specific geometry of the parts.

[0059] Alternatively, see Figure 4 In another exemplary embodiment of the present application, the above step 203 includes the following steps 301 to 303: Step 301, generating a two-dimensional defect heat map based on the two-dimensional feature map and the two-dimensional reconstruction map, each pixel of the two-dimensional defect heat map is assigned an anomaly score, and the anomaly score is used to reflect the difference between the two-dimensional feature map and the two-dimensional reconstruction map of the pixel.

[0060] The area with a higher anomaly score is the area where defects exist on the surface of the die casting to be inspected.

[0061] Step 302: Generate a 3D defect heat map based on the 3D feature map and the 3D reconstruction map, each pixel of the 3D defect heat map is assigned an anomaly score, and the anomaly score is used to reflect the difference between the 3D feature map and the 3D reconstruction map of the pixel.

[0062] Step 303: Determine the abnormal value based on the two-dimensional defect heat map and the three-dimensional defect heat map.

[0063] Alternatively, see Figure 5 In another exemplary embodiment of the present application, the above step 303 is replaced by the following steps 3031 to 3032: Step 3031, summing the two-dimensional defect heat map and the three-dimensional defect heat map to obtain a final defect heat map.

[0064] Step 3032, summing up the abnormal scores of each pixel of the final defect heat map, and determining the sum as the abnormal value.

[0065] Alternatively, see Figure 6 In another exemplary embodiment of the present application, the above step 303 is replaced by the following steps 3033 to 3035: Step 3033: Calculate the sum of the abnormality scores of each pixel in the two-dimensional defect heat map, and record it as the first abnormality score sum.

[0066] Step 3034, calculating the sum of the anomaly scores of each pixel in the three-dimensional defect heat map, and recording it as the second anomaly score sum.

[0067] Step 3035: Calculate the sum of the first anomaly score sum and the second anomaly score sum, and determine the sum as the anomaly value.

[0068] The defect heat map approach is not only applicable to current parts, but also has the ability to be promoted to support future industrial quality control tasks.

[0069] Optionally, in another exemplary embodiment of the present application, the image reconstruction model is a denoising autoencoder (full name in English: Denoising Autoencoder; abbreviated in English: DAE) or a fully convolutional autoencoder based on a residual network.

[0070] Of course, the image reconstruction model can also be other autoencoder models. The autoencoder model is a type of deep learning model. In addition to the autoencoder model, the image reconstruction model can also be other deep learning models.

[0071] Furthermore, the image reconstruction model includes an encoder and a decoder, wherein the encoder is used to map the input two-dimensional feature map and the three-dimensional feature map to a low-dimensional latent space, and the decoder is used to reconstruct the input two-dimensional feature map and the three-dimensional feature map through the low-dimensional latent space, thereby obtaining a two-dimensional reconstructed image and a three-dimensional reconstructed image.

[0072] Denoising autoencoders use self-supervised learning, and only need data on parts with qualified die-casting surfaces to train the model, reducing the reliance on labeled data; Denoising autoencoders learn the features of parts with normal die-casting surfaces, effectively identifying abnormal areas, and remain stable even in cases of uneven lighting and complex die-casting surfaces. Therefore, using the autoencoder model for detection can reduce the need for manual labeling, automatically learn normal features, and efficiently detect abnormal areas.

[0073] This application uses a denoising autoencoder model and can also accelerate parallel computing through GPU to improve image processing speed.

[0074] Optionally, in another exemplary embodiment of the present application, all data during the detection process are recorded for production line quality tracking and analysis.

[0075] Furthermore, the image reconstruction model is obtained by training the initial model of the denoising autoencoder model in advance, and the data used for the training includes the two-dimensional image data and three-dimensional point cloud data of a predetermined number of qualified parts. Of course, before being input into the initial model, the two-dimensional image data and the three-dimensional point cloud data need to be preprocessed to obtain the two-dimensional feature map and the three-dimensional feature map; In addition, the denoising autoencoder is self-supervised learning, so the initial model of the denoising autoencoder can learn the normal features of qualified parts and ignore the abnormal features; Furthermore, Gaussian noise is added when training the initial model to enhance the robustness of the model.

[0076] Furthermore, during the training process, the mean square error (MSE) between the input two-dimensional feature map, three-dimensional feature map and the two-dimensional reconstructed image, three-dimensional reconstructed image is minimized.

[0077] By learning the normal feature distribution of qualified parts, it is possible to accurately identify different types of surface defects in die castings. DAE responds significantly more to abnormal areas (such as defects) than to normal areas, thereby improving detection accuracy.

[0078] This application is described by taking the surface of any die-casting part as an example. When it is necessary to detect the surface of the die-casting parts of different parts, it can be done by replacing some hardware (such as lens focal length, scanner layout).

[0079] In addition, during the process of die-casting surface defect detection, it can be coordinated with a graphics processing unit (GPU) to accelerate calculations and improve image processing speed.

[0080] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the surface defect detection of die castings. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting surface defects of die castings is implemented.

[0081] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0082] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0083] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0084] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0085] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0086] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0087] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0088] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for detecting surface defects of die castings, characterized in that: include: Acquire a two-dimensional feature map and a three-dimensional feature map of the surface of the die casting to be inspected, wherein the two-dimensional feature map and the three-dimensional feature map are used to reflect the actual state of the surface of the die casting to be inspected; Sending the two-dimensional feature map and the three-dimensional feature map to an image reconstruction model to output a two-dimensional reconstruction map and a three-dimensional reconstruction map, wherein the two-dimensional reconstruction map and the three-dimensional reconstruction map are maps corresponding to when the surface of the die casting to be inspected is in a qualified state; Based on the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map, determining an abnormal value of the surface of the die casting to be inspected, wherein the abnormal value is used to reflect the difference between the actual state of the surface of the die casting to be inspected and the qualified state; When the abnormal value is greater than or equal to the predetermined value, it is determined that the surface of the die casting to be inspected is unqualified; or, when the abnormal value is less than the predetermined value, it is determined that the surface of the die casting to be inspected is qualified.

2. The die casting surface defect detection method according to claim 1, characterized in that: The two-dimensional feature map and the three-dimensional feature map are generated by the following method: Collecting two-dimensional image data and three-dimensional point cloud data of the surface of the die casting to be inspected; Performing posture calibration on the two-dimensional image data by using a first predetermined algorithm to obtain the two-dimensional feature map; Aligning the three-dimensional point cloud data to a design model using a second predetermined algorithm, wherein the design model is a model pre-designed for the surface of the die casting to be inspected; The distance between the three-dimensional point cloud data and the surface of the design model is calculated to generate a displacement map, and the displacement map is determined as the three-dimensional feature map.

3. The die casting surface defect detection method according to claim 1, characterized in that: The method of determining the abnormal value of the surface of the die casting to be inspected based on the two-dimensional feature map, the three-dimensional feature map, the two-dimensional reconstruction map and the three-dimensional reconstruction map comprises: Generate a two-dimensional defect heat map based on the two-dimensional feature map and the two-dimensional reconstruction map, each pixel of the two-dimensional defect heat map is assigned an anomaly score, and the anomaly score is used to reflect the difference between the two-dimensional feature map and the two-dimensional reconstruction map of the pixel; generating a 3D defect heat map based on the 3D feature map and the 3D reconstruction map, wherein each pixel of the 3D defect heat map is assigned an anomaly score, and the anomaly score is used to reflect the difference between the 3D feature map and the 3D reconstruction map of the pixel; The outlier is determined based on the two-dimensional defect heat map and the three-dimensional defect heat map.

4. The die casting surface defect detection method according to claim 3, characterized in that: The determining the abnormal value based on the two-dimensional defect heat map and the three-dimensional defect heat map includes: Summing the two-dimensional defect heat map and the three-dimensional defect heat map to obtain a final defect heat map; The abnormal scores of the pixels of the final defect heat map are summed, and the obtained sum is determined as the abnormal value.

5. The die casting surface defect detection method according to claim 3, characterized in that: The determining the abnormal value based on the two-dimensional defect heat map and the three-dimensional defect heat map includes: Calculate the sum of the anomaly scores of each pixel in the two-dimensional defect heat map, and record it as a first anomaly score sum; Calculate the sum of the anomaly scores of each pixel of the three-dimensional defect heat map, and record it as a second anomaly score sum; A sum of the first anomaly score sum and the second anomaly score sum is calculated, and the sum is determined as the anomaly value.

6. The method for detecting surface defects of die castings according to any one of claims 1 to 5, characterized in that: The image reconstruction model is a denoising autoencoder or a fully convolutional autoencoder based on a residual network.

7. The die casting surface defect detection method according to claim 2, characterized in that: The first predetermined algorithm is a generalized Hough transform, and the second predetermined algorithm is an iterative closest point algorithm.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the die casting surface defect detection method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the die casting surface defect detection method described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the die casting surface defect detection method described in any one of claims 1 to 7 is implemented.

Images