Defect detection method and system for metal processing surface of camshaft

Abstract

The invention discloses a defect detection method and system for a camshaft metal machining surface. The method comprises the following steps: acquiring a to-be-detected image, wherein the to-be-detected image comprises a camshaft metal processing surface; processing the to-be-detected image to generate a region-of-interest image indicating the cam surface; the image of the region of interest is input into an anomaly detection model to be processed, an anomaly score graph is output, and an anomaly score in the anomaly score graph represents a probability value of anomaly of a corresponding pixel; selecting a first number of abnormal scores according to a descending order of the abnormal scores, and calculating a mean value of the selected abnormal scores; the defect information of the to-be-detected image is determined by comparing the mean value with a preset threshold value, the defect information comprises whether the to-be-detected image has defects or not and the defect position, and the preset threshold value is determined by analyzing the anomaly detection model. The scheme is low in cost and high in efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and system for detecting defects on a camshaft metal processing surface. Background Art

[0002] The camshaft is a key power transmission component in the internal combustion engine and plays a vital role in the working process of the engine. It is mainly responsible for controlling the opening and closing of the valves, thereby regulating the timing of intake and exhaust, which directly affects the performance, efficiency and emissions of the engine. The manufacturing of the camshaft involves multiple precision machining steps, including blank casting, rough machining, heat treatment, finishing and final inspection. Strict quality control is required in each manufacturing link to ensure the accuracy, strength and durability of the camshaft. However, due to various factors such as material defects, machining errors or improper heat treatment, the metal machined surface of the camshaft may have some defects, such as surface cracks, pits, abnormal wear, profile deviation or uneven hardness. The presence of these defects will seriously affect the performance and life of the engine, and may cause abnormal noise, power loss, and even more serious mechanical failures. Therefore, it is necessary to accurately identify the above defects.

[0003] However, due to the small number of defect samples on the metal processing surface of the camshaft, conventional supervised algorithms, such as the YOLO algorithm, cannot correctly identify all defects.

[0004] In view of this, a new defect detection solution for the metal processing surface of the camshaft is needed. Summary of the invention

[0005] The present application provides a method and system for defect detection of a camshaft metal machined surface, in an effort to solve or at least alleviate at least one of the above problems.

[0006] According to one aspect of the present application, a defect detection method for a camshaft metal processing surface is provided, comprising: acquiring an image to be detected, the image to be detected containing the camshaft metal processing surface; processing the image to be detected to generate an image of a region of interest indicating the cam surface; inputting the image of the region of interest into an abnormality detection model for processing, and outputting an abnormality score map, wherein the abnormality score in the abnormality score map represents the probability value of an abnormality in the corresponding pixel; selecting a first number of abnormality scores in descending order of the abnormality scores and calculating the mean of the selected abnormality scores; determining the defect information of the image to be detected by comparing the mean with a predetermined threshold, wherein the defect information includes whether the image to be detected has a defect and the defect position, and the predetermined threshold is determined by analyzing the abnormality detection model.

[0007] Optionally, in the method according to the present application, the anomaly detection model includes a coupled pre-trained convolutional network component, a converter component, and a multi-layer perceptron component.

[0008] Optionally, in the method according to the present application, the image of the region of interest is input into an anomaly detection model for processing, and an anomaly score map is output, including: inputting the image of the region of interest into a pre-trained convolutional network component, and outputting a multi-scale feature map after processing; inputting the multi-scale feature map into a converter component, and outputting a reconstructed feature map after processing; using the reconstructed feature map and the multi-scale feature map to obtain a reconstruction error; inputting the reconstructed feature map into a multi-layer perceptron component, and after processing, inputting it together with the reconstruction error into an activation layer to obtain an anomaly score map.

[0009] Optionally, in the method according to the present application, the defect information of the image to be detected is determined by comparing the mean and a predetermined threshold, including: if the mean is greater than the predetermined threshold, it is determined that the image to be detected has defects, and the pixel area corresponding to the first number of abnormal scores is the location of the defect; if the mean is not greater than the predetermined threshold, it is determined that the image to be detected has no defects.

[0010] Optionally, in the method according to the present application, determining the predetermined threshold includes: acquiring multiple images of the region of interest as verification samples, and marking the defective areas in the verification samples, generating annotation data for the verification samples, wherein the verification samples include positive samples that do not contain defects and negative samples that contain defects; inputting the verification samples into an anomaly detection model for processing, and outputting an anomaly score map of the verification samples; combining the anomaly score map of the verification samples and their annotation data, calculating the true positive rate and the false positive rate at each pending threshold, and generating a receiver operating characteristic curve based on the true positive rate and the false positive rate; and determining the predetermined threshold by analyzing the receiver operating characteristic curve.

[0011] Optionally, the method according to the present application also includes the step of training anomaly detection model, including: obtaining multiple images of regions of interest as training samples, and marking defective regions in the training samples as labeled data of the training samples, wherein the training samples include positive samples that do not contain defects and negative samples that contain defects; constructing an initial anomaly detection model and model parameters; inputting the training samples into the initial anomaly detection model for processing, adjusting the first parameter by optimizing the reconstruction error, and adjusting the second parameter by comparing the predicted anomaly score map with the labeled data of the training samples, until the training is completed, and finally obtaining the anomaly detection model.

[0012] Optionally, in the method according to the present application, the image to be detected is processed to generate an image of a region of interest indicating the cam surface, including: preprocessing the image to be detected to generate a first image, wherein the preprocessing includes grayscale conversion and binarization; performing morphological processing on the first image and generating a second image by extracting contours; and combining the image to be detected and the second image to generate an image of a region of interest indicating the cam surface.

[0013] Optionally, in the method according to the present application, the first image is morphologically processed, and a second image is generated by extracting contours, including: performing a morphological opening operation on the first image using a predetermined vertical kernel to obtain an intermediate first image; performing a morphological closing operation on the intermediate first image using a predetermined horizontal kernel to obtain a processed first image; and generating a second image by extracting contours that meet predetermined conditions from the processed first image, wherein the predetermined conditions include a contour area threshold and a horizontal coordinate interval of pixels.

[0014] According to another aspect of the present application, a defect detection system for a camshaft metal processing surface is provided, comprising: an image acquisition device, adapted to place a camshaft part on a measuring table and fix it by front and rear cylinders, and photograph the camshaft metal processing surface at different angles to obtain an image to be detected; an image processing device, adapted to process the image to be detected and generate an image of a region of interest indicating the cam surface; an image detection device, adapted to input the image of the region of interest into an abnormality detection model for processing and output an abnormality score map, wherein the abnormality score in the abnormality score map represents a probability value of an abnormality in the corresponding pixel; a defect detection device, adapted to select a first number of abnormality scores from the abnormality score map in descending order of the abnormality scores and calculate the mean of the selected abnormality scores; by comparing the mean with a predetermined threshold, the defect information of the image to be detected is determined, wherein the defect information includes whether there is a defect in the image to be detected and the defect position, wherein the predetermined threshold is determined by analyzing the abnormality detection model.

[0015] Optionally, the system according to the present application also includes a model generation device, which is coupled to the image detection device and is suitable for training and generating the anomaly detection model, wherein the anomaly detection model includes a coupled pre-trained convolutional network component, a converter component and a multi-layer perceptron component; the model generation device is also coupled to the defect detection device, and is also suitable for analyzing the anomaly detection model by verifying samples after the training is completed to determine the predetermined threshold.

[0016] According to another aspect of the present application, a computing device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method as described above.

[0017] According to another aspect of the present application, a computer-readable storage medium storing one or more programs is provided. The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method described above.

[0018] According to another aspect of the present application, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the above-mentioned method when executed by a processor.

[0019] In summary, according to the scheme of the present application, the image to be inspected is first processed, and the ROI area is extracted through image filtering, binarization and other operations to generate an image of the region of interest; then, the image of the region of interest is processed using an abnormality detection model to obtain an abnormality score map; by comparing with a predetermined threshold, it is determined whether there are defects in the image to be inspected, and if there are defects, the defect location. In addition, by judging the six images to be inspected of the metal processing surface of the camshaft, it can be further determined whether the camshaft part has defects.

[0020] In addition, when training the anomaly detection model, an unsupervised learning method is used to train with a small number of negative sample annotations, which effectively overcomes the problem of a small number of defective samples. In addition, the ROC curve is calculated and analyzed for the verification samples to determine the appropriate threshold for judging whether there is a defect.

[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles applied herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. By reading the following detailed description in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. Throughout the application, the same reference numerals generally refer to the same parts or elements.

[0023] Figure 1A schematic diagram of a camshaft metal machining surface defect detection system 100 according to some embodiments of the present application is shown;

[0024] Figure 2 A schematic diagram of a computing device 200 according to some embodiments of the present application is shown;

[0025] Figure 3 A schematic flow chart of a defect detection method 300 for a camshaft metal machined surface according to some embodiments of the present application is shown;

[0026] Figure 4 A schematic diagram of an image to be detected according to an embodiment of the present application is shown;

[0027] Figure 5 According to one embodiment of the present application, Figure 4 Schematic diagram of the first image (a), the second image (b), and the region of interest image (c) during processing;

[0028] Figure 6 A schematic diagram of an anomaly detection model according to an embodiment of the present application is shown;

[0029] Figure 7 A schematic diagram showing annotating an image of a region of interest according to some embodiments of the present application is shown;

[0030] Figure 8 A schematic diagram showing labeling data according to some embodiments of the present application is shown;

[0031] Fig. 9 The Receiver Operating Characteristic curve (ROC) according to some embodiments of the present application is shown. DETAILED DESCRIPTION

[0032] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0033] This application is based on anomaly detection algorithm. It first processes sample images containing metal processing surfaces, extracts regions of interest (ROI) through image filtering, binarization and other operations, and then uses unsupervised learning to train with a small amount of negative sample annotations to obtain an anomaly detection model. The anomaly detection model is used to detect the probability that each pixel in the image of the region of interest has an anomaly. At the same time, after the training is completed, the anomaly detection model is verified using a validation set, and the ROC curve is calculated and analyzed to determine a predetermined threshold. The predetermined threshold is used to finally determine whether each pixel has a defect.

[0034] Thus, in the subsequent inspection process, the camshaft metal parts are placed on the measuring table, and the metal processing surface is photographed multiple times by the camera to obtain multiple images to be inspected. Then, the following processing is performed on each image to be inspected: image processing is performed on the image to be inspected, and the processed image is processed by the abnormality detection model to obtain the probability of abnormality of each pixel; then, by comparing with the predetermined threshold, it is determined whether the cam surface of the image to be inspected has defects. Based on the inspection results of each image to be inspected, it can be further determined whether the camshaft metal parts are qualified.

[0035] According to the method of the present application, it is possible to overcome the problem that there are few defect samples on the camshaft machined surface and that conventional supervised algorithms cannot accurately identify all defects, and the defect detection solution has low cost and high efficiency.

[0036] Figure 1 A schematic diagram of a camshaft metal machining surface defect detection system 100 according to some embodiments of the present application is shown.

[0037] like Figure 1 As shown, the defect detection system 100 for the metal processing surface of the camshaft includes: an image acquisition device 110, an image processing device 120, an image detection device 130, a defect detection device 140 and a model generation device 150.

[0038] In some embodiments, the image processing device 120, the image detection device 130, the defect detection device 140 and the model generation device 150 can be arranged on one or more computing devices, and the computing devices are connected to the image acquisition device 110 to receive the acquired images containing the camshaft metal processing surface.

[0039] The image acquisition device 110 includes a measuring platform, a camera, a light source, a mechanical arm, etc. The camshaft parts are placed on the measuring platform and fixed by front and rear cylinders. The camshaft metal parts are rotated by the mechanical arm so as to take pictures of the camshaft metal processing surface at different angles to obtain the image to be tested.

[0040] In some embodiments, there are two camshaft metal processing surface parts on a camshaft. During the image acquisition process, the metal processing surface of each part can be acquired by moving the measuring table. When acquiring each part, the camshaft part is rotated by the mechanical arm to acquire images of the cam surface at different angles. Specifically, each time an image is taken, the camshaft part is rotated 120° by the mechanical arm, and then the next image is taken. In this way, three images of the metal processing surface can be obtained for each part. That is, for one camshaft part, six images to be tested are acquired.

[0041] Optionally, the light source and the camera are on the same vertical line, and the light source is a white bowl-shaped light source.

[0042] The image acquisition device 110 transmits the acquired images to be inspected to the computing device, which processes each image to be inspected to detect whether there are defects in the images to be inspected. And only when all six images to be inspected have no defects, the camshaft part is confirmed to be qualified.

[0043] First, the image processing device 120 processes the image to be detected to generate an image of a region of interest indicating the cam surface. In some embodiments, the image processing device 120 performs processing such as cropping on the image to be detected to extract the region containing the cam surface and generate an image of a region of interest.

[0044] Afterwards, the image processing device 120 sends the image of the region of interest to the image detection device 130, and the image detection device 130 inputs the image of the region of interest into the anomaly detection model for processing, and outputs an anomaly score map, wherein the anomaly score in the anomaly score map represents the probability value of the existence of an anomaly in the corresponding pixel. In some embodiments, the anomaly detection model includes a coupled pre-trained convolutional network component, a converter component, and a multi-layer perceptron component. In some embodiments, the converter component adopts a U-shaped Transformer architecture, i.e., a U-Transformer model. The anomaly detection model will be described in detail below.

[0045] Afterwards, the image detection device 130 sends the abnormal score map to the defect detection device 140, and the defect detection device 140 selects a first number of abnormal scores from the abnormal score map in descending order of the abnormal scores; by comparing the mean and a predetermined threshold (the predetermined threshold is determined by analyzing the abnormal detection model), the defect information of the image to be detected is determined, and the defect information includes whether the image to be detected has defects and the location of the defects. Specifically, when the mean is greater than the predetermined threshold, it is determined that the image to be detected has defects, and the pixel area corresponding to the first number of abnormal scores is the location of the defects; when the mean is not greater than the predetermined threshold, it is determined that the image to be detected does not have defects.

[0046] According to some embodiments of the present application, after the six images to be detected of the camshaft part are all detected, the defective part of the camshaft part can be further determined based on the defect information of the six images to be detected. Of course, if the six images to be detected of the camshaft part do not have defects, the camshaft part is determined to be qualified.

[0047] At the same time, the image detection device 130 and the defect detection device 140 are respectively coupled to the model generation device 150. On the one hand, the model generation device 150 is responsible for training and generating an anomaly detection model, and providing the trained anomaly detection model to the image detection device 130. On the other hand, after the anomaly detection model is trained, the model generation device 150 analyzes the anomaly detection model by verifying the sample to determine the predetermined threshold.

[0048] According to the detection system 100 of the present application, the problem of few defect samples of the camshaft metal processing surface is overcome, and an unsupervised algorithm is adopted, which does not rely on a large amount of labeled data, and can autonomously learn the characteristics of normal samples, thereby identifying abnormalities or defects of the camshaft metal processing surface. It has the advantages of low cost and high efficiency, and can be used for defect detection of various camshaft metal processing surfaces produced in factories.

[0049] The computing device may be any device with storage and computing capabilities, such as a server, workstation, etc., or a personal computer such as a desktop computer or a notebook computer, but is not limited thereto. In some embodiments, the computing device may also output defect detection results through a visualization screen.

[0050] Figure 2 FIG. 2 shows a schematic diagram of a computing device 200 according to some embodiments of the present application. It should be noted that: Figure 2 The computing device 200 shown is only an example. In practice, the computing device used to implement the method of the present application can be any type of device, and its hardware configuration can be different from the above. Figure 2 The computing device 200 shown is the same as Figure 2 The computing device 200 shown is different. In practice, the computing device used to implement the embodiment of the present application can be Figure 2 The hardware components of the computing device 200 shown are increased or deleted, and the present application does not limit the specific hardware configuration of the computing device.

[0051] like Figure 2In a basic configuration, computing device 200 includes at least one processing unit 202 and system memory 204. According to one aspect, depending on the configuration and type of computing device, processing unit 202 can be implemented as a processor. System memory 204 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 204 includes operating system 205 and program modules 206.

[0052] According to one aspect, operating system 205 is suitable for controlling the operation of computing device 200, for example. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other application programs, and are not limited to any particular application or system. Figure 2 This basic configuration is illustrated in FIG. 2 by those components within dashed line 208. According to one aspect, computing device 200 has additional features or functionality. For example, according to one aspect, computing device 200 includes additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 2 2 is illustrated by removable storage 209 and non-removable storage 210.

[0053] As stated above, according to one aspect, program modules are stored in the system memory 204. According to one aspect, the program modules may include one or more application programs, and the present application does not limit the types of application programs, for example, the application programs may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slide show applications, drawing or computer-aided applications, web browser applications, etc.

[0054] According to one aspect, the examples may be practiced on a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. Figure 2Each or many components shown in can be integrated in a system on chip (SOC) on a single integrated circuit to practice examples. According to one aspect, such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When operated via SOC, the functions described in the present application can be operated via a dedicated logic integrated with other components of a computing device 200 on a single integrated circuit (chip). Other technologies that can perform logical operations (such as AND, OR, and NOT) can also be used to practice embodiments of the present application, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, embodiments of the present application can be practiced in a general-purpose computer or in any other circuit or system.

[0055] According to one aspect, the computing device 200 may also have one or more input devices 212, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, a VR motion capture input device, etc. Output devices 214 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples and other devices may also be used. The computing device 200 may include one or more communication connections 216 that allow communication with other computing devices 218. Examples of suitable communication connections 216 include, but are not limited to: RF transmitters, receivers and / or transceiver circuits; Universal Serial Bus (USB), parallel and / or serial ports.

[0056] The term computer-readable media as used in this application includes computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 204, removable storage 209, and non-removable storage 210 are all examples of computer storage media (i.e., memory storage). Computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other products that can be used to store information and can be accessed by computing device 200. According to one aspect, any such computer storage media may be part of computing device 200. Computer storage media do not include carrier waves or other propagated data signals.

[0057] According to one aspect, communication media is implemented by computer readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0058] In some other embodiments of the present application, the computing device 200 is configured to execute the camshaft metal surface defect detection method 300 according to the present application. The program module 206 includes a plurality of program instructions for executing the method 300, and these program instructions can instruct the processing unit 202 to execute the method 300 of the present application.

[0059] Figure 3 FIG. 3 is a flow chart of a method 300 for detecting defects on a camshaft metal surface according to some embodiments of the present application. It should be noted that: Figure 3 The method 300 shown is similar to Figure 1 The systems 100 shown are complementary to each other, and the related contents are complementary to each other, and will not be described in detail.

[0060] like Figure 3 As shown, the method 300 starts at step S310 .

[0061] In S310, an image to be detected is obtained. The image to be detected includes a metal processing surface of a camshaft, which can be acquired by an image acquisition device 110. For details, please refer to the description of the image acquisition device 110 in the above text. Take a cam surface in the metal processing surface of the camshaft as an example. Figure 4 A schematic diagram of an image to be detected according to an embodiment of the present application is shown.

[0062] In S320 , the image processing device 120 processes the image to be detected and generates an image of a region of interest indicating the cam surface.

[0063] In some embodiments, the image to be detected is processed as follows.

[0064] The first step is to preprocess the image to be detected to generate a first image. The preprocessing includes grayscale conversion and binarization. Specifically, the image to be detected is converted into a grayscale image and binarized (optionally, the binarization threshold is set to 50) to generate a binary image as the first image. Figure 5 Shown for Figure 4 The first image (a) generated by the processing.

[0065] In the second step, the first image is morphologically processed, and the second image is generated by extracting the contour. Specifically, a predetermined vertical kernel (1×305) is used to perform a morphological opening operation on the first image to remove small noise points in the vertical direction, and the first image in the middle is obtained; a predetermined horizontal kernel (31×1) is used to perform a morphological closing operation on the first image in the middle to fill the small gaps in the horizontal direction to obtain the processed first image; and the second image is generated by extracting contours that meet predetermined conditions from the processed first image, wherein the predetermined conditions include a contour area threshold and a horizontal coordinate interval of pixels. In some embodiments, the contour area threshold is set to 1000000 pixels, and the horizontal coordinate interval of pixels is [1200, 2000]. In other words, the contour is searched in the processed first image. If the contour found meets the predetermined conditions: the number of pixels contained in the area contained by the contour (i.e., the contour area) is greater than 1000000 pixels, and the horizontal coordinate interval of the pixels on the contour is [1200, 2000]), then the area contained in the contour is extracted from the processed first image as the second image.

[0066] Specifically, cv.boundingRect can be used to obtain the bounding rectangle (x, y, w, h) of the contour extracted at this time, and the bounding rectangle is extracted from the processed first image as the second image. Figure 5 A second image (b) is generated by processing the first image (Figure (a)).

[0067] The third step is to combine the image to be detected and the second image to generate an image of the region of interest indicating the cam surface.

[0068] According to this embodiment, the second image mainly includes the cam surface area. The applicant has found in research that the second image may ignore the upper and lower extensions of the cam surface end surface in the image to be detected, such as Figure 4 The circled parts A and B are shown in the figure. Therefore, after obtaining the second image, the upper and lower extended parts of the image to be detected are extended to the second image in combination with the image to be detected to form a more complete image of the region of interest. Figure 4 A and B shown in the figure are schematic, and the present embodiment does not limit the area size of the upper and lower extending parts accordingly.

[0069] Specifically, a new blank mask image is created, the height of which can be consistent with the image to be detected, and the width can be slightly wider than the width of the second image (for example, the width is set to w+20). Then the second image is copied to the mask image, usually to the central area of ​​the mask image, that is, a portion of the upper, lower, left and right areas of the mask image are reserved and not filled. After that, the upper and lower extensions of the camshaft end face are extracted from the image to be processed (in some embodiments, 20 columns of pixels are taken on both sides of the left and right sides with the area indicated by the second image as the center as the upper and lower extensions), and image enhancement operations are performed, and then the pixels of these two parts are correspondingly copied to the upper and lower 20 rows of pixels of the mask image. In other words, the pixels of part A in the image to be processed are filled into the pixels of the 0th to 19th rows of the mask image (pixels are taken from part A column by column, and then filled into the mask image by row); similarly, the pixels of part B in the image to be processed are filled into the 20 rows of pixels at the bottom of the mask image (pixels are taken from part B column by column, and then filled into the mask image by row). It should be understood that each row in the mask image contains (w+20) pixels. Therefore, when taking pixels of parts A and B from the image to be processed, it is best to control the number of pixels in each column to (w+20) pixels while including the upper and lower extensions of the camshaft end surface. Figure 5 The figure shows a region of interest image (c) generated by processing the first image (figure (a)).

[0070] In S330, the image detection device 130 inputs the region of interest image into the anomaly detection model for processing, and outputs an anomaly score map, wherein the anomaly score in the anomaly score map represents the probability value of the corresponding pixel having an anomaly.

[0071] According to the implementation of the present application, the anomaly detection model includes a coupled pre-trained convolutional network component, a converter component and a multi-layer perceptron component. Among them, the converter component adopts a U-Transformer model, the multi-layer perceptron component MLP (Multilayer Perceptron) model, and the multi-layer perceptron component finally includes a ReLu activation layer.

[0072] Figure 6 Schematic diagram of an anomaly detection model according to some embodiments of the present application is shown. The anomaly detection model is a hybrid model that combines a convolutional neural network (CNN) and a Transformer architecture. Transformer is a deep learning model that uses an attention mechanism to improve the model training speed, which will not be described in detail here. Figure 6, the transformer component is an inverted pyramid-U-shaped structure. U-Transformer combines U-Net and Transformer, using temporal downsampling in the encoder and upsampling in the decoder to construct the U-Transformer structure. Figure 6 In the example, a three-level encoder and decoder are used. The Split operation refers to dividing the feature data into multiple groups, and the Skip connection operation refers to the skip connection in the residual network. Both are common operations in neural networks and will not be described here.

[0073] Combination Figure 6 , the process of inputting the region of interest image into the anomaly detection model for processing includes: inputting the region of interest image into the pre-trained convolutional network component, and outputting a multi-scale feature map after processing; inputting the multi-scale feature map into the converter component, and outputting a reconstructed feature map after processing; using the reconstructed feature map and the multi-scale feature map to obtain the reconstruction error; inputting the reconstructed feature map into the MLP, and after processing, the output of the MLP and the reconstruction error are input into the activation layer (ReLu activation layer) to obtain the anomaly score map. Among them, the anomaly score map keeps the same size as the input region of interest image, and the anomaly score of each pixel represents the probability value of the pixel having a defect. The larger the score, the greater the probability of the defect.

[0074] According to an embodiment of the present application, method 300 further includes steps (1) to (3) of training an anomaly detection model.

[0075] (1) Obtain multiple region of interest images as training samples. According to the embodiment of the present application, the above processing process (S320) can be performed on the existing image containing the metal processing surface of the camshaft to obtain the region of interest image, which will not be expanded here. The training samples include positive samples without defects and negative samples containing defects.

[0076] At the same time, the defective areas in the training samples are marked as the labeled data of the training samples. Specifically, the Labelme tool is used to semantically annotate the negative samples and mark the defective parts, such as Figure 7 As shown. According to the marked defect part, the corresponding mask image is generated, as shown Figure 8 As shown in , it is used as the labeled data. For the positive sample, the pixel values ​​in the mask image can be directly set to 0 as its labeled data.

[0077] (2) Construct the initial anomaly detection model and model parameters. The network architecture of the anomaly detection model can be referenced Figure 6 shown.

[0078] (3) The training samples are input into the initial anomaly detection model for processing, the first parameter is adjusted by optimizing the reconstruction error, and the second parameter is adjusted by comparing the predicted anomaly score map with the labeled data of the training samples until the training is completed, and finally the anomaly detection model is obtained.

[0079] Specifically, the first parameter includes the parameters of the pre-trained convolutional network component and the parameters of the converter component, and the second parameter includes the parameters of the multi-layer perceptron component. The training process can be divided into two stages. In the first stage, the training sample is input into the model, and after being processed by the pre-trained convolutional network component and the converter component, the reconstruction error is output. The first parameter is adjusted by optimizing the reconstruction error, and the iteration is repeated until the training of the first stage is completed to obtain the trained pre-trained convolutional network component and the converter component. On this basis, the second stage is entered, and the reconstructed feature map and the reconstruction error are successively input into the multi-layer perceptron component, and the abnormal score map is obtained after processing. Combined with the abnormal score map and the labeled data, the parameters of the multi-layer perceptron component are adjusted (for example, the second parameter is adjusted by calculating the loss function of the abnormal score map and the labeled data), and the iteration is repeated until the second stage training is completed. The model at this time is the final anomaly detection model. The training process according to this embodiment is suitable for the case where the number of defective samples is very small.

[0080] After the anomaly detection model is trained, it will be tested and verified to evaluate the performance of the model. This will not be described in detail here.

[0081] In S340, a first number of abnormal scores are selected in descending order of the abnormal scores and an average of the selected abnormal scores is calculated.

[0082] In some embodiments, the first number is 500. That is, the pixel range is set to 500 pixels with the highest abnormality scores, and the average value of the abnormality scores of these 500 pixels is calculated.

[0083] In S350, the defect information of the image to be detected is determined by comparing the mean value with a predetermined threshold value, wherein the defect information includes whether there is a defect in the image to be detected and the defect location, and the predetermined threshold value is determined by analyzing the abnormality detection model.

[0084] Specifically, the defect information of the image to be detected is determined in the following manner: if the mean is greater than a predetermined threshold, it is determined that the image to be detected has defects, and the pixel area corresponding to the first number of abnormal scores is the location of the defect; if the mean is not greater than the predetermined threshold, it is determined that the image to be detected has no defects.

[0085] According to this embodiment, the predetermined threshold is usually determined by evaluating the performance of the anomaly detection model through validation samples after the anomaly detection model is trained. The specific steps include the following (1) to (4).

[0086] (1) Obtain multiple images of regions of interest as verification samples, and annotate the defective regions in the verification samples to generate annotated data of the verification samples, wherein the verification samples include positive samples without defects and negative samples with defects.

[0087] (2) Input the verification sample into the anomaly detection model for processing and output the anomaly score map of the verification sample.

[0088] (3) Combined with the anomaly score map of the validation sample and its annotated data, the true positive rate and false positive rate under each undetermined threshold are calculated, and a receiver operating characteristic curve (i.e., ROC curve) is generated based on the true positive rate and false positive rate.

[0089] The ROC curve is a graphical performance evaluation method used to visualize the performance of a classifier at different classification thresholds. It is mainly composed of two parts: the true positive rate and the false positive rate. The calculation formula is as follows:

[0090] TPR = TP / (TP+FN),

[0091] FPR = FP / (FP + TN),

[0092] Where TP is the true positive example, FN is the false negative example, FP is the false positive example, TN is the true negative example, TPR is the true positive rate, and FPR is the false positive rate.

[0093] The pending thresholds include, for example, 0, 0.1, 0.2, 0.3, ..., 1, and this embodiment does not impose too many restrictions on this. Usually, the value range of the pending threshold is 0-1.

[0094] With the false positive rate FPR as the X-axis and the true positive rate TPR as the Y-axis, record the coordinate points (FPR, TPR) of the true positive rate and false positive rate under each undetermined threshold, and connect all the coordinate points to form an ROC curve. Fig. 9 FIG. 4 shows an ROC curve according to some embodiments of the present application. Fig. 9 As shown, the horizontal axis represents the false positive rate (False Positive Rate), the vertical axis represents the true positive rate (True Positive Rate), and the dotted line represents the diagonal reference line.

[0095] (4) Determine the predetermined threshold by analyzing the ROC curve.

[0096] Specifically, according to the ROC curve, a point on the curve with the maximum true positive rate TPR and the minimum false positive rate FPR is selected, that is, the point closest to (0,1), and the threshold corresponding to the point is the predetermined threshold.

[0097] According to the method 300 of the present application, the image to be inspected is first processed, and the ROI region is extracted by image filtering, binarization and other operations to generate an image of the region of interest; then, the image of the region of interest is processed by the anomaly detection model to obtain an anomaly score map; by comparing with a predetermined threshold, it is determined whether there is a defect in the image to be inspected, and if there is a defect, its defect location. In addition, by judging the six images to be inspected of the metal processing surface of the camshaft, it can be further determined whether the camshaft part has a defect.

[0098] In addition, when training the anomaly detection model, an unsupervised learning method is used to train with a small number of negative sample annotations, which effectively overcomes the problem of a small number of defective samples. At the same time, the ROC curve is calculated and analyzed for the verification samples to determine the appropriate threshold for judging whether there is a defect.

[0099] This solution is low-cost and highly efficient and can be used to detect defects on metal machined surfaces of various camshafts produced in factories.

[0100] This application also discloses:

[0101] A computing device comprises: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method described above.

[0102] A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method described above.

[0103] A computer program product comprises a computer program / instruction, wherein the computer program / instruction implements the above method when executed by a processor.

[0104] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present application, or certain aspects or portions of the method and apparatus of the present application may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present application.

[0105] In the case where the program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the method of the present application according to the instructions in the program code stored in the memory.

[0106] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0107] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present application. According to the above description, it is obvious to construct the structure required for such systems. In addition, the present application is not directed to any specific programming language either. It should be understood that the content of the present application described herein can be realized by various programming languages, and the description of the specific language above is to disclose the preferred implementation of the present application.

[0108] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0109] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0110] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subassemblies. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise clearly stated, each feature disclosed in this specification can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0111] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present application and to form different embodiments.

[0112] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices that are used to implement the functions performed by the elements for the purpose of implementing the disclosure.

[0113] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely refers to different instances of similar objects and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner. In addition, the quantifier "plurality" means "two" and / or "more than two."

[0114] Although the present application is described according to a limited number of embodiments, it will be appreciated by those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the present application thus described. In addition, it should be noted that the language used in this specification is selected primarily for the purpose of readability and teaching, rather than for the purpose of explaining or limiting the subject matter of the present application. Therefore, many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. For the scope of the present application, the disclosure made to the present application is illustrative, not restrictive, and the scope of the present application is defined by the appended claims.

Claims

1. A method for detecting defects on a camshaft metal processing surface, comprising: Acquire an image to be detected, wherein the image to be detected includes a metal processing surface of a camshaft; Processing the image to be detected to generate an image of a region of interest indicating a cam surface; Inputting the region of interest image into an anomaly detection model for processing, and outputting an anomaly score map, wherein the anomaly score in the anomaly score map represents the probability value of an abnormality existing in the corresponding pixel; Select a first number of abnormal scores in descending order of abnormal scores and calculate the mean of the selected abnormal scores; By comparing the mean value with a predetermined threshold, the defect information of the image to be detected is determined, wherein the defect information includes whether there is a defect in the image to be detected and the defect position, and the predetermined threshold is determined by analyzing the abnormality detection model.

2. The method of claim 1, wherein: The anomaly detection model includes a coupled pre-trained convolutional network component, a converter component, and a multi-layer perceptron component.

3. The method of claim 2, wherein: The step of inputting the region of interest image into an anomaly detection model for processing and outputting an anomaly score map comprises: Inputting the region of interest image into a pre-trained convolutional network component, and outputting a multi-scale feature map after processing; Input the multi-scale feature map into a converter component, and output a reconstructed feature map after processing; Using the reconstructed feature map and the multi-scale feature map, a reconstruction error is obtained; The reconstructed feature map is input into a multi-layer perceptron component, and after being processed, it is input into an activation layer together with the reconstruction error to obtain an abnormal score map.

4. The method according to any one of claims 1 to 3, wherein: The step of determining the defect information of the image to be detected by comparing the mean value with a predetermined threshold value includes: If the mean value is greater than a predetermined threshold, it is determined that there is a defect in the image to be detected, and the pixel area corresponding to the first number of abnormal scores is the location of the defect; If the mean value is not greater than a predetermined threshold, it is determined that the image to be detected has no defects.

5. The method according to any one of claims 1 to 4, wherein: Determining the predetermined threshold comprises: Acquire multiple region-of-interest images as verification samples, and annotate defective regions in the verification samples to generate annotated data for the verification samples, wherein the verification samples include positive samples that do not contain defects and negative samples that contain defects; Input the verification sample into an anomaly detection model for processing, and output an anomaly score graph of the verification sample; Combined with the abnormal score graph of the verification sample and its annotated data, the true positive rate and the false positive rate under each pending threshold are calculated, and a receiver operating characteristic curve is generated based on the true positive rate and the false positive rate; by analyzing the receiver operating characteristic curve, the predetermined threshold is determined.

6. The method according to any one of claims 1 to 5, further comprising the step of training an anomaly detection model, comprising: Acquire multiple region-of-interest images as training samples, and annotate defect regions in the training samples as annotated data for the training samples, wherein the training samples include positive samples that do not contain defects and negative samples that contain defects; Build the initial anomaly detection model and model parameters; The training samples are input into the initial anomaly detection model for processing, and the first parameter is adjusted by optimizing the reconstruction error. And by comparing the predicted anomaly score map with the labeled data of the training samples, the second parameter is adjusted until the training is completed, and finally the anomaly detection model is obtained.

7. The method according to any one of claims 1 to 6, wherein: The processing of the image to be detected to generate an image of a region of interest indicating a cam surface includes: Preprocessing the image to be detected to generate a first image, wherein the preprocessing includes grayscale conversion and binarization; Performing morphological processing on the first image and generating a second image by extracting contours; The image to be detected and the second image are combined to generate an image of a region of interest indicating the cam surface.

8. The method of claim 7, wherein: Performing morphological processing on the first image and generating a second image by extracting contours includes: Performing a morphological opening operation on the first image using a predetermined vertical kernel to obtain an intermediate first image; Using a predetermined horizontal kernel to perform a morphological closing operation on the intermediate first image to obtain a processed first image; The second image is generated by extracting contours satisfying predetermined conditions from the processed first image, wherein the predetermined conditions include a contour area threshold and a horizontal coordinate interval of pixels.

9. A camshaft metal surface defect detection system, comprising: The image acquisition device is suitable for placing the camshaft parts on the measuring table and fixing them through the front and rear cylinders, photographing the metal processing surface of the camshaft at different angles to obtain the image to be tested; An image processing device, adapted to process the image to be detected to generate an image of a region of interest indicating a cam surface; An image detection device, adapted to input the region of interest image into an anomaly detection model for processing, and output an anomaly score map, wherein the anomaly score in the anomaly score map represents a probability value of an anomaly existing in the corresponding pixel; The defect detection device is adapted to select a first number of abnormal scores from the abnormal score map in descending order of abnormal scores and calculate the average of the selected abnormal scores; by comparing the average with a predetermined threshold, the defect information of the image to be detected is determined, wherein the defect information includes whether there is a defect in the image to be detected and the defect position, Wherein, the predetermined threshold is determined by analyzing the anomaly detection model.

10. The system according to claim 9, further comprising a model generating device, The model generation device is coupled to the image detection device and is suitable for training and generating the anomaly detection model, wherein the anomaly detection model includes a coupled pre-trained convolutional network component, a converter component, and a multi-layer perceptron component; The model generation device is also coupled to the defect detection device, and is also suitable for analyzing the abnormality detection model through verification samples after training is completed to determine the predetermined threshold.

Images