A method and system for detecting defects on the end face of a camshaft

Abstract

This application discloses a method and system for detecting defects on the end face of a camshaft. The method includes: acquiring eight end face images of the camshaft; classifying the end face images based on the presence or absence of threads at the vertices of the end faces; processing the first and second types of end face images to extract contours from the corresponding end face images; and extracting regions of interest (ROIs) from the corresponding end face images based on the contours to obtain ROI images; inputting the third type of end face images into a semantic segmentation model for processing to extract ROIs to obtain ROI images; selecting a target detection model for detection based on the end face images corresponding to the ROI images to output detection results indicating whether defects exist in the ROI images; and determining that the camshaft end face is defect-free when no defects are found in any of the ROI images. This solution is low-cost and highly efficient.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and system for detecting defects on the end face of a camshaft. Background Technology

[0002] The camshaft is a key power transmission component in an internal combustion engine, playing a crucial role in the engine's operation. It primarily controls the opening and closing of the valves, thereby regulating the timing of intake and exhaust, directly affecting engine performance, efficiency, and emissions. Camshaft manufacturing involves multiple precision machining steps, including blank casting, rough machining, heat treatment, finish machining, and final inspection. Each manufacturing stage requires strict quality control to ensure the camshaft's precision, strength, and durability. However, due to various factors such as material defects, machining errors, or improper heat treatment, defects may appear on the camshaft end face, such as surface cracks, pits, abnormal wear, contour deviations, or uneven hardness. The presence of these defects can severely impact engine performance and lifespan, potentially leading to abnormal noise, power loss, and even more serious mechanical failures.

[0003] Therefore, a defect detection scheme for camshaft end face is needed to accurately identify the above defects. Summary of the Invention

[0004] This application provides a method and system for detecting defects on the end face of a camshaft, in an attempt to solve or at least alleviate at least one of the problems mentioned above.

[0005] According to one aspect of this application, a method for detecting defects on the end face of a camshaft is provided, comprising: acquiring eight end face images of the camshaft; distinguishing each end face image based on whether a thread exists at the apex of the end face; and classifying the end face images, including: classifying the second, fourth, and sixth end face images as first-class end face images (D1-D3); classifying the first, third, and fifth end face images as second-class end face images (X1-X3); and classifying the seventh and eighth end face images as third-class end face images (D4, X4); and further classifying the first-class end face images... The first type of end face image and the second type of end face image are processed separately to extract the contours in the corresponding end face images, and based on the contours, the regions of interest (ROIs) in the corresponding end face images are extracted to obtain ROI images; the third type of end face image is input into a semantic segmentation model for processing to extract the ROIs to obtain ROI images; according to the end face image corresponding to the ROI image, a target detection model is selected for detection to output the detection results, which indicate whether there are defects in the ROI images; when there are no defects in any of the ROI images, it is determined that there are no defects in the end face of the camshaft.

[0006] Optionally, in the method according to this application, the processing of the first type of end face image includes: processing the original first type of end face image to obtain a binary image, and traversing the contours in the binary image to select the contours with area pixels in a first interval; determining the center and radius of the circle using a minimum radius circle function based on the selected contours; generating a first type of mask image based on the original first type of end face image and the determined center and radius; and performing a bitwise AND operation between the original first type of end face image and the first type of mask image to extract the region of interest of the first type of end face image.

[0007] Optionally, in the method according to this application, the processing of the second type of end face image includes: processing the original second type of end face image to obtain a binary image, traversing the contours in the binary image, calculating the minimum contour using the minimum radius circle function, and determining the initial circle center; cropping the original second type of end face image according to a predetermined pixel region with the initial circle center as the center, to obtain a cropped second type of end face image; binarizing the cropped second type of end face image, and selecting the contour of the area pixels in the second interval; determining the circle center and radius using the minimum radius circle function based on the selected contour; generating a second type of mask image based on the original second type of end face image and the determined circle center and radius; and performing a bitwise AND operation on the original second type of end face image and the second type of mask image to extract the region of interest of the second type of end face image.

[0008] Optionally, the method according to this application further includes the step of training a semantic segmentation model, comprising: generating a semantic segmentation dataset by collecting multiple third-class end-face images and their corresponding annotation data, wherein the annotation data indicates the region of interest in the corresponding third-class end-face image; and training an initial semantic segmentation model using the semantic segmentation dataset to generate a semantic segmentation model, wherein the initial semantic segmentation model is based on the UNet model.

[0009] Optionally, in the method according to this application, selecting a target detection model for detection based on the end face image corresponding to the region of interest image includes: if the region of interest image does not correspond to the second end face image, then selecting a first target detection model for detection; if the region of interest image corresponds to the second end face image, then selecting a second target detection model for detection, wherein both the first target detection model and the second target detection model are based on the YOLO model.

[0010] Optionally, the method according to this application further includes the step of training and generating a first object detection model, comprising: generating a first recognition dataset by acquiring the first end face image and the region of interest images corresponding to the third to eighth end face image samples, and labeling the region of interest images with defects; using the first recognition dataset, training an initial object detection model to generate a first object detection model, wherein the initial object detection model includes a backbone network component, a feature extraction component, and a detection head component.

[0011] Optionally, the method according to this application further includes the step of training and generating a second object detection model, comprising: generating a second recognition dataset by acquiring region of interest images corresponding to the second end face image samples, and labeling the region of interest images with defects; improving the initial object detection model to generate an improved object detection model, wherein the initial object detection model includes a backbone network component, a feature extraction component, and a detection head component; and training the improved object detection model using the second recognition dataset to generate the second object detection model.

[0012] Optionally, in the method according to this application, the improvement of the initial target detection model includes: replacing convolutional blocks with spatial depth transformation convolutional blocks in the backbone network component and the feature extraction component, and adding a 3×3 convolutional layer before the spatial depth transformation convolutional blocks; introducing deformable convolutions in the last C2f block of the backbone network component; and introducing a dynamic head module in the detection head component.

[0013] According to another aspect of this application, a camshaft end face defect detection system is provided, comprising:

[0014] The image acquisition device is suitable for placing camshaft parts on a measuring table and fixing them with front and rear cylinders, and taking pictures of the front and rear end faces of the parts to obtain 8 end face images.

[0015] An image classification device is adapted to distinguish end-face images based on whether there are threads at the vertices of the end face, and to classify the end-face images, including: classifying the second, fourth, and sixth end-face images as first-class end-face images (D1-D3), the first, third, and fifth end-face images as second-class end-face images (X1-X3), and the seventh and eighth end-face images as third-class end-face images (D4, X4); a first image processing device is adapted to process the first-class and second-class end-face images respectively to obtain corresponding regions of interest (ROI) images; a second image processing device is adapted to input the third-class end-face images into a semantic segmentation model for processing, extracting the ROI to obtain corresponding ROI images; an image detection device is adapted to select a target detection model for detection based on the end-face images corresponding to the ROI images, and output detection results, the detection results indicating whether there are defects in the ROI images; and a defect determination device is adapted to determine that the end face of the camshaft is defect-free when there are no defects in any of the ROI images.

[0016] According to another aspect of this application, a computing device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the methods described above.

[0017] According to another aspect of this application, a computer-readable storage medium is provided for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method as described above.

[0018] According to another aspect of this application, a computer program product is provided, including a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the method described above.

[0019] In summary, according to the scheme of this application, each end face image is first distinguished based on whether there is a thread at the vertex of the end face, and the end face images are then classified. For different types of end face images, different methods are used to extract the regions of interest (ROIs) indicating the end face portion, generating ROI images. Then, depending on whether the ROI image originates from a second end face image, different target detection models are selected to process the ROI images to detect defects. Based on the detection results of all ROI images, it can be determined whether there are defects on the end face of the camshaft. This detection method is low-cost and highly efficient, and can be used for defect detection on the end faces of various camshafts produced in factories.

[0020] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0021] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles claimed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this application will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this application, the same reference numerals generally refer to the same parts or elements.

[0022] Figure 1 A schematic diagram of a camshaft end face defect detection system 100 according to some embodiments of this application is shown;

[0023] Figure 2 A schematic diagram of a computing device 200 according to some embodiments of this application is shown;

[0024] Figure 3 A schematic flowchart of a camshaft end face defect detection method 300 according to some embodiments of this application is shown;

[0025] Figure 4 The original image (a), grayscale image (b), binary image (c), and region of interest image (d) of a second end face image according to an embodiment of this application are shown.

[0026] Figure 5 The original image (a), grayscale image (b), binary image (c), cropped image (d), binary image of the cropped image (e), and region of interest image (d) of a first end face image according to an embodiment of the present application are shown.

[0027] Figure 6 The original image (a), the annotated mask image (b), and the region of interest image (c) of the eighth end face image (D4) according to an embodiment of this application are shown.

[0028] Figure 7 An architecture diagram of an improved target detection model according to some embodiments of this application is shown. Detailed Implementation

[0029] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0030] Because there are many end faces of the camshaft, and these end faces interfere with each other, after acquiring images of each end face, it is necessary to first extract the region of interest (ROI) of each end face, and then use an object detection algorithm to detect objects on different end faces. The region of interest is the area in the end face image that represents the end face portion.

[0031] It should be understood that the main body of the camshaft is a cylindrical rod of the same length as the cylinder bank. Several cams are fitted onto it to drive the valves. One end of the camshaft is a bearing support point, and the other end is connected to the drive wheel. In the following embodiment, the end connected to the drive wheel is considered the front end of the camshaft, and eight end face images are acquired according to the front-rear relationship of the camshaft.

[0032] Figure 1 A schematic diagram of a camshaft end face defect detection system 100 according to some embodiments of this application is shown.

[0033] like Figure 1 As shown, the camshaft end face defect detection system 100 includes: an image acquisition device 110, an image classification device 120, a first image processing device 130, a second image processing device 140, an image detection device 150, and a defect determination device 160.

[0034] In some embodiments, the image classification device 120, the first image processing device 130, the second image processing device 140, the image detection device 150, and the defect determination device 160 may be arranged on a computing device, which is connected to the image acquisition device 110.

[0035] The image acquisition device 110 includes a measuring platform, a camera, and a light source. The camshaft part is placed on the measuring platform and fixed with front and rear cylinders. Images are taken of the front and rear end faces of the part to obtain eight end face images of the camshaft. Optionally, the light source and camera are on the same horizontal line, and a white bowl-shaped light source is used.

[0036] According to an embodiment of this application, the eight end face images are sequentially named as follows, from the front end to the rear end of the camshaft: first end face image, second end face image, third end face image, fourth end face image, fifth end face image, sixth end face image, seventh end face image, and eighth end face image, following the order from the front end to the rear end of the camshaft.

[0037] The image acquisition device 110 transmits the acquired end face images to the computing device, which processes the end face images to detect whether there are defects in the end face images and further determines whether there are defects in the end face of the camshaft.

[0038] Due to the different positions of the end faces, their outline sizes vary. Therefore, each end face image needs to be processed separately to obtain the required end face ROI region.

[0039] First, the image classification device 120 distinguishes each end face image based on whether there is a thread at the vertex of the end face.

[0040] Specifically, the acquired end face images are divided into eight end faces according to the front and rear of the camshaft: D1, D2, D3, D4, X1, X2, X3, and X4. Among them, D1 to D4 are the rear end faces of the camshaft, and X1 to X4 are the front end faces of the camshaft. In other words, the end face images are distinguished according to whether there are threads at the vertices of the end faces. The end faces with threads are X1-X4, and the end faces without threads are D1-D4. Thus, the images are as follows: first end face image (X1), second end face image (D1), third end face image (X2), fourth end face image (D2), fifth end face image (X3), sixth end face image (D3), seventh end face image (X4), and eighth end face image (D4).

[0041] In this embodiment, the second, fourth, and sixth end face images are designated as first-class end face images (i.e., D1 to D3), the first, third, and fifth end face images are designated as second-class end face images (i.e., X1 to X3), and the seventh and eighth end face images are designated as third-class end face images (i.e., X4 and D4).

[0042] Since the end faces in the first and second types of end face images are complete circular outlines, traditional image processing methods can be used to extract the outline and obtain the region of interest. However, the end faces in the third type of end face images are not regular circular outlines; therefore, a deep learning model is used to extract the region of interest. The specific processing procedure is as follows.

[0043] The first image processing device 130 processes the first type of end-face image and the second type of end-face image respectively to obtain the corresponding region of interest images. During contour extraction, the first type of end-face image only requires contour searching once, while the second type of end-face image, being slightly more complex, requires contour searching twice through fine-tuning. The specific processing methods are detailed below.

[0044] The second image processing device 140 inputs the third type of end-face image into the semantic segmentation model for processing, extracts the region of interest, and obtains the corresponding region of interest image. In some embodiments, the semantic segmentation model is based on the UNet model, and the UNet model is trained using a large number of labeled D4 and X4 end-face images to obtain the semantic segmentation model.

[0045] Subsequently, the image detection device 150 selects a target detection model for detection based on the end face image corresponding to the region of interest image, and outputs a detection result, wherein the detection result indicates whether a defect exists in the region of interest image. In some embodiments, the target detection model is based on the YOLO model, and the YOLO model is trained using a large number of labeled end face images to obtain the target detection model. The target detection model will be described in detail below.

[0046] The defect determination device 160 determines whether there is a defect on the end face based on the detection results. Specifically, when there are no defects in any region of interest images, the defect determination device 160 determines that there is no defect on the end face of the camshaft. In other words, if there is a defect in any region of interest image, the end face of the camshaft is determined to have a defect.

[0047] The detection system 100 according to this application has the advantages of low cost and high efficiency, and can be used for defect detection of various camshaft end faces in factory production.

[0048] The computing device can be any device with storage and computing capabilities, such as a server, workstation, or a personal computer like a desktop or laptop computer, and is not limited thereto. In some embodiments, the computing device can also output defect detection results via a visual screen.

[0049] Figure 2 A schematic diagram of a computing device 200 according to some embodiments of this application is shown. It should be noted that... Figure 2 The computing device 200 shown is merely an example. In practice, the computing device used to implement the method of this application can be any type of device, and its hardware configuration can be similar to... Figure 2 The computing device 200 shown is the same as, and can also be with Figure 2 The computing device 200 shown is different. In practice, the computing device used to implement the embodiments of this application can... Figure 2 The hardware components of the computing device 200 shown may be added or removed. This application does not limit the specific hardware configuration of the computing device.

[0050] like Figure 2In a basic configuration, computing device 200 includes at least one processing unit 202 and system memory 204. According to one aspect, depending on the configuration and type of the computing device, the processing unit 202 may be implemented as a processor. System memory 204 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memory. According to one aspect, system memory 204 includes an operating system 205 and program modules 206.

[0051] According to one aspect, operating system 205 is, for example, suitable for controlling the operation of computing device 200. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 2 The basic configuration is illustrated by the components within the dashed lines 208. According to one aspect, the computing device 200 has additional features or functions. For example, according to one aspect, the computing device 200 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 2 The image is shown by removable storage 209 and non-removable storage 210.

[0052] As stated above, according to one aspect, a program module is stored in system memory 204. According to one aspect, the program module may include one or more applications. This application does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc.

[0053] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 2Each or many of the components shown can be implemented as an example of a system-on-a-chip (SOC) integrated on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operated via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 200. Embodiments of this application can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of this application can be implemented within a general-purpose computer or in any other circuit or system.

[0054] According to one aspect, computing device 200 may also have one or more input devices 212, such as a keyboard, mouse, pen, voice input device, touch input device, VR motion capture input device, etc. It may also include output devices 214, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 200 may include one or more communication connections 216 that allow communication with other computing devices 218. Examples of suitable communication connections 216 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0055] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 204, removable storage 209, and non-removable storage 210 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital universal disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing device 200. According to one aspect, any such computer storage medium can be part of computing device 200. Computer storage media does not include carrier waves or other propagated data signals.

[0056] According to one aspect, a communication medium is implemented by computer-readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, a communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0057] In some other embodiments of this application, computing device 200 is configured to execute camshaft end face defect detection method 300 according to this application. Program module 206 includes multiple program instructions for executing method 300, which can instruct processing unit 202 to execute method 300 of this application.

[0058] Figure 3 A schematic flowchart of a camshaft end face defect detection method 300 according to some embodiments of this application is shown. It should be noted that... Figure 3 Method 300 shown Figure 1 The systems 100 shown are complementary to each other, and the relevant parts will not be elaborated further.

[0059] like Figure 3 As shown, method 300 begins with step S310.

[0060] In S310, eight end face images of the camshaft are acquired, and each end face image is distinguished based on whether there is a thread at the vertex of the end face, and the end face images are classified.

[0061] In this embodiment, after acquiring the end-face image from the image acquisition device 110, considering the differences in the size of the end-face contour due to different end-face positions, the end-face images are first classified to facilitate subsequent processing. The classification of end-face images can be found in the preceding description. Specifically, the classification methods include: classifying the second, fourth, and sixth end-face images as first-class end-face images (D1-D3); classifying the first, third, and fifth end-face images as second-class end-face images (X1-X3); and classifying the seventh and eighth end-face images as third-class end-face images (D4, X4).

[0062] In S320, the first type of end face image and the second type of end face image are processed respectively to extract the contours in the corresponding end face images, and based on the extracted contours, the region of interest in the corresponding end face image is extracted to obtain the region of interest image.

[0063] Since the end face profile with threads is slightly more complex, the processing method for the second type of end face image is to find the profile twice through fine-grained operations to finally determine the region of interest; the processing method for the first type of end face image is to find the profile only once to determine the region of interest. The two processing methods are described below.

[0064] (1) The processing of the first type of end-face images (D1~D3) includes the following steps. It should be understood that the first type of end-face images includes 3 images. The following steps need to be performed on D1, D2, and D3 respectively to determine their respective regions of interest. Then, the regions of interest are cropped from the corresponding end-face images, which are the corresponding region of interest images. That is, when D1 is processed, its binary image, the first type of mask image, and the region of interest are obtained; when D2 is processed, its binary image, the first type of mask image, and the region of interest are obtained; when D3 is processed, its binary image, the first type of mask image, and the region of interest are obtained.

[0065] First, the original first-class end face image is processed to obtain a binary image, and the contours in the binary image are traversed to select the contours with area pixels in the first interval.

[0066] Specifically, the original first-type end-face image is converted into a grayscale image and then binarized to obtain a binary image. Taking D1 as an example, ... Figure 4 The original image (a), grayscale image (b), and binary image (c) of D1 are shown. For each binary image, all contours in the image are traversed, and the number of pixels contained in the area of ​​each contour is counted, recorded as the area pixel value of the contour. Then, contours with area pixels in a first interval (optionally, the first interval is 240000-650000, which is an approximate range obtained by calculating the contour area of ​​several corresponding end face images, but is not limited to this) are selected. In this embodiment, the first interval indicates the area pixel value of the contour with the smaller area in the end face image.

[0067] Next, based on the selected contour, the center and radius are determined using the minimum radius circle function. In this embodiment, existing library functions can be used to calculate the center and radius. For example, the minimum radius circle function (x,y), radius = cv.minEnclosingCircle(contour) in OpenCV can be used to determine the center and radius of the contour circle.

[0068] Then, based on the original first-type end face image and the determined center and radius, a first-type mask image is generated. Specifically, a black mask image of the same size as the original first-type end face image (i.e., the corresponding D1, D2, D3) is created, and an annular region is drawn on the black mask image according to the center and radius determined in the previous step.

[0069] Finally, a bitwise AND operation is performed between the original first-type end-face image and the corresponding first-type mask image to extract the region of interest (ROI) of the first-type end-face image. For example... Figure 4 (d) in the figure shows the region of interest image obtained after processing D1 (i.e., Figure (a)).

[0070] (2) The processing of the second type of end face images (X1 to X3) includes the following steps. Similarly, the second type of end face images includes 3 images. The following steps need to be performed on X1, X2, and X3 respectively to determine their respective regions of interest. Then, the regions of interest are cropped from the corresponding end face images, which are the corresponding region of interest images. This will not be elaborated here.

[0071] First, the original second-type end face image is processed to obtain a binary image. Then, the contours in the binary image are traversed, and the minimum radius circle function is used to calculate the minimum contour to determine the initial circle center.

[0072] Specifically, the original second-type end-face image is converted into a grayscale image and then binarized to obtain a binary image. Taking X1 as an example, as follows... Figure 5 The original image (a), grayscale image (b), and binary image (c) of X1 are shown. Then, a contour lookup function is used to obtain all contours in the binary image, and a minimum radius circle function is used to calculate the minimum contour to obtain the initial circle center. As mentioned earlier, both the contour lookup function and the minimum radius circle function can be existing library functions, which will not be elaborated upon here.

[0073] Next, using the initial center as the center, the original second-type end-face image is cropped according to a predetermined pixel region to obtain the cropped second-type end-face image. Optionally, the size of the predetermined pixel region is 1400x1400. This step cropped the original image, narrowing the search area. Figure 5 (d) shows the cropped X1.

[0074] Next, the cropped second-type end-face image is binarized, and the area pixels are selected from it within the second interval (optionally, the second interval is 6000-8500, which is an approximate range obtained by calculating the contour area of ​​several corresponding cropped end-face images, but is not limited to this). Figure 5 Figure (e) shows the cropped binary image of X1 (i.e., Figure (d)).

[0075] Next, based on the contour selected in the previous step, the center and radius of the circle are determined using the minimum radius circle function. This step is processed in the same way as the previous processing of the first type of end-face image, and will not be repeated here.

[0076] Then, based on the original second-type end face image and the determined center and radius, a second-type mask image is generated.

[0077] Specifically, a black mask image of the same size as the original second-type end face image (i.e., the corresponding X1, X2, X3) is created, and an annular region is drawn on the black mask image according to the center and radius determined in the previous step.

[0078] Finally, a bitwise AND operation is performed between the original second-type end-face image and the corresponding second-type mask image to extract the region of interest (ROI) of the second-type end-face image. For example... Figure 5 Figure (f) shows the region of interest image obtained after processing X1 (i.e., Figure (a)).

[0079] In S330, the third type of end face image is input into the semantic segmentation model for processing, and the region of interest is extracted to obtain the region of interest image.

[0080] In some embodiments according to this application, the semantic segmentation model is based on the UNet model. UNet (full name U-Net) is a deep learning architecture originally proposed by Olaf Ronneberger, Philipp Fischer, and Thomas Brox in 2015 for image segmentation tasks. The network's name comes from its U-shaped architecture, which allows the network to capture multi-scale feature information during encoding and decoding. The network structure of the UNet model will not be elaborated here. In this embodiment, a third-class end-face image is input into the semantic segmentation model for segmentation and extraction, outputting a region of interest (ROI). Cropping the ROI from the third-class end-face image yields the ROI image.

[0081] The method 300 according to this application also includes the step of training a semantic segmentation model, comprising the following three steps.

[0082] The first step involves generating a semantic segmentation dataset by acquiring multiple third-type end-face images and their corresponding annotation data. The annotation data indicates the regions of interest (ROIs) within the corresponding third-type end-face images. Specifically, multiple D4 and X4 images are acquired and semantically annotated to identify the ROIs, forming the semantic segmentation dataset. In some embodiments, an annotation mask is generated using the annotated ROIs, where white areas represent the ROIs. Figure 6The original image (a) and the annotated mask image (b) of the eighth end face image (D4) according to an embodiment of this application are shown.

[0083] The second step is to use the semantic segmentation dataset to train the initial semantic segmentation model and generate a new semantic segmentation model.

[0084] As mentioned earlier, the initial semantic segmentation model is based on UNet, which is the original UNet model. The acquired D4 or X4 data is input into the initial semantic segmentation model for processing to output the region of interest (ROI). The output ROI is compared with the labeled data, the loss function is calculated, and the network parameters are adjusted. This process is repeated iteratively until training is complete, ultimately generating the semantic segmentation model. The training process of the UNet model will not be elaborated here.

[0085] In some embodiments, the semantic segmentation dataset can be further divided into a training set, a test set, and a validation set. The semantic segmentation model is trained using the training set, and then the trained semantic segmentation model is optimized using the test set and the validation set. This application does not impose many restrictions on this.

[0086] According to the embodiments of this application, Figure 6 Figure (a) shows the input semantic segmentation model. The processed region of interest image is shown in Figure (c), which contains three regions of interest and is the end face of D4.

[0087] In S340, a target detection model is selected based on the end face image corresponding to the region of interest image to perform detection and output the detection result, where the detection result indicates whether there is a defect in the region of interest image.

[0088] According to this embodiment, the second end face image (D1) contains a keyway portion, which will cause significant interference to defect detection in the end face region. Therefore, during defect detection, the region of interest image of D1 is treated as an image with more interference and processed separately; the region of interest images of other end faces are treated as images with less interference and processed separately.

[0089] Specifically, if the region of interest (ROI) image does not correspond to the second end-face image (D1), the first target detection model is selected for detection. The ROI image is input into the first target detection model for processing, and the detection result is output. If the ROI image corresponds to the second end-face image (D1), the second target detection model is selected for detection. The ROI image is input into the second target detection model for processing, and the detection result is output. The output typically includes the detected bounding boxes (bboxes) and the confidence score for the category prediction. The detection result is obtained by comparing the output confidence score with a set confidence threshold. For example, the score ranges from 0 to 1; the higher the value, the greater the probability of a defect. Assuming a threshold of 0.5, a score greater than 0.5 indicates the presence of a defect. It should be understood that the above is only illustrative, and the threshold can be set based on experience and actual conditions; this embodiment is not limited to this.

[0090] Both the first and second target detection models are based on the YOLO model.

[0091] YOLO (You Only Look Once) is a deep learning-based object detection algorithm first proposed by Joseph Redmon et al. in 2015. YOLO can identify and locate objects in an image with a single look, offering advantages such as speed, high accuracy, strong interpretability, and wide applicability.

[0092] Optionally, both the first and second object detection models are based on YOLOv8. YOLOv8 can be used as the initial object detection model. More specifically, the first object detection model uses YOLOv8, trained by acquiring corresponding end-face images. The second object detection model improves the YOLOv8 network structure to better adapt to D1 end-face images with more interference, and is trained on this basis to obtain the second object detection model.

[0093] According to an embodiment of this application, method 300 further includes the step of training and generating a first object detection model, which includes the following two steps.

[0094] The first step involves generating a first recognition dataset by acquiring the region of interest (ROI) images corresponding to the first end-face image and the third to eighth end-face image samples, and then labeling the ROI images containing defects. Specifically, a large number of ROI images corresponding to D2, D3, D4, X1, X2, X3, and X4 are acquired to generate the first recognition dataset. The ROI images can be generated through the aforementioned steps S310 to S330, which will not be elaborated here. The labelimg software is then used to label the defect samples in the first recognition dataset, marking the ROI images containing defects and generating a txt file. For example, if a defect exists, it is labeled as 1; if no defect exists, it is labeled as 0. Defects include, for example, surface cracks, pits, abnormal wear, contour deviations, or uneven hardness.

[0095] In some embodiments, the first recognition dataset can be divided into a training set, a test set, and a validation set to complete the training, optimization, and validation of the model, which will not be elaborated here.

[0096] The second step involves training the initial object detection model using the first recognition dataset to generate the first object detection model. This initial object detection model includes a backbone network component, a feature extraction component (Neck), and a detection head component (Head), which will not be elaborated upon here.

[0097] Specifically, the region of interest image from the first recognition dataset is input into the initial object detection model (i.e., YOLOv8) for processing, and the output is a detection result indicating whether a defect exists. Using this detection result and labeled data, the loss function is calculated, and the model parameters are adjusted. This process is repeated iteratively until training is complete, ultimately yielding the first object detection model.

[0098] According to an embodiment of this application, method 300 further includes a step of training and generating a second object detection model, specifically including the following three steps.

[0099] The first step involves generating a second recognition dataset by acquiring the regions of interest (ROIs) corresponding to the second end-face image samples, and then labeling the ROI images containing defects. Specifically, a large number of ROI images corresponding to D1 are acquired to generate the second recognition dataset. The ROI images can be generated through the aforementioned steps S310 to S330, which will not be elaborated here. The labelimg software is then used to label the defect samples in the second recognition dataset, marking the ROI images containing defects and generating a txt file. For example, if a defect exists, it is labeled as 1; if no defect exists, it is labeled as 0.

[0100] Similarly, the second recognition dataset can also be divided into training, testing, and validation sets to complete the training, optimization, and validation of the model, which will not be elaborated here.

[0101] The second step is to improve the initial target detection model to generate an improved target detection model.

[0102] The initial object detection model, taking YOLOv8 as an example, includes a backbone network component, a feature extraction component, and a detection head component. The improvements to the initial object detection model include three points. Figure 7 An architecture diagram of an improved object detection model according to some embodiments of this application is shown below, in conjunction with... Figure 7 Explain the areas for improvement.

[0103] (1) In the backbone network component (Backbone) and the feature extraction component (Neck), the original convolutional block (Conv) is replaced by a spatial depth transformation convolutional block (SPD-Conv), and a 3×3 convolutional layer is added before each spatial depth transformation convolutional block to form a C_SPD block.

[0104] SPD-Conv is a new building block used to replace stride convolutions and... Pooling layer It consists of a Spatial-to-Depth (SPD) layer and a Non-stretched Convolutional (Conv) layer. The SPD layer reduces each spatial dimension of the input feature map to the channel dimension while preserving information within each channel. This is achieved by mapping each pixel or feature of the input feature map to a channel. In this process, the spatial dimension decreases while the channel dimension increases. The Non-stretched Convolutional (Conv) layer is a standard convolutional operation performed after the SPD layer. Unlike stretched convolution, non-stretched convolution does not move across the feature map but performs a convolution operation on each pixel or feature map. This helps reduce the oversampling problem that may occur in the SPD layer and preserves more fine-grained information.

[0105] Meanwhile, in this embodiment, a new 3×3 convolutional layer is introduced before SPD-Conv to form the C_SPD block, as shown below. Figure 7 As shown. This helps the model effectively capture local features, thereby accurately extracting the edge and texture information of the target.

[0106] The improved C_SPD block first performs a convolution operation on the input feature map through a 3×3 convolutional layer (Conv), then transforms it through an SPD layer, and finally convolves the output through a Conv layer. This combination reduces the spatial dimension without losing information, while preserving information within the channels, which helps improve the model's detection performance for low-resolution images and small objects.

[0107] (2) A deformable convolution is introduced in the last C2f block of the backbone network component. In this embodiment, the deformable convolution is selected as DCNv2, so the last C2f block in the backbone network component becomes a C2f_DCNv2 block, as shown below. Figure 7 As shown, DCNv2 improves the model's adaptability to different geometric changes. Besides adding more deformable convolutional layers, it also incorporates a learnable offset parameter Δpn into each element involved in the accumulation. Furthermore, the model learns not only the offset of each element but also its weights, enabling it to adapt. Deformable convolution DCNv2 is an existing module in this field and will not be elaborated upon here.

[0108] The formula for this module is as follows:

[0109] (3) Introduce a dynamic head module into the detection head component, such as... Figure 7 The Dyhead shown in the image. The dynamic head module improves the target detection head by applying a self-attention mechanism on different feature dimensions, enabling it to simultaneously possess scale awareness, spatial awareness, and task awareness capabilities.

[0110] The dynamic head module can dynamically adjust the weights of different feature layers, which is beneficial for extracting multi-scale features. Specifically, a three-dimensional feature tensor F∈R is given in the detection head. L*S*C The enhanced output obtained by applying attention to three different locations is as follows: π L (F)=α C (α S (α l (F)·F)·F).

[0111] The improved target detection model focuses more on local features, enhancing its performance in detecting low-resolution images and small objects, as well as its adaptability to various geometric changes. Therefore, it can detect defects in D1 more accurately.

[0112] about Figure 7Other unspecified blocks, such as SPFF, Upsample, Concat, and detect, are all existing modules of YOLOv8. Due to space limitations, they will not be discussed in detail here.

[0113] The third step is to use the second recognition dataset to train the improved target detection model and generate the second target detection model.

[0114] Specifically, the region-of-interest images from the second recognition dataset are input into the improved object detection model for processing, and the output is a detection result indicating whether a defect exists. Using this detection result and labeled data, the loss function is calculated, and the model parameters are adjusted. This process is repeated iteratively until training is complete, ultimately yielding the second object detection model.

[0115] In S350, if there are no defects in the images of each region of interest, it is determined that there are no defects on the end face of the camshaft.

[0116] In other words, the regions of interest images of the eight end faces of the camshaft are inspected individually. Only when there are no defects on any of the eight end faces will the camshaft be deemed to be defect-free, i.e., the camshaft is qualified.

[0117] According to method 300 of this application, each end face image is first distinguished based on whether there is a thread at the vertex of the end face, and the end face images are classified. For different types of end face images, different methods are used to extract the regions of interest (ROIs) indicating the end face portion, generating ROI images. Then, depending on whether the ROI image originates from a second end face image, different target detection models are selected to process the ROI images to detect defects. Based on the detection results of all ROI images, it can be determined whether there are defects on the end face of the camshaft. This detection method is low-cost, highly efficient, and can be used for defect detection on the end faces of various camshafts produced in factories.

[0118] Optionally, it also includes a computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method as described above.

[0119] Optionally, it also includes a computer program product comprising a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the method as described above.

[0120] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of this application, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing this application.

[0121] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the method of this application according to instructions in the program code stored in the memory.

[0122] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0123] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this application. Based on the above description, the required structure for constructing such a system is obvious. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of this application.

[0124] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0125] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0126] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed herein and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed herein may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0127] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments.

[0128] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the method or method elements forms means for implementing the method or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by the elements for the purposes of this disclosure.

[0129] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, sequence, or any other manner. Furthermore, the quantifier “multiple” means “two” and / or “more than two.”

[0130] Although this application has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of this application described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for interpreting or limiting the subject matter of this application. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of this application is illustrative and not restrictive, and the scope of this application is defined by the appended claims.

Claims

1. A method for detecting defects on the end face of a camshaft, comprising: Eight end face images of the camshaft were acquired, and the end face images were distinguished based on whether there were threads at the vertices of the end faces. The end face images were then classified as follows: the second, fourth, and sixth end face images were classified as the first type of end face images; the first, third, and fifth end face images were classified as the second type of end face images; and the seventh and eighth end face images were classified as the third type of end face images. The first type of end face image and the second type of end face image are processed respectively to extract the contours in the corresponding end face images, and based on the contours, the region of interest in the corresponding end face image is extracted to obtain the region of interest image; The third type of end face image is input into a semantic segmentation model for processing to extract the region of interest, thereby obtaining the region of interest image; Based on the end face image corresponding to the region of interest image, a target detection model is selected for detection to output the detection result, which indicates whether there is a defect in the region of interest image; When there are no defects in the images of each region of interest, it is determined that there are no defects on the end face of the camshaft. The step of selecting a target detection model for detection based on the end face image corresponding to the region of interest image includes: if the region of interest image does not correspond to the second end face image, then selecting a first target detection model for detection; and if the region of interest image corresponds to the second end face image, then selecting a second target detection model for detection.

2. The method as described in claim 1, wherein, The processing of the first type of end face image includes: The original first-class end face image is processed to obtain a binary image, and the contours in the binary image are traversed to select the contours with area pixels in the first interval. Based on the selected profile, the center and radius of the circle are determined using the minimum radius circle function; Based on the original first type of end face image and the determined center and radius, a first type of mask image is generated; The original first-type end face image and the first-type mask image are subjected to a bitwise AND operation to extract the region of interest of the first-type end face image.

3. The method as described in claim 1 or 2, wherein, The processing of the second type of end face image includes: The original second-type end face image is processed to obtain a binary image. The contours in the binary image are traversed, and the minimum radius circle function is used to calculate the minimum contour to determine the initial circle center. Using the initial center as the center, the original second type end face image is cropped according to a predetermined pixel area to obtain the cropped second type end face image; The cropped second-type end face image is binarized, and the contour of the area pixels in the second interval is selected from it; Based on the selected profile, the center and radius of the circle are determined using the minimum radius circle function; Based on the original second-type end face image and the determined center and radius, a second-type mask image is generated; The original second-type end face image and the second-type mask image are subjected to a bitwise AND operation to extract the region of interest of the second-type end face image.

4. The method of claim 1 or 2, further comprising the step of training a semantic segmentation model, including: A semantic segmentation dataset is generated by collecting multiple third-class end-face images and their corresponding annotation data, wherein the annotation data indicates the region of interest in the corresponding third-class end-face images; Using the semantic segmentation dataset, an initial semantic segmentation model is trained to generate a semantic segmentation model, wherein the initial semantic segmentation model is based on the UNet model.

5. The method as described in claim 1 or 2, Both the first target detection model and the second target detection model are based on the YOLO model.

6. The method of claim 5, further comprising the step of training to generate a first object detection model, including: The first recognition dataset is generated by collecting the first end face image and the region of interest images corresponding to the third to eighth end face image samples, and the region of interest images with defects are labeled. Using the first recognition dataset, an initial target detection model is trained to generate a first target detection model, wherein the initial target detection model includes a backbone network component, a feature extraction component, and a detection head component.

7. The method of claim 5, further comprising the step of training to generate a second object detection model, including: A second recognition dataset is generated by acquiring the region of interest image corresponding to the second end face image sample, and the region of interest image with defects is labeled. The initial target detection model is improved to generate an improved target detection model, wherein the initial target detection model includes a backbone network component, a feature extraction component, and a detection head component; Using the second recognition dataset, the improved target detection model is trained to generate a second target detection model.

8. The method of claim 7, wherein, The improvement of the initial target detection model includes: In the backbone network component and the feature extraction component, spatial depth transformation convolutional blocks are used instead of convolutional blocks, and a 3×3 convolutional layer is added before the spatial depth transformation convolutional block; Deformable convolutions are introduced in the last C2f block of the backbone network component; A dynamic head module is introduced into the detection head assembly.

9. A camshaft end face defect detection system, comprising: The image acquisition device is suitable for placing camshaft parts on a measuring table and fixing them with front and rear cylinders, and taking pictures of the front and rear end faces of the parts to obtain 8 end face images. An image classification device is suitable for distinguishing end face images based on whether there are threads at the vertices of the end face, and classifying the end face images, including: classifying the second, fourth, and sixth end face images as first-class end face images, classifying the first, third, and fifth end face images as second-class end face images, and classifying the seventh and eighth end face images as third-class end face images; The first image processing device is adapted to process the first type of end face image and the second type of end face image respectively to obtain the corresponding region of interest image; The second image processing device is adapted to input the third type of end face image into the semantic segmentation model for processing, extract the region of interest, and obtain the corresponding region of interest image; An image detection device is adapted to select a target detection model for detection based on an end face image corresponding to the region of interest image, and output a detection result, wherein the detection result indicates whether there is a defect in the region of interest image; The defect determination device is suitable for determining that the end face of the camshaft is free of defects when no defects are found in the images of each region of interest. The step of selecting a target detection model for detection based on the end face image corresponding to the region of interest image includes: if the region of interest image does not correspond to the second end face image, then selecting a first target detection model for detection; and if the region of interest image corresponds to the second end face image, then selecting a second target detection model for detection.

10. A computing device, comprising: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method as described in any one of claims 1-8.

11. A computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-8.

12. A computer program product comprising a computer program / instructions, wherein, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-8.

Images