Camshaft end face defect detection method and system

Abstract

The invention discloses a camshaft end face defect detection method and system. The method comprises the following steps: collecting eight end face images of the camshaft, distinguishing each end face image according to whether a thread exists at the vertex of the end face, and classifying the end face images; processing the first type of end face images and the second type of end face images to extract contours in the corresponding end face images, and extracting regions of interest in the corresponding end face images based on the contours to obtain regions of interest images; inputting the third-class end face image into a semantic segmentation model for processing, and extracting a region of interest to obtain a region of interest image; according to an end face image corresponding to the region-of-interest image, selecting a target detection model for detection so as to output a detection result indicating whether a defect exists in the region-of-interest image; and judging that the end face of the cam shaft does not have defects when the images of the interested areas do not have defects. The scheme is low in cost and high in efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a camshaft end face defect detection method and system. Background Art

[0002] The camshaft is a key power transmission component in the internal combustion engine and plays a vital role in the working process of the engine. It is mainly responsible for controlling the opening and closing of the valves, thereby regulating the timing of intake and exhaust, which directly affects the performance, efficiency and emissions of the engine. The manufacturing of the camshaft involves multiple precision machining steps, including blank casting, rough machining, heat treatment, finishing and final inspection. Strict quality control is required in each manufacturing link to ensure the accuracy, strength and durability of the camshaft. However, due to various factors such as material defects, machining errors or improper heat treatment, some defects may appear on the end face of the camshaft, such as surface cracks, pits, abnormal wear, profile deviation or uneven hardness. The presence of these defects will seriously affect the performance and life of the engine, and may cause abnormal noise, power loss, and even cause more serious mechanical failures.

[0003] In view of this, a camshaft end face defect detection solution is needed that can accurately identify the above defects. Summary of the invention

[0004] The present application provides a camshaft end face defect detection method and system, in an effort to solve or at least alleviate at least one of the above problems.

[0005] According to one aspect of the present application, a camshaft end face defect detection method is provided, comprising: collecting 8 end face images of the camshaft, and distinguishing each end face image according to whether there is a thread at the vertex of the end face, and classifying the end face images, including: taking the second end face image, the fourth end face image and the sixth end face image as the first type of end face images (D1~D3), taking the first end face image, the third end face image and the fifth end face image as the second type of end face images (X1~X3), and taking the fourth end face image and the eighth end face image as the third type of end face images (D4, X4); taking the first type of end face images The first image and the second type of end face image are processed respectively to extract the contour in the corresponding end face image, and based on the contour, the region of interest in the corresponding end face image is extracted to obtain the region of interest image; the third type of end face image is input into the semantic segmentation model for processing to extract the region of interest to obtain the region of interest image; according to the end face image corresponding to the region of interest image, a target detection model is selected for detection to output a detection result, and the detection result indicates whether there is a defect in the region of interest image; when there is no defect in each region of interest image, it is determined that there is no defect in the end face of the camshaft.

[0006] Optionally, in the method according to the present application, processing of the first type of end face image includes: processing the original first type of end face image to obtain a binary image, and traversing the contours in the binary image to select the contours of area pixels in the first interval; based on the selected contours, using the minimum radius circle function to determine the center and radius of the circle; based on the original first type of end face image and the determined center and radius, generating a first type of mask image; performing a bitwise AND operation on the original first type of end face image and the first type of mask image to extract the region of interest of the first type of end face image.

[0007] Optionally, in the method according to the present application, the processing of the second-type end face image includes: processing the original second-type end face image to obtain a binary image, traversing the contours in the binary image, using the minimum radius circle function to calculate the minimum contour, and determining the initial center of the circle; with the initial center of the circle as the center, cropping the original second-type end face image according to a predetermined pixel area to obtain a cropped second-type end face image; binarizing the cropped second-type end face image, and selecting the contour of the area pixels in the second interval therefrom; based on the selected contour, using the minimum radius circle function to determine the center and radius of the circle; based on the original second-type end face image and the determined center and radius, generating a second-type mask image; performing a bitwise AND operation on the original second-type end face image and the second-type mask image to extract the area of ​​interest of the second-type end face image.

[0008] Optionally, the method according to the present application also includes the step of training and generating a semantic segmentation model, including: generating a semantic segmentation dataset by collecting multiple third-category end-face images and their corresponding annotation data, wherein the annotation data indicates the area of ​​interest in the corresponding third-category end-face images; using the semantic segmentation dataset, training an initial semantic segmentation model to generate a semantic segmentation model, wherein the initial semantic segmentation model is based on a UNet model.

[0009] Optionally, in the method according to the present application, a target detection model is selected for detection based on the end face image corresponding to the image of the region of interest, including: if the image of the region of interest does not correspond to the second end face image, then selecting a first target detection model for detection; if the image of the region of interest corresponds to the second end face image, then selecting a second target detection model for detection, wherein the first target detection model and the second target detection model are both based on the YOLO model.

[0010] Optionally, the method according to the present application also includes the step of training and generating a first target detection model, including: generating a first recognition data set by collecting images of the region of interest corresponding to the second to eighth end face image samples, and marking the images of the region of interest with defects; using the first recognition data set to train the initial target detection model to generate a first target detection model, wherein the initial target detection model includes a backbone network component, a feature extraction component, and a detection head component.

[0011] Optionally, the method according to the present application also includes the step of training and generating a second target detection model, including: generating a second recognition data set by collecting the image of the area of ​​interest corresponding to the first end face image sample, and marking the image of the area of ​​interest with defects; improving the initial target detection model to generate an improved target detection model, wherein the initial target detection model includes a backbone network component, a feature extraction component, and a detection head component; using the second recognition data set to train the improved target detection model to generate a second target detection model.

[0012] Optionally, in the method according to the present application, the improvement of the initial target detection model includes: in the backbone network component and the feature extraction component, a spatial depth conversion convolution block is used to replace the convolution block, and a 3×3 convolution layer is added before the spatial depth conversion convolution block; a deformable convolution is introduced in the last C2f block of the backbone network component; and a dynamic head module is introduced in the detection head component.

[0013] According to another aspect of the present application, a camshaft end surface defect detection system is provided, comprising:

[0014] An image acquisition device is suitable for placing the camshaft part on the measuring table and fixing it with front and rear cylinders, and photographing the front and rear end faces of the part to obtain 8 end face images;

[0015] The image classification device is suitable for distinguishing each end face image according to whether there is a thread at the vertex of the end face, and classifying the end face images, including: taking the second end face image, the fourth end face image and the sixth end face image as the first type of end face images (D1-D3), taking the first end face image, the third end face image and the fifth end face image as the second type of end face images (X1-X3), and taking the fourth end face image and the eighth end face image as the third type of end face images (D4, X4); a first image processing device is suitable for processing the first type of end face images and the second type of end face images respectively to obtain corresponding region of interest images; a second image processing device is suitable for inputting the third type of end face images into a semantic segmentation model for processing, extracting the region of interest, and obtaining the corresponding region of interest image; an image detection device is suitable for selecting a target detection model for detection according to the end face image corresponding to the region of interest image to output a detection result, and the detection result indicates whether there is a defect in the region of interest image; a defect judgment device is suitable for judging that the end face of the camshaft does not have a defect when there is no defect in each region of interest image.

[0016] According to another aspect of the present application, a computing device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method as described above.

[0017] According to another aspect of the present application, a computer-readable storage medium storing one or more programs is provided. The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method described above.

[0018] According to another aspect of the present application, a computer program product is provided, comprising a computer program / instruction, wherein the computer program / instruction implements the above-mentioned method when executed by a processor.

[0019] In summary, according to the scheme of the present application, each end face image is first distinguished based on whether there are threads at the vertices of the end face, and the end face images are classified. For different types of end face images, different methods are used to extract the region of interest indicating the end face portion in the end face image to generate a region of interest image. Afterwards, different target detection models are selected based on whether the region of interest image comes from the second end face image, and the region of interest image is processed to detect defects in the region of interest image. Based on the detection results of all region of interest images, it can be determined whether the end face of the camshaft has defects. This detection method is low-cost and high-efficiency, and can be used for defect detection of various camshaft end faces produced in factories.

[0020] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles applied herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. By reading the following detailed description in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. Throughout the application, the same reference numerals generally refer to the same parts or elements.

[0022] Figure 1 A schematic diagram of a camshaft end surface defect detection system 100 according to some embodiments of the present application is shown;

[0023] Figure 2 A schematic diagram of a computing device 200 according to some embodiments of the present application is shown;

[0024] Figure 3 A schematic flow chart of a camshaft end surface defect detection method 300 according to some embodiments of the present application is shown;

[0025] Figure 4 The figure shows an original image (a), a grayscale image (b), a binary image (c) and an image of a region of interest (d) of a second end surface image according to an embodiment of the present application;

[0026] Figure 5 The original image (a), grayscale image (b), binary image (c), cropped image (d), binary image of the cropped image (e), and region of interest image (d) of a first end face image according to an embodiment of the present application are shown;

[0027] Figure 6 The figure shows an original image (a), a marked mask image (b) and an image of a region of interest (c) of an eighth end surface image (D4) according to an embodiment of the present application;

[0028] Figure 7 An architectural diagram of an improved target detection model according to some embodiments of the present application is shown. DETAILED DESCRIPTION

[0029] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0030] Since there are many camshaft end faces and there is mutual interference between the end faces, after acquiring the images of each end face, it is necessary to first extract the region of interest (ROI) of each end face, and then use the target detection algorithm to perform target detection on different end faces. The region of interest is the area in the end face image that represents the end face part.

[0031] It should be understood that the main body of the camshaft is a cylindrical rod with the same length as the cylinder group. A number of cams are mounted on it to drive the valves. One end of the camshaft is a bearing support point, and the other end is connected to the drive wheel. In the following embodiment, the end connected to the drive wheel is used as the front end of the camshaft, and 8 end surface images are collected according to the front and rear relationship of the camshaft.

[0032] Figure 1 A schematic diagram of a camshaft end surface defect detection system 100 according to some embodiments of the present application is shown.

[0033] like Figure 1 As shown, the camshaft end face defect detection system 100 includes: an image acquisition device 110, an image classification device 120, a first image processing device 130, a second image processing device 140, an image detection device 150 and a defect determination device 160.

[0034] In some embodiments, the image classification device 120 , the first image processing device 130 , the second image processing device 140 , the image detection device 150 and the defect determination device 160 may be arranged on a computing device, and the computing device is connected to the image acquisition device 110 .

[0035] The image acquisition device 110 includes a measuring platform, a camera, a light source, etc. The camshaft part is placed on the measuring platform and fixed with front and rear cylinders, and the front and rear end faces of the part are photographed to obtain 8 end face images of the camshaft. Optionally, the light source and the camera are on the same horizontal line, and the light source is a white bowl-shaped light source.

[0036] According to an embodiment of the present application, in the order from the front end to the rear end of the camshaft, the eight end face images are recorded in sequence as: the first end face image, the second end face image, the third end face image, the fourth end face image, the fifth end face image, the sixth end face image, the seventh end face image, and the eighth end face image.

[0037] The image acquisition device 110 transmits the acquired end surface images to the computing device, which processes the end surface images to detect whether there are defects in the end surface images, and further determines whether there are defects in the end surface of the camshaft.

[0038] Due to the different positions of the end faces, the sizes of their contours are different, so each end face image needs to be processed separately to obtain the required end face ROI area.

[0039] First, the image classification device 120 classifies each end surface image according to whether there is a thread at the vertex of the end surface.

[0040] Specifically, the collected end face images are divided into 8 end faces, namely D1, D2, D3, D4, X1, X2, X3, and X4, according to the front and back of the camshaft, wherein D1 to D4 are the rear end faces of the camshaft, and X1 to X4 are the front end faces of the camshaft. In other words, the end face images are distinguished according to whether there are threads at the vertices of the end faces. The end faces with threads are X1-X4 end faces, and the end faces without threads are D1-D4 end faces, which are: the first end face image (X1), the second end face image (D1), the third end face image (X2), the fourth end face image (D2), the fifth end face image (X3), the sixth end face image (D3), the seventh end face image (X4), and the eighth end face image (D4).

[0041] In this embodiment, the second end face image, the fourth end face image and the sixth end face image are regarded as the first type of end face images (i.e., D1~D3), the first end face image, the third end face image and the fifth end face image are regarded as the second type of end face images (i.e., X1~X3), and the fourth end face image and the eighth end face image are regarded as the third type of end face images (i.e., X4, D4).

[0042] Since the end faces in the first and second types of end face images are complete circular contours, traditional image processing methods can be used to extract the contours and obtain the region of interest. The end faces in the third type of end face images are not regular circular contours, so a deep learning model is used to extract the region of interest. The specific processing process is as follows.

[0043] The first image processing device 130 processes the first type of end face image and the second type of end face image respectively to obtain corresponding region of interest images. In the process of extracting contours, the first type of end face image only needs to find the contour once, while the second type of end face image is slightly more complicated and needs to find the contour twice through fine operations. The specific processing method is described in detail below.

[0044] The second image processing device 140 inputs the third type of end face image into the semantic segmentation model for processing, extracts the region of interest, and obtains the corresponding region of interest image. In some embodiments, the semantic segmentation model is based on the UNet model, and a large number of annotated D4 and X4 end face images are used to train the UNet model to obtain the semantic segmentation model.

[0045] Afterwards, the image detection device 150 selects a target detection model for detection based on the end face image corresponding to the region of interest image to output a detection result, wherein the detection result indicates whether there is a defect in the region of interest image. In some embodiments, the target detection model is based on a YOLO model, and a large number of annotated end face images are used to train the YOLO model to obtain a target detection model. The target detection model will be described in detail below.

[0046] The defect determination device 160 determines whether the end surface has defects based on the detection results. Specifically, when none of the images of the region of interest has defects, the defect determination device 160 determines that the end surface of the camshaft has no defects. In other words, as long as there is a defect in one of the images of the region of interest, it is determined that the end surface of the camshaft has defects.

[0047] The detection system 100 according to the present application has the advantages of low cost and high efficiency, and can be used for defect detection of various camshaft end surfaces produced in factories.

[0048] The computing device may be any device with storage and computing capabilities, such as a server, workstation, etc., or a personal computer such as a desktop computer or a notebook computer, but is not limited thereto. In some embodiments, the computing device may also output defect detection results through a visualization screen.

[0049] Figure 2 FIG. 2 shows a schematic diagram of a computing device 200 according to some embodiments of the present application. It should be noted that: Figure 2 The computing device 200 shown is only an example. In practice, the computing device used to implement the method of the present application can be any type of device, and its hardware configuration can be different from the above. Figure 2 The computing device 200 shown is the same as Figure 2 The computing device 200 shown is different. In practice, the computing device used to implement the embodiment of the present application can be Figure 2 The hardware components of the computing device 200 shown are increased or deleted, and the present application does not limit the specific hardware configuration of the computing device.

[0050] like Figure 2In a basic configuration, computing device 200 includes at least one processing unit 202 and system memory 204. According to one aspect, depending on the configuration and type of computing device, processing unit 202 can be implemented as a processor. System memory 204 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 204 includes operating system 205 and program modules 206.

[0051] According to one aspect, operating system 205 is suitable for controlling the operation of computing device 200, for example. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other application programs, and are not limited to any particular application or system. Figure 2 This basic configuration is illustrated in FIG. 2 by those components within dashed line 208. According to one aspect, computing device 200 has additional features or functionality. For example, according to one aspect, computing device 200 includes additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 2 2 is illustrated by removable storage 209 and non-removable storage 210.

[0052] As stated above, according to one aspect, program modules are stored in the system memory 204. According to one aspect, the program modules may include one or more application programs, and the present application does not limit the types of application programs, for example, the application programs may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slide show applications, drawing or computer-aided applications, web browser applications, etc.

[0053] According to one aspect, the examples may be practiced on a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. Figure 2Each or many components shown in can be integrated in a system on chip (SOC) on a single integrated circuit to practice examples. According to one aspect, such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When operated via SOC, the functions described in the present application can be operated via a dedicated logic integrated with other components of a computing device 200 on a single integrated circuit (chip). Other technologies that can perform logical operations (such as AND, OR, and NOT) can also be used to practice embodiments of the present application, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, embodiments of the present application can be practiced in a general-purpose computer or in any other circuit or system.

[0054] According to one aspect, the computing device 200 may also have one or more input devices 212, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, a VR motion capture input device, etc. Output devices 214 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples and other devices may also be used. The computing device 200 may include one or more communication connections 216 that allow communication with other computing devices 218. Examples of suitable communication connections 216 include, but are not limited to: RF transmitters, receivers and / or transceiver circuits; Universal Serial Bus (USB), parallel and / or serial ports.

[0055] The term computer-readable media as used in this application includes computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 204, removable storage 209, and non-removable storage 210 are all examples of computer storage media (i.e., memory storage). Computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other products that can be used to store information and can be accessed by computing device 200. According to one aspect, any such computer storage media may be part of computing device 200. Computer storage media do not include carrier waves or other propagated data signals.

[0056] According to one aspect, communication media is implemented by computer readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0057] In some other embodiments of the present application, the computing device 200 is configured to execute the camshaft end face defect detection method 300 according to the present application. The program module 206 includes a plurality of program instructions for executing the method 300, and these program instructions can instruct the processing unit 202 to execute the method 300 of the present application.

[0058] Figure 3 FIG. 3 is a flow chart of a camshaft end surface defect detection method 300 according to some embodiments of the present application. It should be noted that: Figure 3 The method 300 shown is similar to Figure 1 The systems 100 shown are complementary to each other, and related details are not described in detail.

[0059] like Figure 3 As shown, the method 300 starts at step S310 .

[0060] In S310 , eight end surface images of the camshaft are collected, and each end surface image is distinguished based on whether there is a thread at the vertex of the end surface, and the end surface images are classified.

[0061] In this embodiment, after the end face images are acquired from the image acquisition device 110, considering that the end face contour sizes are different due to the different end face positions, the end face images are first classified for subsequent processing. For the distinction of the end face images, please refer to the relevant description above. The specific classification method includes: the second end face image, the fourth end face image and the sixth end face image are regarded as the first type of end face images (D1~D3), the first end face image, the third end face image and the fifth end face image are regarded as the second type of end face images (X1~X3), and the fourth end face image and the eighth end face image are regarded as the third type of end face images (D4, X4).

[0062] In S320, the first type of end face image and the second type of end face image are processed respectively to extract contours in the corresponding end face images, and based on the extracted contours, the region of interest in the corresponding end face image is extracted to obtain a region of interest image.

[0063] Since the end face profile with threads is slightly more complicated, the processing method for the second type of end face image is to find the profile twice through fine operation to finally determine the area of ​​interest; the processing method for the first type of end face image is to find the profile only once to determine the area of ​​interest. The following describes the two processing methods respectively.

[0064] (1) The processing of the first type of end face images (D1 to D3) includes the following steps. It should be understood that the first type of end face images includes three images, and the following steps need to be performed on D1, D2, and D3 respectively to determine their respective regions of interest, and then the regions of interest are cropped from the corresponding end face images, which are the images of the respective corresponding regions of interest. That is, when D1 is processed, its binary image, the first type of mask image, and the region of interest are obtained; when D2 is processed, its binary image, the first type of mask image, and the region of interest are obtained; when D3 is processed, its binary image, the first type of mask image, and the region of interest are obtained.

[0065] Firstly, the original first-type end face image is processed to obtain a binary image, and the contours in the binary image are traversed to select the contours with area pixels in the first interval.

[0066] Specifically, the original first-class end face image is converted into a grayscale image and binarized to obtain a binary image. Taking D1 as an example, Figure 4 The original image (a), grayscale image (b) and binary image (c) of D1 are shown. For each binary image, all contours in the binary image are traversed, and the number of pixels contained in the area of ​​each contour is counted, which is recorded as the area pixel value of the contour. After that, the contours with area pixels in the first interval (optionally, the first interval is 240000-650000, which is an approximate range obtained by calculating the contour area of ​​several corresponding end face images, but is not limited to this) are selected. In this embodiment, the first interval indicates the area pixel value of the contour with a smaller area in the end face image.

[0067] Next, based on the selected contour, the minimum radius circle function is used to determine the center and radius. In this embodiment, the existing library function can be used to calculate the center and radius. For example, the minimum radius circle function (x, y) of OpenCV, radius = cv.minEnclosingCircle (contour), is used to determine the center and radius of the contour circle.

[0068] Then, based on the original first-type end face image and the determined center and radius, a first-type mask image is generated. Specifically, a black mask image of the same size as the original first-type end face image (i.e., corresponding D1, D2, D3) is created, and a ring area is drawn on the black mask image according to the center and radius determined in the previous step.

[0069] Finally, the original first-type end face image and the corresponding first-type mask image are bitwise ANDed to extract the region of interest of the first-type end face image. Figure 4 (d) in FIG. 1 shows the image of the region of interest obtained after D1 (ie, FIG. (a)) is processed.

[0070] (2) The processing of the second type of end face images (X1-X3) includes the following steps. Similarly, the second type of end face images includes three images, and the following steps need to be performed on X1, X2, and X3 respectively to determine their respective regions of interest, and then the regions of interest are cropped from the corresponding end face images, that is, the respective corresponding region of interest images, which will not be repeated here.

[0071] Firstly, the original second type end face image is processed to obtain a binary image, and the contours in the binary image are traversed. The minimum contour is calculated using the minimum radius circle function to determine the initial circle center.

[0072] Specifically, the original second-type end face image is converted into a grayscale image and binarized to obtain a binary image. Taking X1 as an example, Figure 5 The original image (a), grayscale image (b) and binary image (c) of X1 are shown. Afterwards, the contour search function is used to obtain all contours in the binary image, and the minimum radius circle function is used to calculate the minimum contour to obtain the initial circle center. As mentioned above, the contour search function and the minimum radius circle function can both use existing library functions, which will not be repeated here.

[0073] Next, the original second-type end face image is cropped according to the predetermined pixel area with the initial circle center as the center to obtain a cropped second-type end face image. Optionally, the size of the predetermined pixel area is 1400x1400. Through this step of processing, the original image is cropped to narrow the search range. Figure 5 (d) shows the cropped X1.

[0074] Next, the cropped second-type end face images are binarized, and contours with area pixels in a second interval (optionally, the second interval is 6000-8500, which is an approximate range obtained by calculating the contour area of ​​several corresponding cropped end face images, but is not limited thereto) are selected. Figure 5 (e) shows the cropped binary image of X1 (ie, Figure (d)).

[0075] Next, based on the contour selected in the previous step, the minimum radius circle function is used to determine the center and radius of the circle. The processing method of this step is the same as the processing of the first type of end face image mentioned above, and will not be repeated here.

[0076] Afterwards, a second type of mask image is generated based on the original second type of end surface image and the determined center and radius.

[0077] Specifically, a black mask image of the same size as the original second-type end surface image (ie, corresponding X1, X2, X3) is created, and a ring area is drawn on the black mask image according to the center and radius determined in the previous step.

[0078] Finally, the original second-type end face image and the corresponding second-type mask image are bitwise ANDed to extract the region of interest of the second-type end face image. Figure 5 (f) in FIG. 1 shows the image of the region of interest obtained after X1 (ie, FIG. (a)) is processed.

[0079] In S330, the third type of end face image is input into the semantic segmentation model for processing, and the region of interest is extracted to obtain a region of interest image.

[0080] In some embodiments according to the present application, the semantic segmentation model is based on the UNet model. UNet (full name U-Net) is a deep learning architecture originally proposed by Olaf Ronneberger, Philipp Fischer and Thomas Brox in 2015 for image segmentation tasks. The name of the network comes from its U-shaped architecture, which enables the network to capture multi-scale feature information during encoding and decoding. The network structure of the UNet model is not described here. In this embodiment, a third-class end face image is input into the semantic segmentation model for segmentation and extraction, and the region of interest is output. The region of interest is cropped out from the third-class end face image to obtain a region of interest image.

[0081] The method 300 according to the present application also includes the step of training and generating a semantic segmentation model, which includes the following three steps.

[0082] In the first step, a semantic segmentation dataset is generated by collecting multiple third-category end-face images and their corresponding annotation data. The annotation data indicates the region of interest in the corresponding third-category end-face image. That is, multiple D4 and X4 are collected and semantically annotated to annotate the region of interest therein as a semantic segmentation dataset. In some embodiments, an annotation mask image is generated using the annotated region of interest, and the ROI region is represented by a white area in the annotation mask image. Figure 6An original image (a) and a marked mask image (b) of an eighth end surface image (D4) according to an embodiment of the present application are shown.

[0083] The second step is to use the semantic segmentation dataset to train the initial semantic segmentation model and generate a semantic segmentation model.

[0084] As mentioned above, the initial semantic segmentation model is based on UNet, which is the original UNet model. The collected D4 or X4 is input into the initial semantic segmentation model for processing to output the region of interest, compare the output region of interest with the labeled data, calculate the loss function, and adjust the network parameters. The whole process is repeated until the training is completed, and finally a semantic segmentation model is generated. The training process of the UNet model is not described here.

[0085] In some embodiments, the semantic segmentation data set can be divided into a training set, a test set, and a validation set. The training set is used to train the semantic segmentation model, and then the test set and the validation set are used to optimize the trained semantic segmentation model. This application does not impose too many restrictions on this.

[0086] According to the embodiments of the present application, Figure 6 Figure (a) in Figure 1 is input into the semantic segmentation model, and the image of the region of interest obtained after processing is shown in Figure (c), which contains a total of 3 regions of interest, which are the end faces of D4.

[0087] In S340, a target detection model is selected to perform detection according to the end face image corresponding to the region of interest image, so as to output a detection result, wherein the detection result indicates whether there is a defect in the region of interest image.

[0088] According to this embodiment, the second end face image (D1) includes the keyway portion, which will cause more interference to the defect detection of the end face area. Therefore, during defect detection, the image of the region of interest of D1 is treated as a type of image with more interference and processed separately; the images of the region of interest of other end faces are treated as a type of image with less interference and processed.

[0089] Specifically, if the image of the region of interest does not correspond to the second end face image (D1), the first target detection model is selected for detection. The image of the region of interest is input into the first target detection model for processing, and the detection result is output. If the image of the region of interest corresponds to the second end face image (D1), the second target detection model is selected for detection. The image of the region of interest is input into the second target detection model for processing, and the detection result is output. Usually the output will include the detected bounding box (bbox) and the confidence (score) of the category prediction. The output confidence is compared with the set confidence threshold to obtain the detection result. For example, the score range is 0 to 1, and the larger the value, the greater the possibility of defects. Set the threshold to 0.5, as long as the score is greater than 0.5, it means that there is a defect. It should be understood that the above is only an illustrative description, and the threshold can be set according to experience and actual conditions. This embodiment is not limited to this.

[0090] Among them, the first target detection model and the second target detection model are both based on the YOLO model.

[0091] The YOLO model (You Only Look Once) is a target detection algorithm based on deep learning, first proposed by Joseph Redmon et al. in 2015. The YOLO model can identify and locate objects in an image with a single look, and has the advantages of fast speed, high accuracy, strong interpretability and wide applicability.

[0092] Optionally, both the first target detection model and the second target detection model are based on YOLOv8. YOLOv8 can be used as the initial target detection model. More specifically, the first target detection model adopts YOLOv8, and YOLOv8 is trained by collecting corresponding end face images to obtain the first target detection model. The second target detection model makes some improvements to the network structure of YOLOv8 so that it can better adapt to the D1 end face images with more interference, and is trained on this basis to obtain the second target detection model.

[0093] According to an embodiment of the present application, method 300 also includes a step of training and generating a first target detection model, which includes the following two steps.

[0094] The first step is to generate a first identification data set by collecting images of the region of interest corresponding to the second to eighth end face image samples, and mark the images of the region of interest with defects. Specifically, a large number of images of the region of interest corresponding to D2, D3, D4, X1, X2, X3, and X4 are collected to generate a first identification data set. The images of the region of interest can be generated by the aforementioned steps S310 to S330, which will not be repeated here. Use labelimg software to label the defective samples in the first identification data set, mark the images of the region of interest with defects, and generate a file in txt format. For example, if there is a defect, it is marked as 1; if there is no defect, it is marked as 0. Defects are, for example, surface cracks, pits, abnormal wear, contour deviations, or uneven hardness.

[0095] In some embodiments, the first recognition data set can also be divided into a training set, a test set, and a validation set to complete the training, optimization, and validation of the model, which will not be elaborated here.

[0096] In the second step, the initial target detection model is trained using the first recognition data set to generate a first target detection model. The initial target detection model includes a backbone network component (Backbone), a feature extraction component (Neck), and a detection head component (Head), which will not be expanded here.

[0097] Specifically, the image of the region of interest in the first recognition data set is input into the initial target detection model (ie, YOLOv8) for processing, and the detection result of whether there is a defect is output. The loss function is calculated using the detection result and the labeled data, and the model parameters are adjusted, and the iteration is repeated until the training is completed, and finally the first target detection model is obtained.

[0098] According to an implementation of the present application, method 300 also includes a step of training and generating a second target detection model, which specifically includes the following three steps.

[0099] The first step is to generate a second identification data set by collecting the region of interest corresponding to the first end face image sample, and mark the region of interest image with defects. Specifically, a large number of region of interest images corresponding to D1 are collected to generate a second identification data set. The region of interest image can be generated through the aforementioned steps S310 to S330, which will not be repeated here. Use labelimg software to label the defective samples in the second identification data set, mark the region of interest image with defects, and generate a txt format file. For example, if there is a defect, it is marked as 1; if there is no defect, it is marked as 0.

[0100] Similarly, the second recognition data set can also be divided into a training set, a test set, and a validation set to complete the training, optimization, and validation of the model, which will not be elaborated here.

[0101] The second step is to improve the initial target detection model and generate an improved target detection model.

[0102] The initial target detection model takes YOLOv8 as an example, including a backbone network component (Backbone), a feature extraction component (Neck), and a detection head component (Head). The initial target detection model is improved in three aspects. Figure 7 The architecture diagram of the improved target detection model according to some embodiments of the present application is shown below. Figure 7 , explain the improvements.

[0103] (1) In the backbone network component (Backbone) and feature extraction component (Neck), the spatial depth conversion convolution block (SPD-Conv) is used to replace the original convolution block (Conv), and a 3×3 convolution layer is added before each spatial depth conversion convolution block to form a C_SPD block.

[0104] SPD-Conv is a new building block to replace the strided convolution and Pooling layer . It consists of a spatial-to-depth (SPD) layer and a non-stride convolution (Conv) layer. Among them, the role of the spatial-to-depth (SPD) layer is to reduce each spatial dimension of the input feature map to the channel dimension while retaining the information within the channel. This can be achieved by mapping each pixel or feature of the input feature map to a channel. In this process, the size of the spatial dimension is reduced, while the size of the channel dimension is increased. The non-stride convolution (Conv) layer is a standard convolution operation that is performed after the SPD layer. Unlike the stride convolution, the non-stride convolution does not move on the feature map, but performs a convolution operation on each pixel or feature map. This helps to reduce the over-downsampling problem that may occur in the SPD layer and retain more fine-grained information.

[0105] At the same time, in the embodiment, a new 3×3 convolution layer is introduced before SPD-Conv to form a C_SPD block, such as Figure 7 This helps the model effectively capture local features and thus accurately extract the edge and texture information of the target.

[0106] After the improved C_SPD block, the input feature map is first convolved through a 3×3 convolutional layer (Conv), then converted through the SPD layer, and then the output result is convolved through the Conv layer again. This combination can reduce the size of the spatial dimension without losing information while retaining the information within the channel, which helps improve the model's detection performance for low-resolution images and small objects.

[0107] (2) Introducing deformable convolution in the last C2f block of the backbone network component. In this embodiment, the deformable convolution selects DCNv2, so that the last C2f block in the backbone network component becomes a C2f_DCNv2 block, as shown in Figure 7 As shown in the figure. DCNv2 can improve the model's adaptability to different geometric changes. In addition to adding more deformable convolution layers, a learnable offset parameter Δpn is added to each element involved in the accumulation. In addition, the model must learn not only the offset of each element, but also the weight of each element to make it adaptive. Deformable convolution DCNv2 belongs to the existing modules in this field and will not be described here.

[0108] The formula for this module is as follows:

[0109] (3) Introduce a dynamic head module (Dynamic Head) into the detection head component (Head), such as Figure 7 The Dyhead module improves the object detection head by applying a self-attention mechanism on different feature dimensions, making it scale-aware, spatially-aware, and task-aware.

[0110] The dynamic head module can dynamically adjust the weights of different feature layers, which is conducive to extracting multi-scale features. Among them, the three-dimensional feature tensor F∈R is given on the detection head. L*S*C , the enhanced output after applying the attention at different positions is as follows: L (F) = α C (α S (α l (F)·F)·F).

[0111] The improved target detection model pays more attention to local features, which can improve the detection performance of the model for low-resolution images and small objects, as well as its adaptability to different geometric changes. Therefore, the defects in D1 can be detected more accurately.

[0112] about Figure 7 Other unexplained blocks, such as SPFF, Upsample, Concat, and detect, all belong to the existing modules of YOLOv8. Due to space limitations, they will not be expanded one by one.

[0113] The third step is to use the second recognition data set to train the improved target detection model to generate a second target detection model.

[0114] Specifically, the image of the region of interest in the second recognition data set is input into the improved target detection model for processing, and the detection result of whether there is a defect is output. The loss function is calculated using the detection result and the labeled data, and the model parameters are adjusted, and the iteration is repeated until the training is completed, and finally the second target detection model is obtained.

[0115] In S350, when there is no defect in each region of interest image, it is determined that there is no defect in the end surface of the camshaft.

[0116] In other words, the images of the regions of interest of the eight end faces of the camshaft will be inspected individually, and only when there are no defects on the eight end faces will it be determined that there are no defects on the end faces of the camshaft, that is, the camshaft is qualified.

[0117] According to the method 300 of the present application, each end face image is first distinguished based on whether there are threads at the vertices of the end face, and the end face images are classified. For different types of end face images, different methods are used to extract the region of interest indicating the end face portion in the end face image to generate a region of interest image. Afterwards, different target detection models are selected based on whether the region of interest image comes from the second end face image, and the region of interest image is processed to detect defects in the region of interest image. Based on the detection results of all region of interest images, it can be determined whether the end face of the camshaft has defects. The detection method is low-cost and high-efficiency, and can be used for defect detection of various camshaft end faces produced in factories.

[0118] Optionally, a computer-readable storage medium storing one or more programs is also included, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes the method as described above.

[0119] Optionally, a computer program product is also included, including a computer program / instruction, wherein the computer program / instruction implements the method described above when executed by a processor.

[0120] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present application, or certain aspects or portions of the method and apparatus of the present application may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present application.

[0121] In the case where the program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the method of the present application according to the instructions in the program code stored in the memory.

[0122] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0123] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present application. According to the above description, it is obvious to construct the structure required for such systems. In addition, the present application is not directed to any specific programming language either. It should be understood that the content of the present application described herein can be realized by various programming languages, and the description of the specific language above is to disclose the preferred implementation of the present application.

[0124] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0125] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0126] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subassemblies. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise clearly stated, each feature disclosed in this specification can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0127] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present application and to form different embodiments.

[0128] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices that are used to implement the functions performed by the elements for the purpose of implementing the disclosure.

[0129] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely refers to different instances of similar objects and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner. In addition, the quantifier "plurality" means "two" and / or "more than two."

[0130] Although the present application is described according to a limited number of embodiments, it will be appreciated by those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the present application thus described. In addition, it should be noted that the language used in this specification is selected primarily for the purpose of readability and teaching, rather than for the purpose of explaining or limiting the subject matter of the present application. Therefore, many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. For the scope of the present application, the disclosure made to the present application is illustrative, not restrictive, and the scope of the present application is defined by the appended claims.

Claims

1. A camshaft end surface defect detection method, comprising: Collecting 8 end face images of the camshaft, and distinguishing each end face image according to whether there is a thread at the vertex of the end face, and classifying the end face images, including: taking the second end face image, the fourth end face image and the sixth end face image as the first type of end face images, taking the first end face image, the third end face image and the fifth end face image as the second type of end face images, and taking the fourth end face image and the eighth end face image as the third type of end face images; Processing the first type of end face images and the second type of end face images respectively to extract contours in the corresponding end face images, and extracting regions of interest in the corresponding end face images based on the contours to obtain region of interest images; Inputting the third type of end face image into a semantic segmentation model for processing, extracting the region of interest, and obtaining a region of interest image; Selecting a target detection model to perform detection according to the end face image corresponding to the image of the region of interest, so as to output a detection result, wherein the detection result indicates whether there is a defect in the image of the region of interest; When there is no defect in each region of interest image, it is determined that there is no defect on the end surface of the camshaft.

2. The method of claim 1, wherein: The processing of the first type of end face image includes: The original first-type end face image is processed to obtain a binary image, and contours in the binary image are traversed to select contours with area pixels in the first interval; Based on the selected contour, the center and radius of the circle are determined using the minimum radius circle function; Generate a first type of mask image based on the original first type of end surface image and the determined center and radius; A bitwise AND operation is performed on the original first-type end face image and the first-type mask image to extract a region of interest of the first-type end face image.

3. The method according to claim 1 or 2, wherein: The processing of the second type of end face image includes: The original second-type end face image is processed to obtain a binary image, and the contours in the binary image are traversed, and the minimum contour is calculated using the minimum radius circle function to determine the initial circle center; Taking the initial circle center as the center, cropping the original second-type end face image according to a predetermined pixel area to obtain a cropped second-type end face image; Binarizing the cropped second type of end face image, and selecting the contour of area pixels in the second interval; Based on the selected contour, the center and radius of the circle are determined using the minimum radius circle function; Generate a second type of mask image based on the original second type of end surface image and the determined center and radius; A bitwise AND operation is performed on the original second-type end face image and the second-type mask image to extract a region of interest of the second-type end face image.

4. The method according to any one of claims 1 to 3, further comprising the step of training and generating a semantic segmentation model, comprising: Generate a semantic segmentation dataset by collecting a plurality of third-category end-face images and their corresponding annotation data, wherein the annotation data indicates a region of interest in the corresponding third-category end-face image; Using the semantic segmentation dataset, an initial semantic segmentation model is trained to generate a semantic segmentation model, wherein the initial semantic segmentation model is based on a UNet model.

5. The method according to any one of claims 1 to 4, wherein: The step of selecting a target detection model for detection according to the end face image corresponding to the region of interest image includes: If the image of the region of interest does not correspond to the second end face image, selecting the first target detection model for detection; If the image of the region of interest corresponds to the second end face image, the second target detection model is selected for detection. The first target detection model and the second target detection model are both based on the YOLO model.

6. The method according to claim 5, further comprising the step of training and generating a first target detection model, comprising: Generate a first recognition data set by collecting images of the region of interest corresponding to the second to eighth end surface image samples, and mark images of the region of interest with defects; The initial target detection model is trained using the first recognition data set to generate a first target detection model, wherein the initial target detection model includes a backbone network component, a feature extraction component, and a detection head component.

7. The method of claim 5, further comprising the step of training and generating a second target detection model, comprising: Generate a second recognition data set by acquiring an image of a region of interest corresponding to the first end face image sample, and mark the image of the region of interest with defects; Improving the initial target detection model to generate an improved target detection model, wherein the initial target detection model includes a backbone network component, a feature extraction component, and a detection head component; The improved target detection model is trained using the second recognition data set to generate a second target detection model.

8. The method of claim 7, wherein: The improvement of the initial target detection model includes: In the backbone network component and the feature extraction component, a spatial depth conversion convolution block is used to replace the convolution block, and a 3×3 convolution layer is added before the spatial depth conversion convolution block; Introducing deformable convolution in the last C2f block of the backbone network component; A dynamic head module is introduced into the detection head assembly.

9. A camshaft end surface defect detection system, comprising: An image acquisition device is suitable for placing the camshaft part on the measuring table and fixing it with front and rear cylinders, and photographing the front and rear end faces of the part to obtain 8 end face images; The image classification device is adapted to distinguish each end face image according to whether there is a thread at the vertex of the end face, and classify the end face images, including: taking the second end face image, the fourth end face image and the sixth end face image as the first type of end face images, taking the first end face image, the third end face image and the fifth end face image as the second type of end face images, and taking the fourth end face image and the eighth end face image as the third type of end face images; A first image processing device, adapted to process the first type of end surface image and the second type of end surface image respectively to obtain corresponding region of interest images; The second image processing device is adapted to input the third type of end face image into the semantic segmentation model for processing, extract the region of interest, and obtain a corresponding region of interest image; An image detection device, adapted to select a target detection model for detection according to the end face image corresponding to the image of the region of interest, so as to output a detection result, wherein the detection result indicates whether there is a defect in the image of the region of interest; The defect determination device is suitable for determining that the end surface of the camshaft has no defects when no defects are present in the images of the regions of interest.

10. A computing device comprising: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1-8.

Images