Part dimensional inspection methods, devices and computer equipment

Abstract

This application relates to a method, apparatus, and computer device for detecting the dimensions of parts. The method involves acquiring target images containing multiple parts of different categories from an image acquisition system; inputting the target images into a target detection model for target detection to obtain the bounding box information and category information of each part; for each part, determining the sub-pixel coordinates corresponding to the bounding box information of the part based on the target image and the bounding box information; and determining the size information of the part based on a preset candidate mapping relationship between pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and bounding box information of the part. The target detection model identifies the bounding box information and category information of all parts in the image at once, eliminating the need to set separate matching templates for different parts and improving localization efficiency.

Description

Technical Field

[0001] This application relates to the field of part size inspection technology, and in particular to a part size inspection method, apparatus and computer equipment. Background Technology

[0002] In industrial production, the accurate measurement of part dimensions is a key step in ensuring product quality.

[0003] In traditional techniques, standard part images are typically cropped and used as matching templates to match target parts in captured images. The contours of target parts are extracted based on threshold segmentation and edge detection algorithms, thereby determining the size of the target parts.

[0004] Because different parts have different sizes and shapes, different matching templates, threshold segmentation parameters and edge detection parameters need to be set for different parts, resulting in low efficiency in part size detection. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, and computer equipment for detecting the dimensions of parts, which can improve the efficiency of detecting the dimensions of parts, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for inspecting the dimensions of a part, including:

[0007] Acquire a target image captured by an image acquisition system; wherein the target image includes multiple different categories of parts to be detected;

[0008] The target image is input into the target detection model for target detection to obtain the bounding box information and category information of each part to be detected;

[0009] For each of the parts to be detected, based on the target image and the bounding box information of the part to be detected, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected are determined; and,

[0010] The size information of the part to be detected is determined based on the preset candidate mapping relationship between pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein, the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

[0011] In one embodiment, the object detection model is trained in the following manner:

[0012] Acquire multiple sample images and sample label data corresponding to each sample image; wherein, each sample image includes multiple sample parts of different categories, and the shooting conditions of sample parts of the same category in different sample images are different, the shooting conditions include lighting conditions and / or the placement angle of the parts, and the sample label data includes the bounding box information and category information of different sample parts in the sample image;

[0013] The initial detection model is trained using the sample images and the corresponding sample label data to obtain the target detection model.

[0014] In one embodiment, the image acquisition system includes a light source, a stage for placing parts, and a camera equipped with a telecentric lens;

[0015] The light source is at a preset angle to the surface of each of the parts to be tested. The preset angle is determined according to a critical value, which is the angle between the light source and the surface of the part to be tested when the surface of the part to be tested produces reflection or shadow. The flatness error of the stage is less than a preset error.

[0016] In one embodiment, the candidate mapping relationship is constructed in the following way:

[0017] Based on the system parameters of the image acquisition system, establish a basic mapping relationship between pixel coordinates and size information;

[0018] For each candidate part, a correction coefficient is determined based on the category information of the candidate part.

[0019] The correction coefficient is used to correct the basic mapping relationship to obtain the candidate mapping relationship between the pixel coordinates and size information of the candidate parts.

[0020] In one embodiment, determining the size information of the part to be detected based on a preset candidate mapping relationship between pixel coordinates and size information, and the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected, includes:

[0021] Based on the category information of the part to be detected, a target mapping relationship matching the part to be detected is selected from different candidate mapping relationships;

[0022] The size information of the part to be detected is determined based on the sub-pixel coordinates corresponding to the target mapping relationship and the bounding box information.

[0023] In one embodiment, the system parameters include internal parameters and external parameters; the internal parameters include focal length and pixel size of a single pixel unit, and the external parameters include the distance between the camera and the stage in the image acquisition system.

[0024] In one embodiment, determining the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the target image and the bounding box information of the part to be detected includes:

[0025] Based on the bounding box information of the part to be detected, determine the image region corresponding to the part to be detected in the target image;

[0026] Based on the image region, determine the sub-pixel coordinates corresponding to the bounding box information of the part to be detected.

[0027] In one embodiment, determining the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the image region includes:

[0028] The image region is binarized to obtain a binary image;

[0029] An edge detection algorithm is used to extract the initial edge point coordinates of the part region in the binary image;

[0030] The initial edge point coordinates are denoised to obtain the target edge point coordinates;

[0031] The subpixel segmentation algorithm is used to fit the coordinates of the target edge points to obtain the subpixel coordinates corresponding to the bounding box information of the part to be detected.

[0032] Secondly, this application also provides a part dimension detection device, comprising:

[0033] An acquisition module is used to acquire a target image captured by an image acquisition system; wherein, the target image includes multiple parts to be detected that fall into different categories;

[0034] The detection module is used to input the target image into the target detection model for target detection, and obtain the bounding box information and category information of each of the parts to be detected;

[0035] The determination module is configured to, for each part to be detected, determine the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the target image and the bounding box information of the part to be detected; and determine the size information of the part to be detected based on a preset candidate mapping relationship between pixel coordinates and size information, and the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0037] Acquire a target image captured by an image acquisition system; wherein the target image includes multiple different categories of parts to be detected;

[0038] The target image is input into the target detection model for target detection to obtain the bounding box information and category information of each part to be detected;

[0039] For each of the parts to be detected, based on the target image and the bounding box information of the part to be detected, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected are determined; and,

[0040] The size information of the part to be detected is determined based on the preset candidate mapping relationship between pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein, the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

[0041] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0042] Acquire a target image captured by an image acquisition system; wherein the target image includes multiple different categories of parts to be detected;

[0043] The target image is input into the target detection model for target detection to obtain the bounding box information and category information of each part to be detected;

[0044] For each of the parts to be detected, based on the target image and the bounding box information of the part to be detected, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected are determined; and,

[0045] The size information of the part to be detected is determined based on the preset candidate mapping relationship between pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein, the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

[0046] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0047] Acquire a target image captured by an image acquisition system; wherein the target image includes multiple different categories of parts to be detected;

[0048] The target image is input into the target detection model for target detection to obtain the bounding box information and category information of each part to be detected;

[0049] For each of the parts to be detected, based on the target image and the bounding box information of the part to be detected, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected are determined; and,

[0050] The size information of the part to be detected is determined based on the preset candidate mapping relationship between pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein, the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

[0051] The aforementioned part size detection method, apparatus, and computer equipment acquire a target image captured by an image acquisition system; wherein the target image includes multiple parts to be detected comprising different categories; the target image is input into a target detection model for target detection to obtain bounding box information and category information for each part to be detected; for each part to be detected, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected are determined based on the target image and the bounding box information of the part to be detected; and the size information of the part to be detected is determined based on a preset candidate mapping relationship between pixel coordinates and size information, and the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts. The above solution achieves several key improvements. First, by acquiring target images containing multiple categories of parts to be detected, it eliminates the need to collect images of each part individually, thus providing the foundation for batch detection. Second, by using a target detection model to identify the bounding box and category information of all parts to be detected in the image at once, it improves positioning efficiency compared to traditional template matching methods, eliminating the need to set separate matching templates for different parts. Third, by determining the sub-pixel coordinates corresponding to the bounding box information, it improves the accuracy of part edge positioning. Fourth, based on the candidate mapping relationship constructed using image acquisition system parameters and part categories, and combined with category information and sub-pixel coordinates, it determines the true size information, achieving accurate size conversion for different categories of parts. This eliminates the need to adjust size conversion parameters for different parts, solving the problem of low detection efficiency and improving size detection accuracy. Furthermore, the multi-category adaptive candidate mapping relationship enhances the scenario adaptability of the detection method. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating a part size detection method in one embodiment;

[0054] Figure 2 This is a schematic diagram of the process for establishing candidate mapping relationships in one embodiment;

[0055] Figure 3 This is a flowchart illustrating the process of determining the sub-pixel coordinates corresponding to the bounding box information of the part to be detected in one embodiment.

[0056] Figure 4 This is a flowchart illustrating the process of determining the sub-pixel coordinates corresponding to the bounding box information of the part to be detected in another embodiment.

[0057] Figure 5 This is a structural block diagram of a part size detection device in one embodiment;

[0058] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] The part size detection method provided in this application embodiment can be applied to various application scenarios of part size detection. This method can be executed by a server or a terminal. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0061] In one exemplary embodiment, such as Figure 1 As shown, a method for inspecting the dimensions of a part is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0062] S101, acquire the target image acquired by the image acquisition system.

[0063] The image acquisition system is a hardware system used to acquire images of the parts to be inspected, enabling clear and distortion-free image acquisition. The target image is an image captured by the image acquisition system that contains at least two or more different categories of parts to be inspected.

[0064] In some alternative implementations, the image acquisition system may include a light source, a stage for placing parts, and a camera equipped with a telecentric lens; the light source is at a preset angle to the surface of each part to be inspected, the preset angle being determined according to a critical value, the critical value being the angle between the light source and the surface of the part to be inspected when the surface of the part to be inspected produces reflection or shadow; the flatness error of the stage is less than a preset error.

[0065] The light source, which provides illumination for image acquisition, can be a high-brightness light-emitting diode (LED) surface light source, a ring LED light source, or a coaxial LED light source, etc. It features adjustable brightness and uniformity compensation, which can improve image clarity. Ring LED light sources are suitable for contour detection of parts, enhancing the brightness of part edges; coaxial LED light sources are suitable for parts with strong surface reflectivity, reducing the impact of reflections on image acquisition. While meeting image acquisition quality requirements, the appropriate light source type can be selected based on the surface characteristics of the part.

[0066] The stage is a platform used to place the parts to be inspected. It is the part-bearing component of the image acquisition system, and its flatness directly affects the placement of the parts and the image acquisition effect.

[0067] A telecentric lens is a special type of optical lens that has object-side telecentric, image-side telecentric, or bilateral telecentric optical characteristics. It can effectively eliminate barrel distortion and pincushion distortion in images, ensuring that the acquired images can accurately reproduce the actual shape and size of the parts.

[0068] The camera can be an industrial camera, or a high-resolution charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) camera. When equipped with a telecentric lens, it can achieve clear and distortion-free acquisition of part images.

[0069] The preset angle is the illumination angle between the light source and the surface of the part to be inspected. The preset angle can avoid reflection and shadow on the surface of the part and can be determined based on the critical value. The critical value is the angle between the light source and the surface of the part when reflection or shadow occurs on the surface of the part to be inspected. It is the reference threshold for determining the preset angle.

[0070] Flatness error is the deviation between the actual plane of the stage and the ideal plane, reflecting the flatness of the stage. The preset error is the maximum flatness error value set in advance to achieve flat placement of parts and high image acquisition accuracy. It serves as the basis for stage selection and debugging.

[0071] In this way, the image acquisition system is set up as a combination of a light source, a stage, and a camera equipped with a telecentric lens. The use of a telecentric lens effectively eliminates the image distortion problem caused by ordinary industrial lenses, so that the acquired target image can truly restore the actual shape and size of the part to be inspected. The preset angle between the light source and the surface of the part is determined according to the critical value, avoiding reflections or shadows on the surface of the part, making the edge contour of the part clearly visible in the target image, improving the contrast and recognition of the image, and facilitating subsequent target detection and edge extraction. At the same time, the flatness error of the stage is controlled within the preset error, so that the part to be inspected is placed flat, avoiding image acquisition deviation and size detection error caused by part tilt. From the hardware level, the accuracy and stability of the entire inspection method are improved.

[0072] For example, an image acquisition system consisting of a light source, a high-precision stage, and a camera with a telecentric lens can be pre-built. Multiple different types of parts to be inspected (such as various accessories of valve core plastic parts) are neatly placed on the stage. The angle and brightness of the light source are adjusted to avoid reflection or shadows on the surface of the parts. The target image containing all the parts to be inspected is captured by an industrial camera. The camera transmits the image data to a host computer for subsequent processing.

[0073] S102, the target image is input into the target detection model for target detection to obtain the bounding box information and category information of each part to be detected.

[0074] Among them, the parts to be inspected are industrial parts that require dimensional accuracy inspection. Different categories of parts to be inspected refer to parts with differences in shape, size, and specifications, such as valve core body, sealing ring, and positioning pin in valve core plastic parts.

[0075] The object detection model is an algorithm based on computer vision and deep learning. It can identify the location and category of target objects from images, output the bounding box information and category information of the target objects, and has the ability to detect multiple targets and multiple categories.

[0076] Bounding box information is used to define the coordinates of the part to be detected in an image. It typically includes the pixel coordinates of the top left and bottom right corners of the bounding box, which can accurately locate the region of each part to be detected in the image.

[0077] Category information is used to identify the type of the part to be inspected, such as part name, specification code, etc.

[0078] For example, the target image received by the host computer can be preprocessed before being input into the trained target detection model. Preprocessing includes operations such as grayscale conversion, Gaussian filtering, and contrast enhancement.

[0079] The object detection model identifies and locates each part to be detected in an image through deep learning steps such as feature extraction, feature fusion, and object prediction. It outputs the bounding box information of each part, such as the pixel coordinates of (x1, y1, x2, y2), and category information, such as "valve core" and "sealing ring".

[0080] S103, for each part to be detected, determine the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the target image and the bounding box information of the part to be detected; and determine the size information of the part to be detected based on the candidate mapping relationship between the preset pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected.

[0081] Among them, subpixel coordinates are coordinate information that breaks through the limitations of image pixel resolution. They locate the edge of the part to the subpixel level (such as 0.1 pixels) rather than the coordinates of a single pixel, which can improve the accuracy of edge positioning.

[0082] The candidate mapping relationship is the correspondence between pixel coordinates and actual physical size information. It is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts. The candidate mapping relationship is related to the hardware parameters of the image acquisition system and can be constructed separately for different categories of parts to achieve accurate conversion from pixel coordinates to actual size.

[0083] System parameters are the inherent hardware parameters and installation parameters of the image acquisition system, including the internal and external parameters of the camera.

[0084] Candidate parts are all categories of parts pre-included in the inspection system, and each category corresponds one-to-one with the category of the part to be inspected. Their category information is used to construct a unique candidate mapping relationship.

[0085] For example, for each part to be detected output by the target detection model, the corresponding part image region can be extracted from the target image based on the bounding box information of the part to be detected. After binarization, edge detection and denoising of the part image region, the coordinates of the edge points of the part are fitted by the sub-pixel segmentation algorithm to obtain the sub-pixel coordinates corresponding to the bounding box information of the part, thereby achieving high-precision positioning of the part edge.

[0086] Furthermore, candidate mapping relationships between pixel coordinates and size information can be pre-constructed based on the system parameters of the image acquisition system and stored in the host computer. Once the category information and sub-pixel coordinates of the part to be inspected are obtained, the corresponding target mapping relationship is matched from the stored candidate mapping relationships based on the category information. The sub-pixel coordinates are substituted into the candidate mapping relationships, and candidate physical size information of the part to be inspected is obtained through coordinate conversion. A correction coefficient is then determined based on the category information, and the candidate physical size information is corrected using this correction coefficient to obtain the true physical size information of the part to be inspected, such as length, width, and diameter.

[0087] In some optional implementations, a target mapping relationship matching the part to be detected can be selected from different candidate mapping relationships based on the category information of the part to be detected; and the size information of the part to be detected can be determined based on the sub-pixel coordinates corresponding to the target mapping relationship and the bounding box information.

[0088] For example, a candidate mapping relationship between pixel coordinates and size information can be pre-constructed for each type of part based on the system parameters of the image acquisition system and the category information of each candidate part, and stored in the host computer. After obtaining the category information and subpixel coordinates of the part to be detected, the corresponding target mapping relationship is matched from the stored candidate mapping relationship according to the category information. The subpixel coordinates are substituted into the target mapping relationship, and the actual physical size information of the part to be detected, such as length, width, and diameter, is obtained through coordinate conversion.

[0089] For example, taking the size inspection of valve core plastic parts as an example, the parts to be inspected include three different types of parts: valve core body, sealing ring, and positioning pin. These three types of parts are placed on the stage of the image acquisition system at the same time, and the target image containing the three types of parts is obtained by taking pictures with a camera.

[0090] In the above embodiments, a matching target mapping relationship is selected from different candidate mapping relationships based on the category information of the part to be inspected, which realizes fast and accurate matching of candidate mapping relationships, improves the efficiency of size conversion, and avoids size detection errors caused by selecting the wrong mapping relationship. Then, the size information is determined based on the target mapping relationship and sub-pixel coordinates. The exclusive conversion relationship is combined with high-precision sub-pixel coordinates, giving full play to the advantages of both. This ensures the relevance of size conversion and improves the accuracy of size detection, solving the problems of fixed parameters and insufficient accuracy in traditional size conversion, and realizing accurate and fast size detection of different types of parts.

[0091] The image is then input into the target detection model, which outputs the bounding box information (50, 60, 150, 160) and category information "valve core" for the valve core, the bounding box information (200, 80, 280, 160) and category information "sealing ring" for the sealing ring, and the bounding box information (300, 100, 320, 200) and category information "locating pin" for the positioning pin.

[0092] For the valve core, the image area can be cropped based on its bounding box information. After processing, the sub-pixel coordinates (50.2, 60.1, 149.8, 159.9) corresponding to its bounding box can be obtained. Similarly, the sub-pixel coordinates of the sealing ring and the positioning pin can be obtained. Candidate mapping relationships M1, M2, and M3 are pre-constructed for the valve core, sealing ring, and positioning pin, respectively. According to the category information, the sub-pixel coordinates of the valve core are substituted into M1, the sealing ring into M2, and the positioning pin into M3. The diameter of the valve core is calculated to be 12.002 mm, the inner diameter of the sealing ring is 8.001 mm, and the length of the positioning pin is 10.003 mm. The dimensional detection of the three types of parts is completed.

[0093] In the above embodiments, firstly, by acquiring target images containing multiple categories of parts to be detected, it is possible to prepare for batch detection without acquiring images of each part individually; secondly, by using a target detection model to identify the bounding box information and category information of all parts to be detected in the image at once, compared with the traditional template matching positioning method, it is not necessary to set separate matching templates for different parts, thus improving positioning efficiency; thirdly, by determining the sub-pixel coordinates corresponding to the bounding box information, the accuracy of part edge positioning is improved; fourthly, based on the candidate mapping relationship constructed by the image acquisition system parameters and part categories, the true size information is determined by combining category information and sub-pixel coordinates, thus realizing accurate size conversion of different categories of parts without adjusting the size conversion parameters for different parts, solving the problem of low detection efficiency, improving size detection accuracy, and the multi-category adaptive candidate mapping relationship enhances the scenario adaptability of the detection method.

[0094] In some alternative implementations, the object detection model can be trained in the following ways:

[0095] First, obtain multiple sample images and the corresponding sample label data for each sample image.

[0096] Each sample image includes multiple sample parts of different categories. The shooting conditions for sample parts of the same category in different sample images are different. The shooting conditions include lighting conditions and / or the placement angle of the parts. The sample label data includes the bounding box information and category information of different sample parts in the sample image.

[0097] The sample images are image data used to train the object detection model. Each image contains multiple sample parts of different categories. The sample parts and the parts to be detected are industrial parts of the same type.

[0098] Sample label data is reference data used for model training by annotating sample images. It contains bounding box information and category information for each sample part in the sample image and serves as the basis for the model to judge whether the detection results are accurate.

[0099] Sample parts are parts used to capture sample images. They are similar control parts to the parts to be detected, and are used to allow the model to learn the image features of different categories of parts.

[0100] Shooting conditions refer to the environment and placement parameters when shooting sample images. For example, lighting conditions, such as strong light, weak light, side light, and backlight, as well as the placement angle of the parts, such as horizontal, tilted at 15°, and tilted at 30°. Different shooting conditions are used to improve the model's anti-interference ability.

[0101] For example, multiple sample parts of the same type but different categories as the part to be tested can be selected, such as valve cores, sealing rings, and positioning pins. Sample parts of different categories are placed on the stage of the image acquisition system in different combinations. The brightness and angle of the light source are adjusted to change the lighting conditions, and the placement angle of the sample parts is adjusted (e.g., 0°-45°) to capture multiple sample images, so that the shooting conditions of the same category of sample parts in different sample images are different. An annotation tool can be used to annotate the bounding box information (pixel coordinates) and category information of each sample part in each sample image to form sample label data corresponding to each sample image. The sample images and sample label data are divided into training set, validation set and test set according to a preset ratio (e.g., 8:1:1).

[0102] Then, the initial detection model is trained using each sample image and the corresponding sample label data to obtain the target detection model.

[0103] The initial detection models were untrained, basic deep learning object detection models, such as untrained You Only Look Once version 9 (YOLOv9), Faster Region-based Convolutional Neural Networks (FasterR-CNN), Single Shot MultiBox Detector (SSD), and Efficient Detection (EfficientDet). These models had the basic structure but had not learned the image features of the parts.

[0104] For example, a suitable initial detection model (such as YOLOv9) can be selected, and the sample images of the training set can be input into the initial detection model. The model extracts features from the images and outputs predicted bounding box information and category information. The prediction results are compared with the sample label data, and the detection error (such as intersection-over-union ratio, category recognition accuracy, etc.) is calculated. The network parameters of the model are adjusted through the backpropagation algorithm, and the training is iterated repeatedly. During the training process, the training effect of the model is verified using a validation set. When the detection accuracy of the model on the validation set reaches a preset threshold (such as mAP≥95%, where mAP (mean Average Precision) is the average precision), the model is finally tested using a test set. After passing the test, the trained target detection model is obtained.

[0105] In the above embodiments, by acquiring sample images containing multiple categories of sample parts under different shooting conditions and labeling them with corresponding sample data, the initial detection model learns the image features of different categories of parts under various lighting and placement angles during the training process. Then, these sample data are used to train the initial detection model to obtain the target detection model. This makes the trained target detection model have strong anti-interference capabilities and can adapt to the complex lighting environment and slight placement deviations of parts in industrial production. This avoids the positioning deviation or failure problem caused by changes in lighting and placement angles in traditional positioning algorithms, and improves the accuracy and stability of target detection.

[0106] In some alternative implementations, see [link to relevant documentation]. Figure 2 , Figure 2 A flowchart for establishing candidate mapping relationships is provided, which includes the following steps:

[0107] S201, Based on the system parameters of the image acquisition system, establish the basic mapping relationship between pixel coordinates and size information.

[0108] In some optional implementations, system parameters may include internal parameters and external parameters; internal parameters may include focal length and pixel size of a single pixel unit, while external parameters may include the distance between the camera and the stage in the image acquisition system.

[0109] Internal parameters are the camera's inherent optical parameters, which do not change with the camera's mounting position or angle. They are inherent properties of the camera and mainly include focal length and pixel size. Focal length is an optical parameter of the camera lens, referring to the distance from the center of the lens to the image plane, usually measured in millimeters (mm), and directly affects the image aspect ratio and field of view. Pixel size is the physical size of a single photosensitive unit in the camera's image plane, usually measured in micrometers (μm), reflecting the camera's pixel resolution. External parameters describe the camera's mounting position and orientation in the world coordinate system, changing with the camera's mounting position, and mainly refer to the distance between the camera and the stage.

[0110] In this way, by clearly dividing the system parameters into internal and external parameters, and defining the internal parameters as focal length and pixel size of a single pixel unit, and the external parameters as the distance between the camera and the stage, the core parameters for constructing the basic mapping relationship are clearly defined, avoiding mapping deviations caused by chaotic parameter selection. The internal parameters are inherent camera parameters, ensuring the stability of the mapping relationship, while the external parameters are adjustable installation parameters, which facilitate timely parameter updates and reconstruction of the mapping relationship after adjusting the camera height according to the detection requirements.

[0111] For example, a camera calibration method can be used to calibrate the image acquisition system, obtaining the camera's internal and external parameters. Based on the camera's imaging principle and perspective transformation formula, a transformation formula between the pixel coordinate system and the world coordinate system is established. This transformation formula is the fundamental mapping relationship between pixel coordinates and size information, denoted as F(x, y), where (x, y) are the pixel coordinates and F(x, y) are the corresponding real size information. The camera calibration method can be the Zhang Zhengyou calibration method, the self-calibration method, or a calibration method based on active vision, etc.

[0112] S202, for each candidate part, determine the correction coefficient corresponding to the candidate part based on the category information of the candidate part.

[0113] For example, a standard part can be selected for each type of candidate part, and its true size information S0 can be obtained by measuring it with a high-precision measuring tool (such as a micrometer or projector). The standard part is placed in an image acquisition system, and the pixel coordinates (x0, y0) of the standard part are obtained after taking an image. (x0, y0) is substituted into the basic mapping relationship to calculate the converted size S1. The correction coefficient k of the candidate part for this type is calculated according to the formula k=S0 / S1, where k is the correction coefficient, S0 is the true size, and S1 is the converted size of the basic mapping relationship. The above operation is repeated for each type of candidate part to obtain the correction coefficient corresponding to each type of candidate part and store it in association with the category information.

[0114] S203, the basic mapping relationship is corrected by a correction coefficient to obtain the candidate mapping relationship between the pixel coordinates and size information of the candidate parts.

[0115] For example, for each type of candidate part, its corresponding correction coefficient k is combined with the basic mapping relationship F(x, y) to obtain the candidate mapping relationship Fk(x, y) = F(x, y) × k for that type of part; the category information of each type of candidate part is associated with the corresponding candidate mapping relationship and stored in the database of the host computer for easy size conversion of the parts to be tested in the future.

[0116] In the above embodiments, a basic mapping relationship is established based on the system parameters of the image acquisition system, providing a unified basis for the size conversion of all types of parts. This avoids the tedious operation of establishing a mapping relationship for each part individually. For each candidate part, a corresponding correction coefficient is determined based on the category information. The conversion deviation caused by the differences in shape, optical characteristics, etc. of different types of parts is taken into account. Finally, the correction coefficient is used to correct the basic mapping relationship to obtain the candidate mapping relationship, so that each type of part has a unique and accurate size conversion relationship, thus improving the accuracy of size conversion.

[0117] In some alternative implementations, see [link to relevant documentation]. Figure 3 , Figure 3 A flowchart illustrating the process of determining the sub-pixel coordinates corresponding to the bounding box information of a part to be inspected is provided, specifically including the following steps:

[0118] S301, Based on the bounding box information of the part to be detected, determine the image region corresponding to the part to be detected in the target image.

[0119] The image region refers to a local image region in the target image that contains only the part to be detected, extracted based on the bounding box information of the part to be detected.

[0120] For example, after obtaining the bounding box information of the part to be detected (such as the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2)), the image processing software on the host computer can perform region cropping in the target image based on these coordinates, and obtain a rectangular image region with (x1, y1) as the upper left corner and (x2, y2) as the lower right corner. This region is the image region corresponding to the part to be detected.

[0121] S302, Based on the image region, determine the sub-pixel coordinates corresponding to the bounding box information of the part to be detected.

[0122] The image region of the extracted part can be preprocessed to extract the coordinates of the edge points. Then, a subpixel segmentation algorithm is used to fit and refine the coordinates of the edge points to obtain the subpixel level coordinates of the edge of the part. Finally, the subpixel coordinates corresponding to the bounding box information of the part can be determined based on the subpixel coordinates of the edge.

[0123] In the above embodiments, the corresponding image region is determined based on the bounding box information of the part to be detected, and a local region containing only the part is cropped from the target image, eliminating interference from other parts and improving the efficiency and accuracy of edge detection. Furthermore, the sub-pixel coordinates corresponding to the bounding box information are determined based on this specific image region, making the extraction of sub-pixel coordinates more targeted, reducing invalid image data processing, and the processing of local regions reduces the computational load of the algorithm, improving the extraction speed of sub-pixel coordinates, thereby improving the efficiency of the entire size detection method.

[0124] In some alternative implementations, see [link to relevant documentation]. Figure 4 , Figure 4 A flowchart illustrating another method for determining the sub-pixel coordinates corresponding to the bounding box information of a part to be inspected is provided, which specifically includes the following steps:

[0125] S401, perform binarization on the image region to obtain a binary image.

[0126] Binarization is an image processing method that converts a grayscale image into a black-and-white binary image. By setting a grayscale threshold, pixels with grayscale values ​​higher than the threshold are set to white as the foreground (i.e., the part area), while those with grayscale values ​​lower than the threshold are set to black as the background, thus separating the part area from the background. The binary image contains only black and white pixels and is the product of binarization, making it easier to extract the edge contours of the part.

[0127] For example, the captured part image region can be converted to grayscale to obtain a grayscale image; then a grayscale threshold can be calculated, and the grayscale image can be binarized according to the grayscale threshold, setting the part region to white and the background region to black, to obtain a binary image containing only the part and the background.

[0128] S402 uses an edge detection algorithm to extract the initial edge point coordinates of the part region in the binary image.

[0129] Among them, edge detection algorithms are used to extract the edges of objects in an image. For example, the Canny Edge Detection (Canny) algorithm can accurately extract the coordinates of edge points in the part region. The initial edge point coordinates are the pixel coordinates of the part edge points extracted from the binary image by the edge detection algorithm, including invalid coordinates caused by a small number of noise points.

[0130] For example, the Canny edge detection algorithm can be used to process binary images. Through steps such as Gaussian filtering to smooth the image, calculating gradient magnitude and direction, non-maximum suppression, double threshold detection, and edge connection, the edge points of the part region in the binary image are extracted, and the pixel coordinates of these edge points are output as the initial edge point coordinates.

[0131] S403, denoise the initial edge point coordinates to obtain the target edge point coordinates.

[0132] The target edge point coordinates are the pixel coordinates obtained after denoising, which only contain valid edge points.

[0133] For example, a median filtering algorithm can be used to denoise the initial edge point coordinates. The coordinates of each edge point are compared with the coordinates of the surrounding neighborhood, and the coordinates of noise points that deviate from the normal edge are removed, while the continuous and valid edge point coordinates are retained to obtain the target edge point coordinates.

[0134] S404 uses a sub-pixel segmentation algorithm to fit the coordinates of the target edge points, thereby obtaining the sub-pixel coordinates corresponding to the bounding box information of the part to be detected.

[0135] Among them, the subpixel segmentation algorithm is an edge extraction algorithm that breaks through the pixel resolution limitation. It can be the Zernike moment subpixel segmentation algorithm, interpolation method (such as bilinear interpolation, cubic interpolation), least squares fitting algorithm, Sobel subpixel edge detection algorithm, etc., which can fit the edge point coordinates to the subpixel level.

[0136] For example, the Zernike moment subpixel segmentation algorithm can be used. The coordinates of the target edge points are substituted into the fitting formula of the algorithm. By calculating the subpixel-level offset of the edge points, the coordinates of the target edge points are refined and fitted to obtain the subpixel coordinates of the part edge. The original bounding box coordinates are then optimized based on the edge subpixel coordinates to obtain the subpixel coordinates corresponding to the bounding box information of the part to be detected.

[0137] Taking the image region processing of the valve core as an example, the grayscale threshold of 128 is calculated for the extracted grayscale image of the valve core. Pixels with a grayscale value greater than or equal to 128 are set to white (valve core), and those less than 128 are set to black (background), resulting in a binary image. The Canny edge detection algorithm is then used to process the binary image to extract the initial edge point coordinates of the valve core, such as (50, 60), (51, 60), (50, 61)...(150, 160), which includes a small number of noise point coordinates (50, 80). A 3×3 neighborhood is used. The median filter is used to denoise the initial edge point coordinates, removing noise points (50, 80), to obtain continuous target edge point coordinates. The target edge point coordinates are then substituted into the Zernike moment subpixel segmentation algorithm to calculate the subpixel offset of each edge point. For example, the offset of (50, 60) is (50.2, 60.1), and the offset of (150, 160) is (149.8, 159.9). Based on these subpixel edge point coordinates, the subpixel coordinates corresponding to the valve core body bounding box information (50.2, 60.1, 149.8, 159.9) are obtained.

[0138] In the above embodiments, binarization of the image region effectively separates the part from the background, avoiding edge extraction deviations caused by background interference. Furthermore, noise points are removed through denoising to obtain the target edge point coordinates, ensuring the effectiveness and continuity of the edge point coordinates. Subpixel segmentation algorithm is then used to fit the target edge point coordinates to obtain subpixel coordinates, improving the accuracy of the bounding box coordinates to the subpixel level. This enhances the accuracy of part edge positioning and provides crucial high-precision coordinate data for subsequent size conversion, meeting the micron-level size inspection requirements in industrial production.

[0139] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0140] Based on the same inventive concept, this application also provides a part size detection device for implementing the part size detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more of the part size detection device embodiments provided below can be found in the limitations of the part size detection method described above, and will not be repeated here.

[0141] In one exemplary embodiment, such as Figure 5 As shown, a part size detection device is provided, comprising:

[0142] The acquisition module 10 is used to acquire the target image acquired by the image acquisition system; wherein, the target image includes multiple parts to be detected that fall into different categories;

[0143] The detection module 20 is used to input the target image into the target detection model for target detection, and obtain the bounding box information and category information of each part to be detected;

[0144] The determination module 30 is used to determine, for each part to be detected, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the target image and the bounding box information of the part to be detected; and to determine the size information of the part to be detected based on the candidate mapping relationship between the preset pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and the bounding box information of the part to be detected; wherein, the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

[0145] The above solution achieves several key improvements. First, by acquiring target images containing multiple categories of parts to be detected, it eliminates the need to collect images of each part individually, thus providing the foundation for batch detection. Second, by using a target detection model to identify the bounding box and category information of all parts to be detected in the image at once, it improves positioning efficiency compared to traditional template matching methods, eliminating the need to set separate matching templates for different parts. Third, by determining the sub-pixel coordinates corresponding to the bounding box information, it improves the accuracy of part edge positioning. Fourth, based on the candidate mapping relationship constructed using image acquisition system parameters and part categories, and combined with category information and sub-pixel coordinates, it determines the true size information, achieving accurate size conversion for different categories of parts. This eliminates the need to adjust size conversion parameters for different parts, solving the problem of low detection efficiency and improving size detection accuracy. Furthermore, the multi-category adaptive candidate mapping relationship enhances the scenario adaptability of the detection method.

[0146] In one embodiment, the device further includes a training module, which is used to:

[0147] Multiple sample images and corresponding sample label data for each sample image are acquired. Each sample image includes multiple sample parts of different categories. The shooting conditions for sample parts of the same category in different sample images are different, including lighting conditions and / or the placement angle of the parts. The sample label data includes the bounding box information and category information of different sample parts in the sample image. The initial detection model is trained using each sample image and the corresponding sample label data to obtain the target detection model.

[0148] In one embodiment, the image acquisition system includes a light source, a stage for placing parts, and a camera equipped with a telecentric lens; the light source is at a preset angle to the surface of each part to be inspected, the preset angle being determined according to a critical value, the critical value being the angle between the light source and the surface of the part to be inspected when the surface of the part to be inspected produces reflection or shadow; the flatness error of the stage is less than a preset error.

[0149] In one embodiment, the device further includes a building module for:

[0150] Based on the system parameters of the image acquisition system, a basic mapping relationship between pixel coordinates and size information is established; for each candidate part, a correction coefficient is determined according to the category information of the candidate part; the basic mapping relationship is corrected using the correction coefficient to obtain the candidate mapping relationship between pixel coordinates and size information of the candidate part.

[0151] In one embodiment, the determining module 30 is specifically used for:

[0152] Based on the category information of the part to be inspected, a target mapping relationship matching the part to be inspected is selected from different candidate mapping relationships; based on the sub-pixel coordinates corresponding to the target mapping relationship and the bounding box information, the size information of the part to be inspected is determined.

[0153] In one embodiment, the system parameters include internal parameters and external parameters; the internal parameters include focal length and pixel size of a single pixel unit, and the external parameters include the distance between the camera and the stage in the image acquisition system.

[0154] In one embodiment, the determining module 30 is specifically used for:

[0155] Based on the bounding box information of the part to be detected, the image region corresponding to the part to be detected in the target image is determined; based on the image region, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected are determined.

[0156] In one embodiment, the determining module 30 is specifically used for:

[0157] The image region is binarized to obtain a binary image; an edge detection algorithm is used to extract the initial edge point coordinates of the part region in the binary image; the initial edge point coordinates are denoised to obtain the target edge point coordinates; a subpixel segmentation algorithm is used to fit the target edge point coordinates to obtain the subpixel coordinates corresponding to the bounding box information of the part to be detected.

[0158] Each module in the aforementioned part size detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0159] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores image data of the parts. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a method for detecting the dimensions of a part.

[0160] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0161] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the part size detection method described in any of the above embodiments.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the part size detection method described in any of the above embodiments.

[0163] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the part size detection method described in any of the above embodiments.

[0164] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0165] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0166] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for inspecting the dimensions of a part, characterized in that, The method includes: Acquire a target image captured by an image acquisition system; wherein the target image includes multiple different categories of parts to be detected; The target image is input into the target detection model for target detection to obtain the bounding box information and category information of each part to be detected; For each of the parts to be detected, based on the target image and the bounding box information of the part to be detected, the sub-pixel coordinates corresponding to the bounding box information of the part to be detected are determined; and, The size information of the part to be detected is determined based on the preset candidate mapping relationship between pixel coordinates and size information, as well as the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein, the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

2. The method according to claim 1, characterized in that, The target detection model is trained in the following way: Acquire multiple sample images and sample label data corresponding to each sample image; wherein, each sample image includes multiple sample parts of different categories, and the shooting conditions of sample parts of the same category in different sample images are different, the shooting conditions include lighting conditions and / or the placement angle of the parts, and the sample label data includes the bounding box information and category information of different sample parts in the sample image; The initial detection model is trained using the sample images and the corresponding sample label data to obtain the target detection model.

3. The method according to claim 1, characterized in that, The image acquisition system includes a light source, a stage for placing parts, and a camera equipped with a telecentric lens; The light source is at a preset angle to the surface of each of the parts to be tested. The preset angle is determined according to a critical value, which is the angle between the light source and the surface of the part to be tested when the surface of the part to be tested produces reflection or shadow. The flatness error of the stage is less than a preset error.

4. The method according to any one of claims 1-3, characterized in that, The candidate mapping relationship is constructed in the following way: Based on the system parameters of the image acquisition system, establish a basic mapping relationship between pixel coordinates and size information; For each candidate part, a correction coefficient is determined based on the category information of the candidate part. The correction coefficient is used to correct the basic mapping relationship to obtain the candidate mapping relationship between the pixel coordinates and size information of the candidate parts.

5. The method according to claim 4, characterized in that, The step of determining the size information of the part to be detected based on a preset candidate mapping relationship between pixel coordinates and size information, and the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected, includes: Based on the category information of the part to be detected, a target mapping relationship matching the part to be detected is selected from different candidate mapping relationships; The size information of the part to be detected is determined based on the sub-pixel coordinates corresponding to the target mapping relationship and the bounding box information.

6. The method according to claim 4, characterized in that, The system parameters include internal parameters and external parameters; the internal parameters include focal length and pixel size of a single pixel unit, and the external parameters include the distance between the camera and the stage in the image acquisition system.

7. The method according to any one of claims 1-3, characterized in that, The step of determining the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the target image and the bounding box information of the part to be detected includes: Based on the bounding box information of the part to be detected, determine the image region corresponding to the part to be detected in the target image; Based on the image region, determine the sub-pixel coordinates corresponding to the bounding box information of the part to be detected.

8. The method according to claim 7, characterized in that, Determining the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the image region includes: The image region is binarized to obtain a binary image; An edge detection algorithm is used to extract the initial edge point coordinates of the part region in the binary image; The initial edge point coordinates are denoised to obtain the target edge point coordinates; The subpixel segmentation algorithm is used to fit the coordinates of the target edge points to obtain the subpixel coordinates corresponding to the bounding box information of the part to be detected.

9. A part size detection device, characterized in that, The device includes: An acquisition module is used to acquire a target image captured by an image acquisition system; wherein, the target image includes multiple parts to be detected that fall into different categories; The detection module is used to input the target image into the target detection model for target detection, and obtain the bounding box information and category information of each of the parts to be detected; The determination module is configured to, for each part to be detected, determine the sub-pixel coordinates corresponding to the bounding box information of the part to be detected based on the target image and the bounding box information of the part to be detected; and determine the size information of the part to be detected based on a preset candidate mapping relationship between pixel coordinates and size information, and the sub-pixel coordinates corresponding to the category information and bounding box information of the part to be detected; wherein the candidate mapping relationship is constructed based on the system parameters of the image acquisition system and the category information of the candidate parts.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images