Defect positioning method, apparatus and electronic device

Abstract

Embodiments of the present application provide a defect positioning method and device and electronic equipment, and relate to the technical field of image processing. A to-be-detected image about a to-be-detected object collected by an infrared camera is obtained; the to-be-detected object is placed on a conveying belt, and the infrared camera is located above the conveying belt; a defect detection is performed on the to-be-detected image, and an image coordinate of the detected defect in an image coordinate system is determined; a first distance between the defect and a first reference point on the to-be-detected object in a first coordinate axis direction and a second distance between the defect and a second reference point on the to-be-detected object in a second coordinate axis direction are determined according to the image coordinate and a conversion relationship about the image coordinate system and a world coordinate system corresponding to the conveying belt; and the defect is positioned in the to-be-detected object based on the first distance, the second distance, the first reference point and the second reference point. The scheme provided by the embodiments of the present application can improve the defect positioning efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a defect localization method, apparatus, and electronic device. Background Technology

[0002] During the automobile manufacturing process, automobile parts are placed in the stamping workshop for pressing. During the pressing process, due to factors such as process and equipment, pressure scratches may be left on the surface of automobile parts.

[0003] To repair pressure scratches caused during the pressing process, quality inspectors typically inspect each automotive component for pressure scratches after the pressing process and pinpoint the location of any detected scratches. However, determining the location of pressure scratches requires a high degree of precision and focus from the quality inspectors, making the manual method of locating pressure scratches time-consuming and consequently resulting in low efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a defect location method, apparatus, and electronic device to improve defect location efficiency. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a defect localization method, the method comprising:

[0006] Acquire an image of the object to be detected captured by an infrared camera; wherein the object to be detected is placed on a conveyor belt, and the infrared camera is located above the conveyor belt;

[0007] Defect detection is performed on the image to be detected, and the image coordinates of the detected defects in the image coordinate system are determined;

[0008] Based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, determine the first distance between the defect and the first reference point on the object to be detected in the direction of the first coordinate axis, and the second distance between the defect and the second reference point on the object to be detected in the direction of the second coordinate axis;

[0009] Based on the first distance, the second distance, the first reference point, and the second reference point, the defect is located in the object to be detected.

[0010] Optionally, in one specific implementation, determining the first distance between the defect and a first reference point on the object to be detected along the first coordinate axis, and the second distance between the defect and a second reference point on the object to be detected along the second coordinate axis, based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, includes:

[0011] Based on the first coordinate of the first coordinate axis in the image coordinates, the coordinate change value of the preset first initial coordinate when the image to be detected where the defect is located is acquired, the preset first initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first reference point in the direction of the first coordinate axis is determined; wherein, the preset first initial coordinate is: the coordinate of the first reference point in the first coordinate axis when the first reference point is first detected in the image to be detected;

[0012] Based on the positional relationship between the acquisition range and the conveyor belt in the direction of the second coordinate axis in the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset second initial coordinates, and the transformation relationship, the second distance between the defect and the second edge point in the direction of the second coordinate axis is determined; wherein, the preset second initial coordinates are: the coordinates of the second reference point in the second coordinate axis.

[0013] Optionally, in one specific implementation, the positive direction of the first coordinate axis of the image coordinate system is the same as the conveying direction of the conveyor belt; the first reference point is the first edge point of the object to be detected that first or last enters the acquisition range of the infrared camera, and the second reference point is the second edge point of the object to be detected where the coordinate of the second coordinate axis is the largest or smallest.

[0014] The step of determining, based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first reference point on the object to be detected along the first coordinate axis, and the second distance between the defect and the second reference point on the object to be detected along the second coordinate axis, includes:

[0015] Based on the first coordinate of the first coordinate axis in the image coordinates, the coordinate change value of the preset third initial coordinate when the image to be detected containing the defect is acquired, the preset third initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first edge point in the direction of the first coordinate axis is determined; wherein, the preset third initial coordinate is: the coordinate of the first edge point in the first coordinate axis when the first edge point is first detected in the image to be detected;

[0016] Based on the positional relationship between the acquisition range and the conveyor belt in the direction of the second coordinate axis in the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset fourth initial coordinate, and the transformation relationship, the second distance between the defect and the second edge point in the direction of the second coordinate axis is determined; wherein, the preset fourth initial coordinate is: the coordinate of the second edge point in the second coordinate axis.

[0017] Optionally, in one specific implementation, determining the first distance between the defect and the first edge point along the first coordinate axis direction based on the first coordinate of the first coordinate axis in the image coordinates, the coordinate change value of the preset third initial coordinate when acquiring the image to be detected where the defect is located, the preset third initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt includes:

[0018] Calculate the sum of the preset third initial coordinates and the coordinate changes of the preset third initial coordinates when acquiring the image of the defect;

[0019] Calculate the absolute value of the difference between the sum and the first coordinate of the first coordinate axis in the image coordinates;

[0020] Based on the absolute value of the difference and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first edge point in the direction of the first coordinate axis is calculated.

[0021] Optionally, in one specific implementation, the method for determining the coordinate change value includes:

[0022] The coordinate change value is determined based on the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt.

[0023] The acquisition sequence number is used to characterize the order of the images to be detected where the defect is located when arranging the multiple frames of images to be detected in order of acquisition time from earliest to latest.

[0024] Optionally, in one specific implementation, determining the second distance between the defect and the second edge point along the second coordinate axis, based on the positional relationship between the acquisition range and the conveyor belt along the second coordinate axis in the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, a preset fourth initial coordinate, and the transformation relationship, includes:

[0025] Based on the positional relationship between the acquisition range and the conveyor belt along the second coordinate axis of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset fourth initial coordinates, and the transformation relationship, the specified coordinates of the defect are determined; wherein, the specified coordinates include the coordinates of the defect along the second coordinate axis of the world coordinate system;

[0026] Calculate the difference between the target coordinates of the second edge point and the specified coordinates, and use it as the second distance between the defect and the second edge point in the direction of the second coordinate axis; wherein the target coordinates are the coordinates of the second edge point in the world coordinate system on the second coordinate axis.

[0027] Optionally, in one specific implementation, gratings are provided on both sides of the conveyor belt, and the gratings are located at the edge of the acquisition range of the infrared camera; the multiple frames of images to be detected are acquired by the infrared camera during the process from when the object to be detected begins to block the grating to when it moves out of the grating area, and the preset third initial coordinates are: the coordinates of the position of the grating on the first coordinate axis of the image coordinate system.

[0028] Optionally, in one specific implementation, there are multiple infrared cameras arranged along the direction of the second coordinate axis, and the acquisition range of the multiple infrared cameras covers the conveyor belt area along the direction of the second coordinate axis.

[0029] Secondly, embodiments of this application provide a defect location device, the device comprising:

[0030] An image acquisition module is used to acquire an image of the object to be detected captured by an infrared camera; wherein the object to be detected is placed on a conveyor belt, and the infrared camera is located above the conveyor belt;

[0031] The detection module is used to perform defect detection on the image to be detected and determine the image coordinates of the detected defects in the image coordinate system;

[0032] The distance determination module is used to determine, based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, a first distance between the defect and a first reference point on the object to be detected in the direction of the first coordinate axis, and a second distance between the defect and a second reference point on the object to be detected in the direction of the second coordinate axis.

[0033] The positioning module is used to locate the defect in the object to be detected based on the first distance, the second distance, the first reference point, and the second reference point.

[0034] Optionally, in one specific implementation, the distance determination module is specifically used for:

[0035] Based on the first coordinate of the first coordinate axis in the image coordinates, the coordinate change value of the preset first initial coordinate when the image to be detected where the defect is located is acquired, the preset first initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first reference point in the direction of the first coordinate axis is determined; wherein, the preset first initial coordinate is: the coordinate of the first reference point in the first coordinate axis when the first reference point is first detected in the image to be detected;

[0036] Based on the positional relationship between the acquisition range and the conveyor belt in the direction of the second coordinate axis in the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset second initial coordinates, and the transformation relationship, the second distance between the defect and the second edge point in the direction of the second coordinate axis is determined; wherein, the preset second initial coordinates are: the coordinates of the second reference point in the second coordinate axis.

[0037] Optionally, in one specific implementation, the positive direction of the first coordinate axis of the image coordinate system is the same as the conveying direction of the conveyor belt; the first reference point is the first edge point of the object to be detected that first or last enters the acquisition range of the infrared camera, and the second reference point is the second edge point of the object to be detected where the coordinate of the second coordinate axis is the largest or smallest.

[0038] The distance determination module includes:

[0039] The first distance determination submodule is used to determine the first distance between the defect and the first edge point in the direction of the first coordinate axis based on the first coordinate of the first coordinate axis in the image coordinate system, the coordinate change value of the preset third initial coordinate when the image to be detected where the defect is located is acquired, the preset third initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt; wherein, the preset third initial coordinate is: the coordinate of the first edge point in the first coordinate axis when the first edge point is first detected in the image to be detected;

[0040] The second distance determination submodule is used to determine the second distance between the defect and the second edge point in the direction of the second coordinate axis based on the positional relationship between the acquisition range and the conveyor belt in the direction of the second coordinate axis in the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset fourth initial coordinates, and the transformation relationship; wherein, the preset fourth initial coordinates are: the coordinates of the second edge point in the second coordinate axis.

[0041] Optionally, in one specific implementation, the first distance determination submodule is specifically used for:

[0042] Calculate the sum of the preset third initial coordinates and the coordinate changes of the preset third initial coordinates when acquiring the image of the defect;

[0043] Calculate the absolute value of the difference between the sum and the first coordinate of the first coordinate axis in the image coordinates;

[0044] Based on the absolute value of the difference and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first edge point in the direction of the first coordinate axis is calculated.

[0045] Alternatively, in one specific implementation,

[0046] The device further includes a coordinate change value determination module, which is specifically used for:

[0047] The coordinate change value is determined based on the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt.

[0048] The acquisition sequence number is used to characterize the order of the images to be detected where the defect is located when arranging the multiple frames of images to be detected in order of acquisition time from earliest to latest.

[0049] Optionally, in one specific implementation, the second distance determination submodule is specifically used for:

[0050] Based on the positional relationship between the acquisition range and the conveyor belt along the second coordinate axis of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset fourth initial coordinates, and the transformation relationship, the specified coordinates of the defect are determined; wherein, the specified coordinates include the coordinates of the defect along the second coordinate axis of the world coordinate system;

[0051] Calculate the difference between the target coordinates of the second edge point and the specified coordinates, and use it as the second distance between the defect and the second edge point in the direction of the second coordinate axis; wherein the target coordinates are the coordinates of the second edge point in the world coordinate system on the second coordinate axis.

[0052] Optionally, in one specific implementation, gratings are provided on both sides of the conveyor belt, and the gratings are located at the edge of the acquisition range of the infrared camera; the multiple frames of images to be detected are acquired by the infrared camera during the process from when the object to be detected begins to block the grating to when it moves out of the grating area, and the preset third initial coordinates are: the coordinates of the position of the grating on the first coordinate axis of the image coordinate system.

[0053] Optionally, in one specific implementation, there are multiple infrared cameras arranged along the direction of the second coordinate axis, and the acquisition range of the multiple infrared cameras covers the conveyor belt area along the direction of the second coordinate axis.

[0054] Thirdly, embodiments of this application provide an electronic device, including:

[0055] Memory, used to store computer programs;

[0056] When a processor executes a program stored in memory, it implements the steps of any of the above method embodiments.

[0057] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above method embodiments.

[0058] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of any of the above method embodiments.

[0059] Beneficial effects of the embodiments in this application:

[0060] As can be seen above, when locating defects, the image coordinates of the detected defect in the image coordinate system can be determined first. Then, based on the image coordinates of the detected defect and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the distances between the defect and the reference point in the directions of the two coordinate axes can be determined. Using these distances, the defect can be located in the image to be inspected. This method, compared to manual location methods, can shorten the location cycle and improve defect location efficiency. Furthermore, using infrared images acquired by an infrared camera for defect detection can avoid poor detection results due to part reflections, improving the accuracy of defect detection and, consequently, the accuracy of defect location. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0062] Figure 1(a)-Figure 1(b) These are schematic diagrams of images of the objects to be detected provided in the embodiments of this application;

[0063] Figure 1(c)-Figure 1(d) These are schematic diagrams showing the positive direction of the first coordinate axis and the specified transmission direction of the image coordinate system provided in the embodiments of this application;

[0064] Figure 2(a) is a schematic diagram of the grating provided in an embodiment of this application;

[0065] Figure 2(b) is a schematic diagram of the installation position of the infrared camera provided in the embodiment of this application;

[0066] Figure 2(c) is a schematic diagram showing the relationship between the acquisition range of the infrared camera provided in the embodiment of this application and the position of the conveyor belt in the direction of the second coordinate axis of the world coordinate system;

[0067] Figures 2(d)-2(e) These are schematic diagrams illustrating specific examples of the second distance provided in the embodiments of this application;

[0068] Figure 2(f)-Figure 2(g) These are schematic diagrams showing the positions of reference points provided in the embodiments of this application;

[0069] Figure 3 A flowchart illustrating a defect location method provided in an embodiment of this application;

[0070] Figure 4 A flowchart illustrating yet another defect location method provided in an embodiment of this application;

[0071] Figures 5(a)-5(b) These are schematic diagrams illustrating defect location examples provided in the embodiments of this application;

[0072] Figure 6 A flowchart illustrating yet another defect location method provided in an embodiment of this application;

[0073] Figure 7 A schematic diagram showing the preset third initial coordinates of the first edge point provided in an embodiment of this application;

[0074] Figure 8 This is a schematic diagram of the structure of a defect location device provided in an embodiment of this application;

[0075] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0077] To repair pressure scratches caused during the pressing process, quality inspectors typically inspect each automotive component for pressure scratches after the pressing process and pinpoint the location of any detected scratches. However, determining the location of pressure scratches requires a high degree of precision and focus from the quality inspectors, making the manual method of locating pressure scratches time-consuming and consequently resulting in low efficiency.

[0078] To address the aforementioned issues, this application provides a defect location method.

[0079] This method is applicable to various scenarios where infrared cameras are needed to locate defects on objects, such as locating defects on the outer panel of a car's front door or on the casing of a laptop computer.

[0080] Furthermore, the executing entity of this method can be any electronic device capable of acquiring infrared images and other data, and processing the acquired data. This electronic device can be an infrared camera with data processing capabilities, or various other electronic devices with data processing capabilities that communicate with the infrared camera, such as mobile phones, laptops, desktop computers, and industrial control computers. Moreover, when the executing entity of this method is an electronic device, this electronic device can be an independent electronic device or a cluster of multiple electronic devices, hereinafter referred to as electronic devices.

[0081] Therefore, the application scenarios and execution entities of the embodiments of this application are not limited.

[0082] This application provides a defect location method, which may include the following steps:

[0083] Acquire an image of the object to be detected captured by an infrared camera; wherein the object to be detected is placed on a conveyor belt, and the infrared camera is located above the conveyor belt;

[0084] Defect detection is performed on the image to be detected, and the image coordinates of the detected defects in the image coordinate system are determined;

[0085] Based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, determine the first distance between the defect and the first reference point on the object to be detected in the direction of the first coordinate axis, and the second distance between the defect and the second reference point on the object to be detected in the direction of the second coordinate axis;

[0086] Based on the first distance, the second distance, the first reference point, and the second reference point, the defect is located in the object to be detected.

[0087] As can be seen above, when locating defects, the image coordinates of the detected defect in the image coordinate system can be determined first. Then, based on the image coordinates of the detected defect and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the distances between the defect and the reference point in the directions of the two coordinate axes can be determined. Using these distances, the defect can be located in the image to be inspected. This method, compared to manual location methods, can shorten the location cycle and improve defect location efficiency. Furthermore, using infrared images acquired by an infrared camera for defect detection can avoid poor detection results due to part reflections, improving the accuracy of defect detection and, consequently, the accuracy of defect location.

[0088] To better understand the defect localization method provided in the embodiments of this application, the relevant concepts involved in the embodiments of this application will first be explained.

[0089] The world coordinate system is a three-dimensional rectangular coordinate system that reflects the position of objects in the real world. The origin of the world coordinate system can be determined based on the actual situation.

[0090] The camera coordinate system is a three-dimensional Cartesian coordinate system with its origin located at the optical center of the camera lens. The X and Y axes are parallel to the two sides of the image plane, respectively, and the Z axis is the optical axis of the lens, perpendicular to the image plane. The image plane is the plane on which the camera forms an image.

[0091] The pixel coordinate system is a two-dimensional Cartesian coordinate system that reflects the arrangement of pixels in the camera's charge-coupled device (CCD). The origin of the pixel coordinate system is located at the top left corner of the image, and the U-axis and V-axis are parallel to the two sides of the image plane, respectively. Furthermore, the unit of the coordinate axes in the pixel coordinate system is pixels (integers).

[0092] The image coordinate system is a two-dimensional Cartesian coordinate system. Because the pixel coordinate system is not conducive to coordinate transformations, the image coordinate system was established. The units of the coordinate axes in the pixel coordinate system can be converted to and from the units of the coordinate axes in the world coordinate system and the camera coordinate system. The origin of the image coordinate system is the intersection of the camera's optical axis and the image plane (or the principal point), which is the center point of the image. The X-axis of the image coordinate system is parallel to the U-axis of the pixel coordinate system, and the Y-axis of the image coordinate system is parallel to the V-axis of the pixel coordinate system. Therefore, the pixel coordinate system and the image coordinate system can be considered as a translation relationship; that is, the pixel coordinate system can be obtained by translation of the image coordinate system. The difference between the two lies in the units of the coordinate axes.

[0093] Next, examples will be given to illustrate the scenarios involved in the camera calibration method provided in the embodiments of this application.

[0094] For some objects with relatively smooth surfaces, such as metal parts like car door panels and front hood panels, when visible light shines on them, the image effect is as shown in Figure 1(a) due to specular reflection. Under this imaging effect, defects on the object cannot be effectively detected. Therefore, in order to effectively detect defects on various objects, an infrared light source can be used to illuminate the object, and an infrared camera can be used to acquire an image of the object, as shown in Figure 1(b). The infrared image of the object can then be used for detection, thereby detecting defects on the object.

[0095] To perform defect detection on each object to be inspected, each object can be placed sequentially on a conveyor belt with a specified conveying direction. As the conveyor belt moves, each object will be transported sequentially to the image acquisition area.

[0096] An infrared light source is installed within the image acquisition area, and an infrared camera is installed above the conveyor belt within this area. Thus, when the object to be inspected enters the image acquisition area along the conveyor belt, the infrared light source illuminates the object on the conveyor belt, and the infrared camera above the conveyor belt captures an image of the object.

[0097] As the conveyor belt moves, the coordinates of a point on the conveyor belt change along one axis of the world coordinate system corresponding to the conveyor belt. Therefore, the axis corresponding to this changing coordinate can be used as the first coordinate, and the other axis as the second coordinate axis. Furthermore, the first coordinate axis can be either the X-axis or the Y-axis. Correspondingly, when the X-axis is the first coordinate axis, the Y-axis is the second coordinate axis; and vice versa.

[0098] Therefore, the infrared camera can be mounted along the second coordinate axis of the world coordinate system. The positive direction of the first coordinate axis of the image coordinate system corresponding to the infrared camera can be the same as or different from the specified transmission direction. Furthermore, the first coordinate axis of the image coordinate system can be either the X-axis or the Y-axis; both are reasonable and are not specifically limited in this embodiment.

[0099] For example, as shown in Figure 1(c), if the positive direction of the first coordinate axis of the image coordinate system is the same as the specified conveying direction of the conveyor belt, then as the conveyor belt moves, the coordinates of point W on the object to be detected in the image coordinate system along the first coordinate axis gradually increase; as shown in Figure 1(d), if the positive direction of the first coordinate axis of the image coordinate system is opposite to the specified conveying direction of the conveyor belt, then as the conveyor belt moves, the coordinates of point W on the object to be detected in the image coordinate system along the first coordinate axis gradually decrease.

[0100] Optionally, the infrared camera can acquire images at a preset acquisition frequency to obtain multiple frames of images of the object to be detected. The preset acquisition frequency can be set according to actual needs, such as acquiring images 5 times per second or 10 times per second, which are all reasonable and are not specifically limited in this embodiment.

[0101] To determine the start time for image acquisition of the object to be inspected on the conveyor belt, a sensing device can be pre-positioned within the image acquisition area. Thus, if the object triggers the sensing device during its transport, the infrared camera can be controlled to begin acquiring images of the object, resulting in multiple frames of images of the object.

[0102] The aforementioned sensing device can be a grating, a radar, a gravity sensor, or other types of sensing devices. All of these are reasonable and are not specifically limited in the embodiments of this application.

[0103] Optionally, in one specific implementation, gratings are provided on both sides of the conveyor belt, and the gratings are located at the edge of the acquisition range of the infrared camera; the multiple frames of images to be detected are acquired by the infrared camera during the process from when the object to be detected begins to block the grating to when it moves out of the grating area, and the preset third initial coordinates are: the coordinates of the position of the grating on the first coordinate axis of the image coordinate system.

[0104] In this specific implementation, gratings can be set on both sides of the conveyor belt, and the gratings are located at the edge of the infrared camera's acquisition range. Thus, when the object to be detected begins to block the gratings, the infrared camera can be controlled to start acquiring an image of the object, and when the object leaves the grating area, the infrared camera can be controlled to stop acquiring an image of the object.

[0105] Based on this, during the process from when the object to be detected begins to block the grating until it moves out of the grating area, the infrared camera can acquire multiple frames of images of the object to be detected, and each frame can be used as the image to be detected for that object. In other words, the image to be detected for the object is the image acquired by the infrared camera when the object is within the grating area.

[0106] For example, as shown in Figure 2(a), gratings can be set on both sides of the conveyor belt used to transport the object to be detected in the image acquisition area. After the object to be detected is placed on the conveyor belt, it can be transported into the grating area as the conveyor belt moves. Thus, when the object to be detected begins to obscure the grating, the infrared camera can be controlled to acquire an image of the object. When the object to be detected is detected to have been transported out of the grating area and no longer obscure the grating, the acquisition of images of the object by the infrared camera is stopped. In this way, multiple frames of images of the object to be detected can be obtained.

[0107] Because the acquisition range of a single infrared camera is too small to cover the entire area of ​​the conveyor belt along the second coordinate axis, the camera can only capture images of a portion of the object to be inspected, not the entire object. In other words, the image captured by a single infrared camera may not be able to accurately detect all defects on the object, and therefore cannot be fully located.

[0108] Based on this, in one optional implementation, there are multiple infrared cameras arranged along the direction of the second coordinate axis, and the acquisition range of the multiple infrared cameras covers the conveyor belt area along the direction of the second coordinate axis.

[0109] In this specific implementation, multiple infrared cameras can be installed above the conveyor belt in the image acquisition area, along the direction of the second coordinate axis of the world coordinate system. In this way, the multiple infrared cameras can be arranged side-by-side along the direction of the second coordinate axis of the world coordinate system, and the acquisition range of the multiple infrared cameras can cover all areas of the conveyor belt along the direction of the second coordinate axis.

[0110] The installation spacing between two adjacent infrared cameras can be set according to the acquisition range of the infrared cameras and the width of the conveyor belt. For example, it can be 30 cm or 50 cm, which are both reasonable and are not specifically limited in this embodiment.

[0111] For example, as shown in Figure 2(b), 13 infrared cameras can be installed side-by-side above the conveyor belt used to transport the object to be inspected, with the positive Y-axis of the image coordinate system corresponding to each infrared camera aligned with the direction of the conveyor belt's movement. This allows the infrared cameras to be arranged side-by-side along the positive X-axis of the image coordinate system, enabling the 13 cameras to capture images of various areas of the conveyor belt in that direction. Furthermore, gratings can be installed below the infrared cameras and on both sides of the conveyor belt, allowing the 13 infrared cameras to begin capturing images of the parts as they begin to obscure the gratings along with the conveyor belt.

[0112] In order to locate the defect on the object to be inspected by using the frames of images captured by the infrared camera, the image coordinate system corresponding to the infrared camera and the world coordinate system corresponding to the conveyor belt can be calibrated in advance, that is, the transformation relationship between the image coordinate system and the world coordinate system can be determined.

[0113] Optionally, the Zhang Zhengyou calibration method can be used to calculate the transformation relationship between the image coordinate system corresponding to the infrared camera and the world coordinate system corresponding to the conveyor belt.

[0114] Specifically, a calibration plate can be placed on a conveyor belt, allowing it to move with the belt. When the calibration plate triggers the grating, an infrared camera can capture a calibration image of the plate. Therefore, using the calibration image captured by the infrared camera, the transformation relationship between the calibration plate's coordinate system (world coordinate system) and the image coordinate system corresponding to the calibration image can be determined.

[0115] The calibration board can be a checkerboard calibration board, a dot calibration board, a CharuCo calibration board, etc., which are all reasonable. In this embodiment of the application, the specific type of calibration board is not specifically limited.

[0116] The aforementioned transformation relationship between the calibration plate coordinate system and the image coordinate system includes a first transformation relationship between the calibration plate coordinate system and the camera coordinate system, and a second transformation relationship between the camera coordinate system and the image coordinate system.

[0117] The process of transforming the calibration plate coordinate system to the camera coordinate system can be achieved through rotation and translation. Thus, as shown in Equation 1, the transformation matrix R1 can be represented by a homogeneous coordinate matrix composed of rotation matrix and translation vector.

[0118] Equation 1 can be expressed as:

[0119]

[0120] Where R is the rotation matrix, t is the translation vector, and X C Y C Z C X represents the coordinates in the camera coordinate system. W Y W Z W This indicates the coordinates in the calibration plate coordinate system.

[0121] Since the upper left corner feature point of the calibration plate can be defined as the origin in the calibration plate coordinate system, the plane of the calibration plate can be exactly aligned with the X-axis. W -Y W Plane coincidence, Z W The axis passes through the plane of the calibration plate and points vertically upwards; at this point, the Z-axis of all feature points on the calibration plate... W Setting Z = 0 simplifies subsequent calculations. Therefore, as shown above, Z can be omitted. W The rotation amount of the shaft is r3.

[0122] The transformation from the camera coordinate system to the image coordinate system is a transformation from a three-dimensional coordinate system to a two-dimensional coordinate system. To solve the transformation relationship between the two, the ordinary image coordinates (x, y) can be extended to homogeneous coordinates (x, y, 1). Thus, as shown in Equation 2, the transformation matrix R2 can be represented by Equation 2.

[0123] Equation 2 can be expressed as:

[0124]

[0125] Where x and y represent coordinates in the image coordinate system, f represents the camera focal length, and X C Y C Z C This represents the coordinates in the camera coordinate system.

[0126] Furthermore, the transformation relationship R between the image coordinate system and the calibration plate coordinate system z It can be represented by Equation 3.

[0127] Equation 3 can be expressed as:

[0128] R Z =R1R2

[0129] Therefore, the transformation relationship between the image coordinate system corresponding to the infrared camera and the world coordinate system corresponding to the conveyor belt can be expressed by R. Z express.

[0130] Furthermore, since each infrared camera has a fixed acquisition range, for each infrared camera, the positional relationship between the acquisition range of the infrared camera and the conveyor belt on the second coordinate axis of the world coordinate system can be predetermined based on the acquisition range of the infrared camera. That is, for each infrared camera, the coordinate range of the conveyor belt area acquired by the acquisition range of the infrared camera on the second coordinate axis of the world coordinate system can be determined.

[0131] When determining the coordinate range of the conveyor belt area captured by each infrared camera on the second coordinate axis of the world coordinate system, the coordinate range of the conveyor belt on the second coordinate axis can be determined first based on the size of the conveyor belt in that direction. Then, for each infrared camera, the conveyor belt area captured by that camera in that direction can be determined based on the coordinate range of the conveyor belt on the second coordinate axis, the camera's installation position, and its acquisition range; that is, the specified coordinate range of the camera's acquisition area on the second coordinate axis of the world coordinate system can be determined. Furthermore, since the infrared cameras are installed side-by-side along the second coordinate axis, their positions remain constant. Therefore, as the conveyor belt moves, the infrared camera always captures an image of the conveyor belt area within that specified coordinate range along the second coordinate axis.

[0132] For example, as shown in Figure 2(c), the conveyor belt area along the second coordinate axis can be divided into regions A and D based on the acquisition range of the infrared camera and the width of the conveyor belt along the second coordinate axis. The coordinate range of the second coordinate axis of the conveyor belt in the world coordinate system is [0, 20]. The infrared camera E can acquire an image of region C, and the coordinate range of region C along the second coordinate axis is [10, 15]. Therefore, the coordinate range of the second coordinate axis of the image acquired by the infrared camera E in the image coordinate system corresponds to the second coordinate range [10, 15] of the second coordinate axis in the world coordinate system.

[0133] Optionally, if multiple infrared cameras are installed above the conveyor belt, the area of ​​the conveyor belt collected by each infrared camera in the direction of the second coordinate axis can be determined according to the arrangement order of the infrared cameras, that is, the collection range of each infrared camera in the specified coordinate range of the second coordinate axis of the world coordinate system can be determined.

[0134] For example, as shown in Figure 2(c), the coordinate range of the second coordinate axis of the conveyor belt in the world coordinate system is [0, 20]. Four infrared cameras are arranged side by side above the conveyor belt. The areas captured by each infrared camera are area A to area D of the conveyor belt, respectively. Thus, the coordinate range of the second coordinate axis of the image captured by each infrared camera in the image coordinate system corresponds to the second coordinate range of the second coordinate axis in the world coordinate system, respectively, which are [0, 5], [6, 10], [11, 15] and [16, 20]. That is, the second coordinate range of the second coordinate axis of the acquisition area of ​​infrared camera A in the world coordinate system is [0, 5]; the second coordinate range of the acquisition area of ​​infrared camera B in the world coordinate system is [6, 10]; the second coordinate range of the acquisition area of ​​infrared camera C in the world coordinate system is [11, 15]; and the second coordinate range of the acquisition area of ​​infrared camera D in the world coordinate system is [16, 20].

[0135] Based on the above description, the following describes a defect location method provided by an embodiment of this application.

[0136] Figure 3 This is a flowchart illustrating a defect location method provided in an embodiment of this application, as shown below. Figure 3 As shown, the method may include the following steps S301-S304.

[0137] S301: Acquire the image of the object to be detected captured by the infrared camera;

[0138] The object to be inspected is placed on a conveyor belt, and the infrared camera is located above the conveyor belt.

[0139] To perform defect detection and location on an object, the object can be placed on a conveyor belt with a specified conveying direction. As the object moves along the conveyor belt, an infrared camera positioned above it captures multiple frames of images. The electronic equipment then acquires these frames and, based on them, determines the image to be inspected.

[0140] In this application, multiple frames of images acquired by the infrared camera can be used as the images to be detected for the object to be detected; alternatively, a portion of the images can be selected from the multiple frames acquired by the infrared camera and used as the images to be detected for the object to be detected. For example, multiple frames can be selected from the multiple frames acquired at preset intervals and used as the images to be detected for the object to be detected. These are all reasonable options and are not specifically limited in this application embodiment.

[0141] S302: Perform defect detection on the image to be inspected and determine the image coordinates of the detected defects in the image coordinate system;

[0142] After acquiring the image to be detected, defect detection can be performed on the image. If a defect is detected in the image, the image coordinates of the detected defect in the image coordinate system corresponding to the infrared camera can be determined.

[0143] The aforementioned defect detection methods can be deep learning methods, similar comparison methods, image feature judgment methods, etc., all of which are reasonable and are not specifically limited in this application embodiment.

[0144] When performing defect detection on an image to be inspected, defect detection can be performed on each frame of the image to be inspected, or a subset of images can be selected from multiple frames of the image to be inspected for defect detection. For example, a subset of images from multiple frames of the image to be inspected can be selected for defect detection at a preset interval. Alternatively, defect detection can be performed on images from multiple frames of the image to be inspected that meet preset image quality requirements. Furthermore, defect detection can be performed on images from multiple frames of the image to be inspected where the area of ​​the object to be inspected reaches a preset ratio. All of these are reasonable and are not specifically limited in the embodiments of this application.

[0145] S303: Based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, determine the first distance between the defect and the first reference point on the object to be inspected in the direction of the first coordinate axis, and the second distance between the defect and the second reference point on the object to be inspected in the direction of the second coordinate axis.

[0146] As the conveyor belt moves, different areas of the object to be detected will sequentially enter the acquisition range of the infrared camera. Therefore, the corresponding area of ​​the object to be detected in each frame of the image captured by the infrared camera will be different. Thus, in different images, the same coordinate range of the first coordinate axis in the image coordinate system can correspond to different areas on the object to be detected. In other words, as the conveyor belt moves, a certain coordinate axis on the first coordinate axis of the image coordinate system can correspond to multiple different points on the object to be detected. While the second coordinate of each point on the object to be detected on the second coordinate axis remains unchanged, its first coordinate on the first coordinate axis will continuously increase or decrease.

[0147] As mentioned earlier, if the positive direction of the first coordinate axis of the image coordinate system corresponding to the infrared camera is the same as the specified conveying direction, then as the conveyor belt moves, the second coordinate of each point on the object to be detected on the second coordinate axis in the image coordinate system remains unchanged, while its first coordinate on the first coordinate axis will continuously increase; if the positive direction of the first coordinate axis of the image coordinate system corresponding to the infrared camera is opposite to the specified conveying direction, then as the conveyor belt moves, the second coordinate of each point on the object to be detected on the second coordinate axis in the image coordinate system remains unchanged, while its first coordinate on the first coordinate axis will continuously decrease.

[0148] Typically, the positive direction of the first coordinate axis of the world coordinate system corresponding to the conveyor belt is the same as the conveying direction. Based on this, the range of coordinates of the infrared camera's acquisition area on the second coordinate axis in the world coordinate system remains unchanged, while the range of coordinates of its acquisition area on the first coordinate axis of the world coordinate system will continuously increase or decrease as the conveyor belt moves. Correspondingly, for each infrared camera, the coordinate values ​​of each point through that infrared camera on the first coordinate axis of the world coordinate system will also continuously increase or decrease as the conveyor belt moves. Therefore, the coordinate values ​​of each point on the first coordinate axis of the image coordinate system will also continuously increase or decrease as the conveyor belt moves.

[0149] For example, if the positive direction of the first coordinate axis of the image coordinate system corresponding to the infrared camera is the same as the specified conveying direction, then as the conveyor belt moves, the first coordinate of each point with the first coordinate m in the first frame image captured by the infrared camera becomes n in the third frame image, while the second coordinate of the aforementioned points remains unchanged. In the first frame image, the point o located on the edge of the object to be detected in the direction of the first coordinate axis and having the largest first coordinate has a first coordinate of b. As the conveyor belt moves, its first coordinate continuously increases in subsequent frames, becoming b1, b2, ..., bn respectively.

[0150] Furthermore, in each frame of the image captured by the infrared camera, there may be a designated point with a first coordinate of m and a second coordinate of n. However, the designated point located in different frames of the image to be detected corresponds to different points in the object to be detected.

[0151] In other words, as the conveyor belt moves, the image coordinates of each point on the object to be detected are constantly changing, thus the same image coordinates will correspond to multiple different points. However, for each point on the object to be detected, the first coordinate of that point in the first coordinate axis direction in the image coordinate system may increase or decrease as the conveyor belt moves, but the actual position of that point on the object to be detected remains fixed.

[0152] Therefore, it is difficult to locate the actual position of any point on the object to be detected based on its coordinates in the image coordinate system. Thus, to determine the actual position of each specified point on the object to be detected, two reference points can be set on the object. In this way, for any point on the object other than the reference points, its actual position on the object can be determined based on its relative position to the two reference points.

[0153] Either of the two reference points can be set according to actual needs. It can be any point inside the object to be detected, or an edge point located on the edge of the object to be detected. Furthermore, when the two reference points are edge points, they can be any edge points selected on the edge of the object to be detected, or edge points on the edge of the object to be detected with maximum or minimum coordinates. For example, a point can be selected on each of two adjacent edges of the object to be detected as the first reference point and the second reference point; the edge point on the object to be detected with the maximum coordinate in the first coordinate axis direction of the world coordinate system can be used as the first reference point, and the edge point on the object to be detected with the maximum coordinate in the second coordinate axis direction of the world coordinate system can be used as the second reference point; the edge point on the object to be detected with the minimum coordinate in the first coordinate axis direction of the world coordinate system can be used as the first reference point, and the edge point on the object to be detected with the minimum coordinate in the second coordinate axis direction of the world coordinate system can be used as the second reference point, and so on.

[0154] Since the actual position of each point on the object to be detected is constant, the first distance between the point and the first reference point in the direction of the first coordinate axis is constant, and the second distance between the point and the second reference point in the direction of the second coordinate axis can be constant.

[0155] For example, if the two reference points are the first edge point and the second edge point of the object to be detected, the first coordinate axis of the image coordinate system is the Y-axis, the second coordinate axis is the X-axis, and the positive direction of the Y-axis is the same as the specified conveyor belt direction, then the point with image coordinates (6, 4) in the first frame image will have image coordinates (6, 9) in the third frame image as the conveyor belt moves. However, regardless of how the image coordinates of this point change, its position in the object to be detected does not change. Therefore, the distance between this point and the first edge point of the object to be detected along the first coordinate axis in the world coordinate system remains unchanged, and the distance between this point and the second edge point of the object to be detected along the second coordinate axis in the world coordinate system also remains unchanged.

[0156] As can be seen above, when it is desired to locate any detected point on the object to be detected, the position of each specified point on the object to be detected can be determined by using the first distance between the point and the first reference point of the object to be detected in the direction of the first coordinate axis of the world coordinate system, and the second distance between the point and the second reference point of the object to be detected in the direction of the second coordinate axis of the world coordinate system.

[0157] Accordingly, after a defect is detected, in order to locate the defect on the object to be inspected, a first distance between the point and a first reference point of the object to be inspected in the direction of the first coordinate axis, and a second distance between the point and a second reference point of the object to be inspected in the direction of the second coordinate axis can be determined. Then, using the first and second distances, the position of each specified point on the object to be inspected can be determined.

[0158] Specifically, after a defect is detected, the first coordinate difference between the defect and the first reference point in the first coordinate axis direction of the image coordinate system can be determined based on the image coordinates of the defect and the image coordinates of two reference points.

[0159] Since the coordinates of each point on the object to be inspected can change along the first coordinate axis, when calculating the first coordinate difference between the defect and the first reference point in the first coordinate axis of the image coordinate system, it is also necessary to calculate the coordinates of the first reference point in the first coordinate axis of the image coordinate system when the infrared camera captures the image of the object to be inspected that detects the defect, or to calculate the coordinates of the defect in the first coordinate axis of the image coordinate system when the infrared camera captures the image of the object to be inspected that detects the first reference point. Thus, after obtaining the coordinates of the defect and the first reference point in the first coordinate axis of the image coordinate system at the same acquisition time, the first coordinate difference between the defect and the first reference point in the first coordinate axis of the image coordinate system can be calculated.

[0160] Furthermore, in order to locate the defect in the world coordinate system, the first coordinate difference needs to be transformed into the world coordinate system. That is, according to the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the absolute value of the first coordinate difference can be converted into the first distance between the defect and the first reference point in the direction of the first coordinate axis in the world coordinate system.

[0161] Accordingly, since the acquisition range of each infrared camera is fixed, and there is a positional relationship between the acquisition range of each infrared camera and the coordinates in the second coordinate axis direction of the world coordinate system, the coordinates of the defect in the second coordinate axis direction of the world coordinate system can be determined based on the positional relationship between the acquisition range of the infrared camera and the conveyor belt in the second coordinate axis direction of the world coordinate system, and the second coordinates of the image coordinates of the defect in the second coordinate axis direction. Furthermore, based on the positional relationship between the acquisition range of the infrared camera and the conveyor belt in the second coordinate axis direction of the world coordinate system, and the second coordinates of the image coordinates of the second reference point in the second coordinate axis direction of the world coordinate system, the coordinates of the second reference point in the second coordinate axis direction of the world coordinate system can be determined. Then, the difference between the coordinates of the defect in the second coordinate axis direction of the world coordinate system and the second coordinates of the second reference point in the second coordinate axis direction of the world coordinate system is calculated, and the absolute value of this second coordinate difference is taken as the second distance between the defect and the second reference point in the second coordinate axis direction of the world coordinate system.

[0162] In other words, the first distance between the aforementioned defect and the first reference point along the first coordinate axis of the world coordinate system is the absolute value of the difference between the coordinates of the defect and the first reference point along the first coordinate axis in the world coordinate system; the second distance between the defect and the second reference point along the second coordinate axis of the world coordinate system is the absolute value of the difference between the coordinates of the defect and the second reference point along the second coordinate axis in the world coordinate system.

[0163] Thus, after performing defect detection on the image to be inspected and determining the image coordinates of the detected defect in the image coordinate system, the first distance between the defect and the first reference point on the object to be inspected in the first coordinate axis direction, and the second distance between the defect and the second reference point on the object to be inspected in the second coordinate axis direction can be determined based on the image coordinates of the defect and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt.

[0164] For example, as shown in Figure 2(f), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is the same as the specified conveying direction of the conveyor belt. The first reference point is point C1 in the object to be detected, and the second reference point is point C2 in the object to be detected. The image to be detected captured by the infrared camera can be acquired. Then, based on the image to be detected, the image coordinates of C1, C2, and the defect are determined. According to the specified conveying direction, the defect point is acquired and detected earlier than C1. Therefore, based on the first coordinate of the defect point in the first coordinate axis direction, the coordinates of the defect point in the first coordinate axis direction when C1 is acquired can be calculated. Then, the first coordinate difference between the coordinates of the defect point in the first coordinate axis direction when C1 is acquired and the coordinates in the first coordinate axis direction of the image coordinates of C1 is calculated. Based on the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the absolute value of the first coordinate difference is converted into the first distance between the defect point and C1 in the first coordinate axis direction.

[0165] Based on the positional relationship between the infrared camera's acquisition range and the conveyor belt along the second coordinate axis of the world coordinate system, the second coordinate of the defect along the second coordinate axis in the image coordinates of the defect, and the coordinate of C2 along the second coordinate axis in the image coordinates of C2, the coordinates of the defect and C2 along the second coordinate axis of the world coordinate system are calculated. The second coordinate difference between the second coordinate difference and the second coordinate difference is then calculated. The absolute value of the second coordinate difference is taken as the second distance between the defect point and C2 along the second coordinate axis.

[0166] For example, as shown in Figure 2(f), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is the same as the specified conveying direction of the conveyor belt. The first reference point is the first edge point C4 of the object to be detected that first enters the acquisition range of the infrared camera, and the second reference point is the second edge point C3 of the object to be detected with the largest coordinate on the second coordinate axis. The image to be detected acquired by the infrared camera can be obtained, and then the image coordinates of C4 and the defect can be determined based on the image to be detected. According to the specified conveying direction, C4 is acquired and detected earlier than the defect point. Therefore, the coordinates of C4 in the first coordinate axis direction when the defect point is acquired can be calculated based on the coordinates of C4 in the first coordinate axis direction. Then, the first coordinate difference between the first coordinate of C4 in the first coordinate axis direction when the defect point is acquired and the first coordinate in the first coordinate axis direction of the image coordinates of the defect point is calculated. Based on the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the absolute value of the above first coordinate difference is converted into the first distance between the defect point and C4 in the first coordinate axis direction.

[0167] Since the coordinates of C4 in the second coordinate axis direction of the world coordinate system are known, the coordinates of the defect in the second coordinate axis direction of the world coordinate system can be calculated based on the acquisition range of the infrared camera, the positional relationship between the conveyor belt in the second coordinate axis direction of the world coordinate system, and the second coordinates in the second coordinate axis direction of the defect image coordinates. The second coordinate difference between the second coordinates of the defect in the second coordinate axis direction of the world coordinate system and the second coordinates of C4 in the second coordinate axis direction of the world coordinate system can also be calculated. The absolute value of the second coordinate difference is then used as the second distance between the defect point and C4 in the second coordinate axis direction.

[0168] For example, as shown in Figure 2(g), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is opposite to the specified conveying direction of the conveyor belt. The first reference point is point C5 in the object to be detected, and the second reference point is point C6 in the object to be detected. The image to be detected captured by the infrared camera can be obtained. Then, based on the image to be detected, the image coordinates of C5, C6, and the defect are determined. According to the specified conveying direction, C5 is acquired and detected earlier than the defect point. Therefore, based on the first coordinate of C5 in the first coordinate axis direction, the coordinate of C5 in the first coordinate axis direction when the defect point is acquired can be calculated. Then, the first coordinate difference between the first coordinate of C5 in the first coordinate axis direction when the defect point is acquired and the first coordinate in the first coordinate axis direction of the image coordinates of the defect point is calculated. Based on the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the absolute value of the first coordinate difference is converted into a first distance between the defect point and C5 in the first coordinate axis direction.

[0169] Based on the positional relationship between the infrared camera's acquisition range and the conveyor belt in the second coordinate axis direction of the world coordinate system, the second coordinate in the second coordinate axis direction of the defect's image coordinates, and the coordinate in the second coordinate axis direction of C6's image coordinates, the coordinates of the defect and C6 in the second coordinate axis direction of the world coordinate system are calculated. The second coordinate difference between the second coordinate difference and the second coordinate difference is then calculated. The absolute value of the second coordinate difference is taken as the second distance between the defect point and C6 in the second coordinate axis direction.

[0170] For example, as shown in Figure 2(g), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is opposite to the specified conveying direction of the conveyor belt. The first reference point is the first edge point C8 where the object to be detected last enters the acquisition range of the infrared camera, and the second reference point is the second edge point C7 where the coordinate of the second coordinate axis of the object to be detected is the smallest. The image to be detected acquired by the infrared camera can be obtained, and then the image coordinates of C8 and the defect can be determined based on the image to be detected. According to the specified conveying direction, the defect point is acquired and detected earlier than C8. Therefore, the coordinates of the defect point in the first coordinate axis direction when C8 is acquired can be calculated based on the first coordinates of the defect point in the first coordinate axis direction. Then, the first coordinate difference between the first coordinates of the defect point in the first coordinate axis direction when C8 is acquired and the first coordinate difference in the first coordinate axis direction of the image coordinates of C8 can be calculated. Based on the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the absolute value of the first coordinate difference is converted into the first distance between the defect point and C8 in the first coordinate axis direction.

[0171] Since the coordinates of C7 in the second coordinate axis direction of the world coordinate system are known, the coordinates of the defect in the second coordinate axis direction of the world coordinate system can be calculated based on the acquisition range of the infrared camera and the positional relationship of the conveyor belt in the second coordinate axis direction of the world coordinate system, as well as the second coordinates in the second coordinate axis direction of the defect image coordinates. The second coordinate difference between the second coordinate difference of the defect and the second coordinate of C7 in the second coordinate axis direction of the world coordinate system can be calculated. The absolute value of the second coordinate difference is then used as the second distance between the defect point and C7 in the second coordinate axis direction.

[0172] Alternatively, in one specific implementation, such as Figure 4 As shown, step S303 above may include the following steps S3031-S3032:

[0173] S3031: Based on the first coordinate of the first coordinate axis in the image coordinates, the coordinate change value of the preset first initial coordinate when acquiring the image of the defect, the preset first initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, determine the first distance between the defect and the first reference point in the direction of the first coordinate axis.

[0174] The preset first initial coordinates are: the coordinates of the first reference point on the first coordinate axis when the first reference point is first detected in the image to be detected;

[0175] S3032: Based on the positional relationship between the acquisition range and the conveyor belt in the second coordinate axis direction of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset second initial coordinates, and the transformation relationship, determine the second distance between the defect and the second edge point in the second coordinate axis direction;

[0176] The preset second initial coordinates are: the coordinates of the second reference point on the second coordinate axis.

[0177] In this specific implementation, as the conveyor belt moves, each part of the object to be inspected will be sequentially conveyed to the image acquisition area. Thus, the infrared camera can acquire each frame of the image to be inspected for each part of the object to be inspected. In this way, when a defect and / or reference point on the object to be inspected is detected in the image to be inspected, the image coordinates of the first reference point, the second reference point, and the defect can be determined.

[0178] Specifically, if the positive direction of the first coordinate axis of the image coordinate system is the same as the specified conveying direction of the conveyor belt, then the first coordinate in the image coordinates of each point on the object to be detected will increase as the conveyor belt moves, while the second coordinate in the image coordinates of each point remains unchanged; if the positive direction of the first coordinate axis of the image coordinate system is opposite to the specified conveying direction of the conveyor belt, then the first coordinate in the image coordinates of each point on the object to be detected will decrease as the conveyor belt moves, while the second coordinate in the image coordinates of each point remains unchanged.

[0179] For the first reference point, the coordinates of the first reference point on the first coordinate axis when it is first detected in the image to be detected can be used as the preset first initial coordinates. Furthermore, the coordinates of the first reference point on the first coordinate axis will increase or decrease as the conveyor belt moves, while the coordinates of the first reference point on the second coordinate axis can remain unchanged.

[0180] As for the second reference point, the coordinates of the second reference point on the second coordinate axis when it is first detected in the image to be detected can be used as the preset second initial coordinates. Furthermore, the coordinates of the second reference point on the second coordinate axis will not increase or decrease as the conveyor belt moves, but will always remain unchanged.

[0181] Correspondingly, for defects, after determining the image coordinates of the defects, the first coordinate of the first coordinate axis in the image coordinates will increase or decrease as the conveyor belt moves, while the second coordinate of the second coordinate axis in the image coordinates can remain unchanged.

[0182] For example, if the image coordinates of the defect are (4, 1), and the specified conveying direction of the conveyor belt is the same as the positive direction of the first coordinate axis of the image coordinate system, then as the conveyor belt moves, the image coordinates of the defect can be updated to (5, 1); if the image coordinates of the defect are (4, 1), and the specified conveying direction of the conveyor belt is opposite to the positive direction of the first coordinate axis of the image coordinate system, then as the conveyor belt moves, the image coordinates of the defect can be updated to (1, 1).

[0183] Furthermore, the image coordinates of the defect and the preset first initial coordinates of the first reference point may be determined based on the same frame of the image to be detected, that is, the image to be detected when the defect is first detected and the image to be detected when the first reference point is first detected may be the same frame of the image to be detected; or they may be determined based on different images to be detected, that is, the image to be detected when the defect is first detected and the image to be detected when the first reference point is first detected may be two frames of the image to be detected.

[0184] Specifically, if the image to be detected that first detects the aforementioned defect and the image to be detected that first detects the first reference point are the same frame of the image to be detected, then based on the image to be detected, the first coordinate of the defect in the image coordinates and the preset first initial coordinate of the first reference point can be determined. Furthermore, as the conveyor belt moves, the change values ​​of the first coordinate and the preset first initial coordinate are the same. Thus, the first coordinate difference between the first coordinate and the preset first initial coordinate can be calculated, and based on the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the absolute value of the first coordinate difference can be converted into the first distance between the defect point and the first reference point in the direction of the first coordinate axis of the world coordinate system.

[0185] If the first image to detect the aforementioned defect and the first image to detect the first reference point are two frames of images to be detected acquired by different infrared cameras at the same acquisition time, then based on the two frames of images to be detected, the first coordinate in the image coordinates of the defect and the preset first initial coordinate of the first reference point can be determined. Furthermore, as the conveyor belt moves, the changes in the first coordinate and the preset first initial coordinate are the same. Therefore, the first coordinate difference between the first coordinate and the preset first initial coordinate can be calculated, and based on the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the absolute value of the first coordinate difference can be converted into the first distance between the defect point and the first reference point in the direction of the first coordinate axis of the world coordinate system.

[0186] If, as the conveyor belt moves, the first image to detect the aforementioned defect is the preceding image A, and the first image to detect the first reference point is the following image B, then when image A is acquired, the first coordinate in the image coordinates of the defect can be determined; when image B is acquired, the preset first initial coordinate of the first reference point can be determined; and when the preset first initial coordinate is determined, the first coordinate of the aforementioned defect has already changed as the conveyor belt moves.

[0187] For example, if the positive direction of the first coordinate axis of the image coordinate system is the same as the specified conveying direction, then when determining the preset first initial coordinate of the first reference point, the first coordinate of the defect has increased by the coordinate change value as the conveyor belt is moving; if the positive direction of the first coordinate axis of the image coordinate system is opposite to the specified conveying direction, then when determining the preset first initial coordinate of the first reference point, the first coordinate of the defect has decreased by the coordinate change value as the conveyor belt is moving.

[0188] Therefore, based on the aforementioned first coordinates, the preset first initial coordinates, the coordinate change values, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect point and the first reference point in the direction of the first coordinate axis of the world coordinate system can be calculated.

[0189] If, as the conveyor belt moves, the first image to be detected at the first reference point is the preceding image C, and the first image to be detected with a defect is the following image D, then when acquiring image C, the preset first initial coordinates of the first reference point can be determined, and when acquiring image D, the first coordinate in the image coordinates of the defect can be determined. Furthermore, when determining the first coordinate, the preset first initial coordinates have been updated as the conveyor belt moves.

[0190] For example, if the positive direction of the first coordinate axis of the image coordinate system is the same as the specified conveying direction, then when determining the first coordinate of the defect, the preset first initial coordinate of the first reference point has increased by the coordinate change value as the conveyor belt moves; if the positive direction of the first coordinate axis of the image coordinate system is opposite to the specified conveying direction, then when determining the first coordinate of the defect, the preset first initial coordinate of the first reference point has decreased by the coordinate change value as the conveyor belt moves.

[0191] Therefore, based on the aforementioned first coordinates, the preset first initial coordinates, the coordinate change values, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect point and the first reference point in the direction of the first coordinate axis of the world coordinate system can be calculated.

[0192] As can be seen above, when the image to be detected that first detects the above-mentioned defect and the image to be detected that first detects the first reference point are not acquired at the same acquisition time, when calculating the first distance between the above-mentioned defect and the first reference point in the direction of the first coordinate axis, it is necessary to consider the coordinate change value of the defect or the first reference point, that is, it is necessary to consider the coordinate change value of the first coordinate or the above-mentioned preset first initial coordinate.

[0193] Therefore, the aforementioned first coordinate or the aforementioned preset first initial coordinate can be used as the reference coordinate, and the other coordinate value between the aforementioned first coordinate and the aforementioned preset first initial coordinate, excluding the reference coordinate, can be used as the variable coordinate. Thus, the coordinate change value of the variable coordinate when the reference coordinate is determined can be calculated. In this way, the coordinate values ​​of the defect and the first reference point respectively along the first coordinate axis at a certain moment can be determined. Furthermore, based on the aforementioned first coordinate, preset first initial coordinate, coordinate change value, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first reference point along the first coordinate axis can be determined.

[0194] Typically, the first coordinate of the defect can be used as the reference coordinate to calculate the coordinate change value of the preset first initial coordinate when the image to be inspected where the defect is located is acquired. Then, the first distance between the defect and the first reference point in the direction of the first coordinate axis can be determined by using the first coordinate, the preset first initial coordinate, the coordinate change value of the preset first initial coordinate when the image to be inspected where the defect is located is acquired, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt.

[0195] Optionally, in one specific implementation, the method for determining the aforementioned coordinate change value may include the following step 11:

[0196] Step 11: Determine the coordinate change value based on the infrared camera's acquisition frequency, the acquisition sequence number of the image to be inspected where the defect is located, and the conveyor belt's moving speed.

[0197] The acquisition sequence number is used to characterize the order in which the defect is located when multiple frames of images to be detected are arranged in order of acquisition time from earliest to latest.

[0198] In this specific implementation, for each point, the change in the coordinates of that point on the first coordinate axis is the change in the coordinates of that point on the first coordinate axis compared to the initial value of the coordinates of that point on the first coordinate axis when the image to be detected where the defect point is located is acquired.

[0199] Correspondingly, for the first reference point, the coordinate change value of the first initial coordinate is preset when the image to be detected where the defect is located is acquired. That is, when the image to be detected where the defect is located is acquired, the change of the coordinate of the first reference point on the first coordinate axis compared with the preset first initial coordinate is acquired.

[0200] Since the aforementioned coordinate change values ​​are based on the movement of the conveyor belt, these coordinate change values ​​can be calculated based on the displacement of the conveyor belt in the image coordinate system when the infrared camera acquires the image of the defect to be detected, compared to when the infrared camera acquires the image of the defect to be detected to determine the preset first initial coordinates.

[0201] Based on this, the coordinate change value of the preset first initial coordinates can be determined when acquiring the image of the defect, according to the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected used to determine the preset first initial coordinates, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt.

[0202] The aforementioned acquisition sequence number can be used to characterize the order of multiple frames of images to be detected arranged in order of acquisition time from earliest to latest, and the sorting of a certain image to be detected.

[0203] Furthermore, since the aforementioned coordinate change is the displacement of the conveyor belt between the acquisition time of the image to be detected used to determine the preset first initial coordinates and the acquisition time of the image to be detected where the defect is located, the displacement of the conveyor belt can be determined based on the difference between the acquisition sequence number of the image to be detected used to determine the preset first initial coordinates and the acquisition sequence number of the image to be detected where the defect is located, and thus, the aforementioned coordinate change value can be determined.

[0204] Therefore, the coordinate change value Y B ’ It can be represented as:

[0205] Y' B =C / FR*Vbelt*Scale

[0206] Where C is the difference in image acquisition sequence number, FR is the preset acquisition frequency, Vbelt is the moving speed of the conveyor belt, and Scale is the transformation relationship between the image coordinate system corresponding to the infrared camera and the world coordinate system corresponding to the conveyor belt.

[0207] In other words, the displacement of the conveyor belt during the time difference between acquiring two frames of images to be inspected, after being converted according to the above conversion relationship, can be used as the coordinate change value of the preset first initial coordinates. The sign of this coordinate change value is related to the relationship between the positive direction of the first coordinate axis of the image coordinate system and the specified conveying direction, as well as the acquisition sequence of the images to be inspected used to determine the preset first initial coordinates and the acquisition sequence of the images to be inspected where the defect is located.

[0208] For example, as shown in Figure 2(f), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is the same as the specified conveying direction of the conveyor belt, and the first reference point is point C1 in the object to be detected. The image to be detected captured by the infrared camera can be acquired, and then, based on the image to be detected, the preset first initial coordinates of C1 and the first coordinates of the defect are determined. According to the specified conveying direction, the defect point is acquired and detected earlier than C1. Therefore, based on the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected used to determine the preset first initial coordinates, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt, the coordinate change value of the preset first initial coordinates when acquiring the image to be detected where the defect is located can be determined. Moreover, the coordinate change value is negative, that is, when acquiring the image to be detected where the defect is located, the coordinate of C1 in the first coordinate axis direction is less than the preset first initial coordinates, and the difference between the two is the absolute value of the coordinate change value.

[0209] For example, as shown in Figure 2(f), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is the same as the specified conveying direction of the conveyor belt. The first reference point is the first edge point C4 of the object to be detected that first enters the acquisition range of the infrared camera. The image to be detected acquired by the infrared camera can be obtained. Then, based on the image to be detected, the preset first initial coordinates of C4 and the first coordinates of the defect are determined. According to the specified conveying direction, C4 is acquired and detected earlier than the defect point. Therefore, based on the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected used to determine the preset first initial coordinates, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt, the coordinate change value of the preset first initial coordinates when acquiring the image to be detected where the defect is located can be determined. Moreover, the coordinate change value is positive, that is, when acquiring the image to be detected where the defect is located, the coordinate of C1 in the first coordinate axis direction is greater than the preset first initial coordinates, and the difference between the two is the absolute value of the coordinate change value.

[0210] For example, as shown in Figure 2(g), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is opposite to the specified conveying direction of the conveyor belt, and the first reference point is point C5 in the object to be detected. The image to be detected captured by the infrared camera can be acquired, and then, based on the image to be detected, the preset first initial coordinates of C5 and the first coordinates of the defect are determined. According to the specified conveying direction, C5 is acquired and detected earlier than the defect point. Therefore, based on the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected used to determine the preset first initial coordinates, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt, the coordinate change value of the preset first initial coordinates when acquiring the image to be detected where the defect is located can be determined. Furthermore, this coordinate change value is negative, meaning that when acquiring the image to be detected where the defect is located, the coordinate of C1 in the first coordinate axis direction is less than the preset first initial coordinate, and the difference between the two is the absolute value of the coordinate change value.

[0211] For example, as shown in Figure 2(g), the positive direction of the Y-axis of the image coordinate system corresponding to the infrared camera is opposite to the specified conveying direction of the conveyor belt. The first reference point is the first edge point C8 where the object to be detected last enters the acquisition range of the infrared camera. The image to be detected acquired by the infrared camera can be obtained. Then, based on the image to be detected, the preset first initial coordinates of C8 and the first coordinates of the defect are determined. According to the specified conveying direction, the defect point is acquired and detected earlier than C8. Therefore, based on the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected used to determine the preset first initial coordinates, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt, the coordinate change value of the preset first initial coordinates when acquiring the image to be detected where the defect is located can be determined. Moreover, the coordinate change value is positive, that is, when acquiring the image to be detected where the defect is located, the coordinate of C8 in the first coordinate axis direction is greater than the preset first initial coordinates, and the difference between the two is the absolute value of the coordinate change value.

[0212] Therefore, after determining the coordinate change value of the preset first initial coordinates when acquiring the image to be inspected where the defect is located, the coordinates of the first reference point in the direction of the first coordinate axis can be determined when acquiring the image to be inspected where the defect is located.

[0213] Next, the first coordinate difference between the first coordinate mentioned above and the first coordinate difference between the first reference point and the first coordinate in the direction of the first coordinate axis when the image to be detected at the location of the defect is acquired can be calculated. Then, the absolute value of the first coordinate difference can be converted using the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, thereby obtaining the first distance between the defect and the first reference point in the direction of the first coordinate axis.

[0214] Accordingly, the second distance between the defect and the second edge point in the second coordinate axis direction can be determined based on the positional relationship between the acquisition range and the conveyor belt in the second coordinate axis direction of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinate system, the preset second initial coordinates, and the transformation relationship.

[0215] As mentioned above, since the acquisition range of each infrared camera is fixed and there is a positional relationship between the acquisition range of each infrared camera and the coordinates in the second coordinate axis direction of the world coordinate system, after determining the second coordinates in the second coordinate axis direction of the defect image coordinates and the preset second initial coordinates of the second reference point in the second coordinate axis, the second distance between the defect and the second edge point in the second coordinate axis direction can be determined based on the positional relationship between the acquisition range and the conveyor belt in the second coordinate axis direction of the world coordinate system, the aforementioned second coordinates, the aforementioned preset second initial coordinates, and the aforementioned transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt.

[0216] Specifically, if the image to be detected that first detects the aforementioned defect and the image to be detected that first detects the second reference point are both images acquired by the same infrared camera, then the second coordinate and the preset second initial coordinate can be transformed into the world coordinate system by utilizing the acquisition range of the infrared camera and the positional relationship of the conveyor belt along the second coordinate axis of the world coordinate system, as well as the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt. Then, the difference between the second coordinate and the preset second initial coordinate in the world coordinate system is calculated, and the absolute value of this difference is used as the second distance between the defect and the second edge point along the second coordinate axis.

[0217] If the image to be detected that initially detects the aforementioned defect and the image to be detected that initially detects the second reference point are not images acquired by an infrared camera, then the second coordinates and the preset second initial coordinates can be transformed into the world coordinate system by utilizing the acquisition range of the two infrared cameras and the positional relationship of the conveyor belt along the second coordinate axis of the world coordinate system, as well as the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt. Then, the difference between the second coordinates in the world coordinate system and the preset second initial coordinates in the world coordinate system is calculated, and the absolute value of this difference is used as the second distance between the defect and the second edge point along the second coordinate axis.

[0218] In other words, the coordinates of the defect on the second coordinate axis of the world coordinate system can be determined first, based on the second coordinates of the defect, the acquisition range of the infrared camera acquiring the image of the defect, the positional relationship of the conveyor belt along the second coordinate axis of the world coordinate system, and the aforementioned transformation relationship. Then, based on the preset second initial coordinates of the second reference point, the acquisition range of the infrared camera acquiring the image of the second reference point, the positional relationship of the conveyor belt along the second coordinate axis of the world coordinate system, and the aforementioned transformation relationship, the coordinates of the second reference point on the second coordinate axis of the world coordinate system can be determined. Afterwards, the second coordinate difference between the second coordinates of the defect and the second reference point on the second coordinate axis of the world coordinate system is calculated, and the absolute value of this second coordinate difference is used as the second distance between the defect and the second reference point along the second coordinate axis.

[0219] S304: Locate defects in the object to be inspected based on the first distance, the second distance, the first reference point, and the second reference point.

[0220] Since the specific locations of the first reference point and the second reference point on the object to be inspected are known, after determining the first distance between the defect and the first reference point in the first coordinate axis direction, and the second distance between the defect and the second reference point in the second coordinate axis direction, the defect can be located in the object to be inspected based on the first distance, the second distance, the first reference point, and the second reference point.

[0221] Optionally, the first coordinates of the first reference point can be determined in the object to be inspected, and then the first coordinates of the defect on the first coordinate axis of the world coordinate system can be determined based on the first distance mentioned above; the second coordinates of the second reference point can be determined in the object to be inspected, and then the second coordinates of the defect on the second coordinate axis of the world coordinate system can be determined based on the second distance mentioned above; in this way, after determining the first and second coordinates of the defect, the position of the defect on the object to be inspected can be determined.

[0222] For example, as shown in Figure 5(a), the first coordinate axis is the Y-axis and the second coordinate axis is the X-axis. In the direction of the Y-axis, the first distance between the defect and the first reference point (e, f, 1) is j, so the Y coordinate of the defect is fj. In the direction of the X-axis, the second distance between the defect and the second reference point (g, h, 1) is i, so the X coordinate of the defect is gi. Thus, the position of the detected defect in the world coordinate system is (gi, fj, 1).

[0223] Optionally, a straight line can be drawn that passes through the first coordinate axis, is parallel to the second coordinate axis, and has a coordinate difference of a first distance from the coordinates of the first reference point on the first coordinate axis. Another straight line can be drawn that passes through the second coordinate axis, is parallel to the first coordinate axis, and has a coordinate difference of a second distance from the coordinates of the second edge point on the second coordinate axis. The intersection of the two straight lines is the location of the defect on the object to be inspected.

[0224] For example, as shown in Figure 5(b), the first coordinate axis is the Y-axis, the second coordinate axis is the X-axis, and the second reference point is point p. A straight line 1 is drawn that passes through the Y-axis, is parallel to the X-axis, and is a distance of the first distance from point q. A straight line 2 is drawn that passes through the X-axis, is parallel to the Y-axis, and is a distance of the second distance from point p. Thus, the intersection of the above straight lines 1 and 2 on the object to be inspected is the determined defect.

[0225] Optionally, after determining the location of the defect on the object to be inspected, the defect can be drawn on the digital model of the object to be inspected.

[0226] Optionally, since the same defect may be detected in multiple frames of images to be detected, defects within a certain range can be merged into the same defect and the defect can be marked on the object to be detected.

[0227] As can be seen above, when locating defects, the image coordinates of the detected defect in the image coordinate system can be determined first. Then, based on the image coordinates of the detected defect and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the distances between the defect and the reference point in the directions of the two coordinate axes can be determined. Using these distances, the defect can be located in the image to be inspected. This method, compared to manual location methods, can shorten the location cycle and improve defect location efficiency. Furthermore, using infrared images acquired by an infrared camera for defect detection can avoid poor detection results due to part reflections, improving the accuracy of defect detection and, consequently, the accuracy of defect location.

[0228] Since both the first and second reference points can be set according to actual needs, and the relationship between the positive direction of the first coordinate axis of the image coordinate system and the specified conveyor direction of the conveyor belt can also be set according to actual needs, after setting the first and second reference points, and the relationship between the positive direction of the first coordinate axis of the image coordinate system and the conveyor direction of the conveyor belt, the first and second distances can be calculated based on the aforementioned relationship between the first and second reference points, the positive direction of the first coordinate axis of the image coordinate system, and the conveyor direction of the conveyor belt. Furthermore, since the calculation principles of the first and second distances remain unchanged, after updating the aforementioned parameters, the specific calculation process of the first and second distances during defect localization can be adjusted accordingly based on the updated relationship between the first and second reference points, the positive direction of the first coordinate axis of the image coordinate system, and the conveyor direction of the conveyor belt.

[0229] The following example illustrates how, in step S303, the positive direction of the first coordinate axis of the image coordinate system is the same as the conveyor belt's conveying direction; the first reference point is the first edge point of the object to be detected that first or last enters the acquisition range of the infrared camera; and the second reference point is the second edge point of the object to be detected whose coordinates on the second coordinate axis are the largest or smallest. Based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first reference point on the object to be detected in the direction of the first coordinate axis, and the second distance between the defect and the second reference point on the object to be detected in the direction of the second coordinate axis are determined.

[0230] Alternatively, in one specific implementation, such as Figure 6 As shown, step S303 above, which determines the first distance between the defect and the first reference point on the object to be inspected along the first coordinate axis, and the second distance between the defect and the second reference point on the object to be inspected along the second coordinate axis, based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, may include the following steps S3033-S3034:

[0231] S3033: Based on the first coordinate of the first coordinate axis in the image coordinate system, the coordinate change value of the preset third initial coordinate when acquiring the image of the defect, the preset third initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, determine the first distance between the defect and the first edge point in the direction of the first coordinate axis.

[0232] The preset third initial coordinates are: the coordinates of the first edge point on the first coordinate axis when the first edge point is first detected in the image to be detected;

[0233] S3034: Based on the positional relationship between the acquisition range and the conveyor belt in the second coordinate axis direction of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset fourth initial coordinates, and the transformation relationship, determine the second distance between the defect and the second edge point in the second coordinate axis direction;

[0234] The preset fourth initial coordinates are: the coordinates of the second edge point on the second coordinate axis.

[0235] In this specific implementation, the positive direction of the first coordinate axis of the image coordinate system corresponding to the infrared camera is the same as the conveying direction of the conveyor belt. The first reference point is set as the first edge point of the object to be detected that enters the acquisition range of the infrared camera first or last, and the second reference point is set as the second edge point of the object to be detected with the largest or smallest coordinate of the second coordinate axis.

[0236] Specifically, when a first edge point is detected for the first time in the image to be detected, the coordinates of the first edge point on the first coordinate axis are preset third initial coordinates; when a second edge point is detected for the first time in the image to be detected, the coordinates of the second edge point on the second coordinate axis are preset fourth initial coordinates. Furthermore, since the coordinates on the second coordinate axis do not change with the movement of the conveyor belt, the coordinates of the second edge point on the second coordinate axis are always preset fourth initial coordinates.

[0237] Furthermore, if the first edge point is the edge point where the object to be detected first enters the acquisition range of the infrared camera, then the first edge point is the point with the largest coordinate on the first coordinate axis of the object to be detected; if the first edge point is the edge point where the object to be detected last enters the acquisition range of the infrared camera, then the first edge point is the point with the smallest coordinate on the first coordinate axis of the object to be detected.

[0238] For example, if the first edge point is the edge point where the object to be detected first enters the acquisition range of the infrared camera, based on the first frame image of the object to be detected acquired by the infrared camera, the maximum value of the first coordinate of the object to be detected on the first coordinate axis in the image coordinate system is determined to be 6, and the points with the first coordinate value of 6 include points A to F, then any point among points A to F can be taken as the first edge point, and the preset initial coordinate of the first edge point is set to 6.

[0239] Since there are gratings on both sides of the conveyor belt, and the infrared camera only starts to acquire the image of the object to be detected when the edge of the object to be detected blocks the grating, if the first edge point is the edge point of the object to be detected that first enters the acquisition range of the infrared camera, then the first edge point is the point on the object to be detected that first blocks the grating, and the preset third initial coordinate of the first edge point on the first coordinate axis of the image coordinate system is the first coordinate of the position of the grating on the first coordinate axis of the image coordinate system.

[0240] Based on this, optional, such as Figure 7 As shown, a calibration plate can be placed on a conveyor belt, and its edge aligned with the grating position. Then, an infrared camera can be controlled to acquire an infrared image of the calibration plate, and the initial coordinates Y of the calibration plate's edge in the Y-axis direction can be determined in the image coordinate system corresponding to the infrared image. raster That is, determining the position of the raster on the first coordinate axis Y of the image coordinate system. raster In this way, the preset initial coordinates Y of the first edge point can be determined. raster .

[0241] If the first edge point is the edge point where the object to be detected last enters the acquisition range of the infrared camera, then the first edge point is the last point on the object to be detected that blocks the grating. Furthermore, the preset third initial coordinate of the first edge point on the first coordinate axis of the image coordinate system is the last coordinate of the grating's position on the first coordinate axis of the image coordinate system.

[0242] After determining the image coordinates of the defect and the preset third initial coordinates, in order to calculate the first distance between the defect and the first edge point in the direction of the first coordinate axis, it is also necessary to calculate the coordinate change value of the preset third initial coordinates when acquiring the image to be detected where the defect is located.

[0243] For the first edge point, the coordinate change value of the first edge point is preset to the third initial coordinate when the image to be detected where the defect is located is acquired. That is, when the image to be detected where the defect is located is acquired, the coordinate change value of the first edge point on the first coordinate axis is compared with the preset third initial coordinate.

[0244] When the first edge point is the edge point where the object to be detected first enters the acquisition range of the infrared camera, the image to be detected with the above-mentioned preset third initial coordinates is determined as the first frame of the image to be detected. Thus, when calculating the coordinate change value of the above-mentioned preset third initial coordinates when acquiring each frame of the image to be detected, it is to calculate the displacement of the conveyor belt in the direction of the first coordinate axis of the image coordinate system compared to the first frame of the image when acquiring the frame of the image to be detected.

[0245] Therefore, the coordinate change value can be determined based on the infrared camera's acquisition frequency, the acquisition sequence number of the image to be inspected where the defect is located, and the conveyor belt's moving speed. Thus, when acquiring the Nth frame of the image to be inspected, the coordinate change value Y' of the preset third initial coordinates will be determined. N It can be represented as:

[0246] Y' N =N / FR*Vbelt*Scale

[0247] Where N is the image acquisition sequence number.

[0248] In other words, the displacement of the conveyor belt during the process of acquiring the first frame of the image to be detected to acquiring the N frames of the image to be detected, after being converted according to the above conversion relationship, can be used as the coordinate change value of the preset third initial coordinate of the first edge point when acquiring the Nth frame of the image to be detected. This coordinate change value is positive.

[0249] Based on this, when the first edge point is the edge point where the object to be detected first enters the acquisition range of the infrared camera, the coordinate change value of the aforementioned preset third initial coordinate can be calculated based on the image acquisition sequence number of the image to be detected where the defect is located, the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt.

[0250] Correspondingly, when the first edge point is the edge point where the object to be detected last enters the acquisition range of the infrared camera, the image to be detected with the above-mentioned preset third initial coordinates is determined to be the last image to be detected. Thus, when calculating the coordinate change value of the above-mentioned preset third initial coordinates when acquiring each frame of the image to be detected, it is to calculate the displacement of the conveyor belt in the first coordinate axis direction of the image coordinate system compared to the last frame of the image to be detected when acquiring that frame of the image to be detected.

[0251] In other words, the displacement of the conveyor belt during the process of acquiring the Nth frame of the image to be detected to acquiring the last frame of the image to be detected, after being converted according to the above conversion relationship, can be used as the coordinate change value of the preset third initial coordinate of the first edge point when acquiring the Nth frame of the image to be detected. This coordinate change value is negative.

[0252] Furthermore, based on the aforementioned preset third initial coordinates and the coordinate change value of the preset third initial coordinates, the coordinates of the first edge point in the direction of the first coordinate axis when the image to be detected where the defect is located is acquired can be calculated. Then, the first coordinate difference between the first coordinate of the defect and the first coordinate of the first edge point in the direction of the first coordinate axis when the image to be detected where the defect is located is calculated, and the absolute value of the first coordinate difference is converted to obtain the first distance between the defect and the first edge point in the direction of the first coordinate axis.

[0253] After determining the image coordinates of the defect and the preset fourth initial coordinates, in order to calculate the second distance between the defect and the second edge point in the direction of the second coordinate axis, it is also necessary to determine the acquisition range of the infrared camera acquiring the image of the defect and the positional relationship of the conveyor belt in the direction of the second coordinate axis of the world coordinate system. In this way, based on the acquisition range of the infrared camera acquiring the image of the defect and the positional relationship of the conveyor belt in the direction of the second coordinate axis of the world coordinate system, as well as the transformation relationship, the coordinates of the second coordinate of the second coordinate axis in the image coordinates on the second axis of the world coordinate system can be determined. Based on the above transformation relationship, the coordinates of the preset fourth initial coordinates on the second axis of the world coordinate system can be determined. Then, the second coordinate difference between the second coordinates on the second axis of the world coordinate system and the coordinates of the preset fourth initial coordinates on the second axis of the world coordinate system can be calculated, and the absolute value of the second coordinate difference can be used as the second distance between the defect and the second edge point in the direction of the second coordinate axis.

[0254] Optionally, in one specific implementation, step S3033 above, determining the first distance between the defect and the first edge point in the first coordinate axis direction based on the first coordinate of the first coordinate axis in the image coordinate system, the coordinate change value of the preset third initial coordinate when acquiring the image to be detected where the defect is located, the preset third initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, may include the following steps 21-23:

[0255] Step 21: Calculate the sum of the coordinate changes of the preset third initial coordinates and the preset third initial coordinates when acquiring the image of the defect to be detected;

[0256] Step 22: Calculate the absolute value of the difference between the sum and the first coordinate of the first axis in the image coordinate system;

[0257] Step 23: Based on the absolute value of the difference and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, calculate the first distance between the defect and the first edge point in the direction of the first coordinate axis.

[0258] In this specific implementation, for each frame of image, after determining the coordinate change value of the preset third initial coordinate when acquiring the frame of image, the sum of the preset third initial coordinate and the coordinate change value of the preset third initial coordinate when acquiring the frame of image to be detected can be calculated as the coordinate of the first edge point in the first coordinate axis in the image coordinate system when acquiring the frame of image to be detected.

[0259] Based on this, the sum of the aforementioned preset third initial coordinates and the coordinate change value of the preset third initial coordinates when acquiring the image to be detected where the defect is located can be calculated as the coordinates of the aforementioned first edge point in the first coordinate axis of the image coordinate system when acquiring the image to be detected where the defect is located.

[0260] Then, the absolute value of the difference between the above sum and the first coordinate of the first coordinate axis in the image coordinates of the defect can be calculated, and based on the above absolute value of the difference and the above transformation relationship, the first distance between the above defect and the above first edge point in the direction of the first coordinate axis can be determined.

[0261] For example, if the first edge point is the edge point where the object to be detected first enters the acquisition range of the infrared camera, the preset third initial coordinates of the first edge point can be represented as: Y raster Therefore, the above sum can be expressed as:

[0262] Y N =Y raster +N / FR*Vbelt*Scale

[0263] The absolute value of the above difference can be expressed as:

[0264] Y defect′ =|Y defect -(Y raster +N / FR*Vbelt*Scale)|

[0265] Among them, Y defect It is the first coordinate of the first coordinate axis in the image coordinate system.

[0266] Therefore, the first distance mentioned above can be expressed as:

[0267] Y defect_world′ =Y defect′ / Scale

[0268] Optionally, in one specific implementation, step S3034 above, determining the second distance between the defect and the second edge point in the second coordinate axis direction based on the positional relationship between the acquisition range and the conveyor belt in the second coordinate axis direction of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinate system, the preset second initial coordinates, and the transformation relationship, may include the following steps 31-32:

[0269] Step 31: Determine the specified coordinates of the defect based on the positional relationship between the acquisition range and the conveyor belt in the direction of the second coordinate axis of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinate system, the preset fourth initial coordinates, and the transformation relationship;

[0270] The specified coordinates include the coordinates of the defect on the second coordinate axis of the world coordinate system;

[0271] Step 32: Calculate the target coordinates of the second edge point and the difference between them and the specified coordinates, which will be used as the second distance between the defect and the second edge point in the direction of the second coordinate axis;

[0272] The target coordinates are the coordinates of the second edge point on the second coordinate axis of the world coordinate system.

[0273] In this specific implementation, after determining the image coordinates of the defect in the image to be detected, the second coordinate of the second coordinate axis in the image coordinates can be converted into the specified coordinates of the defect in the second coordinate axis in the world coordinate system by acquiring the acquisition range of the infrared camera of the image to be detected where the defect is located and the positional relationship of the conveyor belt in the direction of the second coordinate axis in the world coordinate system, as well as the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt.

[0274] Next, the target coordinates of the second edge point in the world coordinate system are determined. Then, the difference between the target coordinates and the specified coordinates can be calculated to obtain the second distance between the defect and the second edge point in the direction of the second coordinate axis.

[0275] Optionally, if the second reference point is the edge point with the smallest coordinate on the second coordinate axis of the object to be inspected, then when the conveyor belt is stationary, the minimum value of each second coordinate on the second coordinate axis of the object to be inspected can be measured as the target coordinate of the second edge point.

[0276] Optionally, if the second reference point is the edge point with the largest coordinate on the second coordinate axis of the object to be inspected, then when the conveyor belt is stationary, the maximum value of each second coordinate on the second coordinate axis of the object to be inspected can be measured as the target coordinate of the second edge point.

[0277] For example, as shown in Figure 2(d), point A is the detected defect, and point a is the second edge point of the object to be detected. The second coordinate of point A in the world coordinate system is 9, and the second coordinate of point a in the world coordinate system is 13. Thus, the second distance between point A and the second edge point a of the object to be detected in the direction of the second coordinate axis is 4.

[0278] In other words, based on the infrared camera's acquisition range and its value range on the second coordinate axis of the world coordinate system, as well as the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the second coordinate in the image coordinates of the defect can be determined, and its corresponding specified coordinate in the world coordinate system can be identified. Then, the target coordinates of the second edge point of the object to be detected can be determined. Furthermore, the difference between the specified coordinates of the defect in the world coordinate system and the target coordinates of the second edge point can be calculated. Therefore, this difference can be used as the second distance between the defect and the second edge point of the object to be detected along the second coordinate axis.

[0279] Optionally, if the second reference point is the edge point with the largest coordinate on the second coordinate axis in the object to be detected, then the above-mentioned second distance can be expressed as:

[0280] X defect_world′ =X defect_world -X edge_world

[0281] Among them, X defect_world X is the second specified coordinate of the defect on the second coordinate axis of the world coordinate system. edge_world The second target coordinates of the second edge point on the second coordinate axis of the world coordinate system.

[0282] For example, as shown in Figure 2(e), the acquisition area of ​​infrared camera E is region C. The X-coordinate of the acquisition area of ​​infrared camera E in the world coordinate system ranges from [10, 15]. The X-coordinate of the image acquired by infrared camera E in the image coordinate system ranges from [0, 20]. Furthermore, a defect H exists in the image, and the X-coordinate of defect H in the image is 5. Based on the range of X-coordinates corresponding to the acquisition area of ​​the infrared camera in the world coordinate system, the X-coordinate of defect H in the world coordinate system can be determined to be 10 + 5 * 5 / 20, that is, the X-coordinate of defect H in the world coordinate system is 11.25. The X-coordinate of the second edge point of the object to be detected on the second coordinate axis of the world coordinate system is 14. Therefore, the second distance of the second edge point of defect H is 2.75.

[0283] Corresponding to the defect location method provided in the above embodiments of this application, this application also provides a defect location device.

[0284] Figure 8 This is a schematic diagram of the structure of a defect location device provided in an embodiment of this application, as shown below. Figure 8 As shown, the defect location device may include the following modules:

[0285] Image acquisition module 810 is used to acquire an image of the object to be detected captured by an infrared camera; wherein the object to be detected is placed on a conveyor belt and the infrared camera is located above the conveyor belt;

[0286] The detection module 820 is used to perform defect detection on the image to be detected and determine the image coordinates of the detected defects in the image coordinate system;

[0287] The distance determination module 830 is used to determine, based on the image coordinates and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, a first distance between the defect and a first reference point on the object to be detected in the direction of the first coordinate axis, and a second distance between the defect and a second reference point on the object to be detected in the direction of the second coordinate axis.

[0288] The positioning module 840 is used to locate the defect in the object to be detected based on the first distance, the second distance, the first reference point, and the second reference point.

[0289] As can be seen above, when locating defects, the image coordinates of the detected defect in the image coordinate system can be determined first. Then, based on the image coordinates of the detected defect and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the distances between the defect and the reference point in the directions of the two coordinate axes can be determined. Using these distances, the defect can be located in the image to be inspected. This method, compared to manual location methods, can shorten the location cycle and improve defect location efficiency. Furthermore, using infrared images acquired by an infrared camera for defect detection can avoid poor detection results due to part reflections, improving the accuracy of defect detection and, consequently, the accuracy of defect location.

[0290] Optionally, in one specific implementation, the distance determination module 830 is specifically used for:

[0291] Based on the first coordinate of the first coordinate axis in the image coordinates, the coordinate change value of the preset first initial coordinate when the image to be detected where the defect is located is acquired, the preset first initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first reference point in the direction of the first coordinate axis is determined; wherein, the preset first initial coordinate is: the coordinate of the first reference point in the first coordinate axis when the first reference point is first detected in the image to be detected;

[0292] Based on the positional relationship between the acquisition range and the conveyor belt in the direction of the second coordinate axis in the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset second initial coordinates, and the transformation relationship, the second distance between the defect and the second edge point in the direction of the second coordinate axis is determined; wherein, the preset second initial coordinates are: the coordinates of the second reference point in the second coordinate axis.

[0293] Optionally, in one specific implementation, the positive direction of the first coordinate axis of the image coordinate system is the same as the conveying direction of the conveyor belt; the first reference point is the first edge point of the object to be detected that first or last enters the acquisition range of the infrared camera, and the second reference point is the second edge point of the object to be detected where the coordinate of the second coordinate axis is the largest or smallest; the distance determination module 830 includes:

[0294] The first distance determination submodule is used to determine the first distance between the defect and the first edge point in the direction of the first coordinate axis based on the first coordinate of the first coordinate axis in the image coordinate system, the coordinate change value of the preset third initial coordinate when the image to be detected where the defect is located is acquired, the preset third initial coordinate, and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt; wherein, the preset third initial coordinate is: the coordinate of the first edge point in the first coordinate axis when the first edge point is first detected in the image to be detected;

[0295] The second distance determination submodule is used to determine the second distance between the defect and the second edge point in the direction of the second coordinate axis based on the positional relationship between the acquisition range and the conveyor belt in the direction of the second coordinate axis in the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset fourth initial coordinates, and the transformation relationship; wherein, the preset fourth initial coordinates are: the coordinates of the second edge point in the second coordinate axis.

[0296] Optionally, in one specific implementation, the first distance determination submodule is specifically used for:

[0297] Calculate the sum of the preset third initial coordinates and the coordinate changes of the preset third initial coordinates when acquiring the image of the defect;

[0298] Calculate the absolute value of the difference between the sum and the first coordinate of the first coordinate axis in the image coordinates;

[0299] Based on the absolute value of the difference and the transformation relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt, the first distance between the defect and the first edge point in the direction of the first coordinate axis is calculated.

[0300] Optionally, in one specific implementation, the device further includes a coordinate change value determination module, which is specifically used for:

[0301] The coordinate change value is determined based on the acquisition frequency of the infrared camera, the acquisition sequence number of the image to be detected where the defect is located, and the moving speed of the conveyor belt.

[0302] The acquisition sequence number is used to characterize the order of the images to be detected where the defect is located when arranging the multiple frames of images to be detected in order of acquisition time from earliest to latest.

[0303] Optionally, in one specific implementation, the second distance determination submodule is specifically used for:

[0304] Based on the positional relationship between the acquisition range and the conveyor belt along the second coordinate axis of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinates, the preset fourth initial coordinates, and the transformation relationship, the specified coordinates of the defect are determined; wherein, the specified coordinates include the coordinates of the defect along the second coordinate axis of the world coordinate system;

[0305] Calculate the difference between the target coordinates of the second edge point and the specified coordinates, and use it as the second distance between the defect and the second edge point in the direction of the second coordinate axis; wherein the target coordinates are the coordinates of the second edge point in the world coordinate system on the second coordinate axis.

[0306] Optionally, in one specific implementation, gratings are provided on both sides of the conveyor belt, and the gratings are located at the edge of the acquisition range of the infrared camera; the multiple frames of images to be detected are acquired by the infrared camera during the process from when the object to be detected begins to block the grating to when it moves out of the grating area, and the preset third initial coordinates are: the coordinates of the position of the grating on the first coordinate axis of the image coordinate system.

[0307] Optionally, in one specific implementation, there are multiple infrared cameras arranged along the direction of the second coordinate axis, and the acquisition range of the multiple infrared cameras covers the conveyor belt area along the direction of the second coordinate axis.

[0308] This application also provides an electronic device, such as... Figure 9 As shown, it includes:

[0309] Memory 901 is used to store computer programs;

[0310] When the processor 902 executes the program stored in the memory 901, it implements the steps of any defect location method provided in the embodiments of this application.

[0311] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 902, communication interface, and memory 901 communicating with each other via the communication bus.

[0312] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0313] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0314] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0315] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0316] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described defect location methods.

[0317] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the defect location methods described above.

[0318] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0319] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0320] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0321] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A defect localization method, characterized by, The method comprises: acquiring a to-be-detected image of a to-be-detected object collected by an infrared camera; wherein the to-be-detected object is placed on a conveying belt, and the infrared camera is located above the conveying belt; performing defect detection on the to-be-detected image, and determining image coordinates of the detected defect in an image coordinate system; determining, according to the image coordinates and a conversion relationship between the image coordinate system and a world coordinate system corresponding to the conveying belt, a first distance between the defect and a first reference point on the to-be-detected object in a first coordinate axis direction of the world coordinate system, and a second distance between the defect and a second reference point on the to-be-detected object in a second coordinate axis direction of the world coordinate system; wherein specific positions of the first reference point and the second reference point on the to-be-detected object are known; locating the defect in the to-be-detected object based on the first distance, the second distance, the first reference point and the second reference point.

2. The method of claim 1, wherein, The determining, according to the image coordinates and a conversion relationship between the image coordinate system and a world coordinate system corresponding to the conveying belt, of a first distance between the defect and a first reference point on the to-be-detected object in a first coordinate axis direction of the world coordinate system, and a second distance between the defect and a second reference point on the to-be-detected object in a second coordinate axis direction of the world coordinate system, comprises: determining, according to a first coordinate of the first coordinate axis in the image coordinates, a coordinate change value of a preset first initial coordinate when the to-be-detected image in which the defect is located is collected, the preset first initial coordinate, and the conversion relationship between the image coordinate system and the world coordinate system corresponding to the conveying belt, the first distance between the defect and the first reference point in the first coordinate axis direction of the world coordinate system; wherein the preset first initial coordinate is a coordinate of the first reference point in the first coordinate axis when the first reference point is first detected in the to-be-detected image; determining, according to a position relationship between a collection range of the infrared camera and the conveying belt in the second coordinate axis direction of the world coordinate system, a second coordinate of the second coordinate axis in the image coordinates, a preset second initial coordinate, and the conversion relationship, the second distance between the defect and the second reference point in the second coordinate axis direction of the world coordinate system; wherein the preset second initial coordinate is a coordinate of the second reference point in the second coordinate axis.

3. The method of claim 2, wherein, A positive direction of the first coordinate axis of the image coordinate system is the same as a conveying direction of the conveying belt; the first reference point is a first edge point of the to-be-detected object that first enters or last exits the collection range of the infrared camera, and the second reference point is a second edge point of the to-be-detected object that has a maximum or minimum coordinate in the second coordinate axis. The method comprises the following steps: According to the first coordinate of the first coordinate axis in the image coordinate, the coordinate change value of the preset third initial coordinate when the image in which the defect is located is collected, the preset third initial coordinate, and the conversion relationship between the image coordinate system and the world coordinate system corresponding to the conveying belt, the first distance between the defect and the first edge point in the first coordinate axis direction of the world coordinate system is determined. According to the position relationship between the collection range of the infrared camera and the position of the conveying belt in the second coordinate axis direction of the world coordinate system, the second coordinate of the second coordinate axis in the image coordinate, the preset fourth initial coordinate, and the conversion relationship, the second distance between the defect and the second edge point in the second coordinate axis direction of the world coordinate system is determined.

4. The method of claim 3, wherein, The method comprises the following steps: The sum value of the preset third initial coordinate and the coordinate change value of the preset third initial coordinate when the image in which the defect is located is collected is calculated. The absolute value of the difference value between the sum value and the first coordinate of the first coordinate axis in the image coordinate is calculated. Based on the absolute value of the difference value and the conversion relationship between the image coordinate system and the world coordinate system corresponding to the conveying belt, the first distance between the defect and the first edge point in the first coordinate axis direction of the world coordinate system is calculated.

5. The method of claim 4, wherein, The determination method of the coordinate change value comprises: According to the collection frequency of the infrared camera, the collection serial number of the image in which the defect is located, and the moving speed of the conveying belt, the coordinate change value is determined. The collection serial number is used to represent the order of the image in which the defect is located when a plurality of images to be detected are arranged in the order from early to late according to the collection time.

6. The method of claim 3, wherein, The method comprises the following steps: According to a position relationship between a collection range of the infrared camera and the conveyor belt in a second coordinate axis direction of the world coordinate system, a second coordinate of the second coordinate axis in the image coordinate, a preset fourth initial coordinate, and the conversion relationship, a designated coordinate of the defect is determined; wherein the designated coordinate includes a coordinate of the defect in the second coordinate axis of the world coordinate system; A difference between a target coordinate of the second edge point and the designated coordinate is calculated as a second distance between the defect and the second edge point in the second coordinate axis direction of the world coordinate system; wherein the target coordinate is a coordinate of the second edge point in the second coordinate axis of the world coordinate system.

7. The method according to any one of claims 3-6, characterized in that, The conveyor belt is provided with gratings on both sides, and the gratings are located at edges of the collection range of the infrared camera; the multiple frames of to-be-detected images are collected by the infrared camera in a process in which the to-be-detected object starts to shield the gratings to the to-be-detected object moving out of the grating area; and the preset third initial coordinate is a coordinate of a position where the grating is located on the first coordinate axis of the image coordinate system.

8. The method of claim 1, wherein, The number of the infrared cameras is multiple, the multiple infrared cameras are arranged along the second coordinate axis direction, and collection ranges of the multiple infrared cameras cover the conveyor belt region in the second coordinate axis direction.

9. A defect localization apparatus, characterized by, The device comprises: An image acquisition module is configured to acquire a to-be-detected image of a to-be-detected object collected by an infrared camera; wherein the to-be-detected object is placed on a conveyor belt, and the infrared camera is located above the conveyor belt; A detection module is configured to perform defect detection on the to-be-detected image, and determine image coordinates of a detected defect in an image coordinate system; A distance determination module is configured to determine, according to the image coordinates and a conversion relationship between the image coordinate system and a world coordinate system corresponding to the conveyor belt, a first distance between the defect and a first reference point on the to-be-detected object in a first coordinate axis direction of the world coordinate system, and a second distance between the defect and a second reference point on the to-be-detected object in a second coordinate axis direction of the world coordinate system; wherein specific positions of the first reference point and the second reference point on the to-be-detected object are known; A positioning module is configured to position the defect in the to-be-detected object based on the first distance, the second distance, the first reference point, and the second reference point.

10. The apparatus of claim 9, wherein, The distance determination module is specifically configured to: determine the first distance between the defect and the first reference point in the first coordinate axis direction of the world coordinate system according to a first coordinate of the first coordinate axis in the image coordinates, a coordinate change value of a preset first initial coordinate when the to-be-detected image in which the defect is located is collected, the preset first initial coordinate, and the conversion relationship between the image coordinate system and the world coordinate system corresponding to the conveyor belt; wherein the preset first initial coordinate is a coordinate of the first reference point in the first coordinate axis when the first reference point is first detected in the to-be-detected image. determine a second distance between the defect and the second reference point in a second coordinate axis direction of the world coordinate system according to a position relationship between a capture range of the infrared camera and the conveyor belt in the second coordinate axis direction of the world coordinate system, a second coordinate of the second coordinate axis in the image coordinate, a preset second initial coordinate, and the conversion relationship, wherein the preset second initial coordinate is a coordinate of the second reference point in the second coordinate axis; and / or, a positive direction of a first coordinate axis of the image coordinate system is the same as a conveying direction of the conveyor belt, the first reference point is a first edge point of the to-be-detected object that enters a capture range of the infrared camera first or last, and the second reference point is a second edge point of the to-be-detected object that has a maximum or minimum coordinate in the second coordinate axis; the distance determination module comprises: a first distance determination submodule, configured to determine a first distance between the defect and the first edge point in a first coordinate axis direction of the world coordinate system according to a first coordinate of the first coordinate axis in the image coordinate, a coordinate change value of a preset third initial coordinate when the to-be-detected image in which the defect is located is captured, the preset third initial coordinate, and a conversion relationship about the image coordinate system and a world coordinate system corresponding to the conveyor belt, wherein the preset third initial coordinate is a coordinate of the first edge point in the first coordinate axis when the first edge point is first detected in the to-be-detected image; a second distance determination submodule, configured to determine a second distance between the defect and the second edge point in a second coordinate axis direction of the world coordinate system according to a position relationship between a capture range of the infrared camera and the conveyor belt in the second coordinate axis direction of the world coordinate system, a second coordinate of the second coordinate axis in the image coordinate, a preset fourth initial coordinate, and the conversion relationship, wherein the preset fourth initial coordinate is a coordinate of the second edge point in the second coordinate axis; and / or, the first distance determination submodule is specifically configured to: calculate a sum value of the preset third initial coordinate and the coordinate change value of the preset third initial coordinate when the to-be-detected image in which the defect is located is captured; calculate a difference absolute value of the sum value and the first coordinate of the first coordinate axis in the image coordinate; based on the difference absolute value and the conversion relationship about the image coordinate system and the world coordinate system corresponding to the conveyor belt, calculate the first distance between the defect and the first edge point in the first coordinate axis direction of the world coordinate system; and / or, the apparatus further comprises a coordinate change value determination module, which is specifically configured to: determine the coordinate change value according to a capture frequency of the infrared camera, a capture serial number of the to-be-detected image in which the defect is located, and a moving speed of the conveyor belt, wherein the capture serial number is used to represent an order of the to-be-detected image in which the defect is located when a plurality of to-be-detected images are arranged in an order from early to late according to capture time; and / or, the second distance determination submodule is specifically configured to: According to a relationship between a collection range of the infrared camera and a position of the conveying belt in a second coordinate axis direction of the world coordinate system, a second coordinate of the second coordinate axis in the image coordinate, a preset fourth initial coordinate, and the conversion relationship, a designated coordinate of the defect is determined; wherein the designated coordinate comprises a coordinate of the defect in the second coordinate axis of the world coordinate system; A difference between the target coordinate of the second edge point and the designated coordinate is calculated as a second distance between the defect and the second edge point in the second coordinate axis direction of the world coordinate system; wherein the target coordinate is a coordinate of the second edge point in the second coordinate axis of the world coordinate system; And / or, The conveying belt is provided with gratings on both sides, and the gratings are located at edges of the collection range of the infrared camera; the multiple frames of images to be detected are collected by the infrared camera in a process in which the object to be detected starts to shield the gratings to move out of the grating area; and the preset third initial coordinate is a coordinate of a position of the grating on the first coordinate axis of the image coordinate system. And / or, The number of the infrared cameras is multiple, the multiple infrared cameras are arranged along the second coordinate axis direction, and collection ranges of the multiple infrared cameras cover the conveying belt area in the second coordinate axis direction.

11. An electronic device, comprising: Comprise: A memory for storing a computer program; A processor for executing the program stored on the memory to implement the method of any one of claims 1-8.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-8.

Images