Key frame detection method, device and equipment, and PCB (Printed Circuit Board) defect detection method, device and equipment

Abstract

The invention belongs to the field of computer vision, and particularly relates to a key frame detection method, device and equipment and a PCB defect detection method, device and equipment, and the method comprises the steps: obtaining a candidate image shot by a camera device when it is detected that a target detection object exists in an image shot by the camera device; pixel points of a preset position of the target detection object in the candidate image are obtained, whether the candidate image meets a first preset condition or not is determined according to the pixel points, and the first preset condition is used for indicating that the pixel points are located in a target area in the candidate image; if the candidate image satisfies the first preset condition, determining whether the candidate image satisfies a second preset condition; and if the candidate image satisfies a second preset condition, determining that the candidate image is a key frame of the target detection object. According to the method, the detection efficiency and reliability of the key frame are effectively improved, meanwhile, the limitation of relying on a hardware sensor is avoided, additional hardware installation is not needed, and the cost is reduced.

Description

Technical Field

[0001] The present application belongs to the field of computer vision, and specifically relates to a key frame detection, PCB board defect detection method, device and equipment. Background Art

[0002] When inspecting PCB boards, key frames need to be extracted first before the subsequent inspection process can be carried out. The current method of extracting key frames depends on the sensor settings. The laser sensor is responsible for detecting specific marks or reflections on the PCB board, communicating with the camera system, and triggering the camera to take pictures of the PCB board to obtain key frames.

[0003] However, the problem with the above method is that the accuracy and stability of the sensor have a direct impact on the detection results. In addition, installing and calibrating the sensor requires expertise and time, which not only increases the maintenance cost and complexity of the equipment, but may also cause inaccurate key frame extraction due to environmental changes or errors in the sensor itself, thus affecting the overall efficiency and accuracy of quality inspection. Summary of the invention

[0004] In response to the above problems, the present application provides a key frame detection, PCB board defect detection method, device and equipment.

[0005] In a first aspect, the present application provides a key frame detection method, the method comprising:

[0006] When it is detected that a target detection object exists in an image captured by a camera device, a candidate image captured by the camera device is acquired, wherein the target detection object moves through a conveyor belt, the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate an image captured by the camera device at a preset frequency after the target detection object is detected;

[0007] Acquire a pixel point at a preset position of the target detection object in the candidate image, and confirm whether the candidate image satisfies a first preset condition based on the pixel point, where the first preset condition is used to indicate that the pixel point is in a target area in the candidate image;

[0008] If the first preset condition is met, confirming whether the candidate image meets a second preset condition, where the second preset condition is used to indicate that the candidate image is the first image corresponding to the target detection object that meets the first preset condition;

[0009] If the candidate image meets the second preset condition, the candidate image is determined to be a key frame of the target detection object.

[0010] In a possible implementation, the determining, according to the pixel points, whether the candidate image satisfies a first preset condition includes:

[0011] Acquire the region where the pixel point is located in the candidate image, wherein the candidate image is an unprocessed image captured by the camera device, and divide the candidate image into two left and right sub-regions by setting a virtual longitudinal center line in the candidate image, and the target detection object moves in the shooting range along a preset movement direction perpendicular to the center line through a conveyor belt;

[0012] Determine whether the pixel point is in a target sub-region of the candidate image, where the target sub-region is used to indicate a first sub-region passed by the target detection object along a preset motion direction;

[0013] If yes, the candidate image does not satisfy the first preset condition;

[0014] If not, obtaining the distance between the pixel point and the center line;

[0015] If the distance between the pixel point and the center line is less than or equal to a first preset threshold, it indicates that the candidate image meets the first preset condition.

[0016] In a possible implementation manner, the acquiring a pixel point at a preset position of the target detection object in the candidate image includes:

[0017] Preprocessing the candidate image to obtain a first standard image;

[0018] Inputting the first standard image into a pre-trained rotation detection model to obtain an output result, wherein the output result includes pixel points at a preset position of the target detection object in the first standard image;

[0019] The output result is restored to obtain a target output result corresponding to the candidate image, wherein the target output result includes pixel points at a preset position of the target detection object in the candidate image.

[0020] In a possible implementation, the target output result further includes a size parameter and a rotation angle of the target detection object in the candidate image, and after determining that the candidate image is a key frame of the target detection object, the method further includes:

[0021] According to the target output result, confirm whether the rotation angle of the target detection object reaches a second preset threshold, wherein the rotation angle is used to indicate the tilt angle of the target detection object in the candidate image;

[0022] If yes, correcting the target detection object in the candidate image so that the rotation angle of the target detection object in the candidate image is less than the second preset threshold; if no, no correction processing is performed;

[0023] According to the size parameters of the target detection object, the target detection object in the candidate image is marked, and the candidate image is cropped to obtain image information containing only the target detection object.

[0024] In a possible implementation manner, if the distance between the pixel point and the center line is greater than the first preset threshold, the method further includes:

[0025] Obtaining a detected target key frame corresponding to the target detection object;

[0026] Determining whether to switch the detection target according to the target key frame and the candidate image, so as to perform key frame detection on the next target detection object on the conveyor belt;

[0027] If the detection target is switched, the detection of the current target detection object is terminated, and key frame detection is performed on the next target detection object.

[0028] In a possible implementation, the step of confirming whether to switch the detection target according to the target key frame and the candidate image includes:

[0029] Mapping the position of a target pixel point of the target detection object in the target key frame into the candidate image, wherein a preset position of the target pixel point on the target detection object is the same as a preset position of the pixel point on the target detection object;

[0030] Obtaining a target distance between the target pixel and a pixel in the candidate image;

[0031] If the target distance is greater than the first preset threshold, the detection target is switched.

[0032] In a second aspect, the present application provides a PCB board defect detection method, the method comprising:

[0033] Acquire a key frame of a target PCB board using the method described in any one of the first aspects;

[0034] According to the key frame of the target PCB board, the target PCB board is subjected to automatic optical inspection to confirm whether the target PCB board has surface defects.

[0035] In a third aspect, the present application provides a key frame detection device, the device comprising:

[0036] A first acquisition module, configured to acquire a candidate image captured by the camera device when a target detection object is detected in an image captured by the camera device, wherein the target detection object moves through a conveyor belt, the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate an image captured by the camera device at a preset frequency after the target detection object is detected;

[0037] A second acquisition module, used to acquire pixel points at a preset position of the target detection object in the candidate image;

[0038] A first processing module, used for confirming whether the candidate image satisfies a first preset condition according to the pixel point, where the first preset condition is used for indicating that the pixel point is in a target area in the candidate image;

[0039] A second processing module, configured to confirm whether the candidate image satisfies a second preset condition if the first preset condition is satisfied, wherein the second preset condition is used to indicate that the candidate image is the first image corresponding to the target detection object that satisfies the first preset condition;

[0040] The second processing module is further configured to determine that the candidate image is a key frame of the target detection object if the candidate image meets a second preset condition.

[0041] In a fourth aspect, the present application provides a PCB board defect detection device, the device comprising:

[0042] An acquisition module, used for acquiring a key frame of a target PCB board by adopting the method described in any one of the first aspects;

[0043] The detection module is used to perform automatic optical detection on the target PCB board according to the key frame of the target PCB board to confirm whether the target PCB board has surface defects.

[0044] In a fifth aspect, the present application provides an electronic device, comprising: at least one processor and a memory; wherein:

[0045] The memory stores computer-executable instructions;

[0046] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method as described in any one of the first aspect and the second aspect.

[0047] In a sixth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, can implement the steps of the method described in any one of the first aspect and the second aspect.

[0048] The key frame detection and PCB board defect detection method, device and equipment provided by the present application, when it is detected that there is a target detection object in the image taken by the camera device, a candidate image taken by the camera device is obtained, wherein the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate the image taken by the camera device at a preset frequency after the target detection object is detected; the pixel points of the preset position of the target detection object in the candidate image are obtained, and it is confirmed whether the pixel points are in the target area in the candidate image; if so, it is confirmed whether the candidate image is the first image corresponding to the target detection object that meets the first preset condition; if so, it is determined that the candidate image is the key frame of the target detection object. This method realizes automatic capture of key frames for the detection object by identifying and analyzing images with a certain shooting frequency, avoids the limitation of relying on hardware sensors, does not require additional hardware installation, and reduces costs. At the same time, by confirming the relative position of the pixel points of the detection object, identifying and extracting key frames, the detection efficiency and reliability of key frames are effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] Figure 1 Schematic diagram of application scenarios provided for this application;

[0051] Figure 2 The process of the key frame detection method provided in the embodiment of the present application Figure 1 ;

[0052] Figure 3 The process of the key frame detection method provided in the embodiment of the present application Figure 2 ;

[0053] Figure 4 A flow chart of a PCB board defect detection method provided in an embodiment of the present application;

[0054] Figure 5 A diagram of a key frame detection device provided by an embodiment of the present invention;

[0055] Figure 6 A diagram of a PCB board defect detection device provided by an embodiment of the present invention;

[0056] Figure 7 A hardware schematic diagram of an electronic device provided by an embodiment of the present invention.

[0057] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein, for example.

[0060] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0061] In modern PCB board AOI quality inspection systems, key frame capture usually involves the cooperation of precision laser sensors and high-speed cameras. When the laser sensor detects the preset trigger signal, the camera will immediately capture high-definition images of the PCB board, which are essential for subsequent quality inspection. However, this solution that relies on hardware sensors has some limitations. First, it requires additional hardware installation on the production line, which may increase the company's operating costs. Secondly, over time, sensors may wear out or fail and need to be replaced regularly, which not only increases maintenance costs but also may cause downtime of the production line.

[0062] In addition, with the development of automation technology, PCB boards are produced on high-speed uniformly moving conveyor belts, which brings new challenges to the extraction of key frames. Traditional sensor-based methods may not be able to adapt to this high-speed dynamic environment, especially when there may be adhesion or other abnormal conditions between PCB boards. These conditions may cause the sensor to fail to accurately trigger the camera's photo action, thereby affecting the integrity and accuracy of quality inspection. Therefore, for PCB board AOI quality inspection on automated production lines, it is necessary to develop more intelligent and adaptable key frame extraction technology to improve production efficiency and quality inspection reliability.

[0063] In view of the above problems, the present application provides a key frame detection method, when it is detected that there is a target detection object in the image taken by the camera device, a candidate image taken by the camera device is obtained, wherein the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate the image taken by the camera device at a preset frequency after the target detection object is detected; the pixel points of the preset position of the target detection object in the candidate image are obtained, and it is confirmed whether the pixel points are in the target area in the candidate image; if so, it is confirmed whether the candidate image is the first image corresponding to the target detection object that meets the first preset condition; if so, it is determined that the candidate image is the key frame of the target detection object. This method realizes automatic capture of key frames for the detection object by identifying and analyzing images with a certain shooting frequency, avoids the limitation of relying on hardware sensors, does not require additional hardware installation, and reduces costs. At the same time, by confirming the relative position of the pixel points of the detection object, identifying and extracting key frames, the detection efficiency and reliability of key frames are effectively improved.

[0064] Furthermore, the present application also provides a PCB board defect detection method, by adopting the above-mentioned key frame detection method, a key frame of a target PCB board is obtained. According to the key frame, an automatic optical inspection (AOI) is performed on the target PCB board to confirm whether the target PCB board has surface defects. The method improves the production efficiency and reliability of PCB board surface detection, while reducing the production cost.

[0065] Figure 1 This is a schematic diagram of the application scenario provided by this application. Figure 1As shown, the figure shows an image continuously shot by a camera at a fixed frequency. In the viewing angle of the camera, the PCB board moves from left to right on the conveyor belt. In the continuously shot images, it is shown that the PCB board enters the shooting range, approaches the center of the picture, passes the center of the picture, and then moves away from the center of the picture. When detecting the key frame of the PCB board, according to the pixel point at the preset position on the PCB board in the image, it is confirmed whether the pixel point is in the target area of ​​the current image. Exemplarily, the preset position in this scene can be set to the center point of the PCB board, and the target area can be represented as the range within the preset distance from the center pixel point of the PCB board to the right side of the center of the current picture. Specifically, when the center pixel point of the PCB board is located on the left side of the center of the current picture, the image (Figure a, Figure b) is not considered to be a key frame of the PCB board, and the subsequent images are continuously detected. Until it is detected that the center pixel point of the PCB board is in the right area of ​​the center of the picture (including the center line position), it is determined whether the distance from the center pixel point of the PCB board to the center of the picture is less than or equal to the preset distance. If so, the current image is a key frame (Figure c), and the coordinates of the center pixel point of the PCB board in the key frame in the image are recorded. After detecting the key frame, you can continue to detect whether the distance between the central pixel point of the PCB board in the subsequent image and the coordinates recorded in the key frame is greater than the preset distance to ensure that the PCB board has moved away from the center of the image. If the distance between the central pixel point of the PCB board and the coordinates recorded in the key frame is greater than the preset distance (Figure d), the detection target is switched to detect the next PCB board in the current image and subsequent images.

[0066] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments may exist independently or in combination with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0067] Figure 2 The process of the key frame detection method provided in the embodiment of the present application Figure 1 The method comprises:

[0068] S201. When it is detected that a target detection object exists in an image taken by a camera device, a candidate image taken by the camera device is acquired.

[0069] The target detection object moves via a conveyor belt, the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate an image captured by the camera device at a preset frequency after the target detection object is detected.

[0070] In this step, the detection of the target object is a prerequisite for determining the key frame. Only when the target object appears in the image can it be further determined whether it is a key frame. Exemplarily, an image recognition algorithm can be used to identify the target detection object, such as a PCB board that needs to be inspected. When the algorithm identifies the target detection object, the image sequence taken by the camera device within a period of time after the target detection object is detected is recorded. Exemplarily, a deep learning algorithm, such as a convolutional neural network (CNN), can be used for recognition. Through training, the model can learn the characteristics of the target detection object, thereby accurately identifying the target detection object in a complex image background.

[0071] Based on the image sequence captured by the camera device within a period of time after the target detection object is detected, each candidate image in the image sequence is detected and analyzed to determine the key frame of the target detection object. Taking the PCB board detection scenario as an example, the PCB board is usually transported by a uniformly moving conveyor belt, and the camera device collects images at a certain frequency at a fixed position above, and the size of the image collected each time remains unchanged. Since the PCB boards on the conveyor belt are arranged in sequence, when the camera device shoots regularly, each PCB board will be captured with a similar number of images. Therefore, when the target PCB board is detected to appear in the image, it means that the key frame of the target PCB board can be determined, and the key frame is determined by analyzing the subsequent image sequences one after another.

[0072] S202: Acquire pixel points at a preset position of the target detection object in the candidate image.

[0073] In this step, an image recognition algorithm can be used to determine the position of the target detection object in the image for the acquired candidate image. The position of the target detection object in the image can be determined by selecting a pixel point at a preset position on the target detection object for identification as a reference for subsequent analysis, for example, selecting the central pixel point of the target detection object in the image. By determining the position of the target detection object, it is further analyzed whether the image is a key frame.

[0074] Exemplarily, the acquiring the pixel point at the preset position of the target detection object in the candidate image includes:

[0075] Preprocessing the candidate image to obtain a first standard image;

[0076] Inputting the first standard image into a pre-trained rotation detection model to obtain an output result, wherein the output result includes pixel points of the target detection object in the first standard image;

[0077] The output result is restored to obtain a target output result corresponding to the candidate image, wherein the target output result includes pixel points of the target detection object in the candidate image.

[0078] It should be noted that the pixel points for determining the preset position of the target detection object can be obtained through the rotation detection model analysis. The rotation detection model represents a deep learning model, such as a convolutional neural network (CNN), and can also use a model with a YOLOv8_obb network structure.

[0079] Preprocessing the candidate image means converting the candidate image to be analyzed into the format type required by the model. Exemplarily, the candidate image can first be affine transformed according to the input image size requirements and data type requirements of the model to transform it into the image size required by the model. After normalization, batch processing, etc., the image is further converted into the data type required by the model. The standardized image is input to the model to obtain the output result. The output result is obtained by the model based on the standardized image analysis, so it may involve the subsequent restoration of the output result so that the output result is represented as the output result of the original candidate image. That is, the model outputs the pixel points of the target detection object of the standardized image, then it is necessary to transform the pixel points according to the original candidate image to obtain the pixel points of the target detection object of the original candidate image.

[0080] S203: Determine whether the candidate image meets a first preset condition according to the pixel points.

[0081] The first preset condition is used to indicate that the pixel point is in a target area in the candidate image.

[0082] In this step, the purpose of setting the first preset condition is to ensure that the captured image contains enough information about the target object, for example, the target object should occupy a certain proportion of the image area. When the center of the target detection object is located in the center or important area of ​​the image, it can be guaranteed that the captured image is the most representative, which helps to avoid image distortion or key information being blocked, thereby ensuring the accuracy of quality inspection. In addition, setting the standard for the area of ​​the target detection object in the picture can also ensure that the central pixel point of the target detection object is located in a stable area of ​​the image, so that even if the conveyor belt is moving, the image will not produce excessive jitter or offset, so that more stable and reliable image data can be obtained.

[0083] S204: If not, determine that the candidate image is not a key frame of the target detection object.

[0084] It should be noted that when it is determined that the candidate image does not meet the first preset condition, it can be further determined whether the target detection object has been determined to have a key frame. If the key frame has not been determined, it is necessary to continue to determine the key frame for the next candidate image.

[0085] S205: If yes, determine whether the candidate image meets a second preset condition.

[0086] The second preset condition is used to indicate that the candidate image is the first image corresponding to the target detection object that satisfies the first preset condition;

[0087] S206: If yes, determine that the candidate image is a key frame of the target detection object.

[0088] S207: If not, the process is the same as step S204.

[0089] It should be noted that when a candidate image meets the first preset condition, it means that the image can be used as a key frame for subsequent analysis. However, in actual scenarios, due to differences in acquisition frequency and other factors, there may be multiple continuously acquired images that meet the first preset condition, and usually only one key frame needs to be extracted for the same object, and there is no need to repeatedly select other key frames. At the same time, to determine the key frame, it is necessary to ensure that a representative image is selected, that is, the most important moment in the object's motion, usually when the target detection object just appears in the target area. Therefore, when the candidate image meets the first preset condition, it is necessary to further confirm whether the candidate image is the first image that meets the first preset condition.

[0090] After confirming the key frame of the target detection object, in general object detection scenarios, it is necessary to further perform quality inspection analysis and judgment on the key frame. It should be noted that in order to increase the accuracy and convenience of subsequent analysis, some processing may be required for the key frame, because in the actual image acquisition scene, the camera device will capture other redundant information such as the background while capturing the target detection object. In addition, the target detection object may not be placed regularly during its movement on the transmission belt, which may cause skewness and the like. At this time, the key frame needs to be corrected. Therefore, when the pixel points of the target detection object in the image are confirmed by the rotation detection model mentioned above, the size parameters of the target detection object in the image, such as width and height, and the rotation angle, can also be output at the same time, so that when the image is detected as a key frame, the image can be processed and corrected.

[0091] Exemplarily, the target output result also includes the size parameter and the rotation angle of the target detection object in the candidate image, and after determining that the candidate image is a key frame of the target detection object, the following further includes:

[0092] According to the target output result, confirm whether the rotation angle of the target detection object reaches a second preset threshold, wherein the rotation angle is used to indicate the tilt angle of the target detection object in the candidate image;

[0093] If yes, correcting the target detection object in the candidate image so that the rotation angle of the target detection object in the candidate image is less than the second preset threshold; if no, no correction processing is performed;

[0094] According to the size parameters of the target detection object, the target detection object in the candidate image is marked, and the candidate image is cropped to obtain image information containing only the target detection object.

[0095] It should be noted that the posture of the target detection object in the image is determined, that is, the angle of the object frame is calculated to determine whether correction is needed. If the rotation angle of the detected object exceeds the set threshold, the image needs to be rotated and corrected to ensure that the object is in a horizontal state. Exemplarily, an image rotation algorithm can be used to correct objects in the image, such as by rotating the center point of the image, and then scaling and cropping to eliminate the tilt of the object. According to the size parameters and rotation angle of the target object, after the target detection object frame is determined by marking, it can be cropped to extract the area containing only the target detection object. In the above manner, redundant information in the key frame can be removed, thereby reducing the amount of calculation for subsequent analysis and improving the accuracy.

[0096] The key frame detection method provided in this embodiment, when it is detected that there is a target detection object in the image taken by the camera device, obtains a candidate image taken by the camera device, wherein the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate the image taken by the camera device at a preset frequency after the target detection object is detected; obtains the pixel point of the preset position of the target detection object in the candidate image, and confirms whether the pixel point is in the target area in the candidate image; if so, confirms whether the candidate image is the first image corresponding to the target detection object that meets the first preset condition; if so, determines that the candidate image is the key frame of the target detection object. This method realizes automatic capture of key frames for the detection object by identifying and analyzing images with a certain shooting frequency, avoids the limitation of relying on hardware sensors, does not require additional hardware installation, and reduces costs. At the same time, by confirming the relative position of the pixel points of the detection object, identifying and extracting key frames, the detection efficiency and reliability of key frames are effectively improved.

[0097] Figure 3 The process of the key frame detection method provided in the embodiment of the present application Figure 2 .like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, the process of confirming whether the candidate image meets the first preset condition is described in detail. The method includes:

[0098] S301: Obtain the area where the pixel point is located in the candidate image.

[0099] Among them, the candidate image is an unprocessed image taken by the camera device. By setting a virtual longitudinal center line in the candidate image, the candidate image is divided into two left and right sub-areas. The target detection object moves in the shooting range through a conveyor belt along a preset movement direction perpendicular to the center line.

[0100] In this step, the specific position of the pixel in the image is determined to determine whether it is within the target sub-area. Among them, the virtual center line can be hypothetical. If it is virtual, it is not a feature of the image itself, so it needs to be created during the image processing process. The specific implementation method can be to identify the position of the virtual center line through an image processing algorithm, such as edge detection or threshold segmentation, and then divide the image into two left and right sub-areas according to the center line. Exemplarily, the preset direction of the target detection object moving through the conveyor belt can be from left to right or from right to left relative to the image, or other movement directions that meet the conditions can be set according to the actual scene.

[0101] S302: Determine whether the pixel point is in the target sub-region of the candidate image.

[0102] The target sub-region is used to indicate the first sub-region passed by the target detection object along a preset movement direction.

[0103] S303: If yes, the candidate image does not meet the first preset condition.

[0104] It should be noted that when the pixel point of the target detection object is in the first sub-area, it means that the target detection object has just entered the shooting range. At this time, due to the differences in the detection objects, there may be other problems such as the target detection object not fully entering the image range, the target detection object is unstable and presents a non-static state due to inertia, etc., and the detection position is not optimal. When the target detection object reaches the second sub-area, that is, the other half of the image, the target detection object is usually in a stable state and has fully entered the image range. At this time, the key favorable position can be found.

[0105] S304: If not, obtain the distance between the pixel point and the center line.

[0106] S305: Determine whether the distance between the pixel point and the center line is less than or equal to a first preset threshold.

[0107] S306: If yes, the candidate image meets the first preset condition.

[0108] S307: If not, the candidate image does not meet the first preset condition.

[0109] It should be noted that the identification of whether the target detection object is in the target area can be achieved by setting a threshold. The first preset threshold can be determined according to the actual scene. The appropriate threshold can be determined by confirming the distance the conveyor belt moves per unit time, the width of the target detection object, the width of the image, and performing a large amount of data testing. Obtaining a suitable first preset threshold can more accurately collect key frames. For different hardware detection scenarios, the corresponding thresholds may be different.

[0110] Exemplarily, if the distance between the pixel point and the center line is greater than the first preset threshold, the method further includes:

[0111] Obtaining a detected target key frame corresponding to the target detection object;

[0112] Determining whether to switch the detection target according to the target key frame and the candidate image, so as to perform key frame detection on the next target detection object on the conveyor belt;

[0113] If the detection target is switched, the detection of the current target detection object is terminated, and key frame detection is performed on the next target detection object.

[0114] It should be noted that in a continuous detection environment, it is necessary to clarify the triggering conditions for the detection of the next target detection object. Such conditions not only need to indicate that the key frame of the current target detection object has been extracted, but also need to ensure that the position of the next target detection object in the image is conducive to key frame detection. Therefore, it is necessary to analyze the image related to the current target detection object, and when it is confirmed that the position of the current target detection object has met the switching detection target, switch to identify the next target detection object and perform key frame analysis.

[0115] Exemplarily, the determining whether to switch the detection target according to the target key frame and the candidate image includes:

[0116] Mapping the position of a target pixel point of the target detection object in the target key frame into the candidate image, wherein a preset position of the target pixel point on the target detection object is the same as a preset position of the pixel point on the target detection object;

[0117] Obtaining a target distance between the target pixel and a pixel in the candidate image;

[0118] If the target distance is greater than the first preset threshold, the detection target is switched.

[0119] It should be noted that by mapping the center point of the PCB board in the key frame to the current candidate image, the distance between the mapped point and the actual center point of the PCB board in the candidate image can be compared. If this distance exceeds a preset threshold, it means that the position of the PCB board has changed significantly and the detection target needs to be switched. Only when the position change of the PCB board exceeds a certain threshold, it is considered necessary to update the key frame, which can reduce unnecessary detection and improve the accuracy of detection.

[0120] The key frame detection method provided in this embodiment determines whether an image is a key frame by determining the distance between the position of the pixel point of the detected object in the image and the center of the image. The automated image analysis and processing improves the efficiency and accuracy of key frame detection and has strong adaptability, and can be effectively applied to different actual detection environments.

[0121] Figure 4 Flow chart of the PCB board defect detection method provided in the embodiment of the present application. Figure 4 As shown, in this embodiment Figure 3 Based on the embodiment, the scenario of applying the above key frame detection method is described in detail. The method includes:

[0122] S401, obtaining a key frame of a target PCB board.

[0123] The key frame is obtained by adopting any one of the steps in the above-mentioned embodiments.

[0124] S402: Performing automatic optical inspection on the target PCB board according to the key frame of the target PCB board to confirm whether the target PCB board has surface defects.

[0125] AOI systems use optical imaging technology to detect defects on the surface of PCB boards, such as cracks, holes, contaminants, etc. In the key frame, due to the obvious change in the position or posture of the PCB board, any surface defects may appear clearer, thereby improving the accuracy of detection. The AOI system inputs the captured image into the computer processing system, and analyzes the image through image processing algorithms to identify and classify any detected defects. The system can compare the defects in the key frame with the standard image or previously stored defect data to determine the nature and size of the defect. The inspection results will be recorded and used for subsequent analysis and processing. If a surface defect is detected, the system may trigger an alarm, stop the conveyor belt, or send the PCB board to a dedicated repair station.

[0126] In the application scenario of this embodiment, the detection efficiency is improved through the aforementioned key frame detection method: only the key frames are detected, which reduces unnecessary image processing, thereby saving time and resources. The detection accuracy is improved: based on the aforementioned key frame detection method, a clear and highly recognizable image can be extracted from the PCB board, and the image quality is improved, which can further improve the accuracy of subsequent surface defect detection. The detection cost is reduced: no other hardware support such as sensors is required, and only automated image analysis is used to achieve process-based key frame detection and actual detection scenarios for quality inspection.

[0127] The PCB board defect detection method provided in this embodiment determines key frames through automated image analysis and performs quality inspection related processes based on the key frames, thereby improving detection efficiency and accuracy while reducing costs.

[0128] Figure 5 A key frame detection device diagram provided by an embodiment of the present invention, such as Figure 5 As shown, the detection device 50 includes a first acquisition module 501 , a second acquisition module 502 , a first processing module 503 and a second processing module 504 .

[0129] A first acquisition module 501 is used to acquire a candidate image captured by the camera device when it is detected that there is a target detection object in the image captured by the camera device, wherein the target detection object moves through a conveyor belt, the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate an image captured by the camera device at a preset frequency after the target detection object is detected;

[0130] A second acquisition module 502 is used to acquire a pixel point at a preset position of the target detection object in the candidate image;

[0131] A first processing module 503, used to confirm whether the candidate image meets a first preset condition according to the pixel point, where the first preset condition is used to indicate that the pixel point is in a target area in the candidate image;

[0132] A second processing module 504 is used to confirm whether the candidate image satisfies a second preset condition if the first preset condition is satisfied, wherein the second preset condition is used to indicate that the candidate image is the first image corresponding to the target detection object that satisfies the first preset condition;

[0133] The second processing module 504 is further configured to determine that the candidate image is a key frame of the target detection object if the candidate image meets a second preset condition.

[0134] Figure 6 A PCB board defect detection device diagram provided by an embodiment of the present invention, such as Figure 6As shown, the detection device 60 includes an acquisition module 601 and a detection module 602 .

[0135] An acquisition module 601 is used to acquire a key frame of a target PCB board by using a method as described in any one of the above embodiments;

[0136] The detection module 602 is used to perform automatic optical detection on the target PCB board according to the key frame of the target PCB board to confirm whether the target PCB board has surface defects.

[0137] Figure 7 Schematic diagram of the hardware of the electronic device provided by the embodiment of the present invention. Figure 6 As shown, the electronic device 70 provided in this embodiment includes: at least one processor 701 and a memory 702. The device 70 also includes a communication component 703. The processor 701, the memory 702 and the communication component 703 are connected via a bus 704.

[0138] In a specific implementation process, at least one processor 701 executes the computer-executable instructions stored in the memory 702, so that at least one processor 701 executes the above method.

[0139] The specific implementation process of the processor 701 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0140] In the above Figure 7 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0141] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0142] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0143] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described above is implemented.

[0144] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0145] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0146] The division of the units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electrical, mechanical or other forms.

[0147] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0148] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0149] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0150] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0151] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A key frame detection method, characterized in that: The method comprises: When it is detected that a target detection object exists in an image captured by a camera device, a candidate image captured by the camera device is acquired, wherein the target detection object moves through a conveyor belt, the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate an image captured by the camera device at a preset frequency after the target detection object is detected; Acquire a pixel point at a preset position of the target detection object in the candidate image, and confirm whether the candidate image satisfies a first preset condition based on the pixel point, where the first preset condition is used to indicate that the pixel point is in a target area in the candidate image; If the first preset condition is met, confirming whether the candidate image meets a second preset condition, where the second preset condition is used to indicate that the candidate image is the first image corresponding to the target detection object that meets the first preset condition; If the candidate image meets the second preset condition, the candidate image is determined to be a key frame of the target detection object.

2. The method according to claim 1, characterized in that The step of confirming whether the candidate image satisfies a first preset condition according to the pixel points includes: Acquire the region where the pixel point is located in the candidate image, wherein the candidate image is an unprocessed image captured by the camera device, and divide the candidate image into two left and right sub-regions by setting a virtual longitudinal center line in the candidate image, and the target detection object moves in the shooting range along a preset movement direction perpendicular to the center line through a conveyor belt; Determine whether the pixel point is in a target sub-region of the candidate image, where the target sub-region is used to indicate a first sub-region passed by the target detection object along a preset motion direction; If yes, the candidate image does not satisfy the first preset condition; If not, obtaining the distance between the pixel point and the center line; If the distance between the pixel point and the center line is less than or equal to a first preset threshold, it indicates that the candidate image meets the first preset condition.

3. The method according to claim 1, characterized in that: The obtaining of the pixel point at the preset position of the target detection object in the candidate image includes: Preprocessing the candidate image to obtain a first standard image; Inputting the first standard image into a pre-trained rotation detection model to obtain an output result, wherein the output result includes pixel points at a preset position of the target detection object in the first standard image; The output result is restored to obtain a target output result corresponding to the candidate image, wherein the target output result includes pixel points at a preset position of the target detection object in the candidate image.

4. The method according to claim 3, characterized in that The target output result also includes the size parameters and rotation angle of the target detection object in the candidate image. After determining that the candidate image is a key frame of the target detection object, the method further includes: According to the target output result, confirm whether the rotation angle of the target detection object reaches a second preset threshold, wherein the rotation angle is used to indicate the tilt angle of the target detection object in the candidate image; If yes, correcting the target detection object in the candidate image so that the rotation angle of the target detection object in the candidate image is less than the second preset threshold; if no, no correction processing is performed; According to the size parameters of the target detection object, the target detection object in the candidate image is marked, and the candidate image is cropped to obtain image information containing only the target detection object.

5. The method according to claim 2, characterized in that: If the distance between the pixel point and the center line is greater than the first preset threshold, the method further includes: Obtaining a detected target key frame corresponding to the target detection object; Determining whether to switch the detection target according to the target key frame and the candidate image, so as to perform key frame detection on the next target detection object on the conveyor belt; If the detection target is switched, the detection of the current target detection object is terminated, and key frame detection is performed on the next target detection object.

6. The method according to claim 5, characterized in that The step of confirming whether to switch the detection target according to the target key frame and the candidate image includes: Mapping the position of a target pixel point of the target detection object in the target key frame into the candidate image, wherein a preset position of the target pixel point on the target detection object is the same as a preset position of the pixel point on the target detection object; Obtaining a target distance between the target pixel and a pixel in the candidate image; If the target distance is greater than the first preset threshold, the detection target is switched.

7. A PCB board defect detection method, characterized in that: The method comprises: Acquire a key frame of a target PCB board using the method described in any one of claims 1 to 6; According to the key frame of the target PCB board, the target PCB board is subjected to automatic optical inspection to confirm whether the target PCB board has surface defects.

8. A key frame detection device, characterized in that: The device comprises: A first acquisition module, configured to acquire a candidate image captured by the camera device when a target detection object is detected in an image captured by the camera device, wherein the target detection object moves through a conveyor belt, the camera device is located at a fixed position above the conveyor belt, and the candidate image is used to indicate an image captured by the camera device at a preset frequency after the target detection object is detected; A second acquisition module, used to acquire pixel points at a preset position of the target detection object in the candidate image; A first processing module, used for confirming whether the candidate image satisfies a first preset condition according to the pixel point, where the first preset condition is used for indicating that the pixel point is in a target area in the candidate image; A second processing module, configured to confirm whether the candidate image satisfies a second preset condition if the first preset condition is satisfied, wherein the second preset condition is used to indicate that the candidate image is the first image corresponding to the target detection object that satisfies the first preset condition; The second processing module is further configured to determine that the candidate image is a key frame of the target detection object if the candidate image meets a second preset condition.

9. A PCB board defect detection device, characterized in that: The device comprises: An acquisition module, used for acquiring a key frame of a target PCB board by using the method described in any one of claims 1 to 6; The detection module is used to perform automatic optical detection on the target PCB board according to the key frame of the target PCB board to confirm whether the target PCB board has surface defects.

10. An electronic device, characterized in that: include: at least one processor and memory; wherein, The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 7.

Images