Workpiece deflection detection method, device and equipment and computer readable storage medium

Abstract

The invention relates to a workpiece deflection detection method, device and equipment and a computer readable storage medium. The method comprises the following steps: inputting an obtained workpiece image of a to-be-detected workpiece into a trained semantic segmentation model for processing, and outputting at least one piece of workpiece region information corresponding to the workpiece image; according to the information of the at least one workpiece area, determining an area center point of each workpiece area included in the workpiece image; determining a target detection area corresponding to each workpiece area according to the area center point of the workpiece area; for any target detection area, carrying out statistics on a gray value of the target detection area to obtain a gray statistical value corresponding to the target detection area; and determining a deflection detection result of the to-be-detected workpiece according to the gray statistical value and a preset gray threshold range corresponding to the gray statistical value. According to the invention, the deflection condition of the workpiece can be accurately detected, and different detection scenes and requirements can be flexibly adapted.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a workpiece deflection detection method, device, equipment and computer-readable storage medium. Background Art

[0002] With the continuous advancement of artificial intelligence technology, automated industrial inspection based on visual algorithms is gradually being applied to workpiece inspection.

[0003] During workpiece inspection, a trained model is usually used to analyze workpiece images to identify potential defects. However, when the workpiece is deflected, the accuracy of the model in defect detection is easily affected, which may lead to missed detection. Summary of the invention

[0004] Based on this, it is necessary to provide a workpiece deflection detection method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems, which can improve the accuracy of workpiece deflection detection and effectively avoid the occurrence of missed detection.

[0005] In a first aspect, the present application provides a method for detecting workpiece deflection, comprising:

[0006] Inputting the acquired workpiece image of the workpiece to be detected into the trained semantic segmentation model for processing, and outputting at least one workpiece region information corresponding to the workpiece image;

[0007] Determine the center point of each workpiece area included in the workpiece image according to at least one workpiece area information;

[0008] According to the center point of the workpiece area, the target detection area corresponding to each workpiece area is determined;

[0009] For any target detection area, the grayscale value of the target detection area is counted to obtain the grayscale statistical value corresponding to the target detection area;

[0010] The deflection detection result of the workpiece to be detected is determined according to the grayscale statistical value and the corresponding preset grayscale threshold range.

[0011] In a second aspect, the present application provides a workpiece deflection detection device, comprising:

[0012] A processing module, used for inputting the acquired workpiece image of the workpiece to be detected into the trained semantic segmentation model for processing, and outputting at least one workpiece region information corresponding to the workpiece image;

[0013] A center point determination module, used to determine the center point of each workpiece area included in the workpiece image according to at least one workpiece area information;

[0014] The area determination module is used to determine the target detection area corresponding to each workpiece area according to the area center point of the workpiece area;

[0015] A statistical module is used to count the grayscale value of any target detection area to obtain the grayscale statistical value corresponding to the target detection area;

[0016] The determination module is used to determine the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the corresponding preset grayscale threshold range.

[0017] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0019] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0020] The above-mentioned workpiece deflection detection method, device, computer equipment, computer-readable storage medium and computer program product, by using a semantic segmentation model to segment the workpiece area from the workpiece image, then determine the corresponding target detection area according to the regional center point of the workpiece area, and determine the workpiece deflection according to the grayscale statistics of the target detection area and the preset grayscale threshold range. By segmenting the workpiece area in the workpiece image, the scheme can filter the background content irrelevant to the workpiece from the image, avoiding the interference of the image background on the subsequent deflection detection. At the same time, by determining the corresponding target detection area according to the regional center point of the workpiece area, and determining the workpiece deflection according to the grayscale statistics of the target detection area and the preset grayscale threshold range, the scheme can focus the deflection detection on the central area of ​​the workpiece area, further reducing the interference of redundant information. Moreover, the scheme specifically performs deflection detection according to the grayscale statistics, so that by pre-setting the corresponding grayscale threshold range, efficient and accurate detection of different deflection conditions of the workpiece can be achieved, without the need to perform corresponding model training or set corresponding templates for different situations, and can also quickly adapt to different detection sites and detection needs by adjusting the grayscale threshold range, with the advantage of high flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A diagram of an application environment of a workpiece deflection detection method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a flow chart of a workpiece deflection detection method provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a process for determining a center point of a workpiece region provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a workpiece image provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of another workpiece image provided by an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a workpiece region provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of a mask graphic provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of a target detection area provided in an embodiment of the present application;

[0029] Fig. 9 A structural block diagram of a workpiece deflection detection device provided in an embodiment of the present application;

[0030] Fig.10 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0031] Fig.11 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0032] Fig.12 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] The workpiece deflection detection method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 can communicate with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices, etc. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0035] like Figure 2 As shown, the embodiment of the present application provides a workpiece deflection detection method, which is applied to Figure 1 The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method comprises the following steps:

[0036] Step S201: inputting the acquired workpiece image of the workpiece to be detected into a trained semantic segmentation model for processing, and outputting at least one workpiece region information corresponding to the workpiece image.

[0037] The workpiece image of the workpiece to be inspected may be an image acquired during an automatic optical inspection (AOI) process of the workpiece to be inspected, and the image may include image content corresponding to the entirety or a portion of the workpiece. Exemplarily, the workpiece to be inspected may be a connecting cable with a Universal Serial Bus (USB) interface, and the workpiece image may include image content corresponding to a connector of the connecting cable.

[0038] The trained semantic segmentation model may be a neural network model trained using an annotated workpiece image. The workpiece region information may include position information of an image region corresponding to the workpiece in the workpiece image, etc. Specifically, when the workpiece image contains content corresponding to multiple components of the workpiece to be detected, the workpiece region information output by the semantic segmentation model may include multiple workpiece region information corresponding to each component.

[0039] Based on the workpiece region information, at least one image region corresponding to at least one workpiece portion can be located from the workpiece image, thereby eliminating interference of background regions in the workpiece image that are irrelevant to the workpiece with subsequent deflection detection.

[0040] Step S202: determining the center point of each workpiece region included in the workpiece image according to at least one workpiece region information.

[0041] According to each workpiece region information, the corresponding workpiece region can be segmented in the workpiece image and then the region center point of each workpiece region can be determined. Specifically, when the workpiece image contains multiple workpiece regions, the region center point corresponding to each workpiece region can be determined separately in this step.

[0042] Step S203, determining the target detection area corresponding to each workpiece area according to the area center point of the workpiece area.

[0043] The target detection area may include the central area of ​​the workpiece area. Exemplarily, the central point of the area may be used as the center of a circle, and the circular area around the central point of the area may be used as the target detection area corresponding to the working area.

[0044] Step S204: for any target detection area, the grayscale value of the target detection area is counted to obtain the grayscale statistical value corresponding to the target detection area.

[0045] For each target detection area determined in the above process, the number of pixels and the grayscale value of each pixel can be obtained, and then the grayscale value of each pixel can be analyzed and counted to obtain the grayscale statistical value corresponding to the target detection area. For example, the grayscale value can be counted by calculating the grayscale average value of the target detection area to obtain the grayscale statistical value. For example, the grayscale statistical value can also be obtained by counting the distribution of the grayscale value of each pixel in the target detection area.

[0046] Step S205 , determining the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the corresponding preset grayscale threshold range.

[0047] In the process of detecting the workpiece to be detected, the standard posture that the workpiece should meet can be determined in advance. When the posture of the workpiece to be detected is consistent with the standard posture, the deflection detection result of the workpiece to be detected can be determined as no deflection. Alternatively, when the posture of the workpiece to be detected is tilted or flipped relative to the standard posture, the deflection detection result of the workpiece to be detected can be determined as deflection.

[0048] It can be understood that under the same lighting conditions and image acquisition conditions, different postures of the workpiece to be detected can cause different effects of the illumination light on the workpiece to be detected, thereby making the grayscale statistics of the target detection area in the workpiece image different. Based on this, when the workpiece to be detected is in a standard posture in the detection environment, the grayscale statistics of the target detection area in the corresponding workpiece image should conform to the numerical range, that is, the corresponding grayscale threshold range set in advance, and then in this step, it can be determined whether the workpiece to be detected is deflected according to whether the grayscale statistics obtained above conform to the preset grayscale threshold range. When the grayscale statistics conform to the corresponding preset grayscale threshold range, the deflection detection result of the workpiece to be detected can be obtained as the workpiece is not deflected, otherwise the deflection detection result of the workpiece to be detected can be obtained as the workpiece is deflected.

[0049] It is understandable that, according to different lighting conditions and image acquisition conditions corresponding to different detection scenes, the preset grayscale threshold range can be adaptively adjusted so that it can reflect the corresponding grayscale statistical value when the workpiece is in a standard posture in the scene. In some embodiments, the preset grayscale threshold range can also be set in combination with the tolerance range of the workpiece deflection in the specific detection scene.

[0050] It can be seen that in the embodiment of the present application, by segmenting the workpiece area in the workpiece image, background content unrelated to the workpiece can be filtered from the image, avoiding interference of the image background on subsequent deflection detection. At the same time, the scheme determines the corresponding target detection area according to the center point of the area of ​​the workpiece area, and determines the workpiece deflection according to the grayscale statistics of the target detection area and the preset grayscale threshold range, so that the deflection detection can be focused on the central area of ​​the workpiece area, further reducing the interference of redundant information. Moreover, the scheme specifically performs deflection detection based on grayscale statistics, so that by pre-setting the corresponding grayscale threshold range, efficient and accurate detection of different deflection conditions of the workpiece can be achieved, without the need for corresponding model training or setting corresponding templates for different situations, and can also quickly adapt to different detection sites and detection needs by adjusting the grayscale threshold range, which has the advantage of high flexibility.

[0051] In some embodiments, the grayscale value of the target detection area is counted to obtain the grayscale statistical value corresponding to the target detection area, including:

[0052] Get the gray value of each pixel in the target detection area;

[0053] According to the gray value of each pixel, the number of white pixels in the target detection area with a preset ratio is counted;

[0054] The grayscale statistical value corresponding to the target detection area is calculated according to the grayscale value of each pixel, the number of pixels in the target detection area, and the number of white pixels in a preset ratio.

[0055] For example, when the grayscale value of the target detection area is counted, the grayscale value of each pixel in the target detection area can be obtained first, and then the pixel points in the target detection area can be divided into white pixel points (pixel points with a grayscale value of 255) and non-white pixel points (pixel points with a grayscale value less than 255) according to the grayscale value of each pixel point. Then, the number of white pixel points can be counted, and a preset number of white pixel points can be reserved to participate in the calculation of the grayscale statistical value.

[0056] For example, the grayscale statistics of the target detection area can be calculated by the following formula:

[0057]

[0058] In the formula, is the grayscale statistical value of the target detection area, is the number of pixels in the target detection area, is the number of white pixels in the target detection area, is the preset quantity ratio, For the The gray value of non-white pixels.

[0059] It can be seen that in this embodiment, for the grayscale value statistics of the target detection area, by applying a preset number of white pixels to the grayscale value statistics, the influence of white pixels on the grayscale statistical values ​​can be reduced, so that the grayscale statistical values ​​can more sensitively reflect the different deflection conditions of the workpiece to be detected, which is conducive to obtaining more accurate deflection detection results.

[0060] In some embodiments, Figure 3 As shown, according to at least one workpiece region information, determining the region center point of each workpiece region included in the workpiece image includes:

[0061] Step S301 : for any workpiece region information, determine the minimum circumscribed rectangle of the corresponding workpiece region according to the workpiece region information.

[0062] The workpiece region information may include coordinate information of each edge point of the workpiece region. Based on this information, the minimum circumscribed rectangle of the workpiece region may be determined in the workpiece image, and the vertex coordinates of the minimum circumscribed rectangle may be obtained.

[0063] Step S302: determining the center point of the workpiece area according to the minimum circumscribed rectangle of the workpiece area.

[0064] According to the vertex coordinates of the minimum enclosing rectangle, the coordinates of the center point of the minimum enclosing rectangle can be calculated. For example, the diagonals of the minimum enclosing rectangle can be connected, and the intersection of the two diagonals can be used as the center point of the minimum enclosing rectangle. After the center point of the minimum enclosing rectangle is obtained, the center point can be used as the area center point of the workpiece area.

[0065] It can be seen that in this embodiment, by determining the minimum circumscribed rectangle of the workpiece area, the center point of the workpiece area can be quickly determined by using its center point.

[0066] In some embodiments, determining the target detection area corresponding to each workpiece area according to the area center point of the workpiece area includes:

[0067] For the center point of any workpiece area, the inscribed circle area of ​​the minimum circumscribed rectangle of the workpiece area is determined with the center point of the workpiece area as the center and 1 / 2 of the smaller value of the width and height of the minimum circumscribed rectangle of the workpiece area as the radius;

[0068] According to the inscribed circle area, a mask graphic of the workpiece image is constructed;

[0069] Using the mask pattern, the target detection area corresponding to the inscribed circle area is extracted from the workpiece image.

[0070] Among them, according to the center point of the workpiece area, it can be used as the center of the circle, and the shortest side of the minimum circumscribed rectangle of the workpiece area (that is, the smaller value of the width and height of the minimum circumscribed rectangle) is used as the radius to obtain the inscribed circle area of ​​the minimum circumscribed rectangle of the workpiece area. Exemplarily, the inscribed circle area can be expressed as:

[0071]

[0072] In the formula, is the center point of the workpiece area, It is 1 / 2 of the shortest side of the minimum circumscribed rectangle of the workpiece area. is a point on the edge of the inscribed circle.

[0073] Then, a mask pattern with the same size as the workpiece image can be constructed based on the inscribed circle area of ​​the minimum circumscribed rectangle of each workpiece area. In the mask pattern, the portion corresponding to each inscribed circle area is marked as 1, and the remaining area is marked as 0. Subsequently, the mask pattern can be ANDed with the workpiece image from which the workpiece area is segmented, so that the target detection area corresponding to the inscribed circle area corresponding to it can be extracted from each workpiece area of ​​the workpiece image.

[0074] It can be seen that in this embodiment, by determining the inscribed circle area of ​​the minimum circumscribed rectangle with the center point of the workpiece area as the center, a target detection area that can focus on reflecting the core part of the workpiece area can be obtained. At the same time, in this embodiment, a mask pattern is constructed according to the inscribed circle area, and the mask pattern can be used to directly extract the target detection area in the workpiece image, which is beneficial to improving the overall processing efficiency. Moreover, in this embodiment, by extracting the target detection area according to the inscribed circle area, the target detection area corresponding to the workpiece to be detected in different postures in the workpiece image can also be made consistent, so that the deflection detection can be performed with the same standard, which is beneficial to improving the accuracy of the deflection detection result.

[0075] In some embodiments, determining the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the preset grayscale threshold range corresponding thereto includes:

[0076] If each grayscale statistical value of at least one grayscale statistical value meets the preset grayscale threshold range corresponding thereto, it is determined that the deflection detection result of the workpiece to be detected is that the workpiece is not deflected; or,

[0077] If there is a grayscale statistical value in at least one grayscale statistical value that does not meet the preset grayscale threshold range corresponding to it, it is determined that the deflection detection result of the workpiece to be detected is workpiece deflection.

[0078] Wherein, the workpiece to be detected may include multiple components, so that the workpiece image may correspondingly include multiple different workpiece areas corresponding to multiple different components of the workpiece to be detected. Exemplarily, still taking the example that the workpiece to be detected may be a connecting line with a Universal Serial Bus (USB) interface, its workpiece image may include workpiece areas corresponding to two different components of the connecting line: a sheath (plug) and a plug (boot). Wherein, for each workpiece area, the area center point corresponding to its workpiece area can be determined in the aforementioned manner, and then the corresponding target detection area can be determined, and the grayscale value of the target detection area can be counted, so that the grayscale statistical value corresponding to each workpiece area can be obtained.

[0079] Among them, for each workpiece area, it is possible to obtain whether the corresponding grayscale statistical value meets the preset grayscale threshold range according to the corresponding preset grayscale threshold range. Specifically, when the workpiece image contains multiple workpiece areas corresponding to multiple parts of the workpiece to be detected, a corresponding preset grayscale threshold range can be set for each workpiece area.

[0080] If the grayscale statistics corresponding to each workpiece area in the workpiece image meet the corresponding preset grayscale threshold range, it can be obtained that the posture of the workpiece to be detected meets the standard posture, that is, no deflection occurs. Therefore, the deflection detection result corresponding to the workpiece to be detected can be obtained as the workpiece is not deflected.

[0081] Alternatively, if there is at least one grayscale statistic corresponding to a workpiece region in the workpiece image that does not conform to its corresponding preset grayscale threshold range, it can be obtained that the posture of the workpiece to be detected does not conform to the standard posture, that is, deflection has occurred. Therefore, the deflection detection result corresponding to the workpiece to be detected can be obtained as workpiece deflection.

[0082] It can be seen that in this embodiment, for the workpiece to be inspected including multiple components, the grayscale statistical value of the workpiece area corresponding to each component in the workpiece image is obtained respectively to determine whether it meets the preset grayscale threshold range, and when the grayscale statistical value of the workpiece area does not meet the corresponding range, the corresponding deflection detection result is obtained as the workpiece deflection, which can improve the comprehensiveness of the deflection detection and effectively avoid the occurrence of missed detection and the like.

[0083] In some embodiments, after determining the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the preset grayscale threshold range corresponding thereto, the method further includes:

[0084] When the deflection detection result is that the workpiece is deflected, a deflection prompt message is sent.

[0085] When the deflection detection result of the workpiece to be detected is that the workpiece is deflected, deflection prompt information can be obtained according to the deflection detection result and sent to the terminal used by the detection personnel. The deflection prompt information can prompt that the workpiece to be detected has deflection, so that the detection personnel can re-inspect the workpiece to be detected after receiving the deflection prompt information. In some embodiments, the deflection prompt information can also include a workpiece image of the workpiece to be detected, so that the detection personnel can promptly know the deflection of the workpiece to be detected.

[0086] It can be seen that in this embodiment, by sending a deflection prompt message when the deflection detection result is a workpiece deflection, the staff can promptly be informed of the deflection of the workpiece, and perform corresponding processing such as re-inspection or resetting, which is beneficial to improving the overall production inspection efficiency of the workpiece.

[0087] In order to further illustrate the workpiece deflection detection method of the present application, the following is a detailed description of the method through an embodiment:

[0088] By way of example, in this embodiment, the workpiece to be detected is taken as an example of a connection cable with a Universal Serial Bus Type-C (USB Type-C) interface to illustrate the workpiece deflection detection method.

[0089] Specifically, the workpiece to be inspected may include components such as a plastic plug and a metal boot, and the corresponding workpiece image may be an image obtained during the automatic optical inspection (AOI) process of the workpiece to be inspected, which may contain image content corresponding to these components. For example, when the workpiece to be inspected is in a standard posture, its workpiece image may be as follows: Figure 4 As shown, when the workpiece to be inspected is deflected, its workpiece image can be as follows Figure 5 shown.

[0090] The trained semantic segmentation model may be a neural network model trained using labeled workpiece images. Figure 6 As shown, the semantic segmentation model can process the input workpiece image, segment therefrom a first workpiece region corresponding to a metal plug (boot) and a second workpiece region corresponding to a plastic sheath (plug), and output workpiece region information corresponding to the first workpiece region and the second workpiece region.

[0091] For each workpiece region, the region center point of the workpiece region in the workpiece image can be determined according to the workpiece region information, and the target detection region corresponding to the workpiece region can be determined according to the region center point.

[0092] For example, Figure 6 As shown, the minimum bounding rectangle corresponding to each workpiece region can be determined according to the workpiece region information, and then the intersection of the diagonal lines of the minimum bounding rectangle is used as the region center point of the workpiece region. For example, the region center point of the first workpiece region corresponding to the metal plug (boot) can be recorded as , the center point of the second workpiece area corresponding to the plastic sheath (plug) can be recorded as At the same time, the side lengths of the minimum bounding rectangle of the corresponding workpiece area can be obtained respectively. For example, the width and height of the minimum bounding rectangle of the first workpiece area can be recorded as , and record the width and height of the minimum circumscribed rectangle of the second workpiece area as .

[0093] Then, the center point of the workpiece area can be used as the center of the circle, and 1 / 2 of the shortest side of the corresponding minimum circumscribed rectangle can be used as the radius to obtain the inscribed circle area of ​​the minimum circumscribed rectangle of the workpiece area. For example, the center of the inscribed circle area of ​​the minimum circumscribed rectangle corresponding to the first workpiece area is , the radius is , the inscribed circle area can be expressed as: ,in is a point on the edge of the inscribed circle area; similarly, the center of the inscribed circle area of ​​the minimum circumscribed rectangle corresponding to the second workpiece area is , the radius is , the inscribed circle area can be expressed as: ,in is a point on the edge of the inscribed circle.

[0094] Based on the inscribed circle area corresponding to each workpiece area, a mask pattern of the workpiece image can be further constructed, and the mask pattern can be used to extract the target detection area corresponding to the inscribed circle area from the workpiece image. Figure 6 The inscribed circle area corresponding to the first workpiece area and the second workpiece area shown in FIG. Figure 7 The mask is shown in FIG. The mask is the same size as the workpiece image, and the portion corresponding to the inscribed circle area is marked as 1, and the rest of the area is marked as 0. By performing an AND operation on the mask and the workpiece image, the target detection area corresponding to each workpiece area and its inscribed circle area can be extracted. For example, the target detection area extracted from the first workpiece area can be as follows: Figure 8 As shown in part (a), the target detection area extracted from the second workpiece area can be Figure 8 As shown in part (b).

[0095] For each target detection area, its grayscale value may be counted to obtain a grayscale statistical value corresponding to the target detection area.

[0096] Among them, for the target detection area corresponding to each workpiece area, it can be determined whether the corresponding grayscale statistical value meets the corresponding preset grayscale threshold range. If the grayscale statistical values ​​of the target detection areas corresponding to all workpiece areas meet the corresponding preset grayscale threshold range, the deflection detection result of the workpiece to be detected can be obtained as the workpiece is not deflected, or if there is at least one workpiece area The corresponding grayscale statistical value does not meet the corresponding preset grayscale threshold range, the deflection detection result of the workpiece to be detected can be obtained as the workpiece is deflected.

[0097] When the deflection detection result of the workpiece to be detected is workpiece deflection, deflection prompt information can be obtained according to the deflection detection result and sent to the terminal used by the detection personnel. The deflection prompt information can prompt that the workpiece to be detected has deflection, so that the detection personnel can re-inspect the workpiece to be detected after receiving the deflection prompt information.

[0098] In this embodiment, by combining deep learning, mathematical modeling and traditional image processing methods, it is possible to accurately detect whether the Type-C data cable connector is deflected during automatic optical inspection. Among them, since the segmentation result of the first stage only retains the Type-C data cable charging head and plastic protective cover to be detected and filters the background, the instability and interference of the traditional algorithm in the presence of the background can be avoided. Moreover, because this method is based on grayscale value comparison to achieve deflection detection, when a large number of different deflection situations occur, or when deflection detection is required in different scenarios and different requirements, there is no need to re-collect the corresponding images to train the model, but only need to change the preset grayscale threshold range according to the specific detection scenario or detection requirement, thereby solving the problem that the traditional technology cannot flexibly adapt to on-site production when using deep models for deflection detection, and effectively improves the detection efficiency of workpiece deflection.

[0099] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0100] Based on the same inventive concept, the embodiment of the present application also provides a workpiece deflection detection device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more workpiece deflection detection device embodiments provided below can refer to the limitations of the workpiece deflection detection method above, and will not be repeated here.

[0101] like Fig. 9 As shown, the embodiment of the present application provides a workpiece deflection detection device 900, comprising:

[0102] The processing module 901 is used to input the acquired workpiece image of the workpiece to be detected into the trained semantic segmentation model for processing, and output at least one workpiece region information corresponding to the workpiece image;

[0103] A center point determination module 902 is used to determine the center point of each workpiece area included in the workpiece image according to at least one workpiece area information;

[0104] The region determination module 903 is used to determine the target detection region corresponding to each workpiece region according to the region center point of the workpiece region;

[0105] A statistics module 904 is used to count the grayscale value of any target detection area to obtain a grayscale statistical value corresponding to the target detection area;

[0106] The determination module 905 is used to determine the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the corresponding preset grayscale threshold range.

[0107] In some embodiments, in terms of performing statistics on the grayscale values ​​of the target detection area to obtain the grayscale statistical values ​​corresponding to the target detection area, the statistical module 904 is specifically used to:

[0108] Get the gray value of each pixel in the target detection area;

[0109] According to the gray value of each pixel, the number of white pixels in the target detection area with a preset ratio is counted;

[0110] The grayscale statistical value corresponding to the target detection area is calculated according to the grayscale value of each pixel, the number of pixels in the target detection area, and the number of white pixels in a preset ratio.

[0111] In some embodiments, in determining the center point of each workpiece region included in the workpiece image according to at least one workpiece region information, the center point determination module 902 is specifically used to:

[0112] For any workpiece region information, determine the minimum circumscribed rectangle of the corresponding workpiece region according to the workpiece region information;

[0113] According to the minimum circumscribed rectangle of the workpiece area, the center point of the workpiece area is determined.

[0114] In some embodiments, in determining the target detection area corresponding to each workpiece area according to the area center point of the workpiece area, the area determination module 903 is specifically used to:

[0115] For the center point of any workpiece area, the inscribed circle area of ​​the minimum circumscribed rectangle of the workpiece area is determined with the center point of the workpiece area as the center and 1 / 2 of the smaller value of the width and height of the minimum circumscribed rectangle of the workpiece area as the radius;

[0116] According to the inscribed circle area, a mask graphic of the workpiece image is constructed;

[0117] Using the mask pattern, the target detection area corresponding to the inscribed circle area is extracted from the workpiece image.

[0118] In some embodiments, in determining the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the preset grayscale threshold range corresponding thereto, the determination module 905 is specifically used to:

[0119] If each grayscale statistical value of at least one grayscale statistical value meets the preset grayscale threshold range corresponding thereto, it is determined that the deflection detection result of the workpiece to be detected is that the workpiece is not deflected; or,

[0120] If there is a grayscale statistical value in at least one grayscale statistical value that does not meet the preset grayscale threshold range corresponding to it, it is determined that the deflection detection result of the workpiece to be detected is workpiece deflection.

[0121] In some embodiments, the workpiece deflection detection device further includes a prompt module, and the prompt module is used to send deflection prompt information when the deflection detection result is workpiece deflection.

[0122] Each module in the above workpiece deflection detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0123] In some embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data involved and applied in the workpiece deflection detection process. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned workpiece deflection detection method are implemented.

[0124] In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned workpiece deflection detection method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0125] Those skilled in the art will understand that Fig.10 or Fig.11The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0126] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0127] In some embodiments, Fig.12 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0128] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0129] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0131] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting workpiece deflection, characterized in that: include: Inputting the acquired workpiece image of the workpiece to be detected into the trained semantic segmentation model for processing, and outputting at least one workpiece region information corresponding to the workpiece image; Determining, according to the at least one workpiece region information, a region center point of each workpiece region included in the workpiece image; Determining the target detection area corresponding to each of the workpiece areas according to the area center points of the workpiece areas; For any of the target detection areas, performing statistics on the grayscale values ​​of the target detection area to obtain a grayscale statistical value corresponding to the target detection area; The deflection detection result of the workpiece to be detected is determined according to the grayscale statistical value and the preset grayscale threshold range corresponding thereto.

2. The method according to claim 1, characterized in that The step of performing statistics on the grayscale values ​​of the target detection area to obtain grayscale statistical values ​​corresponding to the target detection area includes: Obtaining the grayscale value of each pixel in the target detection area; According to the grayscale value of each pixel point, counting the number of white pixels with a preset ratio in the target detection area; A grayscale statistical value corresponding to the target detection area is calculated according to the grayscale value of each pixel, the number of pixels in the target detection area, and the number of white pixels in the preset ratio.

3. The method according to claim 1, characterized in that Determining the area center point of each workpiece area included in the workpiece image according to the at least one workpiece area information includes: For any of the workpiece region information, determining the minimum circumscribed rectangle of the corresponding workpiece region according to the workpiece region information; The center point of the workpiece area is determined according to the minimum circumscribed rectangle of the workpiece area.

4. The method according to claim 3, characterized in that Determining the target detection area corresponding to each workpiece area according to the area center point of the workpiece area includes: For any area center point of the workpiece area, with the area center point of the workpiece area as the center and 1 / 2 of the smaller value of the width and height of the minimum circumscribed rectangle of the workpiece area as the radius, determine the inscribed circle area of ​​the minimum circumscribed rectangle of the workpiece area; Constructing a mask graphic of the workpiece image according to the inscribed circle area; The target detection area corresponding to the inscribed circle area is extracted from the workpiece image using the mask pattern.

5. The method according to claim 1, characterized in that The step of determining the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the preset grayscale threshold range corresponding thereto comprises: If each grayscale statistical value of the at least one grayscale statistical value meets the preset grayscale threshold range corresponding thereto, then it is determined that the deflection detection result of the workpiece to be detected is that the workpiece is not deflected; or, If there is a grayscale statistic value in the at least one grayscale statistic value that does not conform to the preset grayscale threshold range corresponding thereto, it is determined that the deflection detection result of the workpiece to be detected is workpiece deflection.

6. The method according to any one of claims 1 to 5, characterized in that: After determining the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the preset grayscale threshold range corresponding thereto, the method further includes: When the deflection detection result is that the workpiece is deflected, a deflection prompt message is sent.

7. A workpiece deflection detection device, characterized in that: include: A processing module, used for inputting the acquired workpiece image of the workpiece to be detected into the trained semantic segmentation model for processing, and outputting at least one workpiece region information corresponding to the workpiece image; A center point determination module, used to determine the center point of each workpiece area included in the workpiece image according to the at least one workpiece area information; An area determination module, used to determine the target detection area corresponding to each workpiece area according to the area center point of the workpiece area; A statistical module, for performing statistics on the grayscale value of any target detection area to obtain a grayscale statistical value corresponding to the target detection area; The determination module is used to determine the deflection detection result of the workpiece to be detected according to the grayscale statistical value and the preset grayscale threshold range corresponding thereto.

8. The device according to claim 7, characterized in that The device also includes: The prompt module is used to send deflection prompt information when the deflection detection result is workpiece deflection.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images