Processing method and processing device

Abstract

The embodiment of the present application provides a processing method, comprising: obtaining a processed image of a device to be detected, wherein the processed image is obtained by performing a first processing on an initial image of the device to be detected, the first processing at least comprising scaling the initial image according to defect parameters corresponding to the initial image; performing a second processing on the processed image to generate a target image, and analyzing the target image by a target defect detection method to achieve defect detection of the device to be detected. The processing method of the embodiment of the present application is easy to operate, and can conveniently scale the initial image according to the defect parameters to obtain a suitable processed image, thereby improving the recognition speed of the algorithm, while also avoiding reducing the accuracy of the algorithm, and further performing a second processing on the processed image to generate a target image to calibrate the processed image, which can significantly improve the recognition efficiency of the subsequent algorithm.

Description

Technical Field

[0001] The present application relates to the field of detection technology, and in particular to a processing method and a processing device. Background Art

[0002] In industrial defect detection systems, images of objects to be inspected are acquired through cameras and other acquisition devices, and then processed using algorithms to perform defect detection on the images to determine whether there are defects.

[0003] Currently, defect detection typically involves capturing images of objects with ultra-high-definition cameras to obtain high-resolution images, which facilitate subsequent computer detection and analysis. However, due to the high image resolution, defect detection algorithms are slow and inefficient, often requiring powerful machines and streamlined algorithms to achieve the throughput required on automated production lines. Summary of the Invention

[0004] In view of the above problems existing in the prior art, the present application provides a processing method and a processing device. The technical solutions adopted in the embodiments of the present application are as follows:

[0005] On the one hand, the embodiments of the present application provide a processing method, including:

[0006] Obtaining a processed image of the device to be inspected, wherein the processed image is obtained by performing a first processing on an initial image of the device to be inspected, the first processing at least comprising scaling the initial image according to defect parameters corresponding to the initial image;

[0007] The processed image is subjected to a second processing to generate a target image, and the target image is analyzed by a target defect detection method to achieve defect detection of the device to be detected.

[0008] In some embodiments, the method further comprises: obtaining the initial image, comprising:

[0009] Identify the device to be detected using a device recognition model to determine a frame position of the device to be detected;

[0010] The shooting parameters of the acquisition device are adjusted according to the position of the frame to obtain an initial image of the device to be detected within a range corresponding to the shooting parameters, wherein the acquisition device is used to acquire the image of the device to be detected.

[0011] In some embodiments, the acquisition device does not store the initial image of the device to be inspected, and the scaling process of the initial image according to the defect parameters corresponding to the initial image includes:

[0012] determining a target resolution according to defect parameters corresponding to the previewed initial image;

[0013] Adjusting the shooting parameters of the acquisition device according to the target resolution to achieve scaling processing of the previewed initial image, wherein the acquisition device is used to acquire the image of the device to be detected;

[0014] Alternatively, the acquisition device stores the initial image, and the scaling process is performed on the initial image according to the defect parameter corresponding to the initial image, including:

[0015] determining a scaling parameter based on a defect parameter corresponding to the stored initial image;

[0016] The stored initial image is adjusted according to the scaling parameter to implement scaling processing on the initial image.

[0017] In some embodiments, performing a second process on the processed image to generate a target image includes:

[0018] performing component recognition on the processed image to determine positions of at least two target detection components;

[0019] Endpoint matching is performed on the positions of the at least two target detection elements using a first constraint model to determine the positions of all target detection elements on the processed image.

[0020] In some embodiments, performing endpoint matching on the positions of the at least two target detection elements using the first constraint model includes:

[0021] Matching is performed based on the positions of the at least two target detection elements and on the endpoints of the first constraint model, and the angles and side lengths corresponding to the endpoints;

[0022] If the positions of the at least two target detection elements conform to the relationship among the endpoints, angles, and side lengths, the processed image conforms to the first constraint model.

[0023] In some embodiments, when the processed image conforms to the first constraint model, performing endpoint matching on the positions of the at least two target detection elements using the first constraint model to determine the positions of all target detection elements on the processed image includes:

[0024] If there are endpoints in the first constraint model that cannot be matched with the at least two target detection elements, determining that there are unrecognized target detection elements in the processed image;

[0025] A padding operation is performed on the unrecognized target detection elements according to the first constraint model to determine positions of all the target detection elements on the processed image.

[0026] In some embodiments, performing a second process on the processed image to generate the target image includes:

[0027] performing component recognition on the processed image, and determining positions of at least three target detection components on the processed image;

[0028] According to the positions of the at least three target detection elements, the angle of the corresponding processed image is rotated based on the second constraint model to adjust the angle of the processed image.

[0029] In some embodiments, performing a second process on the processed image to generate the target image further includes:

[0030] comparing the positions of the at least three target detection elements in the at least two processed images after adjustment with the second constraint model, and respectively obtaining deviation angles between the at least two processed images and the second constraint model;

[0031] Calculating a mean value of the deviation angles according to the respective deviation angles, and setting a corresponding weight for the corresponding processed image according to the magnitude of the respective deviation angles;

[0032] Each of the adjusted processed images is rotated again according to the deviation angle mean and the weight to generate the target image.

[0033] In some embodiments, scaling the initial image according to the defect parameter corresponding to the initial image includes:

[0034] generating the defect parameters using a defect prediction model according to the annotation parameters corresponding to the initial image;

[0035] Determine a scaling ratio according to the defect parameter to scale the initial image according to the scaling ratio, wherein the defect parameter at least includes the number of downsampling times a, and the scaling ratio is based on 2 a get.

[0036] On the other hand, an embodiment of the present application provides a processing device, including:

[0037] an obtaining module, configured to obtain a processed image of the device to be inspected, wherein the processed image is obtained by performing a first processing on an initial image of the device to be inspected, the first processing at least comprising scaling the initial image according to defect parameters corresponding to the initial image;

[0038] The calibration module performs a second processing on the processed image to generate a target image, and analyzes the target image in a target defect detection manner to achieve defect detection of the device to be detected.

[0039] The beneficial effect of the embodiment of the present application is that: in the scheme of the embodiment of the present application, the processed image of the device to be detected is obtained by performing a first processing on the initial image of the device to be detected. The first processing at least includes scaling the initial image according to the defect parameters corresponding to the initial image to reduce the resolution of the initial image, thereby speeding up the algorithm's processing speed of the image, while ensuring that the processed image reaches the clarity required by the algorithm's detection accuracy. The processed image is then subjected to a second processing to generate a target image to calibrate the image so that the target image can adapt to the requirements of the algorithm and improve the algorithm's recognition efficiency, thereby facilitating the analysis of the target image through a target defect detection method and achieving defect detection of the device to be detected. The processing method of the embodiment of the present application is easy to operate and can conveniently scale the initial image according to the defect parameters to obtain a suitable processed image, thereby improving the algorithm's recognition speed and avoiding reducing the algorithm's accuracy. The processed image is further subjected to a second processing to generate a target image to calibrate the processed image, which can significantly improve the recognition efficiency of the subsequent algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A flowchart of the processing method provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of a treatment method provided by an embodiment of the present application;

[0043] Figure 3 Another schematic diagram of the processing method provided by the embodiment of the present application;

[0044] Figure 4 This is another schematic diagram of the processing method provided by the embodiment of the present application;

[0045] Figure 5 This is another schematic diagram of the processing method provided by the embodiment of the present application;

[0046] Figure 6A block diagram of a processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be clearly and completely described below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] Unless otherwise defined, the technical or scientific terms used in this application should have the usual meanings understood by persons of ordinary skill in the field to which this application belongs. The words "first", "second" and similar terms used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the devices or objects appearing before the word include the devices or objects listed after the word and their equivalents, without excluding other devices or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0049] In order to keep the following description of the embodiments of the present application clear and concise, detailed descriptions of known functions and known components are omitted in this application.

[0050] In the scheme of the embodiment of the present application, a processed image of the device to be detected is obtained by performing a first processing on the initial image of the device to be detected. The first processing at least includes scaling the initial image according to the defect parameters corresponding to the initial image to reduce the resolution of the initial image, thereby speeding up the algorithm's processing speed of the image, while ensuring that the processed image reaches the clarity required by the algorithm's detection accuracy. The processed image is then subjected to a second processing to generate a target image to calibrate the image so that the target image can adapt to the requirements of the algorithm and improve the algorithm's recognition efficiency, thereby facilitating the analysis of the target image through a target defect detection method and achieving defect detection of the device to be detected. The processing method of the embodiment of the present application is easy to operate and can conveniently scale the initial image according to the defect parameters to obtain a suitable processed image, thereby improving the algorithm's recognition speed and avoiding reducing the algorithm's accuracy. The processed image is further subjected to a second processing to generate a target image to calibrate the processed image, which can significantly improve the recognition efficiency of the subsequent algorithm.

[0051] Figure 1 Flowchart showing the processing method of the embodiment of the present application. Figure 1 As shown, the processing method of the embodiment of the present application includes the following steps S200-S300.

[0052] S200, obtaining a processed image of the device to be inspected, wherein the processed image is obtained by performing a first processing on an initial image of the device to be inspected, the first processing at least comprising scaling the initial image according to defect parameters corresponding to the initial image;

[0053] This step aims to obtain a processed image with appropriate resolution and size for the device to be inspected, so as to improve the algorithm's processing speed for the image during subsequent inspection and analysis. The device to be inspected can be a mobile phone, tablet, or other device that needs to be inspected for defects, and this application does not limit this.

[0054] Defect parameters are characteristic parameters used to accurately detect defects. They include defect size and sampling density. Considering the relatively small size of the defect to be detected, for ease of calculation, the defect size can be expressed in pixels. Of course, other units can also be used as long as they do not affect detection accuracy, and this application does not limit this.

[0055] In some embodiments, scaling the initial image according to the defect parameter corresponding to the initial image includes:

[0056] generating the defect parameters using a defect prediction model according to the annotation parameters corresponding to the initial image;

[0057] Determine a scaling ratio according to the defect parameter to scale the initial image according to the scaling ratio, wherein the defect parameter at least includes the number of downsampling times a, and the scaling ratio is based on 2 a get.

[0058] The defect prediction model is pre-trained using defect samples. Defect samples can be selected based on common defects, categorized by size, and then input into the defect prediction model for identification and prediction. Based on the annotation parameters of each defect sample, a defect model is output, and the defect size and sampling density are determined based on the defect model. The annotation parameters for the defect samples can be determined based on the technician's experience and then input. Of course, during the actual inspection process, the defect size and sampling density can also be provided by the technician based on their experience.

[0059] The downsampling times a can be set according to the requirements of the defect detection algorithm used. For example, when a neural network algorithm is used, the downsampling times a is set to 2. The larger the downsampling times used in the downsampling process, the smaller the image resolution will be. In order to avoid the initial image from shrinking too quickly during the scaling process, in some embodiments of the present application, the initial image is downsampled by a multiple of 2. When the downsampling times is 2, if the downsampling times is 1, the initial image is reduced to 1 / 2 of the original size; if the downsampling times is 2, the initial image is reduced to 1 / 4 of the original size; if the downsampling times is a, the initial image is reduced to 1 / 2 of the original size. a Therefore, in order to avoid losing pixels in the defective part during the scaling process, the image scaling ratio is based on 2 a get.

[0060] In some practical applications, the defect parameters also include defect size and sampling density. Specifically, the scaling ratio for the initial image can be calculated using the following formula (1).

[0061] Ratio=2 a *SD / FS (1)

[0062] In formula (1), Ratio represents the scaling ratio, a represents the number of downsampling times, SD represents the sampling density, and FS represents the defect size.

[0063] Initial image resolution FS SD Ratio Processing image resolution Size scaling factor 5472*3648 40 3 0.15 820.8*547.2 44 times 5472*3648 40 6 0.3 1641.6*1094.4 11 times 5472*3648 60 3 0.1 547.2*364.8 100 times 5472*3648 60 6 0.2 1094.4*729.6 25 times 5472*3648 80 3 0.075 410.4*273.6 177 times

[0064] Table 1 Scaling ratio of the initial image by the processing method of the embodiment of the present application under different defect parameter conditions

[0065] Table 1 shows the scaling ratio of the initial image after scaling by the processing method of the embodiment of the present application under different defect parameter conditions, where the number of downsampling times a is 1. For example, as shown in the first row of Table 1, the resolution of the initial image is 5472*3648, the defect size FS in the defect parameter is 40 pixels, and the sampling density SD is selected as sampling once every 3 pixels. Then, the Ratio calculated by formula (1) is 0.15, that is, the length and width of the initial image are reduced to 0.15 times the size of the initial image. Thus, the scaling ratio of the initial image can reach 44 times. According to the experience of technicians, the sampling density SD is 3 or 6, which can ensure that the pixels of the defect part will not be lost too much after scaling, so as to ensure that the accuracy of the subsequent defect monitoring algorithm will not decrease. It can be seen from Table 1 that when the sampling density SD is 3 or 6, the impact on the scaling ratio will be relatively small when the defect size FS is small; and when the defect size FS is large, for example, 80 pixels, the impact on the scaling ratio will be relatively large. Therefore, through the processing method of this application, the initial image is scaled according to the defect parameters to reduce the image resolution and improve the algorithm speed. Moreover, the larger the defect size FS of the defect portion, the higher the scaling factor, the smaller the resolution of the processed image obtained, and the greater the algorithm speed improvement.

[0066] Figure 2 and Figure 3 A schematic diagram shows the scaling effect of defect parts of different defect sizes after being processed by the processing method of an embodiment of the present application. Figure 2 The detection element 1 of the defective part is a screw foreign body. The size of the screw itself is small, and the defect size will also be small. The zoom ratio of the processed image obtained by the processing method of the embodiment of the present application will also be small, thereby improving the processing efficiency of the algorithm while ensuring that the recognition accuracy of the algorithm is not affected. Figure 3 The detection element 2 of the defective part is foam, and the size of the foam is relatively large, that is, the pixels occupied by the defect size are large enough, and the scaling factor of the processing method of the embodiment of the present application will also be large, thereby ensuring that the recognition accuracy of the algorithm is not affected while greatly improving the processing efficiency of the algorithm.

[0067] In the processing method of this embodiment, the initial image of the device to be inspected is scaled according to the defect parameters, which greatly reduces the initial image and can significantly improve the speed of the algorithm. At the same time, it ensures that the pixels of the defective part meet the accuracy of the algorithm without causing misjudgment due to unclear defective parts of the image.

[0068] In some embodiments, before obtaining the processed image, the process further includes step S100 of obtaining an initial image of the device to be detected. That is, the image of the device to be detected is captured by a capture device to obtain the initial image. This step can be implemented as the following steps S110-S120:

[0069] S110, identifying the device to be detected using a device recognition model to determine a frame position of the device to be detected;

[0070] S120: Adjust shooting parameters of a capture device according to the position of the frame to obtain an initial image of the device to be detected within a range corresponding to the shooting parameters, wherein the capture device is used to capture the image of the device to be detected.

[0071] In this application, the device recognition model is pre-trained using device samples. Device samples can be selected based on the device to be tested. For example, when a mobile phone is required for defect detection, a mobile phone can be used as a device sample. The device recognition model is trained by inputting a certain number of device samples for identification and prediction. The model then classifies each device sample based on its shape and size, outputting a corresponding bounding box model. When the output bounding box model meets a pre-set accuracy, the device recognition model training is complete.

[0072] In this embodiment, the device recognition model identifies the features of the device to be detected in the field of view, and compares it with the established device border model based on the identified features such as shape or size. When a matching border model is determined, the border position of the device to be detected is determined based on the border model.

[0073] In this embodiment, the acquisition device can be a device such as a camera that can obtain an image of the device to be detected, and this application does not limit this. After determining the frame position of the device to be detected, the shooting parameters of the acquisition device are adjusted according to the frame position. Taking a camera as an example, during the shooting process, after obtaining the frame position of the device to be detected, the distance between the camera and the device to be detected is adjusted so that the frame of the device to be detected fills the field of view of the camera, and then the aperture size, shutter, exposure compensation, sensitivity and other parameters are adjusted accordingly according to the distance of the camera. After the shooting parameters are adjusted, the shooting is performed to obtain the initial image of the device to be detected.

[0074] In this embodiment, the shooting parameters of the acquisition device are automatically adjusted according to the position of the frame of the device to be detected, so that the area surrounded by the frame of the device to be detected fills the image area as much as possible, so that the part of the image that needs to be detected for defects can also be maximized, and the clarity is high, which is convenient for improving the accuracy of analysis and detection through algorithms. At the same time, there is no need for manual frequent adjustment of the shooting parameters, and the operation is convenient.

[0075] In some embodiments, when the acquisition device does not store the initial image of the device to be inspected, scaling the initial image according to the defect parameters corresponding to the initial image includes the following steps S210-S220:

[0076] S210, determining a target resolution according to defect parameters corresponding to the previewed initial image;

[0077] S220 , adjusting shooting parameters of the acquisition device according to the target resolution to achieve scaling processing of the previewed initial image, wherein the acquisition device is used to acquire the image of the device to be detected.

[0078] This embodiment aims to directly capture a processed image of a suitable resolution size of the device to be detected by an acquisition device. Taking a camera as an example, during the shooting process, the distance between the camera and the device to be detected can be adjusted according to the method involved in steps S110-S120, so that the frame of the device to be detected fills the field of view of the camera, and then the aperture size, shutter, exposure compensation, sensitivity and other parameters are adjusted accordingly according to the distance of the camera to obtain a first preview image of the initial image of the device to be detected. The target resolution is then calculated based on the defect parameters corresponding to the first preview image of the initial image. The defect parameters can be generated using a defect prediction model, or they can be directly input based on the experience of the technician to obtain the defect parameters. The above formula (1) is then used to calculate the zoom ratio, and the target resolution is calculated based on the original resolution of the initial image and the zoom ratio.

[0079] After determining the target resolution, adjust the distance between the camera and the device to be detected again according to the target resolution so that the border of the device to be detected fills the camera's field of view. Then adjust the aperture size, shutter, exposure compensation, sensitivity and other parameters accordingly according to the adjusted distance of the camera. After the adjustment is completed, take a picture to obtain a scaled image of the device to be detected.

[0080] When the acquisition device stores the initial image, the scaling process of the initial image according to the defect parameters corresponding to the initial image includes the following steps S230-S240:

[0081] S230, determining a scaling parameter according to the defect parameter corresponding to the stored initial image;

[0082] S240: Adjust the stored initial image according to the scaling parameter to implement scaling processing on the initial image.

[0083] In this embodiment, after the camera stores the initial image of the device to be inspected, the initial image is scaled according to the corresponding defect parameters to obtain a processed image with appropriate resolution. The scaling parameters are first determined based on the defect parameters corresponding to the initial image. The defect parameters can be generated using a defect prediction model or directly input based on the experience of technicians. After the defect parameters are determined, the scaling ratio can be determined using the above formula (1). The length and width of the initial image are scaled according to the scaling ratio to reduce the initial image, thereby achieving scaling processing of the initial image.

[0084] S300 , performing a second processing on the processed image to generate a target image, and analyzing the target image in a target defect detection manner to achieve defect detection on the device to be inspected.

[0085] In this step, after obtaining a processed image with an appropriate resolution and size, the processed image is subjected to a second processing step. The processed image targeted by this embodiment typically contains multiple target detection elements. However, when using some detection algorithms in the prior art to identify target detection elements, it is possible that the positions of some detection elements may not be accurately identified, or some detection elements may not be identified, resulting in omission of some detection elements. Consequently, during subsequent defect detection, these inaccurate or missed detection elements may result in inaccurate defect detection results.

[0086] In an embodiment of the present application, the second processing may include matching the position of the detected target detection element using the first constraint model to further calibrate the position of the target detection element to improve the accuracy of the algorithm for analyzing and processing the image at the detection element during subsequent defect detection. When matching the detection element of the processed image using the first constraint model, if the position of the labeled detection element appears, it can be determined that there is an unrecognized target detection element. In this case, a fill-in operation can be performed on the unrecognized target detection element.

[0087] Furthermore, continuing with the camera example, during the shooting process, because the placement angle of the device to be detected cannot be consistent with the placement angle of the sample used in the algorithm detection, there will be a certain angle deviation between the initial image and the sample image. However, the angle of the processed image has not changed and is the same as the angle of the initial image. To facilitate the adaptation of subsequent algorithms and improve recognition accuracy, a second processing can be performed on the angle of the processed image using a second constraint model based on the sample image to calibrate the angle of the processed image.

[0088] The processing method of the embodiment of the present application is easy to operate and can conveniently scale the initial image according to the defect parameters to obtain a suitable processed image, thereby improving the recognition speed of the algorithm while avoiding reducing the accuracy of the algorithm. The processed image is further subjected to a second processing to generate a target image to calibrate the processed image, which can significantly improve the recognition efficiency of the subsequent algorithm.

[0089] In some embodiments, the above step S300 may be implemented as the following steps S311-S312:

[0090] S311, performing component recognition on the processed image to determine positions of at least two target detection components;

[0091] S312: Perform endpoint matching on the positions of the at least two target detection elements using a first constraint model to determine the positions of all target detection elements on the processed image.

[0092] This embodiment aims to further calibrate the position of the target detection element using a first constraint model. For a detection device, the position of the target detection element to be detected is determined, and technicians can establish one or more geometric constraint models based on the position of the target detection element. In this embodiment, the first constraint model is one or more geometric constraint models that can include all target detection elements.

[0093] When performing component recognition on the processed image, calculations can be performed using existing component detection algorithms based on the identified characteristic elements to determine the positions of at least two target detection components on the processed image. Endpoint matching is then performed on the positions of the at least two target detection components using the endpoints of the first constraint model to determine the positions of all target detection components on the processed image.

[0094] In some embodiments, performing endpoint matching on the positions of the at least two target detection elements using the first constraint model includes:

[0095] Matching is performed based on the positions of the at least two target detection elements and on the endpoints of the first constraint model, and the angles and side lengths corresponding to the endpoints;

[0096] If the positions of the at least two target detection elements conform to the relationship among the endpoints, angles, and side lengths, the processed image conforms to the first constraint model.

[0097] In this embodiment, based on the geometric shape of the first constraint model, the angles corresponding to the endpoints of the first constraint model and the side lengths between the endpoints are determined. The positions of the at least two target detection elements are then matched based on the endpoints and the corresponding angle and side length features. If the positions of the at least two target detection elements match the corresponding endpoints, angles, and side length features, the positions of the at least two target detection elements are accurate, and the processed image conforms to the first constraint model.

[0098] During the matching process, based on the endpoints and the corresponding angle and side length features, if the position of the target detection element deviates from the corresponding endpoint of the first constraint model, the position of the target detection element is adjusted to the corresponding endpoint, thereby further calibrating the position of the target detection element on the processed image, so that the processed image conforms to the first constraint model, thereby improving the accuracy of subsequent defect detection.

[0099] In some embodiments, when the processed image conforms to the first constraint model, performing endpoint matching on the positions of the at least two target detection elements using the first constraint model to determine the positions of all target detection elements on the processed image includes:

[0100] If there are endpoints in the first constraint model that cannot be matched with the at least two target detection elements, determining that there are unrecognized target detection elements in the processed image;

[0101] A padding operation is performed on the unrecognized target detection elements according to the first constraint model to determine positions of all the target detection elements on the processed image.

[0102] In this embodiment, the first constraint model includes the positions of all components to be inspected. Therefore, if the processed image conforms to the first constraint model, based on the endpoints and corresponding angle and side length features, if the first constraint model contains endpoints that cannot be matched with the at least two target detection components, it can be determined that the processed image contains target detection components that have not been identified by the detection algorithm for the detection components. Based on the endpoints that cannot be matched with the at least two target detection components in the first constraint model, target detection components are added at the corresponding positions in the processed image. This marks the missed target detection components on the processed image to complete the target detection components, thereby determining the positions of all target detection components on the processed image and improving the accuracy of the defect detection algorithm.

[0103] For example, please refer to Figure 4As shown in , the first constraint model includes a triangle. When corresponding to the triangle formed by target detection elements 4, 5, and 6 in the figure, if the positions of target detection elements 4, 5, and 6 in the processed image do not match the endpoints of the triangle in the first constraint model and there is some deviation, the target detection elements with deviations can be adjusted to the corresponding endpoints based on the position of the triangle to calibrate the positions of the target detection elements. It is understandable that the first constraint model can also include geometric constraint models established based on target detection elements in different positions, or geometric constraint models of other shapes, to calibrate the positions of all target detection elements.

[0104] Furthermore, during the matching process, if the target detection element 6 corresponding to one endpoint of the triangle in the first constraint model is not marked on the processed image, it is determined that there is a target detection element in the processed image that has not been identified by the detection algorithm for the detection element. The target detection element is then added based on the position on the processed image corresponding to the triangle endpoint to mark the unidentified target detection element, thereby determining the positions of all target detection elements on the processed image to improve the accuracy of the defect detection algorithm. It is understood that further detection and judgment of unidentified target detection elements can be performed using other geometric constraint models of the first constraint model to obtain the positions of all target detection elements.

[0105] In some embodiments, the above step S300 may be implemented as the following steps S321 - S322 .

[0106] S321, performing component recognition on the processed image, and determining positions of at least three target detection components on the processed image;

[0107] S322 : Rotate the angle of the corresponding processed image based on the second constraint model according to the positions of the at least three target detection elements to adjust the angle of the processed image.

[0108] This embodiment aims to adjust the angle of the processed image based on the second constraint model so that the angle of the processed image is consistent with the second constraint model. When the processed image is subjected to element recognition, calculations can be performed based on some of the identified characteristic elements using some of the current detection element detection algorithms, thereby determining the positions of at least three target detection elements on the processed image. Under the condition that technicians can establish one or more geometric constraint models based on the positions of the target detection elements, the second constraint model can determine a corresponding geometric constraint model based on the positions of the at least three target detection elements, so that the second constraint model can be adapted to the geometric shape composed of the positions of the at least three target detection elements. Parameter information of the positions of the at least three target detection elements on the processed image is obtained, and calculations are performed based on the parameter information of the corresponding endpoints of the at least three target detection elements in the second constraint model to obtain the first image rotation matrix of the processed image, and the processed image is rotated according to the first image rotation matrix, thereby adjusting the angle of the processed image.

[0109] In some embodiments, due to accuracy issues in the detection element algorithm, the positions of at least three target detection elements in the processed image may have some errors, resulting in the adjusted processed image angle still deviating to a greater or lesser extent from the second constraint model. Therefore, to further calibrate the rotated processed image, step S300 further includes the following steps S323-S325.

[0110] S323, comparing the positions of the at least three target detection elements in the at least two processed images after adjustment with the second constraint model, and respectively obtaining deviation angles between the at least two processed images and the second constraint model;

[0111] S324, calculating a mean value of the deviation angles according to the respective deviation angles, and setting a corresponding weight for the corresponding processed image according to the magnitude of the respective deviation angles;

[0112] S325 , rotating each of the adjusted processed images again according to the deviation angle mean and the weight to generate the target image.

[0113] This embodiment aims to further calibrate the angle of the rotated processed image.

[0114] When there is only one rotated processed image, parameter information of the positions of the at least three target detection elements in the adjusted processed image is obtained, and the parameter information is compared with the parameter information of each endpoint of the second constraint model to obtain the deviation angle between the adjusted processed image and the second constraint model; a corresponding weight is taken according to the size of the deviation angle, and this weight can be determined based on the experience of the technician, so that when the second image rotation matrix of the adjusted processed image is calculated, the adjusted processed image is rotated again according to the second image rotation matrix and the weight to generate the target image.

[0115] When the adjusted processed images include at least two images, the position parameter information of the target detection elements in each of the two processed images is obtained respectively, and the parameter information is compared with the parameter information of each endpoint of the second constraint model to obtain at least two deviation angles between each of the at least two processed images and the second constraint model. The mean of the deviation angles is calculated based on the at least two deviation angles. At the same time, corresponding weights are set for the corresponding processed images based on the size of the at least two deviation angles. This weight can be determined based on the difference between the deviation angle and the mean of the deviation angle, or based on the experience of the technician. The third image rotation matrix of the adjusted processed image is then calculated based on the mean of the deviation angles. The adjusted processed image is rotated again based on the third image rotation matrix and the corresponding weights to generate the target image.

[0116] For example, refer to Figure 5 As shown in , when there are adjusted processed images including at least processed image 7 and processed image 8, the position parameter information of the target detection elements in processed image 7 and processed image 8 is obtained respectively, and the obtained position parameter information is compared with the parameter information of each endpoint of the second constraint model of the sample image to obtain two deviation angles of processed image 7 and processed image 8 from the second constraint model respectively. The deviation angle mean is calculated based on the two deviation angles, and corresponding weights are set for the corresponding processed images based on the magnitude of at least two deviation angles. For example, if the deviation angle of processed image 7 is larger than that of processed image 8, the weight corresponding to processed image 7 can be set to 0.6, and the weight corresponding to processed image 8 can be set to 0.3. Then, the third image rotation matrix of the adjusted processed image 7 and processed image 8 is calculated based on the deviation angle mean, and the adjusted processed image 7 and processed image 8 are rotated again based on the third image rotation matrix and the corresponding weights to generate target image 9 and target image 10.

[0117] Based on the same inventive concept, the present application also provides a processing device, Figure 6 FIG. 1 is a block diagram of a processing device according to an embodiment of the present application. Figure 6 As shown, the display device includes:

[0118] An obtaining module 10 is configured to obtain a processed image of the device to be inspected, wherein the processed image is obtained by performing a first processing on an initial image of the device to be inspected, wherein the first processing at least includes scaling the initial image according to defect parameters corresponding to the initial image;

[0119] The calibration module 20 performs a second processing on the processed image to generate a target image, and analyzes the target image in a target defect detection manner to achieve defect detection of the device to be detected.

[0120] The processing device in the embodiment of the present application can implement the processing method provided in any embodiment of the present application through the functional modules configured therein.

[0121] Furthermore, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent devices, modifications, omissions, combinations (e.g., solutions that intersect various embodiments), adaptations, or changes. The devices in the claims are to be interpreted broadly based on the language employed in the claims and are not limited to the examples described in this specification or during the prosecution of this application, which examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, with the true scope and spirit being indicated by the following claims and the full scope of their equivalents.

[0122] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of their solutions) can be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a specific disclosed embodiment. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of this application should be determined with reference to the appended claims and the full scope of equivalents to which these claims are entitled.

[0123] The above describes in detail several embodiments of the present application, but the present application is not limited to these specific embodiments. Those skilled in the art can make various variations and modifications to the embodiments based on the concept of the present application, and these variations and modifications should all fall within the scope of protection claimed by the present application.

Claims

1. A processing method, wherein: include: Obtaining a processed image of the device to be inspected, wherein the processed image is obtained by performing a first processing on an initial image of the device to be inspected, the first processing at least comprising scaling the initial image according to defect parameters corresponding to the initial image; Performing a second processing on the processed image to generate a target image, and analyzing the target image using a target defect detection method to achieve defect detection of the device to be inspected; The acquisition device does not store the initial image of the device to be inspected, and the scaling process of the initial image according to the defect parameters corresponding to the initial image includes: determining a target resolution according to defect parameters corresponding to the previewed initial image; Adjusting the shooting parameters of the acquisition device according to the target resolution to achieve scaling processing of the previewed initial image, wherein the acquisition device is used to acquire the image of the device to be detected; Alternatively, the acquisition device stores the initial image, and the scaling process is performed on the initial image according to the defect parameter corresponding to the initial image, including: determining a scaling parameter based on a defect parameter corresponding to the stored initial image; The stored initial image is adjusted according to the scaling parameter to implement scaling processing on the initial image.

2. The method according to claim 1, further comprising: Obtaining the initial image includes: Identify the device to be detected using a device recognition model to determine a frame position of the device to be detected; The shooting parameters of the acquisition device are adjusted according to the position of the frame to obtain an initial image of the device to be detected within a range corresponding to the shooting parameters, wherein the acquisition device is used to acquire the image of the device to be detected.

3. The method according to claim 1, wherein The performing a second processing on the processed image to generate a target image includes: performing component recognition on the processed image to determine positions of at least two target detection components; Endpoint matching is performed on the positions of the at least two target detection elements using a first constraint model to determine the positions of all target detection elements on the processed image.

4. The method according to claim 3, wherein: The performing endpoint matching on the positions of the at least two target detection elements using the first constraint model includes: Matching is performed based on the positions of the at least two target detection elements and on the endpoints of the first constraint model, and the angles and side lengths corresponding to the endpoints; If the positions of the at least two target detection elements conform to the relationship among the endpoints, angles, and side lengths, the processed image conforms to the first constraint model.

5. The method according to claim 4, wherein, when the processed image conforms to the first constraint model, performing endpoint matching on the positions of the at least two target detection elements using the first constraint model to determine the positions of all target detection elements on the processed image comprises: If there are endpoints in the first constraint model that cannot be matched with the at least two target detection elements, determining that there are unrecognized target detection elements in the processed image; A padding operation is performed on the unrecognized target detection elements according to the first constraint model to determine positions of all the target detection elements on the processed image.

6. The method according to claim 1, wherein The performing a second processing on the processed image to generate the target image includes: performing component recognition on the processed image, and determining positions of at least three target detection components on the processed image; According to the positions of the at least three target detection elements, the angle of the corresponding processed image is rotated based on the second constraint model to adjust the angle of the processed image.

7. The method according to claim 6, wherein: The performing a second processing on the processed image to generate the target image further includes: comparing the positions of the at least three target detection elements in the at least two processed images after adjustment with the second constraint model, and respectively obtaining deviation angles between the at least two processed images and the second constraint model; Calculating a mean value of the deviation angles according to the respective deviation angles, and setting a corresponding weight for the corresponding processed image according to the magnitude of the respective deviation angles; Each of the adjusted processed images is rotated again according to the deviation angle mean and the weight to generate the target image.

8. The method according to any one of claims 1 to 7, wherein: The scaling process of the initial image according to the defect parameters corresponding to the initial image includes: generating the defect parameters using a defect prediction model according to the annotation parameters corresponding to the initial image; Determine a scaling ratio according to the defect parameter to scale the initial image according to the scaling ratio, wherein the defect parameter at least includes the number of downsampling times a, and the scaling ratio is based on 2 a get.

9. A processing device, wherein: include: an obtaining module, configured to obtain a processed image of the device to be inspected, wherein the processed image is obtained by performing a first processing on an initial image of the device to be inspected, the first processing at least comprising scaling the initial image according to defect parameters corresponding to the initial image; a calibration module, performing a second processing on the processed image to generate a target image, and analyzing the target image by a target defect detection method to achieve defect detection of the device to be detected; The acquisition device does not store the initial image of the device to be inspected, and the scaling process of the initial image according to the defect parameters corresponding to the initial image includes: determining a target resolution according to defect parameters corresponding to the previewed initial image; Adjusting the shooting parameters of the acquisition device according to the target resolution to achieve scaling processing of the previewed initial image, wherein the acquisition device is used to acquire the image of the device to be detected; Alternatively, the acquisition device stores the initial image, and the scaling process is performed on the initial image according to the defect parameter corresponding to the initial image, including: determining a scaling parameter based on a defect parameter corresponding to the stored initial image; The stored initial image is adjusted according to the scaling parameter to implement scaling processing on the initial image.

Images