Defect detection methods, devices and systems

Abstract

This application discloses a defect detection method, apparatus, and system. The method includes: obtaining a first image, wherein the first image is an image acquired by an image acquisition device when an output device is in a first posture; obtaining a second image based on the first image, wherein the second image is an image corresponding to a second posture of the output device; wherein the display of a target defect on the image output area of ​​the output device differs between the first image corresponding to the first posture and the second image corresponding to the second posture; and obtaining a defect detection result on the image output area of ​​the output device based on the display of the target defect in the second image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a defect detection method, apparatus and system. Background Technology

[0002] Currently, image detection is performed using side-view images of the display screen to detect the location of defects in the display screen.

[0003] However, currently, when taking side-view photos of a display screen, defect detection can only be performed from a fixed angle, which may miss defects that can only be detected from other angles, thus causing inaccurate defect detection. Summary of the Invention

[0004] In view of the above, this application provides a defect detection method, apparatus, and system, as follows:

[0005] A defect detection method, comprising:

[0006] Obtain a first image, which is an image acquired by the image acquisition device when the output device is in a first posture;

[0007] A second image is obtained based on the first image, wherein the second image is the image corresponding to the output device being in a second posture.

[0008] Wherein, in the first image corresponding to the output device in the first posture and in the second image corresponding to the second posture, the display of the target defect on the image output area of ​​the output device is different;

[0009] The defect detection result on the image output area of ​​the output device is obtained based on the display of the target defect in the second image.

[0010] In the above method, preferably, the second pose corresponds to the target type of the target defect, so that the defect detection result obtained based on the second image can characterize whether there is a defect corresponding to the target type on the image output area.

[0011] In the above method, preferably, the visibility of the target defect corresponding to the target type on the image output area in the first image is less than the visibility of the target defect corresponding to the target type on the image output area in the second image.

[0012] The above method, preferably, involves obtaining a second image based on the first image, including:

[0013] Using an image processing model, the first image is processed to obtain a second image corresponding to the output device being in at least one second pose;

[0014] The image processing model is trained based on input samples and output samples. The input samples include sample images corresponding to the output device in the first pose, and the output samples include sample images corresponding to the output device in at least one second pose.

[0015] The above method, preferably, involves obtaining a second image based on the first image, including:

[0016] Using an image processing model, the first image is processed according to the second pose to obtain a second image corresponding to the output device being in the second pose;

[0017] The image processing model is trained based on input samples and output samples. The input samples include: the sample image corresponding to the output device in the first pose and the pose parameters of the target pose. The output samples include: the sample image corresponding to the output device in the target pose.

[0018] In the above method, preferably, the target pose has one or more, so that the image processing model can output the image corresponding to the output device being in one or more second poses.

[0019] The above method, preferably, involves obtaining a second image based on the first image, including:

[0020] Using the image processing model corresponding to the second pose, the first image is processed to obtain the second image corresponding to the output device being in the second pose;

[0021] The image processing model is trained based on input samples and output samples. The input samples include sample images corresponding to the output device in the first pose, and the output samples include sample images corresponding to the output device in the second pose.

[0022] In the above method, preferably, both the first posture and the second posture are represented by the angle between the plane where the image output area is located and the image acquisition direction of the image acquisition device.

[0023] A defect detection device, comprising:

[0024] The first acquisition unit is used to acquire a first image, wherein the first image is an image acquired by the image acquisition device when the output device is in a first posture;

[0025] The second obtaining unit is configured to obtain a second image based on the first image, wherein the second image is an image corresponding to the output device being in a second posture.

[0026] Wherein, in the first image corresponding to the output device in the first posture and in the second image corresponding to the second posture, the display of the target defect on the image output area of ​​the output device is different;

[0027] A region detection unit is used to obtain defect detection results on the image output region of the output device based on the display of the target defect in the second image.

[0028] A defect detection system, comprising:

[0029] Image acquisition equipment, used to acquire images from output devices;

[0030] An electronic device is configured to: acquire a first image, wherein the first image is an image acquired by the image acquisition device when the output device is in a first posture; acquire a second image based on the first image, wherein the second image is an image corresponding to the output device in a second posture; wherein the display of a target defect on the image output area of ​​the output device differs between the first image corresponding to the output device in the first posture and the second image corresponding to the output device in the second posture; and acquire a defect detection result on the image output area of ​​the output device based on the display of the target defect in the second image.

[0031] As can be seen from the above technical solutions, in the defect detection method, apparatus, and system disclosed in this application, after acquiring a first image, a second image is obtained by processing the first image. The visibility of the target type defect detected in the second image by the defect detection method of this application is greater than that of the target type defect in the first image. Therefore, in the embodiments of this application, a first image at a specific angle is acquired by an image acquisition device, and then processed to obtain a second image at another angle. Based on the different second images displaying defects, defect detection can be performed at corresponding angles, thereby improving the accuracy of defect detection. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A flowchart of a defect detection method provided in Embodiment 1 of this application;

[0034] Figure 2a This is a schematic diagram of the image acquisition and output device of the image acquisition device in the embodiments of this application;

[0035] Figure 2b This is a schematic diagram illustrating the application scenario of defect detection for output devices in this application;

[0036] Figure 3 This is a schematic diagram showing the visibility of the target defect in the first and second images;

[0037] Figure 4 This is an application illustration of the image processing model determining at least one second image in the embodiments of this application;

[0038] Figure 5 This is an example diagram illustrating the training of the first image processing model in an embodiment of this application;

[0039] Figure 6 This is a schematic diagram illustrating the application of the image processing model in this application to determine the second image according to the second pose.

[0040] Figure 7 This is an example diagram illustrating the training of the second image processing model in an embodiment of this application;

[0041] Figure 8 This is a schematic diagram illustrating the application of the image processing model in determining the second image in the embodiments of this application;

[0042] Figure 9 This is an example diagram illustrating the training of the third image processing model in an embodiment of this application;

[0043] Figure 10 This is a schematic diagram of the structure of a defect detection device provided in Embodiment 2 of this application;

[0044] Figure 11 This is a schematic diagram of the structure of a defect detection system provided in Embodiment 3 of this application;

[0045] Figure 12 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application;

[0046] Figures 13-14 These are example diagrams illustrating the training of the image processing model according to embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] refer to Figure 1 The diagram shown is a flowchart illustrating the implementation of a defect detection method according to Embodiment 1 of this application. This method is applicable to electronic devices capable of image processing, and these devices can be configured within a defect detection system. The technical solution in this embodiment is primarily used to detect screen defects in a display screen, thereby improving the accuracy of defect detection.

[0049] Specifically, the method in this embodiment may include the following steps:

[0050] Step 101: Obtain the first image.

[0051] The first image is the image captured by the image acquisition device when the output device is in a first posture. The image acquisition device can be an industrial camera or webcam, etc., to capture the image on the screen of the output device. Since the image acquisition device has a specific acquisition direction, and the output device is in a first posture relative to the image acquisition device, the image acquisition device can capture the image output area of ​​the output device in the first posture. The first posture is represented by the angle between the plane containing the image output area and the image acquisition direction of the image acquisition device. In practical applications, this angle is usually not 90 degrees. Figure 2a As shown.

[0052] Specifically, in this embodiment, when the image acquisition device performs defect detection on the output devices before they leave the factory, it acquires images of the output devices that are sequentially conveyed on the inspection production line, thereby acquiring a first image of the output device in a first posture, such as... Figure 2b .

[0053] It should be noted that the first posture is represented by the angle between the plane where the image output area is located and the image acquisition direction of the image acquisition device, such as a 45-degree angle.

[0054] Output devices can be screens, monitors, tablets, televisions, or mobile phones, or other devices with screens.

[0055] Step S102: Obtain the second image based on the first image.

[0056] The second image is the image corresponding to the output device in the second posture. The second image does not need to be obtained by the image acquisition device in the second posture; it can be obtained directly by processing the first image, so as to obtain images from other angles based on the image captured by the image acquisition device at a fixed angle.

[0057] The second pose here is represented by the angle between the plane where the image output area is located and the image acquisition direction of the image acquisition device, such as a 60-degree angle or a 75-degree angle.

[0058] In this embodiment, the first pose and the second pose are different. The difference between the first pose and the second pose can be that the angle between the plane representing the image output area and the image acquisition direction of the image acquisition device, as represented by the first pose, is different from the angle between the plane representing the image output area and the image acquisition direction of the image acquisition device, as represented by the second pose. For example, the angle represented by the first pose is 45 degrees, while the angle represented by the second pose is 60 degrees.

[0059] Specifically, the display of the target defect on the image output area of ​​the output device differs between the first image corresponding to the output device in the first posture and the second image corresponding to the output device in the second posture. In other words, the state of the target defect on the image output area of ​​the output device in the first image, as perceived by the human eye or defect detection algorithm, is different from the state of the target defect on the image output area of ​​the output device in the second image, as perceived by the human eye or defect detection algorithm.

[0060] Since the defects visible in images taken from different angles vary, some defect types can only be detected by images taken from a specific angle. Therefore, for target defects that cannot be detected by the human eye or defect detection algorithms, i.e., the visibility of the target defect in the first image is low, it is necessary to process them to make the target defect in the obtained second image more easily detectable by the human eye or detection algorithms. In other words, the visibility of the target type corresponding to the target defect in the image output area of ​​the first image is less than the visibility of the target type corresponding to the target defect in the image output area of ​​the second image.

[0061] For example: Figure 3 As shown, the visibility of defect r in the first image w1 when it is in the first posture at a 45-degree angle cannot be detected by the human eye or by the defect detection algorithm. Therefore, it is necessary to determine the second posture at a 60-degree angle that can detect defect r, and process the first image to obtain the second image w2 when it is in the second posture at a 60-degree angle. At this time, the visibility of defect r in the second image w2 is greater than the visibility of defect r in the first image w1.

[0062] Step S103: Obtain the defect detection result on the image output area of ​​the output device based on the display of the target defect in the second image.

[0063] Specifically, in this embodiment, a defect detection algorithm can be used to determine whether there is a defect in the second image, thereby obtaining a defect detection result that ensures whether the image output area contains the target defect.

[0064] As can be seen from the above scheme, in the defect detection method provided in Embodiment 1 of this application, after acquiring the first image, the first image is processed to obtain the second image. The visibility of the target type defect detected in the second image by the defect detection method of this application is greater than that of the target type defect in the first image. Therefore, in this embodiment of the application, by acquiring a first image from a specific angle using an image acquisition device and processing it, a second image from another angle can be obtained. Based on the different second images displaying defects, defect detection can be performed at corresponding angles, thereby improving the accuracy of defect detection.

[0065] Furthermore, the second pose corresponds to the target type of the target defect, so that the defect detection result obtained based on the second image can characterize whether a defect of the target type exists in the image output area. In other words, since the defects displayed in images taken from different angles are different, some feature types of defects can only be detected through images taken from a specific angle. Therefore, it can be determined that the required second pose will differ depending on the defect type. Based on this, a mapping relationship between different defect types and different second poses is pre-created, so that the defect detection result corresponding to the second image obtained after processing the first pose and placing the output device in the second pose can characterize whether a defect of the target type exists in the image output area.

[0066] For example, target types include type A, type B, and type C. Each target type of defect corresponds to a second pose. For instance, a type A defect corresponds to a 60-degree angle between the plane containing the image output area and the image acquisition direction of the image acquisition device, so that the defect detection result obtained based on the second image corresponding to the 60-degree angle can characterize whether a defect corresponding to type A exists in the image output area; a type B defect corresponds to a 75-degree angle between the plane containing the image output area and the image acquisition direction of the image acquisition device, so that the defect detection result obtained based on the second image corresponding to the 75-degree angle can characterize whether a defect corresponding to type B exists in the image output area; and a type C defect corresponds to an 80-degree angle between the plane containing the image output area and the image acquisition direction of the image acquisition device, so that the defect detection result obtained based on the second image corresponding to the 80-degree angle can characterize whether a defect corresponding to type C exists in the image output area.

[0067] Specifically, in the defect detection process, the target defect to be detected is identified, such as dead pixels, leaks, or blue screens. Then, the second pose corresponding to the target type of the target defect is determined. The first image is then processed based on the second pose to determine the second image corresponding to the output device in the second pose. Thus, the defect detection result corresponding to the target defect in the obtained second image can clearly determine whether there is a defect of the target type on the image output area.

[0068] For example: Figure 3 As shown, the visibility of type A defect r in the first image w1 when it is in the first posture at a 45-degree angle cannot be detected by the human eye or defect detection algorithm. Therefore, it is necessary to determine the second posture at a 60-degree angle corresponding to the type of type A defect r, and process the first image to obtain the second image w2 when it is in the second posture at a 60-degree angle. At this time, the visibility of type A defect r in the second image w2 is greater than the visibility of type A defect r in the first image w1.

[0069] In the first implementation, step 102, when obtaining the second image based on the first image, can be achieved in the following way:

[0070] Using an image processing model, the first image is processed to obtain a second image corresponding to the output device being in at least one second pose.

[0071] The image processing model is trained based on input samples and output samples. The input samples include sample images corresponding to the output device in a first pose, and the output samples include sample images corresponding to the output device in at least one second pose.

[0072] For example, the first pose represents the angle between the plane containing the image output area of ​​the first image and the image acquisition direction of the image acquisition device, which is 45 degrees. The second pose represents the angle between the plane containing the image output area of ​​the first image and the image acquisition direction of the image acquisition device, which is 60 degrees, 70 degrees, and 80 degrees. Taking the image processing model shown above as an example, the image acquisition device captures the first image a1 at the first pose of 45 degrees. Then, the image processing model processes the input first image to obtain the second images b1, b2, and b3 corresponding to the output device at 60 degrees, 70 degrees, and 80 degrees, respectively. Figure 4 As shown.

[0073] In this embodiment, the process of training an image processing model based on input samples and output samples includes: acquiring sample images corresponding to each output device in a first pose and sample images corresponding to each output device in at least one second pose; then, the electronic device trains the input samples using its own deep learning model until the output results are completely consistent with the output samples, and determines that the currently trained deep learning model is the trained image processing model.

[0074] Specifically, a model is constructed based on a deep learning algorithm, and initial model parameters are configured. The initial model parameters are trained using input samples, and the output images corresponding to each second pose are generated. Then, a discriminator determines whether the output image corresponding to each second pose is consistent with the sample image in the output samples for that second pose. If any output image corresponding to a second pose is inconsistent with the sample image in the output samples for that second pose, it indicates that the deep learning model has not been trained successfully. At this point, based on the difference between the output result and the output sample, the initial model parameters are adjusted, and the process returns to continue training the adjusted initial model parameters using input samples. This process continues until it is determined that all output images corresponding to each second pose are consistent with the sample images in the output samples for that second pose. This indicates that the deep learning model has been trained successfully, and the currently trained deep learning model is considered a successfully trained image processing model. Figure 5 As shown.

[0075] It should be noted that the deep learning model algorithm shown in this embodiment can be an adversarial network (GAN) model algorithm, or other model algorithms.

[0076] For example: when the first posture is 45 degrees, the input sample is the sample image corresponding to the output device being in the first posture at 45 degrees; when the second posture is 60 degrees, 75 degrees and 80 degrees, the output sample is the sample image corresponding to the output device being in the second posture at 60 degrees, 75 degrees and 80 degrees respectively.

[0077] In this embodiment, the initial model parameters are trained using input samples, and output images corresponding to the second poses at angles of 60 degrees, 75 degrees, and 80 degrees are generated. Then, a discriminator determines whether the output image corresponding to the second pose at 60 degrees is completely identical to the sample image corresponding to the second pose at 60 degrees, the output image corresponding to the second pose at 75 degrees is completely identical to the sample image corresponding to the second pose at 75 degrees, and the output image corresponding to the second pose at 80 degrees is completely identical to the sample image corresponding to the second pose at 80 degrees. If it is determined that the output image corresponding to any angle of the second pose is inconsistent with the sample image corresponding to that angle, the initial model parameters are adjusted based on the difference between the output image and the sample image at that angle, and the process returns to continue training the adjusted initial model parameters using input samples. This process continues until the output image corresponding to the second pose at a 60-degree angle is completely identical to the sample image corresponding to the second pose at a 60-degree angle, the output image corresponding to the second pose at a 75-degree angle is completely identical to the sample image corresponding to the second pose at a 75-degree angle, and the output image corresponding to the second pose at a 80-degree angle is completely identical to the sample image corresponding to the second pose at a 80-degree angle. At this point, the deep learning model is considered to have completed training. The currently trained deep learning model is then used as the trained image processing model, enabling it to process the image corresponding to the first pose at a 45-degree angle to obtain the corresponding images for the second poses at 60-degree, 75-degree, and 80-degree angles.

[0078] As shown in the image processing model above, the image processing model trained using sample images corresponding to the first pose and at least one sample image corresponding to the second pose can process a first image under a fixed first pose and output one or more second images under the second pose. This allows for defect detection at corresponding angles by displaying different second images of defects, thereby improving the accuracy of defect detection.

[0079] In the second implementation, step 102, when obtaining the second image based on the first image, can be achieved in the following way:

[0080] Using an image processing model, the first image is processed according to the second pose to obtain the second image corresponding to the output device being in the second pose;

[0081] The image processing model is trained based on input samples and output samples. The input samples include: the sample image corresponding to the output device in the first pose and the pose parameters of the target pose. The output samples include: the sample image corresponding to the output device in the target pose.

[0082] For example, the first pose represents the angle between the plane containing the image output area of ​​the first image and the image acquisition direction of the image acquisition device, which is 45 degrees. The second pose represents the angle between the plane containing the image output area of ​​the first image and the image acquisition direction of the image acquisition device, which is 60 degrees. Taking the image processing model shown above as an example, the image acquisition device captures a first image c1 at a first pose of 45 degrees. Then, in the electronic device, the image processing model processes the input first image according to the second pose of 60 degrees to obtain the second image d1 corresponding to the output device being at the second pose of 60 degrees. Figure 6 As shown.

[0083] In this embodiment, the process of training an image processing model based on input and output samples includes: acquiring sample images and pose parameters of the target pose corresponding to each output device in a first pose, and sample images corresponding to the output device in the target pose; first, encoding the sample images to determine the image features corresponding to the sample images; then, combining the image features and pose parameters of the target pose; training the combined image features and pose parameters of the target pose using a deep learning model until the output result is completely consistent with the output sample, and determining the currently trained deep learning model as the trained image processing model.

[0084] Specifically, the process involves constructing a model based on a deep learning algorithm and configuring initial model parameters. Sample images are encoded to determine corresponding image features. These features are then combined with the target pose parameters. The initial model parameters are trained using the combined image features and target pose parameters, outputting the image corresponding to the target pose. A discriminator then determines whether the output image matches the sample image in the output samples. If they do not match, the deep learning model is not fully trained. Based on the difference between the output and the output samples, the initial model parameters are adjusted, and the process continues to train the adjusted initial model parameters using the combined image features and target pose parameters. This process is repeated until the output image matches the sample image in the output samples. At this point, the deep learning model is considered fully trained, and the currently trained deep learning model is considered a successfully trained image processing model. Figure 7 As shown.

[0085] It should be noted that the deep learning model algorithm shown in this embodiment can be an adversarial network (GAN) model algorithm, or other model algorithms.

[0086] For example: Figure 9 As shown, when the target pose is 60 degrees, the output sample is the sample image corresponding to the output device being at the target pose of 60 degrees. When the first pose is 45 degrees, the input sample is the sample image corresponding to the output device being at the first pose of 45 degrees and the pose parameters of the target pose of 60 degrees.

[0087] In this embodiment, the sample image corresponding to the first pose at a 45-degree angle is encoded to determine the image feature g corresponding to the sample image. Then, the image feature g is combined with the pose parameters of the target pose at a 60-degree angle. The initial model parameters are trained using the combined image feature g and the pose parameters of the target pose at a 60-degree angle, and the output image corresponding to the target pose at a 60-degree angle is output. Then, a discriminator determines whether the output image of the target pose at a 60-degree angle is completely identical to the sample image of the target pose at a 60-degree angle. If they are determined to be different, the initial model parameters are adjusted based on the difference between the output image of the target pose at a 60-degree angle and the sample image, and the process returns to continue training the initial model parameters using the combined image feature g and the pose parameters of the target pose at a 60-degree angle. This process continues until the output image of the target pose at 60 degrees is completely identical to the sample image corresponding to the target pose at 60 degrees. This indicates that the deep learning model has been trained successfully, and the currently trained deep learning model is determined to be the trained image processing model. This allows the trained image processing model to process the image corresponding to the first pose at 45 degrees, thereby obtaining the image of the target pose at 60 degrees.

[0088] As shown in the image processing model above, the image processing model, constructed using sample images corresponding to the first pose, the pose parameters of the target pose, and sample images of the output device in the target pose, can process the first image in a fixed first pose and output a second image in the target pose. This allows for defect detection at corresponding angles by displaying different second images of defects, thereby improving the accuracy of defect detection.

[0089] In the third implementation, step 102, when obtaining the second image based on the first image, can be achieved in the following way:

[0090] Using the image processing model corresponding to the second pose, the first image is processed to obtain the second image corresponding to the output device in the second pose;

[0091] The image processing model is trained based on input samples and output samples. The input samples include sample images corresponding to the output device in the first pose, and the output samples include sample images corresponding to the output device in the second pose.

[0092] It should be noted that there are multiple image processing models, with one image processing model corresponding to one second pose.

[0093] For example, the angle between the plane representing the image output area of ​​the first image in the first pose and the image acquisition direction of the image acquisition device is 45 degrees, and the angle between the plane representing the image output area of ​​the first image in the second pose and the image acquisition direction of the image acquisition device is 60 degrees. An image processing model U corresponding to the second pose of 60 degrees is found from N image processing models. Based on this, the image acquisition device captures a first image e1 at the first pose of 45 degrees. Then, the image processing model U is used in the electronic device to process the input first image e1 to obtain the second image f1 corresponding to the output device at 60 degrees. Figure 8 As shown, N is a positive integer greater than or equal to 1.

[0094] In this embodiment, the process of training an image processing model based on input samples and output samples includes: obtaining N input samples and output samples corresponding to each input sample; then, for each input sample, training the input sample using a deep learning model until the output result is completely consistent with the output sample corresponding to the input sample, determining that the currently trained deep learning model is the trained image processing model, and then determining N image processing models, each of which is used to output an image at a specific angle, i.e., a specific second pose.

[0095] Specifically, for each input sample, a model is constructed based on a deep learning algorithm, and initial model parameters are configured. The initial model parameters are trained using the input sample, outputting the image corresponding to the second pose. Then, a discriminator determines whether the output image corresponding to the second pose matches the sample image in the corresponding output sample of the input sample. If they do not match, the deep learning model is not fully trained. At this point, based on the difference between the output image and the corresponding output sample, the initial model parameters are adjusted, and the process returns to continue training the adjusted initial model parameters using the input sample. This process continues until the output image corresponding to the second pose matches the sample image in the corresponding output sample of the input sample. At this point, the deep learning model is considered fully trained, and the currently trained deep learning model is determined to be a successfully trained image processing model. This process then determines N trained image processing models, such as... Figure 9As shown.

[0096] It should be noted that the deep learning model algorithm shown in this embodiment can be an adversarial network (GAN) model algorithm, or other model algorithms.

[0097] For example, input sample x1 is the image corresponding to the output device in a first pose at a 45-degree angle, and input sample x2 is also the image corresponding to the output device in a first pose at a 45-degree angle. Output sample y1, corresponding to input sample x1, is the image corresponding to the output device in a second pose at a 60-degree angle, and output sample y2, corresponding to input sample x2, is the image corresponding to the output device in a second pose at a 75-degree angle. A deep learning model is trained on input samples x1 and x2 respectively. When the output result corresponding to input sample x1 matches the output sample y1, the currently trained deep learning model is determined to be a completed image processing model; similarly, when the output result corresponding to input sample x2 matches the output sample y2, the currently trained deep learning model is also determined to be a completed image processing model.

[0098] As shown in the image processing model above, N image processing models are constructed using N sample images corresponding to the first pose and sample images corresponding to the output device in the target pose for each input sample. The second pose is used to select the corresponding image processing model, and the selected model processes the acquired first image at a specific angle to obtain a second image in the second pose. This allows for defect detection at corresponding angles by displaying different second images of defects, thereby improving the accuracy of defect detection.

[0099] refer to Figure 10 This is a schematic diagram of a defect detection device provided in Embodiment 2 of this application. This device can be configured in an electronic device capable of image processing, which can be configured within a defect detection system. The technical solution in this embodiment is mainly used to detect screen defects in a display screen, thereby improving the accuracy of defect detection.

[0100] Specifically, the device in this embodiment may include the following units:

[0101] The first acquisition unit 1001 is used to acquire a first image, which is an image acquired by the image acquisition device when the output device is in a first posture;

[0102] The second obtaining unit 1002 is used to obtain a second image based on the first image, wherein the second image is the image corresponding to the output device being in a second posture.

[0103] Among them, the display of the target defect on the image output area of ​​the output device is different in the first image corresponding to the first posture and the second image corresponding to the second posture.

[0104] The region detection unit 1003 is used to obtain the defect detection result on the image output area of ​​the output device based on the display of the target defect in the second image.

[0105] As can be seen from the above scheme, in the defect detection device provided in Embodiment 2 of this application, after acquiring the first image, the second image is obtained by processing the first image. The visibility of the target type defect detected in the second image by the defect detection method of this application is greater than that of the target type defect in the first image. Therefore, in this embodiment of the application, by acquiring a first image from a specific angle using an image acquisition device and processing it, a second image from another angle can be obtained. Based on the different second images displaying defects, defect detection can be performed at corresponding angles, thereby improving the accuracy of defect detection.

[0106] In one implementation, the second pose corresponds to the target type of the target defect, so that the defect detection result obtained based on the second image can characterize whether there is a defect of the target type on the image output area.

[0107] In one implementation, the visibility of the target defect corresponding to the target type in the image output area of ​​the first image is less than the visibility of the target defect corresponding to the target type in the image output area of ​​the second image.

[0108] In one implementation, the second obtaining unit 1002 is specifically used to: process the first image using an image processing model to obtain a second image corresponding to the output device being in at least one second pose; wherein the image processing model is trained based on input samples and output samples, the input samples include: sample images corresponding to the output device being in a first pose, and the output samples include: sample images corresponding to the output device being in at least one second pose.

[0109] In one implementation, the second obtaining unit 1002 is specifically used to: process the first image according to the second pose using an image processing model to obtain a second image corresponding to the output device being in the second pose; wherein, the image processing model is trained based on input samples and output samples, the input samples include: the sample image corresponding to the output device being in the first pose and the pose parameters of the target pose, and the output samples include: the sample image corresponding to the output device being in the target pose.

[0110] In one implementation, there are one or more target poses, such that the image processing model can output an image corresponding to the output device being in one or more second poses.

[0111] In one implementation, the second obtaining unit 1002 is specifically used for: obtaining a second image based on a first image, including:

[0112] Using the image processing model corresponding to the second pose, the first image is processed to obtain the second image corresponding to the output device in the second pose;

[0113] The image processing model is trained based on input samples and output samples. The input samples include sample images corresponding to the output device in the first pose, and the output samples include sample images corresponding to the output device in the second pose.

[0114] In one implementation, both the first and second poses are represented by the angle between the plane containing the image output area and the image acquisition direction of the image acquisition device.

[0115] It should be noted that the specific implementation of each unit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0116] refer to Figure 11 This is a schematic diagram of a defect detection system provided in Embodiment 3 of this application. The system includes the following structure:

[0117] Image acquisition device 1101 is used to acquire images from the output device;

[0118] Electronic device 1102 is used to: acquire a first image, wherein the first image is an image acquired by an image acquisition device when the output device is in a first posture; acquire a second image based on the first image, wherein the second image is an image corresponding to the output device in a second posture; wherein the display of the target defect on the image output area of ​​the output device is different in the first image corresponding to the output device in the first posture and the second image corresponding to the output device in the second posture; and acquire a defect detection result on the image output area of ​​the output device based on the display of the target defect in the second image.

[0119] Furthermore, such as Figure 12 The diagram shown is a structural schematic of an electronic device provided in Embodiment 4 of this application. The electronic device can be an electronic device capable of image processing. The electronic device can be configured in a defect detection system. The technical solution in this embodiment is mainly used to detect screen defects of a display screen in order to improve the accuracy of defect detection.

[0120] Specifically, the electronic device in this embodiment may include the following structure:

[0121] The memory 1201 is used to store computer programs and data generated during the execution of computer programs;

[0122] The processor 1202 is configured to execute a computer program to: acquire a first image, wherein the first image is an image acquired by an image acquisition device when the output device is in a first posture; acquire a second image based on the first image, wherein the second image is an image corresponding to the output device in a second posture; wherein the display of the target defect on the image output area of ​​the output device is different in the first image corresponding to the output device in the first posture and the second image corresponding to the output device in the second posture; and acquire a defect detection result on the image output area of ​​the output device based on the display of the target defect in the second image.

[0123] As can be seen from the above technical solutions, in the electronic device provided in Embodiment 4 of this application, a second image is obtained by processing a first image. The visibility of the target type defect detected in the second image by the defect detection method of this application is greater than that of the target type defect in the first image. Therefore, in this embodiment of the application, a first image at a specific angle is acquired by an image acquisition device, and then processed to obtain a second image at another angle. Based on the different second images displaying defects, defect detection at corresponding angles can be performed, thereby improving the accuracy of defect detection.

[0124] As can be seen from the above scheme, in the defect detection system provided in Embodiment 3 of this application, a first image is acquired through an image acquisition device, and then the first image is processed by an electronic device to obtain a second image. The visibility of the target type defect detected in the second image by the defect detection method of this application is greater than that of the target type defect in the first image. Therefore, in this embodiment of the application, by acquiring a first image from a specific angle through an image acquisition device and processing it, a second image from another angle can be obtained. Based on the different second images displaying defects, defect detection can be performed at corresponding angles, thereby improving the accuracy of defect detection.

[0125] It should be noted that the specific implementation of the processor in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0126] Specifically, the technical solution in this application is described in detail below:

[0127] First, a dataset is collected. For each type of defect, an image 1 that can be captured from a side angle and an image 2 taken from a fixed angle on the production line are collected, which are the real data. The images 1 and 2 of the same type of defect, i.e. the real data, are used as image pairs. The image pairs of multiple defects are then used to form a dataset for subsequent model training. Each type of defect corresponds to a side angle.

[0128] Next, the technical solution in this application uses the side-view angle as a constraint to generate a Generative Adversarial Network (GAN), and trains it on the collected dataset. This allows the generation of images of the corresponding defect from side-view angles using images at fixed angles. In other words, training the GAN network with the collected dataset yields a model that can generate side-view images from other angles. Figure 13 As shown, the adversarial network GAN mainly consists of a generator and a discriminator. The generator refers to the image processing model shown above, and the discriminator refers to the discriminator shown above.

[0129] Specifically, the Generator is trained using the dataset obtained above and specific side-view angle data. An adversarial learning mechanism is established to optimize the Generator based on the discrimination result (0 or 1) output by the Discriminator. Specifically, the Encoder in the Generator encodes the image 1 of the defect's side-view angle in the dataset to generate corresponding image features h1, and combines these image features with the specific side-view angle data h2. The Decoder in the Generator generates the corresponding output image based on the combined image features and the target pose's pose parameters h2, such as... Figure 14 As shown, the discriminator can evaluate the similarity score (0 or 1) between the output image and the ground truth data, i.e., the fixed-angle image 2 actually taken on the production line. If the score is 0, the parameters of the generator are adjusted, and the corresponding image features are re-encoded until the discriminator evaluates the similarity score between the output image and the ground truth data to be 1. At this point, the generator is determined to be an image recognition model that can be deployed on the production line.

[0130] Based on this, it can be seen that by using the image recognition model generated above to generate side images from other angles from images taken at a fixed angle, it is possible to detect defects in the output device.

[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0132] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0134] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A defect detection method, comprising: Obtain a first image, which is an image acquired by the image acquisition device when the output device is in a first posture; A second image is obtained based on the first image, wherein the second image is the image corresponding to the output device being in a second posture. Wherein, in the first image corresponding to the output device in the first posture and in the second image corresponding to the second posture, the display of the target defect on the image output area of ​​the output device is different; the second posture corresponds to the target type of the target defect, so that the defect detection result obtained based on the second image can characterize whether there is a defect corresponding to the target type on the image output area; The defect detection result on the image output area of ​​the output device is obtained based on the display of the target defect in the second image.

2. The method according to claim 1, wherein the visibility of the target defect corresponding to the target type on the image output area in the first image is less than the visibility of the target defect corresponding to the target type on the image output area in the second image.

3. The method according to claim 1, wherein obtaining the second image based on the first image comprises: Using an image processing model, the first image is processed to obtain a second image corresponding to the output device being in at least one second pose; The image processing model is trained based on input samples and output samples. The input samples include sample images corresponding to the output device in the first pose, and the output samples include sample images corresponding to the output device in at least one second pose.

4. The method according to claim 1 or 2, wherein obtaining the second image based on the first image comprises: Using an image processing model, the first image is processed according to the second pose to obtain a second image corresponding to the output device being in the second pose; The image processing model is trained based on input samples and output samples. The input samples include: the sample image corresponding to the output device in the first pose and the pose parameters of the target pose. The output samples include: the sample image corresponding to the output device in the target pose.

5. The method according to claim 4, wherein the target pose has one or more, such that the image processing model can output an image corresponding to the output device being in one or more second poses.

6. The method according to claim 1, wherein obtaining the second image based on the first image comprises: Using the image processing model corresponding to the second pose, the first image is processed to obtain the second image corresponding to the output device being in the second pose; The image processing model is trained based on input samples and output samples. The input samples include sample images corresponding to the output device in the first pose, and the output samples include sample images corresponding to the output device in the second pose.

7. The method according to claim 1 or 2, wherein both the first posture and the second posture are represented by the angle between the plane where the image output area is located and the image acquisition direction of the image acquisition device.

8. A defect detection device, comprising: The first acquisition unit is used to acquire a first image, wherein the first image is an image acquired by the image acquisition device when the output device is in a first posture; The second obtaining unit is configured to obtain a second image based on the first image, wherein the second image is an image corresponding to the output device being in a second posture. Wherein, in the first image corresponding to the output device in the first posture and in the second image corresponding to the second posture, the display of the target defect on the image output area of ​​the output device is different; the second posture corresponds to the target type of the target defect, so that the defect detection result obtained based on the second image can characterize whether there is a defect corresponding to the target type on the image output area; A region detection unit is used to obtain defect detection results on the image output region of the output device based on the display of the target defect in the second image.

9. A defect detection system, comprising: Image acquisition equipment, used to acquire images from output devices; An electronic device for: acquiring a first image, wherein the first image is an image acquired by the image acquisition device in a first posture relative to the output device; A second image is obtained based on the first image, the second image being an image corresponding to the output device in a second posture; wherein, the display of the target defect on the image output area of ​​the output device differs between the first image corresponding to the output device in the first posture and the second image corresponding to the output device in the second posture; a defect detection result on the image output area of ​​the output device is obtained based on the display of the target defect in the second image; the second posture corresponds to the target type of the target defect, so that the defect detection result obtained based on the second image can characterize whether a defect corresponding to the target type exists on the image output area.

Images