A method, device and electronic device for detecting screen foreign matter based on binocular vision

Abstract

The embodiments of the present application disclose a method, device, and electronic device for detecting screen foreign matter based on binocular vision, which are used to improve the accuracy of detecting screen defects. The method of the embodiment of the present application includes: photographing the target screen through a binocular vision system to obtain a first image and a second image; converting the second image into a front view image according to a preset main-side view conversion relationship; performing defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information; performing parallax detection on the defect information of the first binary image and the second binary image according to the partition information of the target screen to obtain a dust removal picture; performing parallax detection on the defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defect picture; screening the initial defect picture according to the dust removal picture to obtain a target defect picture.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of screen detection technology, and in particular to a method, device, and electronic device for detecting screen foreign matter based on binocular vision. Background Art

[0002] With the continuous advancement of display technology and the widespread use of flat and curved displays, especially in large monitors, TVs, and smartphones, the detection of screen surface defects has become a critical component of production quality control. Traditional defect detection methods typically rely on two-dimensional image processing technology, primarily identifying surface defects using frontal images. This method is effective for identifying defects on flat displays, but when applied to curved displays, detection accuracy often decreases significantly due to geometric deformation of the screen.

[0003] Existing technologies primarily rely on analyzing images captured from a single perspective through image comparison and edge detection. This approach can provide relatively accurate defect location when processing straight surfaces. However, due to the large parallax difference in curved surfaces, traditional methods often result in misjudgments or missed detections, making them ineffective in adapting to the unique geometric characteristics of curved displays. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, and electronic device for detecting screen foreign matter based on binocular vision, which can improve the accuracy of detecting screen defects.

[0005] A first aspect of the embodiments of the present application provides a method for detecting foreign matter on a screen based on binocular vision, comprising:

[0006] The target screen is photographed by a binocular vision system to obtain a first image and a second image, wherein the first image is a front view image and the second image is a side view image;

[0007] Converting the second image into a front view image according to a preset main-side view conversion relationship;

[0008] Acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image;

[0009] Performing defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information;

[0010] performing parallax detection on the defect information of the first binary image and the second binary image based on the partition information of the target screen to obtain a dust removal picture; wherein the partition information is used to distinguish between a straight surface portion and a curved surface portion of the target screen; and the dust removal picture represents defects on the screen;

[0011] performing parallax detection on defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defect image;

[0012] The initial defective image is screened according to the dust removal image to obtain a target defective image.

[0013] Optionally, performing parallax detection on defect information of the first binary image and the second binary image according to partition information of the target screen to obtain the dust removal picture includes:

[0014] Determining partition attributes of defect information of the first binary image and the second binary image according to partition information of the target screen, wherein the partition attributes include straight surfaces and curved surfaces;

[0015] A dust removal picture is obtained by performing parallax detection on defect information of the first binary image and the second binary image according to the partition attribute.

[0016] Optionally, determining the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen includes:

[0017] Obtaining partition lines of partition information of the target screen;

[0018] Calculating a target distance between defect information of the first binary image and the second binary image and the partition line;

[0019] A partition attribute of defect information of the first binary image and the second binary image is determined according to the target distance.

[0020] Optionally, performing disparity detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain the dust removal picture includes:

[0021] determining a disparity threshold according to the partition attribute;

[0022] Disparity detection is performed on defect information of the first binary image and the second binary image according to the disparity threshold to obtain a dust removal picture.

[0023] Optionally, determining the disparity threshold according to the partition attribute includes:

[0024] When the partition attribute is a straight surface, determining the preset threshold to be a disparity threshold;

[0025] When the partition attribute is a curved surface, the target distance is substituted into a threshold calculation formula to calculate the disparity threshold.

[0026] Optionally, the disparity threshold includes a matching value threshold, a center offset threshold, and a length ratio threshold.

[0027] Optionally, before converting the second image into a front view image according to a preset main-side view conversion relationship, the method further includes:

[0028] The calibration image is photographed through a binocular vision system to obtain a main view dot matrix image and a side view dot matrix image;

[0029] A main-view and side-view conversion relationship is calculated based on the main-view dot matrix diagram and the side-view dot matrix diagram.

[0030] A second aspect of the embodiments of the present application provides a screen foreign body detection device based on binocular vision, comprising:

[0031] a shooting unit, configured to shoot the target screen through a binocular vision system to obtain a first image and a second image, wherein the first image is a front view image and the second image is a side view image;

[0032] a conversion unit, configured to convert the second image into a front view image according to a preset main view to side view conversion relationship;

[0033] an acquiring unit, configured to acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image;

[0034] a defect detection unit, configured to perform defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information;

[0035] a first disparity detection unit configured to perform disparity detection on defect information of the first binary image and the second binary image based on partition information of the target screen, to obtain a dust removal image; wherein the partition information is used to distinguish between a straight surface portion and a curved surface portion of the target screen; and the dust removal image represents defects on the screen;

[0036] a second disparity detection unit, configured to perform disparity detection on defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defective image;

[0037] The screening unit is used to screen the initial defective image according to the dust removal image to obtain a target defective image.

[0038] Optionally, the first disparity detection unit includes:

[0039] a determination subunit, configured to determine partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen, wherein the partition attributes include a straight surface and a curved surface;

[0040] The detection subunit is configured to perform parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain a dust removal picture.

[0041] Optionally, the determining subunit includes:

[0042] An acquisition module, configured to acquire partition lines of partition information of the target screen;

[0043] A calculation module, configured to calculate a target distance between the defect information of the first binary image and the second binary image and the partition line;

[0044] A first determining module is configured to determine a partition attribute of defect information of the first binary image and the second binary image according to the target distance.

[0045] Optionally, the detection subunit includes:

[0046] A second determining module, configured to determine a disparity threshold according to the partition attribute;

[0047] The detection module is configured to perform disparity detection on defect information of the first binary image and the second binary image according to the disparity threshold value to obtain a dust removal picture.

[0048] Optionally, the second determining module is specifically configured to:

[0049] When the partition attribute is a straight surface, determining the preset threshold to be a disparity threshold;

[0050] When the partition attribute is a curved surface, the target distance is substituted into a threshold calculation formula to calculate the disparity threshold.

[0051] Optionally, the disparity threshold includes a matching value threshold, a center offset threshold, and a length ratio threshold.

[0052] Optionally, the device further includes a conversion relationship unit, wherein the conversion relationship unit is configured to:

[0053] The calibration image is photographed through a binocular vision system to obtain a main view dot matrix image and a side view dot matrix image;

[0054] A main-view and side-view conversion relationship is calculated based on the main-view dot matrix diagram and the side-view dot matrix diagram.

[0055] A third aspect of the embodiments of the present application provides an electronic device, including:

[0056] processor, memory, input and output units, and buses;

[0057] The processor is connected to the memory, the input and output unit, and the bus;

[0058] A program is stored in the memory, and the processor calls the program to execute the method in the first aspect and any possible implementation of the first aspect.

[0059] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the computer executes the method in the first aspect and any possible implementation of the first aspect.

[0060] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0061] In this embodiment, the front view image and the converted side view image provide defect data from different perspectives, ensuring comprehensive inspection of the target screen. Combined with the target screen's partitioning information, disparity detection can be performed separately for straight and curved surface areas, effectively identifying and addressing defects in these areas, thereby avoiding misjudgments caused by curved surface distortion. This differentiated processing significantly improves overall inspection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of an embodiment of a method for detecting foreign matter on a screen based on binocular vision in an embodiment of the present application;

[0063] Figure 2 This is a flow chart of an embodiment of disparity detection based on partition information in the present application;

[0064] Figure 3 This is a flow chart of an embodiment of determining partition attributes according to partition information in the present application;

[0065] Figure 4 This is a flow chart of an embodiment of disparity detection based on partition information in the present application;

[0066] Figure 5 This is a flow chart of an embodiment of determining a disparity threshold according to a partition attribute in an embodiment of the present application;

[0067] Figure 6 A schematic diagram of a flow chart of an embodiment of calculating the main-side view conversion relationship in an embodiment of the present application;

[0068] Figure 7This is a structural diagram of an embodiment of a screen foreign body detection device based on binocular vision in an embodiment of the present application;

[0069] Figure 8 This is a structural diagram of an embodiment of an electronic device in the embodiments of the present application. DETAILED DESCRIPTION

[0070] The embodiments of the present application provide a method, device, and electronic device for detecting screen foreign matter based on binocular vision, which are used to improve the accuracy of detecting screen defects.

[0071] The method of the present application can be applied to a server, a terminal or other device with logic processing capabilities, and the present application does not limit this. For the convenience of description, the following description is based on an example in which the execution subject is a server.

[0072] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0073] See also Figure 1 In the embodiment of the present application, an embodiment of the screen foreign body detection method based on binocular vision includes:

[0074] 101. Photograph the target screen through a binocular vision system to obtain a first image and a second image, wherein the first image is a front view image and the second image is a side view image.

[0075] The server uses a binocular vision system (consisting of at least two camera modules) to capture images of the target screen from the front and side. The first image, captured from the front, represents the target screen's frontal view. The second image, captured from the side, represents the target screen's side view. The binocular vision system provides different perspectives, helping the server obtain more comprehensive geometric information about the screen.

[0076] Specifically, the target screen is horizontally fixed at the shooting position of the binocular vision system, the main view camera is set directly above the shooting position, and the main view camera is used to shoot the front view image of the target screen from directly above (i.e., the first image), and the oblique view camera is set diagonally above the shooting position, and the oblique view camera is used to shoot the oblique view image of the target screen from diagonally above (i.e., the second image).

[0077] 102. Convert the second image into a front view image according to a preset main view to side view conversion relationship.

[0078] The server converts the secondary image (the side view) into a front view based on a pre-set transformation relationship between the primary and side views. This transformation relationship is designed by analyzing the screen geometry, the relative positions of the cameras, and the imaging principle. The server uses this transformation relationship to adjust the perspective of the side view image to make it appear as if it were captured from the front. This is achieved through an image transformation algorithm.

[0079] 103. Obtain a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image.

[0080] The server performs binarization and grayscale processing on the first image (the front view image) and the second image (the converted front view image), respectively. Binary images highlight target areas or defect areas by converting pixel values ​​to black and white, while grayscale images retain the grayscale information and provide more detail. During this process, the server generates these images using image processing algorithms (such as thresholding and grayscale conversion).

[0081] 104. Perform defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information.

[0082] The server performs defect detection on the four previously acquired images (two binary images and two grayscale images). Defect detection algorithms (such as edge detection, morphological processing, and region growing) are applied to these images to identify flaws or anomalies. Binary images help highlight obvious defects, while grayscale images can detect more subtle defects.

[0083] 105. Perform parallax detection on the defect information of the first binary image and the second binary image based on the partition information of the target screen to obtain a dust removal image. The partition information is used to distinguish between the straight surface and the curved surface of the target screen. The dust removal image represents the defects on the screen.

[0084] The server performs parallax detection on the defect information in the first and second binary images based on the target screen's partitioning information (distinguishing between straight and curved surfaces). Parallax detection analyzes the differences in defect locations between the two images to deduce the specific spatial location of the defect, thereby generating a dust removal image. The dust removal image reflects the specific distribution of defects on the target screen.

[0085] 106. Perform parallax detection on defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defect image.

[0086] The server further performs parallax detection on the defect information in the first and second grayscale images based on the partition information. Parallax detection helps analyze the three-dimensional distribution of defects and obtain an initial defect image. This step provides a more detailed defect analysis based on the detailed information in the grayscale images.

[0087] 107. The initial defect image is screened according to the dust removal image to obtain a target defect image.

[0088] Finally, the server uses the dust-removed image as a reference to filter the initial defect image, removing irrelevant portions (i.e., the portion of the initial defect image corresponding to the dust-removed image), retaining the target defect area. The target defect image is then considered to be an under-display defect. By comparing it with the dust-removed image, the server can further eliminate noise and irrelevant background information to accurately identify the target defect.

[0089] By screening the dust removal images, the server can filter out irrelevant interference information, accurately identify the target defect area, improve the accuracy of defect identification, and reduce the possibility of false detection.

[0090] In this embodiment, the front view image and the converted side view image provide defect data from different perspectives, ensuring comprehensive inspection of the target screen. Combined with the target screen's partitioning information, the server can perform disparity inspections on both straight and curved surfaces, effectively identifying and addressing defects in these areas and avoiding misjudgments caused by curved surface distortion. This differentiated processing significantly improves overall inspection accuracy.

[0091] See also Figure 2 In some embodiments of the present application, step 105 in the above embodiment performs parallax detection on defect information of the first binary image and the second binary image according to the partition information of the target screen to obtain the dust removal image, which may include the following steps:

[0092] 201. Determine partition attributes of defect information of the first binary image and the second binary image according to partition information of the target screen, where the partition attributes include straight surface and curved surface.

[0093] The server classifies defects in the first and second binary images based on the target screen's partitioning information (straight and curved areas). Partition attributes are defined based on the target screen's geometry, and defects in straight and curved areas require different treatments. Partition information typically includes the screen's structural divisions, and the server uses this data to label defects in each area.

[0094] By clearly distinguishing defects in straight and curved surface areas, the server can process defect information more specifically, improve the accuracy and efficiency of subsequent defect detection, and avoid analysis deviations caused by the characteristics of different areas.

[0095] 202. Perform parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain a dust removal picture.

[0096] The server performs disparity detection on the defect information in the first and second binary images based on the partition attributes (straight or curved) determined in the previous step. Disparity detection compares the defect information in the two images to estimate the depth and spatial location of the defect, thereby generating a dust removal image. Disparity detection accurately locates the spatial location of the defect based on depth differences in the images, especially in straight and curved areas, where disparity distribution may differ.

[0097] Parallax detection, combined with zoning attributes, provides accurate defect location and depth information, generating a dust removal image. This process allows the server to more effectively distinguish defects on straight and curved surfaces, ensuring accurate defect analysis and ultimately outputting a precise dust removal image, providing reliable data for subsequent defect processing.

[0098] In this embodiment, the server can accurately identify and locate defects on the screen based on parallax detection and partition information in complex screen shapes (such as straight surfaces and curved surfaces).

[0099] See also Figure 3 In some embodiments of the present application, step 201 in the above embodiment, determining the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen, may include the following steps:

[0100] 301. Obtain partition lines of partition information of a target screen.

[0101] The server first needs to obtain the partition lines of the target screen. These partition lines define different areas of the screen (such as straight areas and curved areas). Partition lines are typically generated by the geometric model set during screen design and represent the boundaries of different parts. The server extracts this partition line information based on design data or sensor feedback. Partition lines can be based on physical characteristics or functional requirements (for example, the boundaries between straight and curved areas). In one possible embodiment, there may be two partition lines, corresponding to two of the four long sides of the target screen. The partition lines are used to distinguish between the straight and curved areas at the edges of the two long sides of the target screen.

[0102] By accurately acquiring the partition lines, the server can clearly identify the different areas of the target screen, thereby providing basic data support for subsequent defect positioning and ensuring that the defect information can be correctly matched to the screen area.

[0103] 302. Calculate target distances between defect information of the first binary image and the second binary image and a partition line.

[0104] In this step, the server uses image processing algorithms (such as edge detection and morphological transformation) to calculate the distance between the defect information and the partition lines in the first and second binary images. Each defect area corresponds to a target distance. This distance value is used to determine whether the defect area belongs to a straight surface area or a curved surface area. By calculating these distances, the server can determine the position of the defect relative to the straight surface area or the curved surface area, and thus determine the partition attribute of the defect.

[0105] By calculating the target distance between the defect information and the partition line, the server can accurately assess the location of the defect in the screen area, thereby helping to accurately classify the defect information and providing a clear basis for subsequent parallax detection and defect positioning.

[0106] 303. Determine partition attributes of defect information of the first binary image and the second binary image according to the target distance.

[0107] According to the target distance calculated previously, the server maps the relationship between the defect information and the partition line. Specifically, in one possible embodiment, the target screen can be divided into two parts according to the long side, each of which includes a partition line. For the first area, if the left side of the partition line is the curved surface of the target screen and the right side is the straight surface of the target screen, the partition line is used as the X-axis origin, negative coordinates represent the curved surface, positive coordinates represent the straight surface, and the target distance is obtained by subtracting the X-axis origin from the coordinates of the defect area. When the target distance is negative, it indicates that the partition attribute of the corresponding defect area is a curved surface. When the target distance is positive, it indicates that the partition attribute of the corresponding defect area is a straight surface. For the second area, if the left side of the partition line is the straight surface of the target screen and the right side is the curved surface of the target screen, the partition line is used as the X-axis origin, negative coordinates represent the straight surface, positive coordinates represent the curved surface, and the target distance is obtained by subtracting the X-axis origin from the coordinates of the defect area. When the target distance is negative, it indicates that the partition attribute of the corresponding defect area is a straight surface. When the target distance is positive, it indicates that the partition attribute of the corresponding defect area is a curved surface.

[0108] In this way, the server is able to assign a clear partition attribute to each defect based on its relative position to the partition line.

[0109] By matching the target distance with the partition attributes, the server can accurately determine the area (straight or curved) to which each defect belongs. This provides clear data support for subsequent parallax detection and defect processing, ensuring that defects in different areas can be properly analyzed and processed.

[0110] In this embodiment, by combining defect information with the distance relationship between partition lines, the server can accurately distinguish defects in straight surface areas and curved surface areas in a complex screen structure, thereby achieving more accurate defect detection and positioning.

[0111] See also Figure 4 In some embodiments of the present application, step 202 in the above embodiment performs parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain the dust removal image, which may include the following steps:

[0112] 401. Determine a disparity threshold according to a partition attribute.

[0113] The server sets a disparity threshold for each region based on the determined partition attributes (straight or curved). The disparity threshold refers to the maximum positional deviation allowed for defect information between two images during disparity detection. Due to the different geometric characteristics of straight and curved surfaces, the disparity threshold must be adjusted based on the region.

[0114] By setting appropriate disparity thresholds based on the characteristics of different areas (straight and curved surfaces), the server can ensure a more accurate and efficient disparity detection process, thereby reducing misjudgments and ensuring accurate spatial positioning of defects.

[0115] 402 : Perform disparity detection on defect information of the first binary image and the second binary image according to the disparity threshold to obtain a dust removal picture.

[0116] After determining the disparity threshold, the server performs disparity detection on the defect information in the first and second binary images. Disparity detection compares the defects in the two images and analyzes the differences in their depth and position. The server estimates the actual spatial location of the defect by detecting the offset between the two images and determines whether the defect is within an acceptable range based on the disparity threshold. Finally, a dust removal image is generated, which shows the specific location and area of ​​the defect on the target screen. The defect in the dust removal image can be considered to be dust on the target screen, not foreign matter underneath.

[0117] Through parallax detection, combined with parallax thresholds set based on partition attributes, the server can accurately identify and locate defects on the target screen. The dust removal image clearly displays the spatial distribution of defects, especially in complex screen geometries, effectively avoiding false and missed detections and improving defect detection accuracy.

[0118] In this embodiment, it is ensured that the parallax detection process can take into account the characteristics of different areas of the target screen, thereby more accurately identifying and locating defects, generating accurate dust removal images, and ultimately providing reliable data support for defect processing.

[0119] See also Figure 5 In some embodiments of the present application, step 401 in the above embodiment may include the following steps of determining the disparity threshold according to the partition attributes:

[0120] 501. When the partition attribute is a straight surface, determine that the preset threshold is a disparity threshold.

[0121] When processing a straight-face area, the server can set a fixed, preset threshold as the disparity threshold because its geometric characteristics result in minimal parallax variation. This threshold is typically based on actual testing or design requirements, ensuring that parallax deviations within this range are considered normal and do not affect accurate defect location. The server applies this preset value to disparity detection in the straight-face area.

[0122] By using a fixed, preset threshold, the server can quickly perform disparity detection in the direct-view area and avoid detection delays caused by small disparity changes. This simplifies the algorithm, improves processing efficiency, and ensures accurate defect location in the direct-view area.

[0123] 502. When the partition attribute is a curved surface, the target distance is substituted into the threshold calculation formula to calculate the disparity threshold.

[0124] For curved surface areas, parallax varies significantly due to their geometric characteristics. Therefore, the server needs to dynamically calculate the parallax threshold based on the target distance (i.e., the distance between the defect and the partition line). Typically, the parallax threshold increases with distance because the parallax range is greater at greater distances. The server then applies the target distance to the pre-set threshold calculation formula to determine the appropriate parallax threshold for the area. The threshold calculation formula is as follows:

[0125] Formula 1

[0126] Formula 2

[0127] in, is the threshold coefficient. 、 、 is a constant coefficient. is the target distance. It is a constant and can be set to 1 in practical applications. is the disparity threshold, is the preset threshold. In practical applications, when the partition attribute is a straight surface, it can also be substituted into the threshold calculation formula for calculation. The threshold calculation formula is suitable for defect areas of straight surfaces and curved surfaces.

[0128] By calculating dynamic thresholds based on target distance, the server can flexibly adapt to the parallax characteristics of the curved surface area, ensuring that defect detection can be accurately processed at every location in space, avoiding the impact of defect positioning accuracy due to excessively large or small parallax ranges.

[0129] In this embodiment, the server can dynamically adjust the disparity threshold according to the geometric characteristics of different areas (straight surface or curved surface), thereby achieving more accurate defect detection on the entire screen. Especially in complex curved surface areas, the adjustment of the disparity threshold can effectively improve the detection accuracy.

[0130] In the embodiment of the present application, the disparity threshold may include a matching value threshold, a center offset threshold, and a length ratio threshold. For a defective area, if the matching value is greater than the matching value threshold, the defective area is determined to be dust on the target screen. If the center offset is less than the center offset threshold, the defective area is determined to be dust on the target screen. If the length ratio is less than the length ratio threshold, the defective area is determined to be dust on the target screen.

[0131] When calculating the matching value, you can use the matching value calculation formula, which is as follows:

[0132] Formula 3

[0133] Where R is the feature matching value corresponding to the two defect locations of the main squint, TF is the texture feature of the corresponding defect area, SF is the shape feature of the corresponding defect area, and α and β are weighting parameters for the corresponding features. Specifically, TF is generally calculated using LBP, GLCM, surf, etc., while the shape feature SF is calculated using a combination of HU moments, simple shape descriptors, or Fourier descriptors.

[0134] When calculating the center offset, you can use the center offset calculation formula, which is as follows:

[0135] Formula 4

[0136] in, is the center offset. The center coordinates of the main and strabismus defect areas are ( C Mx ,C My )and( C Sx ,C Sy ),Right now,( C Mx ,C My ) is the center coordinate of a defect area in the first binary image, ( C Sx ,C Sy ) is the center coordinate of a defect area in the second binary image.

[0137] When calculating the length ratio, you can use the length ratio calculation formula to calculate, the length ratio calculation formula is as follows:

[0138] Formula 5

[0139] in, is the length ratio. The length of the defect area in the longitudinal direction of the main and oblique view is L M , L S ,Right now, L M is the length in the longitudinal direction of a defect area in the first binary image, L S is the length in the longitudinal direction of a defect area in the second binary image. L S Greater than L M .

[0140] In the process of performing parallax detection on the defect information of the first binary image and the second binary image to obtain the dust removal picture, when detecting a defect area, the detection is performed in the order of whether the detection matching value is greater than the matching value threshold, whether the detection center offset is less than the center offset threshold, and whether the detection length ratio is less than the length ratio threshold. In this process, once the conditions are met, the detection of the defect area is terminated and the defect area is included in the dust removal picture.

[0141] It should be noted that in some embodiments of the present application, in executing step 106, parallax detection is performed on the defect information of the first grayscale image and the second grayscale image according to the partition information to obtain the specific process of the initial defective image. Please refer to the aforementioned steps 201-202, 301-303, 401-402, 501-502, which will not be repeated here.

[0142] It should be noted that, in the process of performing parallax detection on the defect information of the first grayscale image and the second grayscale image to obtain the initial defective picture, when detecting a certain defective area, the detection is carried out in the order of whether the matching value is less than or equal to the matching value threshold, whether the center offset is greater than or equal to the center offset threshold, and whether the length ratio is greater than or equal to the length ratio threshold. In this process, if the prerequisite is met, the detection of the defective area will continue. If all conditions are met, the defective area is determined to be a real foreign object under the screen.

[0143] See also Figure 6In some embodiments of the present application, before step 102 in the above embodiment converts the second image into a front view image according to the preset main-side view conversion relationship, the following steps may be further included:

[0144] 601. The calibration image is photographed by a binocular vision system to obtain a main view dot matrix image and a side view dot matrix image.

[0145] The server uses a binocular vision system to capture a calibration pattern, producing two dot maps: a main view dot map and a side view dot map. A calibration pattern is a known reference pattern or graphic, typically consisting of a series of calibration points at known locations. Using the binocular vision system, the main view dot map is captured from the front, while the side view dot map is captured from the side. These dot maps serve as reference points for subsequent image geometric transformations.

[0146] By capturing the calibration image and obtaining the main and side view dot maps, the server provides the necessary benchmark data for calculating the perspective conversion relationship. These dot maps can help calibrate the camera perspective, ensuring that subsequent image conversion is more accurate.

[0147] 602. Calculate the main view and side view conversion relationship based on the main view dot matrix and the side view dot matrix.

[0148] The server uses the calibration points in the main view dot matrix and the side view dot matrix to calculate the main view and side view transformation relationship through geometric transformation algorithms (such as perspective transformation, homography matrix calculation, etc.). This process is based on the correspondence between the two dot matrices. The server obtains a transformation matrix or transformation formula through mathematical calculation. This formula can convert the side view image into the main view image and ensure that the two are aligned in space. It should be noted that in the embodiment of the present application, based on the main view dot matrix and the side view dot matrix, the server can calculate the main view and side view transformation relationship through multiple formulas, formula combinations, or custom formulas. It is only necessary to ensure the accuracy of the main view and side view transformation relationship.

[0149] By accurately calculating the main-side view transformation, the server ensures that images captured from the side are correctly converted to frontal views, improving the accuracy of image analysis. In subsequent steps, this transformation ensures geometric consistency between images, reducing errors caused by perspective differences.

[0150] This embodiment provides basic data for accurately calculating the view conversion relationship, so that the server can correctly convert the side view image into the front view image in the subsequent steps, thereby ensuring high accuracy of the image conversion.

[0151] See also Figure 7 In the embodiment of the present application, an embodiment of a screen foreign body detection device based on binocular vision includes:

[0152] The shooting unit 701 is used to shoot the target screen through a binocular vision system to obtain a first image and a second image, where the first image is a front view image and the second image is a side view image.

[0153] The conversion unit 702 is configured to convert the second image into a front view image according to a preset main view to side view conversion relationship.

[0154] The acquisition unit 703 is configured to acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image.

[0155] The defect detection unit 704 is configured to perform defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information.

[0156] The first disparity detection unit 705 is configured to perform disparity detection on the defect information of the first binary image and the second binary image based on the partition information of the target screen, thereby obtaining a dust removal image. The partition information is used to distinguish between the straight and curved surfaces of the target screen. The dust removal image represents defects on the screen.

[0157] The second disparity detection unit 706 is configured to perform disparity detection on the defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defective image.

[0158] The screening unit 707 is configured to screen the initial defective image according to the dust removal image to obtain a target defective image.

[0159] In this embodiment, the front view image and the converted side view image provide defect data from different perspectives, ensuring comprehensive inspection of the target screen. Combined with the target screen's partitioning information, the screen foreign object detection device can perform disparity detection on both straight and curved surfaces, effectively identifying and addressing defects in these areas and avoiding misjudgments caused by curved surface distortion. Through this differentiated processing, the screen foreign object detection device significantly improves overall detection accuracy.

[0160] Optionally, the first disparity detection unit 705 includes:

[0161] The determination subunit is used to determine the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen, where the partition attributes include straight surface and curved surface.

[0162] The detection subunit is used to perform parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain a dust removal picture.

[0163] Optionally, the determination subunit includes:

[0164] The acquisition module is used to obtain the partition lines of the partition information of the target screen.

[0165] The calculation module is used to calculate the target distance between the defect information of the first binary image and the second binary image and the partition line.

[0166] The first determining module is configured to determine a partition attribute of defect information of the first binary image and the second binary image according to the target distance.

[0167] Optionally, the detection subunit includes:

[0168] The second determining module is configured to determine a disparity threshold according to a partition attribute.

[0169] The detection module is used to perform disparity detection on defect information of the first binary image and the second binary image according to a disparity threshold value to obtain a dust removal picture.

[0170] Optionally, the second determining module is specifically configured to:

[0171] When the partition attribute is a straight surface, the preset threshold is determined to be a disparity threshold.

[0172] When the partition attribute is a curved surface, the target distance is substituted into the threshold calculation formula to calculate the disparity threshold.

[0173] Optionally, the disparity threshold includes a matching value threshold, a center offset threshold, and a length ratio threshold.

[0174] Optionally, the device further includes a conversion relationship unit, which is configured to:

[0175] The calibration image is photographed by a binocular vision system to obtain a main view dot matrix image and a side view dot matrix image.

[0176] The main view and side view conversion relationship is calculated based on the main view dot matrix and the side view dot matrix.

[0177] In this implementation, the functions of each unit and module are the same as those mentioned above. Figures 1 to 6 The steps in the illustrated embodiment correspond to each other and will not be repeated here.

[0178] See also Figure 8 In the embodiments of the present application, an electronic device includes:

[0179] Processor 801, memory 802, input and output unit 803 and bus 804;

[0180] The processor 801 is connected to the memory 802, the input and output unit 803 and the bus 804;

[0181] The memory 802 stores a program, and the processor 801 calls the program to execute Figures 1 to 6Steps in the illustrated embodiment.

[0182] In this embodiment, the function of the processor 801 is the same as that of the aforementioned Figures 1 to 6 The steps in the illustrated embodiment correspond to each other and will not be repeated here.

[0183] The embodiment of the present application further provides a computer-readable storage medium having a program stored thereon, which, when executed on a computer, causes the computer to execute the aforementioned Figures 1 to 6 A method in any possible embodiment.

[0184] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0186] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0187] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0188] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A method for detecting screen foreign matter based on binocular vision, characterized in that: include: The target screen is photographed by a binocular vision system to obtain a first image and a second image, wherein the first image is a front view image and the second image is a side view image; Converting the second image into a front view image according to a preset main-side view conversion relationship; Acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image; Performing defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information; performing parallax detection on the defect information of the first binary image and the second binary image based on the partition information of the target screen to obtain a dust removal picture; wherein the partition information is used to distinguish between a straight surface portion and a curved surface portion of the target screen; and the dust removal picture represents defects on the screen; performing parallax detection on defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defect image; screening the initial defective image according to the dust removal image to obtain a target defective image; The method of performing parallax detection on defect information of the first binary image and the second binary image according to the partition information of the target screen to obtain a dust removal picture includes: Determining partition attributes of defect information of the first binary image and the second binary image according to partition information of the target screen, wherein the partition attributes include straight surfaces and curved surfaces; performing parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain a dust removal picture; The determining, based on the partition information of the target screen, partition attributes of the defect information of the first binary image and the second binary image includes: Obtaining partition lines of partition information of the target screen; Calculating a target distance between defect information of the first binary image and the second binary image and the partition line; A partition attribute of defect information of the first binary image and the second binary image is determined according to the target distance.

2. The method according to claim 1, characterized in that The performing of parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain the dust removal picture includes: determining a disparity threshold according to the partition attribute; Disparity detection is performed on defect information of the first binary image and the second binary image according to the disparity threshold to obtain a dust removal picture.

3. The method according to claim 2, characterized in that The determining of the disparity threshold according to the partition attribute includes: When the partition attribute is a straight surface, determining the preset threshold to be a disparity threshold; When the partition attribute is a curved surface, the target distance is substituted into a threshold calculation formula to calculate the disparity threshold.

4. The method according to claim 2 or 3, characterized in that The disparity threshold includes a matching value threshold, a center offset threshold, and a length ratio threshold.

5. The method according to claim 1, characterized in that Before converting the second image into a front view image according to the preset main-side view conversion relationship, the method further includes: The calibration image is photographed through a binocular vision system to obtain a main view dot matrix image and a side view dot matrix image; A main-view and side-view conversion relationship is calculated based on the main-view dot matrix diagram and the side-view dot matrix diagram.

6. A screen foreign body detection device based on binocular vision, characterized in that: include: a shooting unit, configured to shoot the target screen through a binocular vision system to obtain a first image and a second image, wherein the first image is a front view image and the second image is a side view image; a conversion unit, configured to convert the second image into a front view image according to a preset main view to side view conversion relationship; an acquiring unit, configured to acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image; a defect detection unit, configured to perform defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information; a first disparity detection unit configured to perform disparity detection on defect information of the first binary image and the second binary image based on partition information of the target screen, to obtain a dust removal image; wherein the partition information is used to distinguish between a straight surface portion and a curved surface portion of the target screen; and the dust removal image represents defects on the screen; a second disparity detection unit, configured to perform disparity detection on defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defective image; a screening unit, configured to screen the initial defective image according to the dust removal image to obtain a target defective image; The first disparity detection unit includes: a determination subunit, configured to determine partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen, wherein the partition attributes include a straight surface and a curved surface; a detection subunit, configured to perform parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain a dust removal picture; The determining subunit includes: An acquisition module, configured to acquire partition lines of partition information of the target screen; A calculation module, configured to calculate a target distance between the defect information of the first binary image and the second binary image and the partition line; A first determining module is configured to determine a partition attribute of defect information of the first binary image and the second binary image according to the target distance.

7. An electronic device, characterized in that: include: processor, memory, input and output units, and buses; The processor is connected to the memory, the input and output unit, and the bus; A program is stored in the memory, and the processor calls the program to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 5.

Images