Screen foreign matter detection method and device based on binocular vision and electronic equipment

Abstract

The embodiment of the invention discloses a screen foreign matter detection method and device based on binocular vision and electronic equipment, and is used for improving the precision of detecting screen body defects. The method comprises the following steps: shooting a target screen body through a binocular vision system to obtain a first image and a second image; converting the second image into a front view image according to a preset main-side view conversion relation; performing defect detection on the first binary image, the first grayscale image, the second binary image and the second grayscale image to obtain corresponding defect information; according to the partition information of the target screen body, performing parallax detection on the defect information of the first binary image and the second binary image to obtain a dust removal picture; according to the partition information, performing parallax detection on defect information of the first grayscale image and the second grayscale image to obtain an initial defect picture; and screening the initial defect picture according to the dust removal picture to obtain a target defect picture.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of screen detection, and in particular, to a method, device, and electronic device for detecting foreign objects on a screen based on binocular vision. Background Art

[0002] With the continuous development of display technology and the wide application of flat and curved display screens, especially in large monitors, televisions, and smartphones, the detection of defects on the screen surface has become an important part of production quality control. Traditional defect detection methods usually rely on two-dimensional image processing technology and mainly identify defects on the screen surface through frontal images. This method can identify defects well when dealing with flat display screens, but when applied to curved display screens, the detection accuracy often drops significantly due to the geometric deformation of the screen body.

[0003] In the prior art, defect detection mainly relies on images obtained from a single perspective and analyzes them through methods such as image comparison and edge detection. This method can provide relatively accurate defect localization when dealing with straight areas, but due to the large parallax in curved areas, traditional methods often make misjudgments or miss detections during the detection process and cannot effectively adapt to the special geometric characteristics of curved display screens. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, and electronic device for detecting foreign objects on a screen based on binocular vision, which can improve the accuracy of detecting screen defects.

[0005] The first aspect of the embodiments of the present application provides a method for detecting foreign objects on a screen based on binocular vision, including: Taking pictures of a target screen through a binocular vision system to obtain a first image and a second image, where the first image is a frontal image and the second image is a side view image; Converting the second image into a frontal image according to a preset main-side view conversion relationship; Obtaining a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image; Performing defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information; Performing parallax detection on the defect information of the first binary image and the second binary image according to the partition information of the target screen to obtain a dust removal picture; where the partition information is used to distinguish the straight part and the curved part of the target screen; the dust removal picture represents the defects on the screen; Performing parallax detection on the defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defect picture; Screen the initial defect image according to the dust removal image to obtain a target defect image.

[0006] Optionally, the parallax detection of the defect information of the first binary image and the second binary image according to the partition information of the target screen body to obtain a dust removal image includes: Determine the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen body, where the partition attributes include straight surfaces and curved surfaces; Perform parallax detection on the defect information of the first binary image and the second binary image according to the partition attributes to obtain a dust removal image.

[0007] Optionally, determining the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen body includes: Obtain the partition lines of the partition information of the target screen body; Calculate the target distances between the defect information of the first binary image and the second binary image and the partition lines; Determine the partition attributes of the defect information of the first binary image and the second binary image according to the target distances.

[0008] Optionally, the parallax detection of the defect information of the first binary image and the second binary image according to the partition attributes to obtain a dust removal image includes: Determine the parallax threshold according to the partition attributes; Perform parallax detection on the defect information of the first binary image and the second binary image according to the parallax threshold to obtain a dust removal image.

[0009] Optionally, determining the parallax threshold according to the partition attributes includes: When the partition attribute is a straight surface, determine the preset threshold as the parallax threshold; When the partition attribute is a curved surface, substitute the target distance into the threshold calculation formula to calculate the parallax threshold.

[0010] Optionally, the parallax threshold includes a matching degree value threshold, a center offset threshold, and a length ratio threshold.

[0011] Optionally, before converting the second image into a front view image according to the preset main side view conversion relationship, the method further includes: Capture a calibration image through a binocular vision system to obtain a front view dot matrix image and a side view dot matrix image; Calculate the main side view conversion relationship according to the front view dot matrix image and the test dot matrix image.

[0012] In a second aspect of the embodiments of the present application, a screen foreign object detection device based on binocular vision is provided, including: A shooting unit for shooting a target screen through a binocular vision system to obtain a first image and a second image, where the first image is a front view image and the second image is a side view image; A conversion unit for converting the second image into a front view image according to a preset main side view conversion relationship; An acquisition unit for acquiring a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image; A defect detection unit for performing defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information; A first parallax detection unit for performing parallax detection on the defect information of the first binary image and the second binary image according to the partition information of the target screen to obtain a dust removal picture; where the partition information is used to distinguish the straight part and the curved part of the target screen; the dust removal picture represents the defects on the screen; A second parallax detection unit for performing parallax detection on the defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defect picture; A screening unit for screening the initial defect picture according to the dust removal picture to obtain a target defect picture.

[0013] Optionally, the first parallax detection unit includes: A determination subunit for determining the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen, where the partition attributes include straight and curved; A detection subunit for performing parallax detection on the defect information of the first binary image and the second binary image according to the partition attributes to obtain a dust removal picture.

[0014] Optionally, the determination subunit includes: An acquisition module for acquiring the partition line of the partition information of the target screen; A calculation module for calculating the target distance between the defect information of the first binary image and the second binary image and the partition line; A first determination module for determining the partition attributes of the defect information of the first binary image and the second binary image according to the target distance.

[0015] Optionally, the detection subunit includes: A second determination module for determining a parallax threshold according to the partition attributes; A detection module, configured to perform parallax detection on the defect information of the first binary image and the second binary image according to the parallax threshold to obtain a dust removal screen.

[0016] Optionally, the second determination module is specifically configured to: When the partition attribute is a straight surface, determine the preset threshold as the parallax threshold; When the partition attribute is a curved surface, substitute the target distance into the threshold calculation formula to calculate the parallax threshold.

[0017] Optionally, the parallax threshold includes a matching degree value threshold, a center offset threshold, and a length ratio threshold.

[0018] Optionally, the device further includes a conversion relationship unit, and the conversion relationship unit is configured to: Shoot a calibration map through a binocular vision system to obtain a main view dot matrix map and a side view dot matrix map; Calculate the main-side view conversion relationship according to the main view dot matrix map and the test dot matrix map.

[0019] A third aspect of the embodiments of the present application provides an electronic device, including: A processor, a memory, an input-output unit, and a bus; The processor is connected to the memory, the input-output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method in the first aspect and any possible implementation manner of the first aspect.

[0020] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the computer is caused to execute the method in the first aspect and any possible implementation manner of the first aspect.

[0021] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages: In the embodiments of the present application, the front view image and the converted side view image provide defect data from different perspectives, ensuring comprehensive detection of the target screen body. Combining the partition information of the target screen body, parallax detection can be performed on the straight surface and curved surface regions respectively, effectively identifying and processing the defects in these regions, thereby avoiding misjudgment caused by curved surface distortion. Through this differential processing, the overall detection accuracy can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic flowchart of an embodiment of a method for detecting foreign objects on a screen based on binocular vision in the embodiments of the present application; Figure 2 Schematic diagram of a process for parallax detection according to partition information in an embodiment of the present application; Figure 3 Schematic diagram of a process for determining partition attributes according to partition information in an embodiment of the present application; Figure 4 Schematic diagram of a process for parallax detection according to partition information in an embodiment of the present application; Figure 5 Schematic diagram of a process for determining a parallax threshold according to partition attributes in an embodiment of the present application; Figure 6 Schematic diagram of a process for calculating the conversion relationship of the main side view in an embodiment of the present application; Figure 7 Schematic diagram of the structure of a device for detecting foreign objects on a screen based on binocular vision in an embodiment of the present application; Figure 8 Schematic diagram of the structure of an electronic device in an embodiment of the present application. Detailed implementation manners

[0023] The embodiments of the present application provide a method, a device, and an electronic device for detecting foreign objects on a screen based on binocular vision, which are used to improve the accuracy of detecting screen defects.

[0024] The method of the present application can be applied to a server, a terminal, or other devices with logical processing capabilities. In this regard, the present application is not limited. For the sake of convenience of description, the following description will be made taking the server as the execution subject as an example.

[0025] Next, the embodiments in the present application will be described with reference to the accompanying drawings.

[0026] Please refer to Figure 1 , an embodiment of the method for detecting foreign objects on a screen based on binocular vision in the embodiments of the present application includes: 101. Shoot a target screen through a binocular vision system to obtain a first image and a second image, where the first image is a front view image and the second image is a side view image.

[0027] The server shoots the target screen through a binocular vision system (including at least two camera modules). The two cameras capture images from the front and the side respectively. The first image is taken from the front and represents the front view image of the target screen. The second image is taken from the side and represents the side view image of the target screen. The binocular vision system can provide different perspectives to help the server obtain more comprehensive geometric information of the screen.

[0028] Specifically, the target screen is horizontally fixed at the shooting position of the binocular vision system. The main camera is set directly above the shooting position and is used to capture the front view image (i.e., the first image) of the target screen from directly above. The oblique camera is set obliquely above the shooting position and is used to capture the oblique view image (i.e., the second image) of the target screen from obliquely above.

[0029] 102. Convert the second image into a front view image according to the preset conversion relationship between the main view and the side view.

[0030] The server converts the second image (side view image) into a front view image according to the preset conversion relationship between the main view and the side view. The conversion relationship is designed by analyzing the geometric shape of the screen, the relative position of the cameras, and the imaging principle. The server uses this conversion relationship to adjust the viewing angle information of the side view image so that it looks like it was taken from the front. This can be achieved through image transformation algorithms.

[0031] 103. Obtain the first binary image and the first grayscale image corresponding to the first image, and the second binary image and the second grayscale image corresponding to the second image.

[0032] The server performs binarization and grayscale processing on the first image (front view image) and the second image (converted front view image) respectively. Binary images are obtained by converting the pixel values in the image into black and white to highlight the target area or defect area, while grayscale images retain the grayscale information in the image and can provide more details. In this process, the server uses image processing algorithms (such as threshold processing, grayscale transformation, etc.) to generate these images.

[0033] 104. Perform defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain the corresponding defect information.

[0034] The server performs defect detection on the four previously obtained images (two binary images and two grayscale images). Defect detection algorithms (such as edge detection, morphological processing, region growing, etc.) are applied to these images to identify flaws or abnormal parts in the images. Binary images help highlight obvious defects, while grayscale images can handle relatively subtle defects.

[0035] 105. Perform parallax detection on the defect information of the first binary image and the second binary image according to the partition information of the target screen to obtain a dust removal picture. The partition information is used to distinguish the straight part and the curved part of the target screen. The dust removal picture represents the defects on the screen.

[0036] The server performs parallax detection on the defect information in the first and second binary images according to the partition information of the target screen (distinguishing between the flat and curved parts). The parallax detection calculates the specific spatial position of the defect by analyzing the difference in the defect positions in the two images, thereby generating a dust removal screen, which is the specific distribution of the defects on the target screen.

[0037] 106. Perform parallax detection on the defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defect screen.

[0038] The server further performs parallax detection on the defect information in the first and second grayscale images according to the partition information. The parallax detection helps analyze the three-dimensional distribution of the defects to obtain an initial defect screen. This step can provide a more delicate defect analysis based on the detailed information of the grayscale images.

[0039] 107. Screen the initial defect screen according to the dust removal screen to obtain a target defect screen.

[0040] Finally, the server uses the dust removal screen as a reference to screen the initial defect screen, removing the irrelevant parts (i.e., the parts corresponding to the position of the dust removal screen in the initial defect screen), and retaining the target defect area. The target defect screen can be considered as the defect under the screen. By comparing with the dust removal screen, the server can further eliminate noise or irrelevant background information and accurately identify the target defect.

[0041] Through the screening of the dust removal screen, the server can filter out irrelevant interference information, accurately identify the target defect area, improve the accuracy of defect identification, and reduce the possibility of false detection.

[0042] In this embodiment, the front view image and the converted side view image provide defect data from different perspectives, ensuring a comprehensive detection of the target screen. Combining the partition information of the target screen, the server can perform parallax detection on the flat and curved regions respectively, effectively identify and process the defects in these regions, thereby avoiding misjudgment caused by curved surface distortion. Through this differential processing, the server can significantly improve the overall detection accuracy.

[0043] Please refer to Figure 2 , in some embodiments of the present application, the parallax detection performed on the defect information of the first binary image and the second binary image according to the partition information of the target screen in step 105 of the above embodiment to obtain the dust removal screen may include the following steps: 201. Determine the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen, and the partition attributes include flat and curved surfaces.

[0044] The server classifies the defect information in the first binary image and the second binary image according to the partition information (the straight surface area and the curved surface area) of the target screen body. The determination of the partition attribute is defined based on the geometric structure of the target screen body, and the defects in the straight surface part and the curved surface part need to be processed in different ways. The partition information usually includes the structural division data of the screen body, and the server marks the defect information of each area according to these data.

[0045] By clearly distinguishing the defects in the straight surface and curved surface areas, the server can process the defect information more pertinently, improve the accuracy and efficiency of subsequent defect detection, and avoid analysis deviations caused by the characteristics of different areas.

[0046] 202. Perform parallax detection on the defect information of the first binary image and the second binary image according to the partition attribute to obtain the dust removal picture.

[0047] The server performs parallax detection on the defect information in the first binary image and the second binary image according to the partition attribute (straight surface or curved surface) determined in the previous step. The parallax detection process estimates the depth and spatial position of the defect by comparing the defect information in the two images, thereby generating a dust removal picture. Parallax detection can accurately locate the spatial position of the defect according to the depth difference in the image, especially in the straight surface and curved surface areas, where different parallax distributions may exist.

[0048] Parallax detection can combine the partition attribute to provide accurate defect position and depth information, and generate a dust removal picture. Through this process, the server can more effectively distinguish the defects in the straight surface and curved surface areas, ensure the accuracy of defect analysis, and finally output an accurate dust removal picture, providing reliable data for subsequent defect processing.

[0049] In this embodiment, the server can accurately identify and locate the defects on the screen body based on parallax detection and partition information in a complex screen body form (such as a straight surface and a curved surface).

[0050] Please refer to Figure 3 , in some embodiments of the present application, step 201 in the above embodiment determines the partition attribute of the defect information of the first binary image and the second binary image according to the partition information of the target screen body, which may include the following steps: 301. Obtain the partition line of the partition information of the target screen body.

[0051] The server first needs to obtain the partition lines of the target screen body, which define different regions of the screen body (such as the straight surface region and the curved surface region). The partition lines are usually generated by the geometric model set during the screen body design and represent the boundaries of different parts. The server extracts the information of these partition lines according to the design data or sensor feedback. The partition lines can be based on physical characteristics or functional requirements (for example, the boundary between the straight surface and the curved surface region). In a possible embodiment, there can be two partition lines, and these two partition lines correspond to two long sides of the four sides of the target screen body. The partition lines are used to distinguish the straight surface region and the curved surface region at the edges of the two long sides of the target screen body.

[0052] By accurately obtaining the partition lines, the server can clearly identify different regions of the target screen body, thereby providing basic data support for subsequent defect location and ensuring that the defect information can be correctly corresponded to the screen body region.

[0053] 302. Calculate the target distance between the defect information of the first binary image and the second binary image and the partition line.

[0054] In this step, the server calculates the distance between the defect information in the first binary image and the second binary image and the partition line through image processing algorithms (such as edge detection, morphological transformation, etc.). Among them, each defect region corresponds to a target distance. This distance value is a standard for measuring whether the defect region belongs to the straight surface region or the curved surface region. By calculating these distances, the server can determine the position of the defect relative to the straight surface region or the curved surface region, thereby judging its partition attribute.

[0055] By calculating the target distance between the defect information and the partition line, the server can accurately evaluate the position of the defect in the screen body region, and then help to accurately classify the defect information, providing a clear basis for subsequent parallax detection and defect location.

[0056] 303. Determine the partition attribute of the defect information of the first binary image and the second binary image according to the target distance.

[0057] Based on the previously calculated target distance, the server maps the relationship between the defect information and the partition line. Specifically, in a possible implementation, the target screen can be divided into two parts along the long side, and each part includes a partition line. For the first region, if the curved surface of the target screen is on the left side of the partition line and the straight surface is on the right side, then the partition line is used as the origin of the X-axis, with negative coordinates representing the curved surface and positive coordinates representing the straight surface. The target distance is obtained by subtracting the X-axis origin from the coordinates of the defect region. When the target distance is negative, it indicates that the partition attribute of the corresponding defect region is the curved surface, and when the target distance is positive, it indicates that the partition attribute of the corresponding defect region is the straight surface. For the second region, if the straight surface of the target screen is on the left side of the partition line and the curved surface is on the right side, then the partition line is used as the origin of the X-axis, with negative coordinates representing the straight surface and positive coordinates representing the curved surface. The target distance is obtained by subtracting the X-axis origin from the coordinates of the defect region. When the target distance is negative, it indicates that the partition attribute of the corresponding defect region is the straight surface, and when the target distance is positive, it indicates that the partition attribute of the corresponding defect region is the curved surface.

[0058] In this way, the server can assign a clear partition attribute to each defect according to the relative position between the defect and the partition line.

[0059] By matching the target distance with the partition attribute, the server can accurately determine the region (straight surface or curved surface) to which each defect belongs. This provides clear data support for subsequent parallax detection and defect processing, ensuring that defects in different regions can be properly analyzed and processed.

[0060] In this embodiment, by combining the distance relationship between the defect information and the partition line, the server can accurately distinguish the defects in the straight surface region and the curved surface region in a complex screen structure, thereby achieving more accurate defect detection and positioning.

[0061] Please refer to Figure 4 , in some embodiments of the present application, the parallax detection of the defect information of the first binary image and the second binary image according to the partition attribute in step 202 of the above embodiment to obtain the dust removal screen may include the following steps: 401. Determine the parallax threshold according to the partition attribute.

[0062] The server sets the parallax threshold for each region according to the determined partition attribute (straight surface or curved surface). The parallax threshold refers to the maximum position deviation allowed for the defect information in the two images during the parallax detection process. Since the geometric characteristics of the straight surface and the curved surface regions are different, the parallax threshold also needs to be adjusted according to the region.

[0063] By setting appropriate parallax thresholds according to the characteristics of different regions (flat surfaces and curved surfaces), the server can ensure that the parallax detection process is more accurate and efficient, thereby reducing misjudgments and ensuring accurate spatial positioning of defects.

[0064] 402. Perform parallax detection on the defect information of the first binary image and the second binary image according to the parallax threshold to obtain a dust removal image.

[0065] After determining the parallax threshold, the server performs parallax detection on the defect information in the first binary image and the second binary image. Parallax detection compares the defects in the two images and analyzes their depth and position differences. The server estimates the actual spatial position of the defect by detecting the offset of the defect in the image between the two images and determines whether the defect is within an acceptable range according to the parallax threshold. Finally, a dust removal image is generated, and the dust removal image is the specific position and area of the defect on the target screen. The defects in the dust removal image can be considered as dust on the target screen rather than foreign objects under the target screen.

[0066] Through parallax detection and combined with the parallax threshold set according to the partition attributes, the server can accurately identify and locate the defects on the target screen. The dust removal image can clearly show the spatial distribution of the defects, especially in complex screen geometries, effectively avoiding false detections and missed detections and improving the accuracy of defect detection.

[0067] In this embodiment, it is ensured that the parallax detection process can take into account the characteristics of different regions of the target screen, thereby more accurately identifying and locating defects, generating an accurate dust removal image, and finally providing reliable data support for defect processing.

[0068] Please refer to Figure 5 , in some embodiments of the present application, step 401 in the above embodiment of determining the parallax threshold according to the partition attributes may include the following steps: 501. When the partition attribute is a flat surface, determine the preset threshold as the parallax threshold.

[0069] When the server processes the flat surface area, due to the small parallax change caused by the geometric characteristics of the flat surface area, a fixed preset threshold can be directly set as the parallax threshold. This preset threshold is usually based on actual tests or design requirements to ensure that parallax deviations within this range are considered normal and will not affect the accurate positioning of defects. The server applies this preset value to the parallax detection of the flat surface area.

[0070] By using a fixed preset threshold, the server can quickly perform parallax detection in the flat surface area and avoid detection delays caused by small parallax changes. This can simplify the algorithm, improve processing efficiency, and ensure accurate defect positioning in the flat surface area.

[0071] 502. When the partition attribute is a curved surface, the target distance is substituted into the threshold calculation formula to calculate the parallax threshold.

[0072] For the curved surface area, due to the geometric characteristics of the curved surface area, the parallax changes greatly. Therefore, the server needs to dynamically calculate the parallax threshold according to the target distance (i.e., the distance between the defect and the partition line). Usually, the parallax threshold increases with the increase of the distance because the change range of the parallax is also larger at a farther distance. The server substitutes the target distance into the preset threshold calculation formula to calculate the parallax threshold suitable for this area. The threshold calculation formula is as follows: Formula 1 Formula 2 Among them, is the threshold coefficient. , , are constant coefficients. is the target distance. is a constant, which can be set to 1 in actual applications. is the parallax threshold, is the preset threshold. In actual applications, when the partition attribute is a flat surface, it can also be substituted into the threshold calculation formula for calculation, and the threshold calculation formula adapts to the defect areas of flat surfaces and curved surfaces.

[0073] By calculating the dynamic threshold according to the target distance, the server can flexibly adapt to the parallax characteristics of the curved surface area, ensure that the defect detection can be accurately processed at each position in space, and avoid affecting the accuracy of defect positioning due to too large or too small parallax range.

[0074] In this embodiment, the server can dynamically adjust the parallax threshold according to the geometric characteristics (flat surface or curved surface) of different areas, so as to achieve more accurate defect detection on the entire screen body. Especially in complex curved surface areas, the adjustment of the parallax threshold can effectively improve the detection accuracy.

[0075] In the embodiment of the present application, the parallax threshold may include a matching degree value threshold, a center offset threshold, and a length ratio threshold. Among them, for a certain defect area, when the matching degree value is greater than the matching degree value threshold, it is determined that the defect area is dust on the target screen body. When the center offset is less than the center offset threshold, it is determined that the defect area is dust on the target screen body. When the length ratio is less than the length ratio threshold, it is determined that the defect area is dust on the target screen body.

[0076] When calculating the matching degree value, it can be calculated according to the matching degree value calculation formula. The matching degree value calculation formula is as follows: Formula 3 Among them, R is the feature matching degree value corresponding to two defect positions of the main strabismus, TF is the texture feature of the corresponding calculated defect area, SF is the shape feature of the corresponding calculated defect area, and α and β are the weighting parameters of the corresponding features. Specifically, TF is generally calculated using LBP, GLCM, surf, etc., and the shape feature SF is calculated using one of the combination of HU moments, simple shape descriptors, or Fourier descriptors.

[0077] When calculating the center offset, it can be calculated according to the center offset calculation formula, and the center offset calculation formula is as follows: Formula 4 Among them, is the center offset. The center coordinates of the main and strabismus defect areas are ( C Mx ,C My ) and ([[]]END]] C Sx ,C Sy ), that is, ( C Mx ,C My ) is the center coordinate of a certain defect area in the first binary image, and ( C Sx ,C Sy ) is the center coordinate of a certain defect area in the second binary image.

[0078] When calculating the length ratio, it can be calculated according to the length ratio calculation formula, and the length ratio calculation formula is as follows: Formula 5 Among them, is the length ratio. The lengths of the main and strabismus corresponding defect areas in the longitudinal direction are L M , L S , that is, L M is the length of a certain defect area in the longitudinal direction in the first binary image, L S is the length of a certain defect area in the longitudinal direction in the second binary image. Generally, L S is greater than L M .

[0079] In the process of performing parallax detection on the defect information of the first binary image and the second binary image to obtain the dust removal screen, when detecting a certain defect area, it is carried out in the order of whether the detection matching degree value is greater than the matching degree value threshold, whether the detection center offset is less than the center offset threshold, and whether the detection length ratio is less than the length ratio threshold. During this process, once the condition is met, the detection of the defect area is ended, and the defect area is included in the dust removal screen.

[0080] It should be noted that in the embodiments of the present application, in some embodiments of the present application, in the specific process of performing parallax detection on the defect information of the first grayscale image and the second grayscale image according to the partition information in step 106 to obtain the initial defect screen, reference can be made to the foregoing steps 201-202, 301-303, 401-402, 501-502, which will not be elaborated here.

[0081] It should be noted that in the process of performing parallax detection on the defect information of the first grayscale image and the second grayscale image to obtain the initial defect screen, when detecting a certain defect area, it is carried out in the order of whether the detection matching degree value is less than or equal to the matching degree value threshold, whether the detection center offset is greater than or equal to the center offset threshold, and whether the detection length ratio is greater than or equal to the length ratio threshold. During this process, if the precondition is met, the detection of the defect area continues, and if all conditions are met, the defect area is determined to be a real foreign object under the screen.

[0082] Please refer to Figure 6 , in some embodiments of the present application, before step 102 in the above embodiments converts the second image into a front view image according to the preset main side view conversion relationship, the following steps may further be included: 601. Use the binocular vision system to photograph the calibration pattern to obtain the front view dot matrix diagram and the side view dot matrix diagram.

[0083] The server uses the binocular vision system to photograph a calibration pattern and obtains two dot matrix diagrams: the front view dot matrix diagram and the side view dot matrix diagram. The calibration pattern is a known reference pattern or figure, usually including a series of marked points with known positions. Through the binocular vision system, the front view dot matrix diagram is obtained from the front view angle, while the side view dot matrix diagram is obtained from the side view angle. The role of these dot matrix diagrams is to provide reference points for subsequent image geometric transformation.

[0084] By photographing the calibration pattern and obtaining the front view and side view dot matrix diagrams, the server provides the necessary reference data for calculating the view angle conversion relationship. These dot matrix diagrams can help calibrate the camera view angle and ensure more accurate subsequent image conversion.

[0085] 602. Calculate the main side view conversion relationship according to the front view dot matrix diagram and the test dot matrix diagram.

[0086] The server uses the calibration points in the front view dot matrix map and the side view dot matrix map, and through geometric transformation algorithms (such as perspective transformation, homography matrix calculation, etc.) to calculate the conversion relationship between the front and side views. This process is based on the corresponding relationship between the two dot matrix maps. The server obtains a conversion matrix or transformation formula through mathematical calculations, and this formula can convert the side view image into the front view image to ensure their alignment in space. It should be noted that in the embodiments of the present application, according to the front view dot matrix map and the test dot matrix map, the server can calculate the conversion relationship between the front and side views through various formulas or formula combinations or custom formulas, as long as the accuracy of the conversion relationship between the front and side views is ensured.

[0087] By accurately calculating the conversion relationship between the front and side views, the server can ensure that the image taken from the side can be correctly converted into the front view image, thereby improving the accuracy of image analysis. In subsequent steps, this conversion relationship will ensure the geometric consistency between the images and reduce the errors caused by the perspective difference.

[0088] This embodiment provides the basic data for accurately calculating the view conversion relationship, enabling the server to correctly convert the side view image into the front view image in subsequent steps, thus ensuring the high precision of image conversion.

[0089] Please refer to Figure 7 , an embodiment of the screen foreign object detection device based on binocular vision in the embodiments of the present application includes: A shooting unit 701, configured to shoot a target screen body through a binocular vision system to obtain a first image and a second image, where the first image is a front view image and the second image is a side view image.

[0090] A conversion unit 702, configured to convert the second image into a front view image according to a preset conversion relationship between the front and side views.

[0091] An acquisition unit 703, configured to acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image.

[0092] A defect detection unit 704, configured to perform defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information.

[0093] A first parallax detection unit 705, configured to perform parallax detection on the defect information of the first binary image and the second binary image according to the partition information of the target screen body to obtain a dust removal picture. Among them, the partition information is used to distinguish the straight part and the curved part of the target screen body. The dust removal picture represents the defects on the screen.

[0094] The second parallax detection unit 706 is configured to perform parallax detection on the defect information of the first grayscale image and the second grayscale image according to the partition information, so as to obtain an initial defect screen.

[0095] The screening unit 707 is configured to screen the initial defect screen according to the dust removal screen to obtain a target defect screen.

[0096] In this embodiment, the front view image and the converted side view image provide defect data from different perspectives, ensuring comprehensive detection of the target screen body. Combining the partition information of the target screen body, the screen foreign object detection device can perform parallax detection on the straight and curved surface areas respectively, effectively identify and process the defects in these areas, thereby avoiding misjudgment caused by curved surface distortion. Through this differential processing, the screen foreign object detection device can significantly improve the overall detection accuracy.

[0097] Optionally, the first parallax detection unit 705 includes: A determination subunit, configured to determine the partition attributes of the defect information of the first binary image and the second binary image according to the partition information of the target screen body, where the partition attributes include straight surfaces and curved surfaces.

[0098] A detection subunit, configured to perform parallax detection on the defect information of the first binary image and the second binary image according to the partition attributes to obtain a dust removal screen.

[0099] Optionally, the determination subunit includes: An acquisition module, configured to acquire the partition lines of the partition information of the target screen body.

[0100] A calculation module, configured to calculate the target distances between the defect information of the first binary image and the second binary image and the partition lines.

[0101] A first determination module, configured to determine the partition attributes of the defect information of the first binary image and the second binary image according to the target distances.

[0102] Optionally, the detection subunit includes: A second determination module, configured to determine a parallax threshold according to the partition attributes.

[0103] A detection module, configured to perform parallax detection on the defect information of the first binary image and the second binary image according to the parallax threshold to obtain a dust removal screen.

[0104] Optionally, the second determination module is specifically configured to: When the partition attribute is a straight surface, determine the preset threshold as the parallax threshold.

[0105] When the partition attribute is a curved surface, substitute the target distance into the threshold calculation formula to calculate the parallax threshold.

[0106] Optionally, the parallax threshold includes a matching degree value threshold, a central offset threshold, and a length ratio threshold.

[0107] Optionally, the apparatus further includes a conversion relationship unit, and the conversion relationship unit is configured to: Capture a calibration map through a binocular vision system to obtain a main view dot matrix map and a side view dot matrix map.

[0108] Calculate the main-side view conversion relationship according to the main view dot matrix map and the test dot matrix map.

[0109] In this embodiment, the functions of each unit and module correspond to the steps in the foregoing Figures 1 to 6 illustrated embodiment, and will not be elaborated herein.

[0110] Please refer to Figure 8 , an embodiment of the electronic device in the embodiment of the present application includes: A processor 801, a memory 802, an input / output unit 803, and a bus 804; The processor 801 is connected to the memory 802, the input / output unit 803, and the bus 804; A program is stored on the memory 802, and the processor 801 calls the program to execute Figures 1 to 6 the steps in the illustrated embodiment.

[0111] In this embodiment, the function of the processor 801 corresponds to the steps in the foregoing Figures 1 to 6 illustrated embodiment, and will not be elaborated herein.

[0112] The embodiment of the present application further provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the computer is caused to execute the method in any one of the foregoing Figures 1 to 6 possible implementation manners.

[0113] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, apparatuses, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0114] In several embodiments provided by the present application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division manners in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the couplings, direct couplings, or communication connections shown or discussed with each other may be through some interfaces, and the indirect couplings or communication connections of the apparatuses or units may be in electrical, mechanical, or other forms.

[0115] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in various embodiments of the present application may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0117] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, read-only memory), random access memories (RAM, random access memory), magnetic disks, or optical discs that can store program codes.

Claims

1. A screen foreign body detection method based on binocular vision, characterized in that: include: The target screen is photographed by a binocular vision system to obtain a first image and a second image, wherein the first image is a front view image and the second image is a side view image; According to a preset main-side view conversion relationship, converting the second image into a front view image; Acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image; Performing defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information; According to the partition information of the target screen, parallax detection is performed on the defect information of the first binary image and the second binary image to obtain a dust removal picture; wherein the partition information is used to distinguish between a straight surface part and a curved surface part of the target screen; and the dust removal picture represents defects on the screen; According to the partition information, disparity detection is performed on defect information of the first grayscale image and the second grayscale image to obtain an initial defect image; The initial defective picture is screened according to the dust removal picture to obtain a target defective picture.

2. The method according to claim 1, characterized in that: The method of performing parallax detection on defect information of the first binary image and the second binary image according to the partition information of the target screen to obtain a dust removal picture includes: Determining partition attributes of defect information of the first binary image and the second binary image according to partition information of the target screen, wherein the partition attributes include a straight surface and a curved surface; The dust removal picture is obtained by performing parallax detection on defect information of the first binary image and the second binary image according to the partition attribute.

3. The method according to claim 2, characterized in that The determining, according to the partition information of the target screen, the partition attributes of the defect information of the first binary image and the second binary image comprises: Obtaining partition lines of partition information of the target screen; Calculating a target distance between defect information of the first binary image and the second binary image and the partition line; A partition attribute of defect information of the first binary image and the second binary image is determined according to the target distance.

4. The method according to claim 3, characterized in that The performing of parallax detection on defect information of the first binary image and the second binary image according to the partition attribute to obtain the dust removal picture includes: determining a disparity threshold according to the partition attribute; A dust removal picture is obtained by performing disparity detection on defect information of the first binary image and the second binary image according to the disparity threshold.

5. The method according to claim 4, characterized in that The determining of the disparity threshold according to the partition attribute comprises: When the partition attribute is a straight surface, determining the preset threshold to be a disparity threshold; When the partition attribute is a curved surface, the target distance is brought into a threshold calculation formula to calculate the disparity threshold.

6. The method according to claim 4 or 5, characterized in that: The disparity threshold includes a matching value threshold, a center offset threshold and a length ratio threshold.

7. The method according to claim 1, characterized in that Before converting the second image into a front view image according to the preset main view to side view conversion relationship, the method further includes: The calibration image is photographed by a binocular vision system to obtain a main view dot matrix image and a side view dot matrix image; The main view and side view conversion relationship is calculated according to the main view dot matrix diagram and the test dot matrix diagram.

8. A screen foreign body detection device based on binocular vision, characterized in that: include: A shooting unit, used for shooting a target screen through a binocular vision system to obtain a first image and a second image, wherein the first image is a front view image and the second image is a side view image; A conversion unit, configured to convert the second image into a front view image according to a preset main view to side view conversion relationship; an acquisition unit, configured to acquire a first binary image and a first grayscale image corresponding to the first image, and a second binary image and a second grayscale image corresponding to the second image; a defect detection unit, configured to perform defect detection on the first binary image, the first grayscale image, the second binary image, and the second grayscale image to obtain corresponding defect information; A first parallax detection unit is used to perform parallax detection on defect information of the first binary image and the second binary image according to partition information of the target screen, so as to obtain a dust removal picture; wherein the partition information is used to distinguish a straight surface part and a curved surface part of the target screen; and the dust removal picture represents defects on the screen; A second disparity detection unit, configured to perform disparity detection on defect information of the first grayscale image and the second grayscale image according to the partition information to obtain an initial defective picture; The screening unit is used to screen the initial defective picture according to the dust removal picture to obtain a target defective picture.

9. An electronic device, characterized in that: include: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; A program is stored in the memory, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images