Glasses combined image quality detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a glasses combined image quality detection method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a calibration image collected by a binocular camera, wherein the calibration image comprises a left eye image and a right eye image; performing coordinate extraction on the left eye image and the right eye image to obtain a left eye corner pixel coordinate and a right eye corner pixel coordinate which are matched with each other; for each group of matched left eye corner point pixel coordinates and right eye corner point pixel coordinates, performing coordinate projection on the left eye corner point pixel coordinates or the right eye corner point pixel coordinates based on parameters of the binocular camera to obtain a group of projected target corner point pixel coordinates and reference corner point pixel coordinates; and determining the glasses combination quality of the intelligent glasses according to each group of target angular point pixel coordinates and reference angular point pixel coordinates, thereby solving the problems of low glasses combination precision judgment accuracy, manpower resource consumption and the like, improving the combination precision judgment accuracy, being free of manual participation, being capable of realizing automation, and being objective and accurate in detection result.

Description

Technical Field

[0001] The present application relates to the field of automated detection technology, and in particular to a method, device, electronic device, and storage medium for detecting the quality of eyeglass image formation. Background Art

[0002] With the development of science and technology, smart glasses are increasingly used in daily life; smart glasses can be virtual reality (VR) glasses, augmented reality (AR) glasses, etc. In smart glasses, the precise alignment of the left and right screens (i.e., "image convergence") is a core technical indicator to ensure user experience. If there is a deviation in the physical posture (rotation and translation) of the two screens, it will cause the virtual image and the real scene to be misaligned, ghosting, or abnormal binocular vision, causing user dizziness, visual fatigue and other problems. Especially in industrial-grade glasses (such as remote maintenance and precision assembly scenarios), the image convergence accuracy needs to be controlled at the sub-pixel level (for example, ≤1.0 pixel). The current technology is to manually judge the image convergence accuracy, which consumes time and human resources, has low accuracy, is not objective enough, and cannot be implemented in large quantities. Summary of the Invention

[0003] The present application provides a method, device, electronic device and storage medium for detecting the image quality of glasses to improve the accuracy of image accuracy judgment. The image quality of glasses can be judged in batches objectively and accurately, saving time and human resources.

[0004] According to one aspect of the present application, a method for detecting the quality of eyeglass image formation is provided, comprising:

[0005] Acquire a calibration image captured by a binocular camera, wherein the calibration image includes a left-eye image and a right-eye image, and the binocular camera is set on one side of the smart glasses;

[0006] Extracting coordinates of the left-eye image and the right-eye image to obtain matching left-eye corner pixel coordinates and right-eye corner pixel coordinates;

[0007] For each set of matched left corner pixel coordinates and right corner pixel coordinates, coordinate projection is performed on the left corner pixel coordinates or the right corner pixel coordinates based on the parameters of the binocular camera to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates;

[0008] The image quality of the smart glasses is determined according to each group of target corner pixel coordinates and reference corner pixel coordinates.

[0009] According to another aspect of the present application, a device for detecting the quality of eyeglass image formation is provided, comprising:

[0010] An image acquisition module is used to acquire a calibration image collected by a binocular camera, wherein the calibration image includes a left eye image and a right eye image. The binocular camera is set on one side of the smart glasses;

[0011] A coordinate extraction module is used to extract the coordinates of the left eye image and the right eye image to obtain the matching left eye corner pixel coordinates and right eye corner pixel coordinates;

[0012] A coordinate projection module is used to perform coordinate projection on each set of matched left corner pixel coordinates and right corner pixel coordinates based on the parameters of the binocular camera to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates;

[0013] The quality detection module is used to determine the quality of the glasses image of the smart glasses according to each group of target corner point pixel coordinates and reference corner point pixel coordinates.

[0014] According to another aspect of the present application, an electronic device is provided, comprising:

[0015] at least one processor, and a memory communicatively coupled to the at least one processor;

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the glasses image quality detection method described in any embodiment of the present application.

[0017] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the glasses image quality detection method described in any embodiment of the present application when executed.

[0018] According to another aspect of the present application, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, it implements the glasses image quality detection method described in any embodiment of the present application.

[0019] The technical solution of the embodiment of the present application is to obtain a calibration image captured by a binocular camera, wherein the calibration image includes a left-eye image and a right-eye image, and the binocular camera is set on one side of the smart glasses, and the smart glasses form the calibration image by projecting a calibration plate; coordinates of the left-eye image and the right-eye image are extracted to obtain matching left-eye corner pixel coordinates and right-eye corner pixel coordinates; for each set of matching left-eye corner pixel coordinates and right-eye corner pixel coordinates, based on the parameters of the binocular camera, coordinate projection is performed on the left-eye corner pixel coordinates or the right-eye corner pixel coordinates to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates; and the image quality of the smart glasses is determined according to each set of the target corner pixel coordinates and the reference corner pixel coordinates. After the smart glasses are projected through the calibration plate to form a calibration image, the calibration image is collected by the binocular camera, simulating the observation of the calibration image by the human eye. The collection by the binocular camera can realize the batch detection of the quality of glasses image combination, saving time and human resources; by extracting the coordinates of the left eye image and the right eye image, the matching left eye corner point pixel coordinates and the right eye corner point pixel coordinates are obtained, and the left eye corner point pixel coordinates or the right eye corner point pixel coordinates are projected based on the parameters of the binocular camera to obtain a set of coordinates in the same coordinate system, and finally the image quality is evaluated according to the target corner point pixel coordinates and the reference corner point pixel coordinates obtained by projection, and the glasses image quality of the smart glasses is determined. The result is more objective and accurate, which can effectively improve the accuracy of the judgment of the image accuracy. The method does not require human participation and can be automatically implemented, and the detection result is objective and accurate; it solves the problems of low accuracy in judging the accuracy of glasses image combination and human resource consumption.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 This is a flow chart of a method for detecting the quality of eyeglass image formation according to the first embodiment of the present application;

[0023] Figure 2 This is a flow chart of a method for detecting the quality of eyeglass image formation according to the second embodiment of the present application;

[0024] Figure 3 1 is a schematic structural diagram of a device for detecting the quality of eyeglass image formation according to the third embodiment of the present application;

[0025] Figure 4 Schematic diagram of the structure of an electronic device for implementing the glasses image quality detection method of an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Example 1

[0029] Figure 1 This is a flow chart of a method for detecting the quality of eyeglass image formation provided in the first embodiment of the present application. This embodiment is applicable to the case of detecting the quality of eyeglass image formation. The method can be performed by an eyeglass image formation quality detection device. The eyeglass image formation quality detection device can be implemented in the form of hardware and / or software. The eyeglass image formation quality detection device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0030] S101. Acquire a calibration image captured by a binocular camera, where the calibration image includes a left-eye image and a right-eye image. The binocular camera is set on one side of the smart glasses.

[0031] In this embodiment, a binocular camera is an imaging device that simulates the principles of human binocular vision. It typically consists of two parallel cameras, one simulating the left eye and the other the right eye. These cameras, referred to as the left and right cameras, respectively, capture two images when the binocular camera observes a scene. The calibration image can be understood as the image projected by the smart glasses. The left-eye image can be understood as the image displayed by the left eye during the simulation of human vision, i.e., the image seen by the left camera. The right-eye image can be understood as the image displayed by the right eye during the simulation of human vision, i.e., the image seen by the right camera.

[0032] A binocular camera is pre-installed and placed on one side of the smart glasses, that is, on the side where the image projected by the smart glasses can be viewed. The smart glasses project a calibration plate to form and display a calibration image. The calibration plate can be an April Tag plate, a checkerboard plate, a circular plate, or other similar plates. The plate includes one or more corner points, so the calibration image formed includes corner points. The binocular camera simulates the human eye observing and gazing at the calibration image and captures the calibration image. The resulting calibration image includes a left image and a right image, corresponding to the image observed by the left eye and the image observed by the right eye, respectively.

[0033] S102 : Extract coordinates of the left eye image and the right eye image to obtain matching pixel coordinates of the left eye corner point and the right eye corner point.

[0034] In this embodiment, the left-eye corner point pixel coordinates may be understood as the coordinate information of the corner point in the left-eye image; the right-eye corner point pixel coordinates may be understood as the coordinate information of the corner point in the right-eye image.

[0035] Coordinates are extracted from the left and right images to obtain the pixel coordinates of the left corner point in the left image and the pixel coordinates of the right corner point in the right image. A determination is made as to whether the left corner point pixel coordinates and the right corner point pixel coordinates are the same corner point. If so, the left corner point pixel coordinates and the right corner point pixel coordinates are determined to be a set of matching left corner point pixel coordinates and right corner point pixel coordinates. Both the left and right images may contain multiple corner point pixel coordinates. If the left and right images contain multiple corner point pixel coordinates, the matching left corner point pixel coordinates and right corner point pixel coordinates need to be determined by analyzing the corner point pixel coordinates. For example, the matching left corner point pixel coordinates and right corner point pixel coordinates may be determined by analyzing one or more pieces of information, such as positions and identifiers, of the multiple left corner point pixel coordinates in the left image and the multiple right corner point pixel coordinates in the right image. For example, there are 3 left-eye corner point pixel coordinates in the left-eye image, which are identified as 1, 2 and 3 respectively, and there are 2 right-eye corner point pixel coordinates in the right-eye image, which are identified as 1 and 2 respectively. The two corner point pixel coordinates with the same identification can be considered as matching corner point pixel coordinates, that is, the left-eye corner point pixel coordinate identified as 1 and the right-eye corner point pixel coordinate identified as 1 are a set of matching left-eye corner point pixel coordinates and right-eye corner point pixel coordinates, the left-eye corner point pixel coordinate identified as 2 and the right-eye corner point pixel coordinate identified as 2 are a set of matching left-eye corner point pixel coordinates and right-eye corner point pixel coordinates, and there are two sets of matching left-eye corner point pixel coordinates and right-eye corner point pixel coordinates in total.

[0036] S103. For each set of matched left corner pixel coordinates and right corner pixel coordinates, coordinate projection is performed on the left corner pixel coordinates or the right corner pixel coordinates based on the binocular camera parameters to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates.

[0037] In this embodiment, the binocular camera parameters can be one or more types of parameters, such as intrinsic parameters and extrinsic parameters. The target corner pixel coordinates can be understood as the coordinates obtained by projecting the left corner pixel coordinates or the right corner pixel coordinates into another coordinate system, where the other coordinate system is a coordinate system different from the coordinate system in which the corner pixel coordinates are projected. The reference corner pixel coordinates can be understood as the coordinates of the corner point used as a reference.

[0038] For each set of matched left corner pixel coordinates and right corner pixel coordinates, coordinate projection can be performed in the following manner: one of the left corner pixel coordinates and the right corner pixel coordinates is selected for projection, a coordinate can be randomly selected for projection, or a coordinate can be selected for projection according to certain rules. For example, the coordinate selection rule is: pre-setting the priority of the left corner pixel coordinates and the right corner pixel coordinates, and selecting one of the left corner pixel coordinates and the right corner pixel coordinates for projection based on the priority, etc. Based on the parameters of the binocular camera, the selected left corner pixel coordinates or the right corner pixel coordinates are projected to obtain the target corner pixel coordinates, and the unprojected right corner pixel coordinates or the left corner pixel coordinates are used as the reference corner pixel coordinates; projection is performed in the above manner to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates. The target corner pixel coordinates and the reference corner pixel coordinates in the embodiment of the present application are coordinates in the same coordinate system.

[0039] S104: Determine the image quality of the smart glasses according to the target corner pixel coordinates and the reference corner pixel coordinates.

[0040] In this embodiment, the image quality of the smart glasses can be rated as pass / fail, excellent / good / poor, level one / level two / level three, or can be specifically scored, etc. The image quality can be described in various ways. The target corner pixel coordinates of each group are compared with the reference corner pixel coordinates to determine whether the target corner pixel coordinates of each group match the reference corner pixel coordinates. The image quality of the smart glasses is then determined based on the judgment result. For example, if the target corner pixel coordinates and the reference corner pixel coordinates coincide or coincide within an allowable error range, the target corner pixel coordinates and the reference corner pixel coordinates are considered to match. If more than a certain number of target corner pixel coordinates match the reference corner pixel coordinates, the image quality of the smart glasses is determined to be pass. Otherwise, the image quality of the smart glasses is determined to be fail. Alternatively, the degree of matching between the target corner pixel coordinates and the reference corner pixel coordinates is calculated. For example, the degree of matching may be the distance between the target corner pixel coordinates and the reference corner pixel coordinates. For example, the distance may be the Manhattan distance, the Chebyshev distance, etc. Different degrees of matching correspond to different scores. After the degree of matching is calculated, the score corresponding to the degree of matching is determined by the correspondence between the pre-stored degree of matching and the score. A score may be determined for each set of target corner pixel coordinates and reference corner pixel coordinates. The average, maximum, minimum, etc. of the scores of each set are taken to obtain the final score. The final score is directly used as the glasses image quality. Alternatively, the final score is compared with the set score threshold. If the score is higher than the score threshold, the glasses image quality of the smart glasses is determined to be qualified. Otherwise, the glasses image quality of the smart glasses is determined to be unqualified.

[0041] The eyeglasses image quality detection method provided in the embodiments of the present application solves the problems of low accuracy in judging eyeglass image accuracy and wasted human resources. After the smart glasses project a calibration image through a calibration plate, the calibration image is captured by a binocular camera. This simulates human eye observation of the calibration image and captures the calibration image through the binocular camera, which can achieve batch detection and save time and human resources. By extracting coordinates from the left and right eye images, matching left and right eye corner pixel coordinates are obtained. The left or right eye corner pixel coordinates are projected based on the binocular camera parameters to obtain a set of coordinates in the same coordinate system. Finally, the image quality is evaluated based on the projected target corner pixel coordinates and reference corner pixel coordinates to determine the image quality of the smart glasses. The result is more objective and accurate, which can effectively improve the accuracy of image accuracy judgment. The method does not require human intervention and can complete the evaluation with only a single shot. The image processing is time-saving and can be automated, making it suitable for automated production line detection and meeting the needs of large-scale factory inspection. The deployment threshold and maintenance cost are low, and only a binocular camera is required for implementation.

[0042] Example 2

[0043] Figure 2 This is a flow chart of a method for detecting the quality of glasses image quality provided in the second embodiment of this application. This embodiment is refined based on the above embodiment. Figure 2 As shown, the method includes:

[0044] S201. Acquire a calibration image captured by a binocular camera, where the calibration image includes a left-eye image and a right-eye image. The binocular camera is set on one side of the smart glasses.

[0045] Optionally, the calibration image is formed by projecting a calibration plate, and the parameters of the calibration plate are determined according to the type of smart glasses.

[0046] The parameters of the calibration plate can be one or more parameters such as tag density, size, and arrangement. A tag generally refers to a special mark or label used to assist in the camera calibration process. Considering the differences in the structural design and camera viewing angles of different smart glasses, the parameters of the calibration plate can be selected according to the type of smart glasses to improve calibration efficiency and accuracy. For example, the parameters of the calibration plate corresponding to different types of smart glasses can be pre-set and stored in a data table. When testing the image quality of the smart glasses, the data table can be queried to determine the parameters of the calibration plate based on the type of smart glasses.

[0047] The binocular camera in the embodiment of the present application can also be set to be adjustable, which can be adjusted automatically or manually. Taking into account the differences between different smart glasses models and user wearing habits, the binocular camera can be adjusted by simulating the position of the human eye. The parameters such as the baseline between the cameras, the installation height, and the pitch angle can be adjusted according to the shape of the glasses and the typical eye position model of the user group, thereby improving the detection accuracy. For example, the same type of smart glasses corresponds to the same set of installation structure parameters of the binocular camera, different types of smart glasses correspond to the same set of installation structure parameters of the binocular camera, or different types of smart glasses can also correspond to the same set of installation structure parameters of the binocular camera. The installation structure parameters can be one or more parameters such as the height of the binocular camera, the distance between the left and right cameras of the binocular camera, the pitch angle of the binocular camera, and the distance between the binocular camera and the smart glasses. When evaluating the image quality of smart glasses, the type of smart glasses is first determined, and then a determination is made as to whether the type of smart glasses matches the mounting structure parameters of the current binocular camera. For example, the mounting structure parameters of the binocular camera corresponding to each type of smart glasses are pre-stored and a corresponding relationship is established. During the image quality test, the type of smart glasses is determined, the mounting structure parameters of the current binocular camera are obtained and recorded as mounting structure parameters 1. Based on the pre-stored corresponding relationship, the mounting structure parameters of the binocular camera corresponding to the type of smart glasses are determined and recorded as mounting structure parameters 2. Mounting structure parameters 1 and 2 are compared to see if they are the same. If they are the same, it is determined that the type of smart glasses matches the mounting structure parameters of the current binocular camera. If they are different, it is determined that the type of smart glasses does not match the mounting structure parameters of the current binocular camera. If they do match, the image quality test is performed. If not, the mounting structure parameters of the binocular camera that matches the type of smart glasses are obtained, and control instructions are generated based on these mounting structure parameters to adjust the structure of the binocular camera. If they match, a calibration image can be directly acquired. To reduce the number of adjustments and avoid repeated adjustments, when testing the image quality of glasses in batches, the same type of smart glasses can be tested collectively. This allows multiple smart glasses to be tested for image quality after adjusting the binocular camera structure once, saving adjustment time. When adjusting the binocular camera structure, the type of smart glasses can be determined by taking an image of the smart glasses and analyzing it, by scanning a QR code or barcode containing information about the smart glasses type, or by manual input.

[0048] S202: Extract coordinates from the left-eye image to obtain at least one candidate left-eye corner pixel coordinate.

[0049] In this embodiment, the candidate left-eye corner pixel coordinates can be understood as the coordinates of the corner points contained in the left-eye image. The left-eye image may contain multiple corner points, that is, the number of candidate left-eye corner pixel coordinates may be one or more. The coordinates of the corner points in the left-eye image are extracted. The coordinate extraction can be implemented according to a set algorithm or rule. Through the coordinate extraction, at least one candidate left-eye corner pixel coordinate is obtained. Exemplarily, the coordinate extraction algorithm can be the accelerated segmentation test feature (Features from Accelerated Segment Test, FAST) corner detection algorithm, the scale-invariant feature transform (Scale-Invariant Feature Transform, SIFT) algorithm, etc.

[0050] S203: Extract coordinates of the right eye image to obtain at least one candidate right eye corner pixel coordinate.

[0051] In this embodiment, the candidate right corner pixel coordinates can be understood as the coordinates of a corner point contained in the right image. The right image may contain multiple corner points, i.e., the number of candidate right corner pixel coordinates may be one or more. Coordinates of the corner points in the right image are extracted. Coordinate extraction can be performed according to a predetermined algorithm or rule. Through coordinate extraction, at least one candidate right corner pixel coordinate is obtained.

[0052] S204: Match each candidate left corner pixel coordinate with each candidate right corner pixel coordinate, and select at least one set of matched left corner pixel coordinates and right corner pixel coordinates according to the matching result.

[0053] When there are more than one candidate left corner pixel coordinate and candidate right corner pixel coordinate, the candidate left corner pixel coordinates and candidate right corner pixel coordinates need to be matched. For example, each corner point in the calibration plate has a corresponding identifier, and the corner point pixel coordinates are matched based on the corner point identifier. Through matching, one or more sets of matching candidate left corner pixel coordinates and candidate right corner pixel coordinates can be determined, and at least one set of matching candidate left corner pixel coordinates and candidate right corner pixel coordinates is selected as the final matched left corner pixel coordinates and candidate right corner pixel coordinates.

[0054] S205. For each set of matched left corner point pixel coordinates and right corner point pixel coordinates, use the left corner point pixel coordinates as the coordinates to be projected and the right corner point pixel coordinates as the reference corner point pixel coordinates, or use the right corner point pixel coordinates as the coordinates to be projected and the left corner point pixel coordinates as the reference corner point pixel coordinates.

[0055] In this embodiment, the coordinates to be projected can be understood as the coordinates of the corner point that need to be projected into another coordinate system. Both the left corner point pixel coordinates and the right corner point pixel coordinates can be used as the coordinates to be projected. The left corner point pixel coordinates are used as the coordinates to be projected, and the right corner point pixel coordinates are used as the reference corner point pixel coordinates. Alternatively, the right corner point pixel coordinates are used as the coordinates to be projected, and the left corner point pixel coordinates are used as the reference corner point pixel coordinates.

[0056] S206 , projecting the coordinates to be projected based on the parameters of the binocular camera to obtain pixel coordinates of the target corner points.

[0057] The parameters of the binocular camera can be one or more of intrinsic parameters and extrinsic parameters. The coordinates to be projected are projected based on the parameters of the binocular camera, and the coordinates to be projected are projected into another coordinate system to obtain the pixel coordinates of the target corner points.

[0058] S207 , determining a reprojection error according to each group of target corner pixel coordinates and reference corner pixel coordinates.

[0059] In this embodiment, the reprojection error can be understood as the error value of the projected coordinates. The positions of each group of target corner pixel coordinates and the reference corner pixel coordinates are compared, and the reprojection error is determined based on the degree of positional overlap. For example, the difference between the horizontal coordinate or vertical coordinate of each group of target corner pixel coordinates and the reference corner pixel coordinates is calculated, and the maximum, minimum, or average value of each group of differences is calculated, and the calculation result is used as the reprojection error; or the average value of the difference between the horizontal coordinate and vertical coordinate of the target corner pixel coordinates and the reference corner pixel coordinates is calculated, and the maximum, minimum, or average value of each group of average values ​​is calculated, and the calculation result is used as the reprojection error.

[0060] S208: If the re-projection error is within the error range, determine that the image quality of the smart glasses is qualified.

[0061] In this embodiment, the error range can be pre-set, for example, based on accuracy requirements, the distance between the binocular camera and the smart glasses, etc. The reprojection error is compared with the error range. If the reprojection error is within the error range, it can be considered that the corner points are highly coincident in the same coordinate system, and the image quality of the smart glasses is determined to be acceptable.

[0062] S209: If the re-projection error is not within the error range, determine that the image quality of the smart glasses is unqualified.

[0063] Compare the size of the reprojection error and the error range. If the reprojection error is not within the error range, it can be considered that the corner points do not coincide in the same coordinate system, and the image quality of the smart glasses is determined to be unqualified.

[0064] In an embodiment of the present application, the left eye detected by the left camera in the binocular camera is projected onto the corner point of the calibration plate on the left screen, and is projected onto the corresponding position on the right screen detected by the right camera through the parameters of the binocular camera, and the reprojection error is calculated. If the binoculars of the smart glasses are not in alignment, the reprojection error will be relatively large. In this case, it means that the postures of the left and right screens are inconsistent, and the smart glasses need to be re-adjusted. If the smart glasses are in alignment, the reprojection error will be very small at this time, which means that the postures of the left and right screens are consistent. The consistency of the postures of the left and right screens is directly reflected by the size of the reprojection error, and the production line rework or user maintenance is guided. This method only requires taking one image, which is simple and accurate compared to other methods, and can give clear quantitative evaluation results. It is suitable for the requirements of smart glasses alignment evaluation in industrial scenarios.

[0065] Optionally, the coordinates to be projected are projected based on the parameters of the binocular camera to obtain the pixel coordinates of the target corner points, including steps A1-A2:

[0066] A1. When the coordinates to be projected are the pixel coordinates of the left eye corner, project the coordinates to the right eye pixel plane of the binocular camera based on the extrinsic parameters of the binocular camera, the intrinsic parameters of the left eye, and the first camera pose relationship to obtain the pixel coordinates of the target corner. The first camera pose relationship is the pose relationship between the left eye camera in the binocular camera and the calibration plate projected by the smart glasses.

[0067] In this embodiment, the first camera pose relationship can be understood as the pose relationship between the left camera and the calibration plate. The binocular camera includes a left camera and a right camera. Both the left camera and the right camera have corresponding intrinsic references. The left camera's intrinsic reference is denoted as the left-eye intrinsic reference, and the right camera's intrinsic reference is denoted as the right-eye intrinsic reference.

[0068] The first camera pose relationship is pre-determined based on the positions of the left camera and the calibration plate projected by the smart glasses. The binocular camera's extrinsic parameters, left-eye intrinsic parameters, and right-eye intrinsic parameters can all be pre-determined. Once the binocular camera's extrinsic parameters, left-eye intrinsic parameters, and right-eye intrinsic parameters are determined accordingly. When the coordinates to be projected are the left-eye corner pixel coordinates, the binocular camera's extrinsic parameters, left-eye intrinsic parameters, and the relationship between the first camera pose are determined. Based on a pre-determined projection formula, the coordinates to be projected are projected onto the binocular camera's right-eye pixel plane. This means that the image captured by the left camera is projected onto the coordinate plane of the right camera. The target corner pixel coordinates are obtained through coordinate projection.

[0069] For example, the present invention provides a coordinate projection calculation formula:

[0070]

[0071] in, is the pixel coordinate of the left corner point, K clis the left eye internal reference of the left eye camera, is the first camera pose relationship, is the external parameter of the binocular camera, is the pixel coordinate of the target corner point.

[0072] A2. When the coordinates to be projected are the pixel coordinates of the right eye corner point, based on the extrinsic parameters of the binocular camera, the intrinsic parameters of the right eye, and the position relationship of the second camera, the coordinates to be projected are projected onto the left eye pixel plane of the binocular camera to obtain the pixel coordinates of the target corner point. The position relationship of the second camera is the position relationship between the right eye camera in the binocular camera and the calibration plate projected by the smart glasses.

[0073] The position relationship of the second camera is pre-determined based on the position of the right-eye camera and the calibration plate projected by the smart glasses. When the coordinates to be projected are the pixel coordinates of the right-eye corner point, the extrinsic parameters of the binocular camera, the right-eye intrinsic parameters, and the position relationship of the second camera are determined. According to the predetermined projection formula, the coordinates to be projected are projected onto the left-eye pixel plane of the binocular camera. That is, the image captured by the right-eye camera is projected onto the coordinate plane of the left-eye camera. The pixel coordinates of the target corner point are obtained through coordinate projection.

[0074] For example, the present invention provides a coordinate projection calculation formula:

[0075]

[0076] in, is the pixel coordinate of the right eye corner, K cr is the right eye internal reference of the right eye camera, is the second camera pose relationship, is the external parameter of the binocular camera, is the pixel coordinate of the target corner point.

[0077] Optionally, determining the reprojection error based on the target corner pixel coordinates and the reference corner pixel coordinates includes: calculating the distance between each group of target corner pixel coordinates and the reference corner pixel coordinates, and determining the reprojection error based on each distance.

[0078] The distance between the target corner pixel coordinates and the reference corner pixel coordinates is calculated using a pre-set distance formula. For example, the Euclidean distance between the target corner pixel coordinates and the reference corner pixel coordinates is calculated. For each set of target corner pixel coordinates and reference corner pixel coordinates, the distance between the target corner pixel coordinates and the reference corner pixel coordinates is calculated. When there are multiple sets of target corner pixel coordinates and reference corner pixel coordinates, multiple distances can be obtained. The average, weighted sum, maximum, minimum, and other values ​​of the calculated distances are used as the reprojection error. Determining the reprojection error using distance is accurate and objective, and the implementation process is simple and easy to implement. The units of distance can be pixels, mm, cm, etc.; the units of reprojection error can also be pixels, mm, cm, etc.

[0079] For example, the error range can be a threshold value t, where the value of t can be pre-set, for example, 1.0 pixel, where pixels are used as units. If the reprojection error is greater than t, the image quality of the smart glasses is considered unqualified and requires recalibration; otherwise, the image quality of the smart glasses is qualified.

[0080] The glasses image quality detection method provided in the embodiment of the present application solves the problems of low accuracy in judging the accuracy of glasses image and waste of human resources. After the smart glasses form a calibration image through projection by the calibration plate, the calibration image is collected by the binocular camera to simulate the observation of the calibration image by the human eye. The collection by the binocular camera can realize batch detection and save time and human resources. Based on the internal parameters, external parameters and camera posture relationship of the binocular camera, the pixel coordinates of the left corner point or the right corner point are projected to obtain a set of coordinates in the same coordinate system. According to the projected target corner point pixel coordinates and the reference angle pixel coordinates, the target corner point pixel coordinates and the reference angle pixel coordinates are collected. The reprojection error is calculated based on the pixel coordinates of the points, and then the image quality is evaluated based on the reprojection error. The reprojection error based on the real optical path and the spatial corner points of the calibration plate is used as the evaluation standard. It can directly reflect the image state, avoid the error accumulation or human subjective judgment uncertainty in traditional indirect evaluation methods, and can effectively improve the reliability of image accuracy judgment. This method does not require human intervention and can complete the evaluation with only a single shot. The image processing is time-saving and can be automated, making it suitable for automated inspection on production lines and meeting the needs of large-scale factory inspection. The deployment threshold and maintenance cost are low, and it can be implemented with only a binocular camera.

[0081] Example 3

[0082] Figure 3 This is a schematic diagram of the structure of a device for detecting the quality of glasses image quality provided in Example 3 of this application. Figure 3 As shown, the device includes: an image acquisition module 31, a coordinate extraction module 32, a coordinate projection module 33 and a quality detection module 34.

[0083] An image acquisition module 31 is configured to acquire a calibration image captured by a binocular camera, the calibration image including a left eye image and a right eye image, wherein the binocular camera is disposed on one side of the smart glasses;

[0084] A coordinate extraction module 32 is used to extract coordinates of the left eye image and the right eye image to obtain matching left eye corner pixel coordinates and right eye corner pixel coordinates;

[0085] A coordinate projection module 33 is configured to perform coordinate projection on each set of matched left corner pixel coordinates and right corner pixel coordinates based on the parameters of the binocular camera to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates.

[0086] The quality detection module 34 is used to determine the quality of the glasses' combined image according to each group of the target corner pixel coordinates and the reference corner pixel coordinates.

[0087] The eyeglasses image quality detection method provided in the embodiments of the present application solves the problems of low accuracy in judging eyeglass image accuracy and wasted human resources. After the smart glasses project a calibration image through a calibration plate, the calibration image is captured by a binocular camera. This simulates human eye observation of the calibration image and captures the calibration image through the binocular camera, which can achieve batch detection and save time and human resources. By extracting coordinates from the left and right eye images, matching left and right eye corner pixel coordinates are obtained. The left or right eye corner pixel coordinates are projected based on the binocular camera parameters to obtain a set of coordinates in the same coordinate system. Finally, the image quality is evaluated based on the projected target corner pixel coordinates and reference corner pixel coordinates to determine the image quality of the smart glasses. The result is more objective and accurate, which can effectively improve the accuracy of image accuracy judgment. The method does not require human intervention and can complete the evaluation with only a single shot. The image processing is time-saving and can be automated, making it suitable for automated production line detection and meeting the needs of large-scale factory inspection. The deployment threshold and maintenance cost are low, and only a binocular camera is required for implementation.

[0088] Optionally, the coordinate extraction module 32 includes:

[0089] a left-eye coordinate extraction unit, configured to extract coordinates from the left-eye image to obtain at least one candidate left-eye corner pixel coordinate;

[0090] a right eye coordinate extraction unit, configured to extract coordinates from the right eye image to obtain at least one candidate right eye corner pixel coordinate;

[0091] The coordinate matching unit is used to match the pixel coordinates of each candidate left corner point with the pixel coordinates of each candidate right corner point, and select at least one set of matched left corner point pixel coordinates and right corner point pixel coordinates according to the matching results.

[0092] Optionally, the coordinate projection module 33 includes:

[0093] a projection coordinate selection unit, configured to use the left corner point pixel coordinates as the coordinates to be projected and the right corner point pixel coordinates as the reference corner point pixel coordinates, or to use the right corner point pixel coordinates as the coordinates to be projected and the left corner point pixel coordinates as the reference corner point pixel coordinates;

[0094] The projection unit is used to project the coordinates to be projected based on the parameters of the binocular camera to obtain the pixel coordinates of the target corner point.

[0095] Optionally, the projection unit is specifically used to: when the coordinates to be projected are the pixel coordinates of the left corner point, based on the external parameters and left-eye internal parameters of the binocular camera and the first camera pose relationship, project the coordinates to be projected to the right-eye pixel plane of the binocular camera to obtain the pixel coordinates of the target corner point, and the first camera pose relationship is the pose relationship between the left-eye camera in the binocular camera and the calibration plate projected by the smart glasses; when the coordinates to be projected are the pixel coordinates of the right corner point, based on the external parameters and right-eye internal parameters of the binocular camera and the second camera pose relationship, project the coordinates to be projected to the left-eye pixel plane of the binocular camera to obtain the pixel coordinates of the target corner point, and the second camera pose relationship is the pose relationship between the right-eye camera in the binocular camera and the calibration plate projected by the smart glasses.

[0096] Optionally, the quality detection module 34 includes:

[0097] a projection error determining unit, configured to determine a reprojection error based on each group of target corner point pixel coordinates and reference corner point pixel coordinates;

[0098] a first quality detection unit, configured to determine that the image quality of the smart glasses is qualified if the reprojection error is within an error range;

[0099] The second quality detection unit is configured to determine that the image quality of the smart glasses is unqualified if the reprojection error is not within an error range.

[0100] Optionally, the projection error determination unit is specifically used to calculate the distance between each group of target corner point pixel coordinates and reference corner point pixel coordinates, and determine the reprojection error according to each distance.

[0101] Optionally, the calibration image is formed by projecting a calibration plate, and parameters of the calibration plate are determined according to the type of the smart glasses.

[0102] The glasses image quality detection device provided in the embodiments of the present application can execute the glasses image quality detection method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0103] Example 4

[0104] Figure 4 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein. The electronic device in the embodiment of the present application can execute the glasses image quality detection method provided in any embodiment of the present application to detect the image quality of the smart glasses. The electronic device can be connected to a binocular camera to obtain a calibration image captured by the binocular camera. The electronic device can also include a binocular camera, which captures the calibration image and sends it to the processor for corresponding processing.

[0105] like Figure 4 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., which is communicatively connected to the at least one processor 41. The memory stores a computer program that can be executed by the at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 into the random access memory (RAM) 43. Various programs and data required for the operation of the electronic device 40 can also be stored in the RAM 43. The processor 41, ROM 42, and RAM 43 are connected to each other via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0106] Multiple components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0107] The processor 41 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the method for detecting the quality of eyeglass image formation.

[0108] In some embodiments, the eyeglasses image quality detection method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the eyeglasses image quality detection method described above can be performed. Alternatively, in other embodiments, processor 41 can be configured to execute the eyeglasses image quality detection method in any other appropriate manner (e.g., via firmware).

[0109] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for detecting the quality of eyeglass image formation as described in any embodiment of the present application.

[0112] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0115] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0116] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0117] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for detecting the quality of eyeglass image formation, characterized in that: include: Acquire a calibration image captured by a binocular camera, wherein the calibration image includes a left-eye image and a right-eye image, and the binocular camera is set on one side of the smart glasses; Extracting coordinates of the left-eye image and the right-eye image to obtain matching left-eye corner pixel coordinates and right-eye corner pixel coordinates; For each set of matched left corner pixel coordinates and right corner pixel coordinates, coordinate projection is performed on the left corner pixel coordinates or the right corner pixel coordinates based on the parameters of the binocular camera to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates; The image quality of the smart glasses is determined according to each group of target corner pixel coordinates and reference corner pixel coordinates.

2. The method according to claim 1, characterized in that The extracting coordinates of the left eye image and the right eye image to obtain matching left eye corner pixel coordinates and right eye corner pixel coordinates includes: Extracting coordinates of the left-eye image to obtain at least one candidate left-eye corner pixel coordinate; Extracting coordinates of the right eye image to obtain at least one candidate right eye corner pixel coordinate; Matching is performed on the candidate left corner point pixel coordinates and the candidate right corner point pixel coordinates, and at least one set of matched left corner point pixel coordinates and right corner point pixel coordinates is selected according to the matching result.

3. The method according to claim 1, characterized in that The method of performing coordinate projection on the left corner point pixel coordinates or the right corner point pixel coordinates based on the parameters of the binocular camera to obtain a set of projected target corner point pixel coordinates and reference corner point pixel coordinates includes: Using the left corner point pixel coordinates as the coordinates to be projected and the right corner point pixel coordinates as the reference corner point pixel coordinates, or using the right corner point pixel coordinates as the coordinates to be projected and the left corner point pixel coordinates as the reference corner point pixel coordinates; The coordinates to be projected are projected based on the parameters of the binocular camera to obtain the pixel coordinates of the target corner points.

4. The method according to claim 3, characterized in that The projecting the coordinates to be projected based on the parameters of the binocular camera to obtain the pixel coordinates of the target corner point includes: When the coordinates to be projected are left-eye corner pixel coordinates, projecting the coordinates to be projected onto the right-eye pixel plane of the binocular camera based on the extrinsic parameters and left-eye intrinsic parameters of the binocular camera and a first camera pose relationship to obtain the target corner pixel coordinates, wherein the first camera pose relationship is a pose relationship between the left-eye camera in the binocular camera and the calibration plate projected by the smart glasses; When the coordinates to be projected are the pixel coordinates of the right eye corner point, the coordinates to be projected are projected onto the left eye pixel plane of the binocular camera based on the extrinsic parameters and right eye intrinsic parameters of the binocular camera and the second camera pose relationship to obtain the pixel coordinates of the target corner point. The second camera pose relationship is the pose relationship between the right eye camera in the binocular camera and the calibration plate projected by the smart glasses.

5. The method according to claim 1, wherein The determining of the glasses image quality of the smart glasses according to each group of the target corner pixel coordinates and the reference corner pixel coordinates includes: Determine a reprojection error based on the target corner pixel coordinates and the reference corner pixel coordinates of each group; If the reprojection error is within the error range, determining that the image quality of the smart glasses is qualified; If the reprojection error is not within the error range, it is determined that the image quality of the smart glasses is unqualified.

6. The method according to claim 5, characterized in that Determining the reprojection error according to each group of target corner pixel coordinates and reference corner pixel coordinates includes: The distance between each group of target corner pixel coordinates and the reference corner pixel coordinates is calculated, and a reprojection error is determined based on each of the distances.

7. The method according to any one of claims 1 to 6, characterized in that The calibration image is formed by projecting a calibration plate, and the parameters of the calibration plate are determined according to the type of the smart glasses.

8. A device for detecting the quality of glasses image, characterized in that: include: An image acquisition module is used to acquire a calibration image collected by a binocular camera, wherein the calibration image includes a left eye image and a right eye image. The binocular camera is set on one side of the smart glasses; A coordinate extraction module is used to extract the coordinates of the left eye image and the right eye image to obtain the matching left eye corner pixel coordinates and right eye corner pixel coordinates; A coordinate projection module is used to perform coordinate projection on each set of matched left corner pixel coordinates and right corner pixel coordinates based on the parameters of the binocular camera to obtain a set of projected target corner pixel coordinates and reference corner pixel coordinates; The quality detection module is used to determine the quality of the glasses image of the smart glasses according to each group of target corner point pixel coordinates and reference corner point pixel coordinates.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the glasses image quality detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the glasses image quality detection method according to any one of claims 1 to 7 when executed.

Images