Crosshair card detection method, apparatus, intelligent device and readable storage medium

Abstract

This application discloses a method, apparatus, smart device, and readable storage medium for detecting crosshair cards, belonging to the field of smart wearable technology. The method includes acquiring a crosshair card displayed in an optical imaging module; determining the center point of the connected domain of the crosshair card, the center point of the acquired image, and a rotation deviation angle; and determining the sub-pixel center point of the crosshair card based on an affine transformation matrix established between the rotation deviation angle, the center point of the acquired image, and the center point of the connected domain. By applying the crosshair card detection method of this application to head-mounted wearable devices, the detection efficiency and accuracy of crosshair cards are improved, thereby enhancing the quality control of head-mounted wearable devices.

Description

Technical Field

[0001] This application relates to the field of smart wearable technology, and in particular to a crosshair card detection method, apparatus, smart device, and computer-readable storage medium. Background Technology

[0002] AR (Augmented Reality) and VR (Virtual Reality) head-mounted wearable devices are more sensitive to optical axis deviation than other optical devices due to their near-eye optics. The magnitude of the deviation in the overall optical imaging module directly affects the wearer's comfort and immersion.

[0003] To analyze the wearability effect of the optical imaging module, a crosshair detection method can be used. Crosshair detection refers to detecting the center point of the acquired crosshair, which is often achieved through template matching algorithms. However, the accuracy and speed of template matching detection are significantly limited by the hardware detection environment. Furthermore, because the camera's imaging plane and the display module (optical imaging module) are prone to relative rotation, the camera acquires a rotated crosshair instead of a true crosshair, which also affects the accuracy and speed of detection. Therefore, current crosshair detection methods suffer from low accuracy and speed. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, intelligent device, and computer-readable storage medium for detecting crosshair charts, aiming to solve the technical problems of low accuracy and speed in current crosshair chart detection methods.

[0005] To achieve the above objectives, this application provides a crosshair card detection method, which is applied to a head-mounted wearable device; the head-mounted wearable device includes an optical imaging module;

[0006] The method includes the following steps:

[0007] The crosshair chart displayed in the optical imaging module is captured;

[0008] Determine the center point of the connected domain of the crosshair chart, the center point of the acquired image, and the rotation deviation angle;

[0009] The sub-pixel center point of the crosshair is determined based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected region.

[0010] Optionally, the step of determining the sub-pixel center point of the crosshair card based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected component includes:

[0011] Establish an affine transformation matrix from the center point of the connected component to the center point of the acquired image, including the rotation deviation angle;

[0012] Determine the transpose affine transformation matrix corresponding to the affine transformation matrix;

[0013] Based on the affine transformation matrix and the transposed affine transformation matrix, subpixel line fitting is performed on the acquired image corresponding to the crosshair to determine the subpixel center point of the crosshair.

[0014] Optionally, the step of performing sub-pixel line fitting on the acquired image corresponding to the crosshair based on the affine transformation matrix and the transposed affine transformation matrix includes:

[0015] The acquired image corresponding to the crosshair card is translated and rotated according to the affine transformation matrix to obtain a translated positive acquired image;

[0016] The horizontal edge extraction image and the vertical edge extraction image of the translation positive acquisition image are obtained respectively.

[0017] The set of outer contour points in the horizontal direction of the edge extraction image and the set of outer contour points in the vertical direction of the edge extraction image are determined respectively.

[0018] The transpose affine transformation matrix is ​​applied to the outer contour point set in the horizontal direction and the outer contour point set in the vertical direction, respectively, to fit a first fitting line corresponding to the outer contour point set in the horizontal direction and a second fitting line corresponding to the outer contour point set in the vertical direction.

[0019] Optionally, the step of determining the sub-pixel center point of the crosshair card includes:

[0020] Determine the intersection point between the first fitted line and the second fitted line;

[0021] The intersection point is determined as the sub-pixel center point of the crosshair card.

[0022] Optionally, the step of determining the set of outer contour points in the horizontal direction of the edge extraction image and the set of outer contour points in the vertical direction of the edge extraction image includes:

[0023] Morphological operations are performed on the edge extraction images in the horizontal and vertical directions respectively to obtain double bright lines in the horizontal and vertical directions.

[0024] The set of outer contour points in the horizontal direction and the set of outer contour points in the vertical direction are determined based on the double bright lines in the horizontal direction and the double bright lines in the vertical direction, respectively.

[0025] Optionally, after the step of determining the sub-pixel center point of the crosshair, the method further includes:

[0026] Determine the coordinate deviation between the sub-pixel center point and the center point of the acquired image;

[0027] The visual level of the optical imaging module is determined based on the coordinate deviation.

[0028] Optionally, the step of determining the center point of the connected domain of the cross-shaped chart includes:

[0029] Determine the acquired image corresponding to the crosshair card, and determine the binarized acquired image corresponding to the acquired image;

[0030] According to the preset connected component filtering conditions, the binarized acquired image is denoised to obtain a denoised cross-connected component image.

[0031] The center point of the cross-shaped connected region image is determined as the center point of the connected region of the cross-shaped chart.

[0032] Furthermore, to achieve the above objectives, this application also provides a crosshair card detection device, the crosshair card detection device comprising:

[0033] The image acquisition module is used to acquire the crosshair chart displayed in the optical imaging module;

[0034] The image processing module is used to determine the center point of the connected domain of the crosshair, the center point of the acquired image, and the rotation deviation angle; and to determine the sub-pixel center point of the crosshair based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected domain.

[0035] In addition, to achieve the above objectives, this application also provides an intelligent device, including a processor, a storage unit, and a crosshair card detection program stored on the storage unit and executable by the processor, wherein when the crosshair card detection program is executed by the processor, it implements the steps of the crosshair card detection method as described above.

[0036] This application also provides a computer-readable storage medium storing a crosshair card detection program, wherein when the crosshair card detection program is executed by a processor, it implements the steps of the crosshair card detection method as described above.

[0037] The crosshair card detection method in this application, after the image acquisition device such as a camera acquires the crosshair card displayed by the optical imaging module, only needs to obtain the center point of the acquired image of the crosshair card, which is actually the center coordinate of the camera of the image acquisition device. After setting certain connectivity conditions and performing simple preprocessing, the center point of the connected domain of the crosshair card is obtained. Considering some application scenarios where there is relative rotation between the camera imaging plane and the display module, the rotation deviation angle of the crosshair card also needs to be obtained. An affine transformation matrix is ​​established between the rotation deviation angle, the center point of the acquired image, and the center point of the connected domain. Based on this affine transformation matrix, sub-pixel fitting processing can be performed on the acquired image of the crosshair card to obtain the accurate sub-pixel center point of the crosshair card. Because the image processing of the crosshair card in this application is extremely simple, it does not require the high hardware and computing power requirements of template matching, thus significantly improving the efficiency of crosshair card detection. Furthermore, this application detects the crosshair card from a sub-pixel perspective, so its accuracy is also much higher than traditional detection methods. This application can also improve the accuracy and speed of detection in scenarios where there is relative rotation between the camera imaging plane and the display module. It can perform a positive crosshair rotation process on the rotated crosshair pattern by adjusting the rotation deviation angle. Compared to traditional template matching, this application has a wider range of applications, enabling 100% inspection of optical devices such as head-mounted wearable devices with high speed and accuracy, rather than selective sampling, thus significantly improving the quality control of head-mounted wearable devices. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the hardware operating environment of the smart device involved in the embodiments of this application;

[0039] Figure 2 This is a flowchart illustrating the first embodiment of the crosshair card detection method of this application;

[0040] Figure 3 This is a detailed flowchart of step S30 in one embodiment of the crosshair card detection method of this application;

[0041] Figure 4 This is a detailed flowchart of step S33 in one embodiment of the crosshair card detection method of this application;

[0042] Figure 5 This is a flowchart illustrating the subpixel line fitting process involved in the crosshair detection method of this application.

[0043] Figure 6 This is a photographic illustration of the cross-shaped card involved in the cross-shaped card detection method of this application;

[0044] Figure 7 This is a schematic diagram of the preprocessing workflow involved in the crosshair card detection method of this application.

[0045] Figure 8 This is a schematic diagram illustrating the coordinate transformation involved in the cross-shaped card detection method of this application;

[0046] Figure 9 This is a schematic diagram of the x-direction edge extraction involved in the crosshair detection method of this application;

[0047] Figure 10 This is a schematic diagram of edge extraction in the y-direction involved in the crosshair detection method of this application;

[0048] Figure 11 This is a schematic diagram of the frame structure of the cross-shaped card detection device of this application.

[0049] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0051] This application proposes an intelligent device. The intelligent device can be any data processing device with image acquisition and processing capabilities, such as a personal computer, workstation, server, etc., with image acquisition and processing capabilities, and is not limited thereto.

[0052] like Figure 1 As shown, Figure 1 This is a schematic diagram of the hardware operating environment of the smart device involved in the embodiments of this application.

[0053] like Figure 1As shown, the intelligent device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a storage unit 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display or an input unit such as a control panel; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The storage unit 1005 may be a high-speed RAM storage unit or a stable storage unit (non-volatile memory), such as a disk storage unit. The storage unit 1005 may also optionally be a storage device independent of the aforementioned processor 1001. The storage unit 1005, as a computer storage medium, may include a crosshair card detection program.

[0054] Those skilled in the art will understand that Figure 1 The hardware structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0055] Continue to refer to Figure 1 , Figure 1 The storage unit 1005, which is a computer-readable storage medium, may include an operating system, a user interface module, a network communication module, and a crosshair card detection program.

[0056] exist Figure 1 In this module, the network communication module is mainly used to connect to the server and communicate with it for data; while the processor 1001 can call the crosshair card detection program stored in the storage unit 1005 and perform the following operations:

[0057] The crosshair chart displayed in the optical imaging module is captured;

[0058] Determine the center point of the connected domain of the crosshair chart, the center point of the acquired image, and the rotation deviation angle;

[0059] The sub-pixel center point of the crosshair is determined based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected region.

[0060] Furthermore, the processor 1001 can call the crosshair card detection program stored in the memory 1005 and also perform the following operations:

[0061] Establish an affine transformation matrix from the center point of the connected component to the center point of the acquired image, including the rotation deviation angle;

[0062] Determine the transpose affine transformation matrix corresponding to the affine transformation matrix;

[0063] Based on the affine transformation matrix and the transposed affine transformation matrix, subpixel line fitting is performed on the acquired image corresponding to the crosshair to determine the subpixel center point of the crosshair.

[0064] Furthermore, the processor 1001 can call the crosshair card detection program stored in the memory 1005 and also perform the following operations:

[0065] The acquired image corresponding to the crosshair card is translated and rotated according to the affine transformation matrix to obtain a translated positive acquired image;

[0066] The horizontal edge extraction image and the vertical edge extraction image of the translation positive acquisition image are obtained respectively.

[0067] The set of outer contour points in the horizontal direction of the edge extraction image and the set of outer contour points in the vertical direction of the edge extraction image are determined respectively.

[0068] The transpose affine transformation matrix is ​​applied to the outer contour point set in the horizontal direction and the outer contour point set in the vertical direction, respectively, to fit a first fitting line corresponding to the outer contour point set in the horizontal direction and a second fitting line corresponding to the outer contour point set in the vertical direction.

[0069] Furthermore, the processor 1001 can call the crosshair card detection program stored in the memory 1005 and also perform the following operations:

[0070] Determine the intersection point between the first fitted line and the second fitted line;

[0071] The intersection point is determined as the sub-pixel center point of the crosshair card.

[0072] Furthermore, the processor 1001 can call the crosshair card detection program stored in the memory 1005 and also perform the following operations:

[0073] Morphological operations are performed on the edge extraction images in the horizontal and vertical directions respectively to obtain double bright lines in the horizontal and vertical directions.

[0074] The set of outer contour points in the horizontal direction and the set of outer contour points in the vertical direction are determined based on the double bright lines in the horizontal direction and the double bright lines in the vertical direction, respectively.

[0075] Furthermore, the processor 1001 can call the crosshair card detection program stored in the memory 1005 and also perform the following operations:

[0076] Determine the coordinate deviation between the sub-pixel center point and the center point of the acquired image;

[0077] The visual level of the optical imaging module is determined based on the coordinate deviation.

[0078] Furthermore, the processor 1001 can call the crosshair card detection program stored in the memory 1005 and also perform the following operations:

[0079] Determine the acquired image corresponding to the crosshair card, and determine the binarized acquired image corresponding to the acquired image;

[0080] According to the preset connected component filtering conditions, the binarized acquired image is denoised to obtain a denoised cross-connected component image.

[0081] The center point of the cross-shaped connected region image is determined as the center point of the connected region of the cross-shaped chart.

[0082] To facilitate understanding of the following embodiments of this application, the main technical solutions of this application are briefly described below:

[0083] This application is mainly applied to scenarios where there is relative rotation between the camera imaging plane and the display module. The reasons for this relative rotation include: factors in the detection environment of the crosshair card that prevent the camera from physically facing the display module to take pictures; VR / AR and other head-mounted wearable devices need a certain rotation angle to detect the crosshair card in order to improve the user's viewing experience; and other scenarios that require a certain angle of relative rotation between the camera imaging plane and the display module.

[0084] After an image acquisition device such as an industrial camera acquires the crosshair displayed by the optical imaging module, the center point of the acquired image (the center point of the acquired image) can be obtained. The acquired image is preprocessed according to certain connected component conditions to obtain the center point of the (crosshair) connected component, and the rotation deviation angle of the acquired crosshair is obtained. Then, the translation information in the affine transformation matrix obtained by subtracting the two center points is added to the rotation information in the affine transformation matrix. Based on this affine transformation matrix, the center point of the connected component is translated to the center point of the acquired image, and then rotated to obtain the translated and aligned acquired image. The edges in the x and y directions of the translated and aligned acquired image are extracted, and morphological operations are performed to obtain the contour point sets in the x and y directions respectively. The transpose of the affine transformation matrix is ​​used to revert the contour point sets in the x and y directions back to the coordinate system before translation and alignment. The reverted contour point sets are fitted to obtain two intersecting lines, and the intersection point is the sub-pixel center point, that is, the more accurate center coordinate point of the crosshair. This application achieves more accurate optical axis detection of the optical imaging module and eliminates the need for time-consuming algorithms such as template matching, thus significantly improving the detection efficiency of crosshair cards.

[0085] Based on the hardware structure of the aforementioned smart device, various embodiments of the crosshair card detection method of this application are proposed.

[0086] This application provides a method for detecting crosshair charts.

[0087] Please refer to Figure 2 , Figure 2 This is a schematic flowchart of the first embodiment of the crosshair card detection method of this application; in the first embodiment of this application, the crosshair card detection method is applied to a head-mounted wearable device; the head-mounted wearable device includes an optical imaging module;

[0088] The crosshair card detection method includes the following steps:

[0089] Step S10: Acquire the crosshair pattern card displayed in the optical imaging module;

[0090] In this embodiment, the head-mounted wearable device can be an AR or VR device. Since the distance between the user's eyes and the screen is very close when using these devices, this places extremely high demands on the optical imaging module. The optical axis offset should be minimized as much as possible to improve the user's comfort, immersion and other viewing experience.

[0091] The crosshair pattern displayed by the optical imaging module of a head-mounted wearable device can be captured using an image acquisition device such as an industrial camera (hereinafter referred to as a camera for convenience). The optical imaging module includes at least a display screen and a lens positioned in front of the display screen.

[0092] A specific data acquisition scenario could be as follows: an industrial camera is pointing at a display screen to photograph a crosshair pattern, and the image of the crosshair pattern displayed on the screen is captured through lenses in the optical imaging module. The purpose of this is to simulate the actual viewing experience of the human eye when wearing a head-mounted wearable device.

[0093] For the concept of the cross chart, please refer to [link / reference]. Figure 6 This can be considered a cross pattern rotated at a certain angle relative to the upright cross. The reason for this phenomenon is as described above and will not be repeated here. It can be seen that the grayscale distribution of the cross pattern card is that the grayscale value increases from the endpoints of the cross inwards.

[0094] Step S20: Determine the center point of the connected domain of the crosshair card, the center point of the acquired image, and the rotation deviation angle;

[0095] After capturing the crosshair card, the captured image is obtained, such as... Figure 6 As shown, the center point of the captured image on the crosshair chart refers to the center point (x0, y0) of the captured image, which is actually the center point of the camera coordinate system.

[0096] To determine the center point of a connected component in an acquired image, the acquired image can be preprocessed for noise reduction and connectivity selection as needed, thereby obtaining the center point (x) of the connected component. i y i ).

[0097] In one embodiment, step S20, the step of determining the center point of the connected domain of the cross-shaped chart, includes:

[0098] Step a: Determine the acquired image corresponding to the crosshair card, and determine the binarized acquired image corresponding to the acquired image;

[0099] Step b: According to the preset connected component filtering conditions, the binarized acquired image is denoised to obtain a denoised cross-connected component image.

[0100] Step c: Determine the center point of the cross-shaped connected region image as the center point of the connected region of the cross-shaped chart.

[0101] To facilitate understanding of this embodiment and step S20, refer to... Figure 7 Explanation:

[0102] After the camera captures the crosshair card, it obtains an image of the crosshair card. The captured image is then binarized to obtain a corresponding binarized image, which presents a clear black and white effect, highlighting the outline of the target and facilitating the detection of the crosshair card.

[0103] Based on the binarized acquired image, a certain connected component algorithm (such as the two-pass scanning method, seed filling method, etc.) can be used to break the cross-shaped connected component and select the required cross-shaped connected regions based on the cross-shaped connected component features and setting the required connected component selection conditions. In other words, the cross-shaped connected regions are marked from the binarized acquired image, and other irrelevant image noise is removed to obtain the denoised cross-shaped connected component image. After obtaining the cross-shaped connected component image, its image center point, i.e., the coordinates of the connected component center (x, y), can be calculated. i y i ).

[0104] It should be noted that, as can be seen Figure 7 The center point of the connected domain (the vertex where θ is located) is often not the coordinate of the cross center point, so it is necessary to further optimize the center point coordinates to obtain a more accurate sub-pixel center point.

[0105] Furthermore, continue to refer to Figure 7 Regarding the rotation deviation angle, as mentioned above, a cross-connected region can be selected based on the acquired image, and the set of contour coordinate points of this cross-connected region can be obtained (corresponding to the schematic diagram of contour points in the figure). Then, a preset rotation rectangle fitting algorithm can be used to obtain the rotation deviation angle θ. Specifically, the preset rotation rectangle fitting algorithm can include the `boundingRect` and `minAreaRect` functions from OpenCV (a cross-platform computer vision library). These two functions can obtain the bounding rectangle of the cross-connected region, and the offset angle of the bounding rectangle relative to the positive rectangle (the rectangle without rotation offset). This offset angle is also the rotation deviation angle θ of the cross-shaped chart.

[0106] By performing the above-described preprocessing on the acquired image through this embodiment of the application, the consistency of the algorithm during subsequent image processing and the simplicity of coordinate transformation can be ensured, thereby improving the detection efficiency of the crosshair card.

[0107] Step S30: Determine the sub-pixel center point of the crosshair card based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected domain.

[0108] The affine transformation matrix established between the rotation deviation angle, the center point of the acquired image, and the center point of the connected component reflects how to move from one coordinate system to another and how the acquired crosshair card rotates. In this embodiment, the affine transformation matrix can be obtained by moving from the center point of the connected component to the center point of the acquired image and adding a rotation deviation angle. Based on this affine transformation matrix, a series of edge extraction and morphological operations are performed. Then, the transpose of the affine transformation matrix is ​​used to return to the original coordinate system before translation, and sub-pixel line fitting is performed in the x and y directions to obtain two intersecting lines. The corresponding intersection point is the sub-pixel level center point.

[0109] Please refer to Figure 3 In one embodiment, step S30, which involves determining the sub-pixel center point of the crosshair card based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image, and the center point of the connected component, includes:

[0110] Step S31: Establish an affine transformation matrix from the center point of the connected component to the center point of the acquired image and including the rotation deviation angle;

[0111] The translation information of the affine transformation matrix is ​​obtained by calculating the coordinate changes from the center point of the connected component to the center point of the acquired image. In other words, the translation information of the affine transformation matrix is ​​obtained by subtracting the coordinates of the center point of the connected component from the center point of the acquired image. Adding the rotation deviation angle to the same affine transformation matrix gives the affine transformation matrix rotation information as well.

[0112] To better understand the role of the affine transformation matrix in the embodiments, please refer to... Figure 8 For example, Figure 8 The diagram has two center points, left and right. The center point (x) in the left diagram is the center point of the connected domain. i y i The bottom right corner is the center point (x0, y0) of the acquired image. An affine transformation is performed from left to right. That is, the center point coordinates are translated based on the translation information in the affine transformation matrix, and the rotation information (rotation deviation angle) in the affine transformation matrix is ​​used to rotate the crosshair in the acquired image until the crosshair is rotated into a regular crosshair.

[0113] Step S32: Determine the transpose affine transformation matrix corresponding to the affine transformation matrix;

[0114] Transposing the affine transformation matrix yields the transposed affine transformation matrix.

[0115] Step S33: Based on the affine transformation matrix and the transposed affine transformation matrix, perform subpixel line fitting on the acquired image corresponding to the crosshair card to determine the subpixel center point of the crosshair card.

[0116] Under the action of the affine transformation matrix and the transposed affine transformation matrix, the crosshair in the acquired image is first translated and rotated, then the edges are extracted, morphological operations are performed, and connected components are determined to obtain the outer contour point set in the x and y directions. These outer contour point sets are then returned to the original coordinates for sub-pixel fitting, thereby detecting the center point (coordinates) of the crosshair more accurately at the sub-pixel level.

[0117] Specifically, please refer to Figure 4 In one embodiment, step S33 includes:

[0118] Step S330: Translate and rotate the acquired image corresponding to the crosshair according to the affine transformation matrix to obtain a translated positive acquired image;

[0119] After determining the affine transformation matrix, refer to the example described above. Figure 8 The role of the affine transformation matrix is ​​to translate and rotate the acquired image to obtain a translated and aligned image, such as... Figure 8 As shown in the right figure.

[0120] Step S331: Obtain the edge extraction images of the translation positive acquisition image in the horizontal axis direction and the edge extraction images in the vertical axis direction, respectively.

[0121] The horizontal axis, or x, and the vertical axis, or y, correspond to the two directions of a two-dimensional coordinate system.

[0122] In digital image processing, an edge refers to the processing of the image's outline. An edge is defined as a region where the grayscale value changes drastically at its boundary.

[0123] By using a preset edge detection algorithm (such as the Sobel operator), edge detection and extraction in the x-direction and y-direction of the translated positive acquisition image can be performed separately, resulting in edge extraction images in the horizontal and vertical directions.

[0124] Step S332: Determine the set of outer contour points in the horizontal direction of the edge extraction image and the set of outer contour points in the vertical direction of the edge extraction image, respectively.

[0125] Having obtained the edge extraction images along the horizontal and vertical axes, we can directly obtain the corresponding outer contour point sets along the horizontal and vertical axes, respectively. Each point set contains the coordinates of its respective points.

[0126] To reduce interference from other pixels and improve the accuracy of subpixel fitting, morphological operations can be performed on the edge extraction images along the horizontal and vertical axes, specifically closing operations.

[0127] That is, in one embodiment, step S332 includes:

[0128] Step d: Perform morphological operations on the edge extraction images in the horizontal and vertical directions respectively to obtain double bright lines in the horizontal and vertical directions.

[0129] Please refer to Figure 9 and Figure 10 Some general morphological functions can be used to perform morphological operations on the edge extraction images along the horizontal and vertical axes. For example, this can be done through closing operations. After performing morphological operations, for the edge extraction image along the horizontal axis, such as... Figure 9 As shown, six bright lines (patterns with higher grayscale values) were retained. Similarly, for edge extraction images along the vertical axis, as... Figure 10 As shown, six bright lines are also retained. Continue to refer to... Figure 9 Following a similar connected component approach as described in the above embodiments, vertically connecting each of the four middle bright lines in pairs vertically results in two bright lines along the horizontal axis, which are the two middle vertical bright lines. The same principle applies to... Figure 10 Double bright lines were also obtained in the vertical direction.

[0130] Step e: Determine the set of outer contour points in the horizontal direction and the set of outer contour points in the vertical direction based on the double bright lines in the horizontal direction and the double bright lines in the vertical direction, respectively.

[0131] Obtaining the pixel outer contour point set of the double bright lines along the horizontal axis yields the outer contour point set along the horizontal axis. Similarly, obtaining the pixel outer contour point set of the double bright lines along the vertical axis yields the outer contour point set along the vertical axis.

[0132] Step S333: Apply the transposed affine transformation matrix to the set of outer contour points in the horizontal direction and the set of outer contour points in the vertical direction, respectively, to fit and obtain the first fitting line corresponding to the set of outer contour points in the horizontal direction and the second fitting line corresponding to the set of outer contour points in the vertical direction.

[0133] The transpose affine transformation matrix is ​​applied to the outer contour point sets in the horizontal and vertical directions, respectively, so that the outer contour point sets in the horizontal and vertical directions return to the coordinate system of the original acquired image. Then, the outer contour point sets in the horizontal and vertical directions are fitted respectively. The purpose of the fitting is to obtain a straight line, a first fitted line and a second fitted line, that is, to fit a total of two straight lines, and these two straight lines intersect.

[0134] For the method of fitting a straight line based on a point set, the least squares method of line fitting can be performed using the fitline function, so that the first and second fitted lines are closer to the horizontal and vertical axes of symmetry of the cross chart, thus making the intersection point more accurate.

[0135] In one embodiment, step S30, the step of determining the sub-pixel center point of the crosshair card, includes:

[0136] Step f: Determine the intersection point between the first fitted line and the second fitted line;

[0137] Step g: Determine the intersection point as the sub-pixel center point of the crosshair card.

[0138] The first and second fitted lines are often perpendicular to each other. They are compared to a single point, and this intersection point is more precise than the pixel level, that is, the sub-pixel center point. Compared to the traditional template matching traversal method of finding the crosshair, this method can only locate the pixel level with greater precision.

[0139] In one embodiment, after step S30, the method further includes:

[0140] Step h: Determine the coordinate deviation between the sub-pixel center point and the center point of the acquired image;

[0141] Step i: Determine the visual level of the optical imaging module based on the coordinate deviation.

[0142] Ideally, the center point of the captured image in an optical imaging module should correspond precisely to the center point of the crosshair pattern displayed on the screen. However, factors such as screen quality and assembly processes can cause deviations from this expectation. For example, if the lens module of a wearable headband experiences a positional misalignment during assembly, the location of an image as perceived by the human eye will not be the actual location displayed on the screen, resulting in a poor viewing experience. In the case of automated crosshair pattern detection, this manifests as the center of the captured crosshair pattern not being the center of the displayed crosshair pattern.

[0143] In this application, after obtaining the precise subpixel center point in the acquired image, in order to evaluate the quality of the optical imaging module, the subpixel center point can be subtracted from the center point of the acquired image to obtain a coordinate deviation. A certain visual level or quality level is preset, and each level corresponds to a deviation range. Then, based on the coordinate deviation, it is seen that the coordinate deviation is within the deviation range, thereby determining the visual level or quality level. This helps the inspection personnel to evaluate the optical imaging module, improves inspection efficiency, and reduces labor costs.

[0144] In addition, it can help inspectors determine how to adjust the optical imaging module based on coordinate deviations, so as to achieve a more ideal imaging effect and a better user experience.

[0145] The crosshair card detection method in this application, after the image acquisition device such as a camera acquires the crosshair card displayed by the optical imaging module, only needs to obtain the center point of the acquired image of the crosshair card, which is actually the center coordinate of the camera of the image acquisition device. After setting certain connectivity conditions and performing simple preprocessing, the center point of the connected domain of the crosshair card is obtained. Considering some application scenarios where there is relative rotation between the camera imaging plane and the display module, the rotation deviation angle of the crosshair card also needs to be obtained. An affine transformation matrix is ​​established between the rotation deviation angle, the center point of the acquired image, and the center point of the connected domain. Based on this affine transformation matrix, sub-pixel fitting processing can be performed on the acquired image of the crosshair card to obtain the accurate sub-pixel center point of the crosshair card. Because the image processing of the crosshair card in this application is extremely simple, it does not require the high hardware and computing power requirements of template matching, thus significantly improving the efficiency of crosshair card detection. Furthermore, this application detects the crosshair card from a sub-pixel perspective, so its accuracy is also much higher than traditional detection methods. This application can also improve the accuracy and speed of detection in scenarios where there is relative rotation between the camera imaging plane and the display module. It can perform a positive crosshair rotation process on the rotated crosshair pattern by adjusting the rotation deviation angle. Compared to traditional template matching, this application has a wider range of applications, enabling 100% inspection of optical devices such as head-mounted wearable devices with high speed and accuracy, rather than selective sampling, thus significantly improving the quality control of head-mounted wearable devices.

[0146] To facilitate understanding of the implementation process of the main technical solutions in this application, please refer to... Figure 5 .like Figure 5 As shown:

[0147] Establish central coordinates (x i, y i The affine transformation matrix M from the center coordinates (x0, y0) to the center coordinates (x0, y0) and includes the rotational deviation angle θ;

[0148] Calculate the inverse transformation matrix M -1 ;

[0149] Apply the affine transformation matrix M to the image (acquired image);

[0150] x-direction:

[0151] Perform edge extraction in the x-direction;

[0152] Perform morphological operations in the x-direction;

[0153] Find the coordinate points of the edge contour in the x-direction;

[0154] M -1 Contour points acting in the x-direction;

[0155] Perform contour point-line fitting in the x-direction to form the L1 line equation;

[0156] y direction:

[0157] Perform edge extraction in the y-direction;

[0158] Perform morphological operations in the y-direction;

[0159] Find the coordinate points of the edge contour in the y-direction;

[0160] M -1 Contour points acting in the y-direction;

[0161] Perform contour point-line fitting in the y-direction to form the equation of the L2 line;

[0162] Solving by intersecting L1 and L2, we get (x) f ,y f ), the (x) f ,y f This refers to the sub-pixel center point mentioned in the above embodiments.

[0163] In addition, refer to Figure 11 , Figure 11 This is a schematic diagram of the frame structure of the crosshair card detection device of this application. This application also proposes a crosshair card detection device, which includes:

[0164] Image acquisition module A10 is used to acquire the crosshair chart displayed in the optical imaging module;

[0165] Image processing module A20 is used to determine the center point of the connected domain of the crosshair, the center point of the acquired image, and the rotation deviation angle; and to determine the sub-pixel center point of the crosshair based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected domain.

[0166] Optionally, the image processing module A20 is further configured to:

[0167] Establish an affine transformation matrix from the center point of the connected component to the center point of the acquired image, including the rotation deviation angle;

[0168] Determine the transpose affine transformation matrix corresponding to the affine transformation matrix;

[0169] Based on the affine transformation matrix and the transposed affine transformation matrix, subpixel line fitting is performed on the acquired image corresponding to the crosshair to determine the subpixel center point of the crosshair.

[0170] Optionally, the image processing module A20 is further configured to:

[0171] The acquired image corresponding to the crosshair card is translated and rotated according to the affine transformation matrix to obtain a translated positive acquired image;

[0172] The horizontal edge extraction image and the vertical edge extraction image of the translation positive acquisition image are obtained respectively.

[0173] The set of outer contour points in the horizontal direction of the edge extraction image and the set of outer contour points in the vertical direction of the edge extraction image are determined respectively.

[0174] The transpose affine transformation matrix is ​​applied to the outer contour point set in the horizontal direction and the outer contour point set in the vertical direction, respectively, to fit a first fitting line corresponding to the outer contour point set in the horizontal direction and a second fitting line corresponding to the outer contour point set in the vertical direction.

[0175] Optionally, the image processing module A20 is further configured to:

[0176] Determine the intersection point between the first fitted line and the second fitted line;

[0177] The intersection point is determined as the sub-pixel center point of the crosshair card.

[0178] Optionally, the image processing module A20 is further configured to:

[0179] Morphological operations are performed on the edge extraction images in the horizontal and vertical directions respectively to obtain double bright lines in the horizontal and vertical directions.

[0180] The set of outer contour points in the horizontal direction and the set of outer contour points in the vertical direction are determined based on the double bright lines in the horizontal direction and the double bright lines in the vertical direction, respectively.

[0181] Optionally, the image processing module A20 is further configured to:

[0182] Determine the coordinate deviation between the sub-pixel center point and the center point of the acquired image;

[0183] The visual level of the optical imaging module is determined based on the coordinate deviation.

[0184] Optionally, the image processing module A20 is further configured to:

[0185] Determine the acquired image corresponding to the crosshair card, and determine the binarized acquired image corresponding to the acquired image;

[0186] According to the preset connected component filtering conditions, the binarized acquired image is denoised to obtain a denoised cross-connected component image.

[0187] The center point of the cross-shaped connected region image is determined as the center point of the connected region of the cross-shaped chart.

[0188] The specific implementation of the cross-shaped card detection device in this application is basically the same as the embodiments of the cross-shaped card detection method described above, and will not be repeated here.

[0189] Furthermore, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a crosshair card detection program, wherein when executed by a processor, the crosshair card detection program implements the steps of the crosshair card detection method as described above.

[0190] The method implemented when the crosshair card detection program is executed can be referred to in various embodiments of the crosshair card detection method of this application, and will not be repeated here.

[0191] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0192] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0193] These computer program instructions may also be stored in a computer-readable storage unit that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage unit produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0194] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0195] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. This application can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0196] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0197] The above description is merely a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural transformations made based on the inventive concept of this application and the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. A method for detecting crosshair charts, characterized in that, The crosshair detection method is applied to a head-mounted wearable device; the head-mounted wearable device includes an optical imaging module; The method includes the following steps: Collect the crosshair chart displayed in the optical imaging module; Determine the center point of the connected domain of the crosshair chart, the center point of the acquired image, and the rotation deviation angle; The sub-pixel center point of the crosshair is determined based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected region; The step of determining the sub-pixel center point of the crosshair card based on the rotation deviation angle, the affine transformation matrix established between the center point of the acquired image and the center point of the connected component includes: Establish an affine transformation matrix from the center point of the connected component to the center point of the acquired image, including the rotation deviation angle; Determine the transpose affine transformation matrix corresponding to the affine transformation matrix; The acquired image corresponding to the crosshair card is translated and rotated according to the affine transformation matrix to obtain a translated positive acquired image; The horizontal edge extraction image and the vertical edge extraction image of the translation positive acquisition image are obtained respectively. The set of outer contour points in the horizontal direction of the edge extraction image and the set of outer contour points in the vertical direction of the edge extraction image are determined respectively. The transposed affine transformation matrix is ​​applied to the outer contour point set in the horizontal direction and the outer contour point set in the vertical direction, respectively, to fit a first fitting line corresponding to the outer contour point set in the horizontal direction and a second fitting line corresponding to the outer contour point set in the vertical direction, so as to determine the sub-pixel center point of the cross-shaped card.

2. The crosshair detection method as described in claim 1, characterized in that, The step of determining the sub-pixel center point of the crosshair card includes: Determine the intersection point between the first fitted line and the second fitted line; The intersection point is determined as the sub-pixel center point of the crosshair card.

3. The crosshair detection method as described in claim 1, characterized in that, The steps of determining the horizontal axis outer contour point set of the horizontal axis edge extraction image and the vertical axis outer contour point set of the vertical axis edge extraction image respectively include: Morphological operations are performed on the edge extraction images in the horizontal and vertical directions respectively to obtain double bright lines in the horizontal and vertical directions. The set of outer contour points in the horizontal direction and the set of outer contour points in the vertical direction are determined based on the double bright lines in the horizontal direction and the double bright lines in the vertical direction, respectively.

4. The crosshair detection method as described in claim 1, characterized in that, After the step of determining the sub-pixel center point of the crosshair, the method further includes: Determine the coordinate deviation between the sub-pixel center point and the center point of the acquired image; The visual level of the optical imaging module is determined based on the coordinate deviation.

5. The crosshair detection method as described in claim 1, characterized in that, The step of determining the center point of the connected domain of the cross-shaped chart includes: Determine the acquired image corresponding to the crosshair card, and determine the binarized acquired image corresponding to the acquired image; According to the preset connected component filtering conditions, the binarized acquired image is denoised to obtain a denoised cross-connected component image. The center point of the cross-shaped connected region image is determined as the center point of the connected region of the cross-shaped chart.

6. A crosshair card detection device, characterized in that, The crosshair card detection device includes: The image acquisition module is used to acquire the crosshair chart displayed in the optical imaging module; The image processing module is used to determine the center point of the connected component of the crosshair chart, the center point of the acquired image, and the rotation deviation angle; based on the affine transformation matrix established between the rotation deviation angle, the center point of the acquired image, and the center point of the connected component, determine the sub-pixel center point of the crosshair chart; the image processing module is also used to establish an affine transformation matrix from the center point of the connected component to the center point of the acquired image and including the rotation deviation angle; determine the transpose affine transformation matrix corresponding to the affine transformation matrix; and translate and rotate the acquired image corresponding to the crosshair chart according to the affine transformation matrix to obtain a flat... The image is shifted and acquired; the edge extraction images in the horizontal direction and the vertical direction of the shifted and acquired image are obtained respectively; the outer contour point set in the horizontal direction of the edge extraction image and the outer contour point set in the vertical direction of the edge extraction image are determined respectively; the transposed affine transformation matrix is ​​applied to the outer contour point set in the horizontal direction and the outer contour point set in the vertical direction respectively to fit a first fitting line corresponding to the outer contour point set in the horizontal direction and a second fitting line corresponding to the outer contour point set in the vertical direction, so as to determine the sub-pixel center point of the cross-shaped card.

7. A smart device, characterized in that, The intelligent device includes a memory, a processor, and a crosshair card detection program stored in the memory and executable on the processor, wherein: when the crosshair card detection program is executed by the processor, it implements the steps of the crosshair card detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a crosshair card detection program, wherein when the crosshair card detection program is executed by a processor, it implements the steps of the crosshair card detection method as described in any one of claims 1 to 5.

Images