Cross chart detection method and device, intelligent device and readable storage medium

Abstract

The application discloses a cross chart detection method and device, intelligent equipment and a readable storage medium, and belongs to the technical field of intelligent wearing; the method comprises the following steps: collecting a cross chart displayed in an optical imaging module; determining a collection image center point and a connected domain center point of the cross chart; and determining a sub-pixel center point of the cross chart according to an affine transformation matrix established between the collection image center point and the connected domain center point. The cross chart detection method in the application is applied to a head-mounted wearable device, so that the detection efficiency and precision of the cross chart are improved, and the product control of the head-mounted wearable device is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent wearing, in particular to a cross chart detection method and device, intelligent equipment and a computer readable storage medium. BACKGROUND

[0002] AR(Augmented Reality, augmented reality), VR(Virtual Reality, virtual reality) and other head-mounted wearable devices are more sensitive to optical axis offset than other optical devices because of near-eye optics. The optical imaging module offset of the whole machine directly affects the comfort and immersion of the wearer.

[0003] In order to analyze the wearing effect of the optical imaging module on the user, a cross chart detection method can be used. The detection of the cross chart refers to the detection of the center point of the collected cross chart, which is usually realized by a template matching algorithm. The precision and speed of the cross chart detection by the template matching algorithm are greatly limited by the hardware detection environment. Therefore, the current cross chart detection method has the technical problems of low precision and speed. SUMMARY

[0004] The main purpose of the present application is to provide a cross chart detection method, device, intelligent equipment and computer readable storage medium, which aims to solve the technical problems of low precision and speed of the current cross chart detection method.

[0005] To achieve the above purpose, the present application provides a cross chart detection method, which is applied to a head-mounted wearable device; the head-mounted wearable device comprises an optical imaging module;

[0006] The method comprises the following steps:

[0007] Collecting a cross chart displayed in the optical imaging module;

[0008] Determining the center point of the collected image and the center point of the connected domain of the cross chart;

[0009] According to the affine transformation matrix established between the center point of the collected image and the center point of the connected domain, the sub-pixel center point of the cross chart is determined.

[0010] Optionally, the step of determining the center point of the connected domain of the cross chart comprises:

[0011] Determining the collected image corresponding to the cross chart, and determining the binary collected image corresponding to the collected image;

[0012] According to a preset screening connected domain condition, the binarized acquisition image is denoised to obtain a cross connected domain image after denoising;

[0013] The image center point of the cross connected domain image is determined as the connected domain center point of the cross chart.

[0014] Optionally, the step of determining the sub-pixel center point of the cross chart according to the affine transformation matrix established between the acquisition image center point and the connected domain center point comprises:

[0015] The affine transformation matrix from the connected domain center point to the acquisition image center point is established;

[0016] The transpose affine transformation matrix corresponding to the affine transformation matrix is determined;

[0017] According to the affine transformation matrix and the transpose affine transformation matrix, sub-pixel line fitting is performed on the acquisition image corresponding to the cross chart to determine the sub-pixel center point of the cross chart.

[0018] Optionally, the step of performing sub-pixel line fitting on the acquisition image corresponding to the cross chart according to the affine transformation matrix and the transpose affine transformation matrix comprises:

[0019] According to the affine transformation matrix, the acquisition image corresponding to the cross chart is translated to obtain a translated acquisition image;

[0020] The horizontal axis direction edge extraction image of the translated acquisition image in the horizontal axis direction and the vertical axis direction edge extraction image in the vertical axis direction are respectively acquired;

[0021] The horizontal axis direction outer contour point set of the horizontal axis direction edge extraction image and the vertical axis direction outer contour point set of the vertical axis direction edge extraction image are respectively determined;

[0022] The transpose affine transformation matrix is respectively applied to the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set to fit to obtain a first fitting line corresponding to the horizontal axis direction outer contour point set and a second fitting line corresponding to the vertical axis direction outer contour point set.

[0023] Optionally, the step of determining the sub-pixel center point of the cross chart comprises:

[0024] The intersection point between the first fitting line and the second fitting line is determined;

[0025] The intersection point is determined as the sub-pixel center point of the cross chart.

[0026] Optionally, the step of determining the horizontal outer contour point set of the horizontal edge extraction image and the vertical outer contour point set of the vertical edge extraction image respectively, comprises:

[0027] performing morphological operations on the horizontal edge extraction image and the vertical edge extraction image respectively to obtain horizontal double bright lines and vertical double bright lines;

[0028] determining a horizontal outer contour point set and a vertical outer contour point set according to the horizontal double bright lines and the vertical double bright lines respectively.

[0029] Optionally, after the step of determining the sub-pixel center point of the cross graph card, the method further comprises:

[0030] determining a coordinate deviation between the sub-pixel center point and the acquisition image center point;

[0031] determining a visual level of the optical imaging module based on the coordinate deviation.

[0032] In addition, to achieve the above object, the present application also provides a cross graph card detection device, the cross graph card detection device comprising:

[0033] an image acquisition module, configured to acquire a cross graph card displayed in an optical imaging module;

[0034] an image processing module, configured to determine an acquisition image center point and a connected domain center point of the cross graph card, and determine a sub-pixel center point of the cross graph card according to an affine transformation matrix established between the acquisition image center point and the connected domain center point.

[0035] In addition, to achieve the above object, the present application also provides an intelligent device, comprising a processor, a storage unit, and a cross graph card detection program stored in the storage unit and executable by the processor, wherein when the cross graph card detection program is executed by the processor, the steps of the cross graph card detection method as described above are implemented.

[0036] The present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a cross graph card detection program, and when the cross graph card detection program is executed by a processor, the steps of the cross graph card detection method as described above are implemented.

[0037] The cross graph card detection method in the technical scheme of the application, after the camera and the like image acquisition device collects the cross graph card displayed by the optical imaging module, only needs to obtain the center point of the collected cross graph card image, that is, the camera center coordinates of the image acquisition device, and set certain connectivity conditions to obtain the cross graph card connected domain center point through simple preprocessing, establish the affine transformation matrix between the collected image center point and the connected domain center point, and based on the affine transformation matrix, the collected cross graph card image can be processed through sub-pixel fitting, so as to obtain the accurate sub-pixel center point of the cross graph card. The image processing of the cross graph card in the application is extremely simple, and does not need high hardware demand and computing power demand like template matching, so the efficiency of detecting the cross graph card is greatly improved, and the application detects the cross graph card from the angle of sub-pixel, so the accuracy is also much higher than that of the traditional detection method. Compared with the traditional template matching, the application has a wider application scene, and can realize 100% detection of the optical device such as the head-mounted wearable device at a high speed and high accuracy, instead of selective sampling inspection, greatly improving the quality control of the head-mounted wearable device. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The structure schematic diagram of the hardware running environment of the smart device involved in the embodiment scheme of the application;

[0039] Figure 2 The flowchart of the first embodiment of the cross graph card detection method of the application;

[0040] Figure 3 The detailed flowchart of step S30 involved in an embodiment of the cross graph card detection method of the application;

[0041] Figure 4 The detailed flowchart of step S33 involved in an embodiment of the cross graph card detection method of the application;

[0042] Figure 5 The process flowchart of the sub-pixel line fitting implementation process involved in the cross graph card detection method of the application;

[0043] Figure 6 The actual shooting schematic diagram of the cross graph card involved in the cross graph card detection method of the application;

[0044] Figure 7 The preprocessed flowchart disassembled image schematic diagram involved in the cross graph card detection method of the application;

[0045] Figure 8 The coordinate conversion schematic diagram involved in the cross graph card detection method of the application;

[0046] Figure 9 The x-direction edge extraction schematic diagram involved in the cross graph card detection method of the application;

[0047] Figure 10 This is a schematic diagram of edge extraction in the y-direction involved in the crosshair detection method of this application;

[0048] Figure 11 This is a schematic diagram of the frame structure of the cross-shaped card detection device of this application.

[0049] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0051] This application proposes an intelligent device. The intelligent device can be any data processing device with image acquisition and processing capabilities, such as a personal computer, workstation, server, etc., with image acquisition and processing capabilities, and is not limited thereto.

[0052] like Figure 1 As shown, Figure 1 This is a schematic diagram of the hardware operating environment of the smart device involved in the embodiments of this application.

[0053] like Figure 1 As shown, the intelligent device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a storage unit 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display or an input unit such as a control panel; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The storage unit 1005 may be a high-speed RAM storage unit or a stable storage unit (non-volatile memory), such as a disk storage unit. The storage unit 1005 may also optionally be a storage device independent of the aforementioned processor 1001. The storage unit 1005, as a computer storage medium, may include a crosshair card detection program.

[0054] Those skilled in the art will understand that Figure 1 The hardware structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0055] Continue to refer to Figure 1 , Figure 1The storage unit 1005 as a computer readable storage medium can include an operating system, a user interface module, a network communication module, and a cross graph card detection program.

[0056] In Figure 1 The network communication module is mainly used for connecting a server and communicating data with the server, and the processor 1001 can call the cross graph card detection program stored in the storage unit 1005 and perform the following operations:

[0057] Collecting a cross graph card displayed in the optical imaging module;

[0058] Determining a collection image center point and a connected domain center point of the cross graph card;

[0059] According to an affine transformation matrix established between the collection image center point and the connected domain center point, determining a sub-pixel center point of the cross graph card.

[0060] Further, the processor 1001 can call the cross graph card detection program stored in the storage unit 1005 and further perform the following operations:

[0061] Determining a collection image corresponding to the cross graph card and determining a binary collection image corresponding to the collection image;

[0062] According to a preset filtering connected domain condition, performing noise reduction on the binary collection image to obtain a cross connected domain image after noise reduction;

[0063] Determining an image center point of the cross connected domain image as the connected domain center point of the cross graph card.

[0064] Further, the processor 1001 can call the cross graph card detection program stored in the storage unit 1005 and further perform the following operations:

[0065] Establishing an affine transformation matrix from the connected domain center point to the collection image center point;

[0066] Determining a transposed affine transformation matrix corresponding to the affine transformation matrix;

[0067] According to the affine transformation matrix and the transposed affine transformation matrix, performing sub-pixel line fitting on the collection image corresponding to the cross graph card to determine a sub-pixel center point of the cross graph card.

[0068] Further, the processor 1001 can call the cross graph card detection program stored in the storage unit 1005 and further perform the following operations:

[0069] According to the affine transformation matrix, performing translation on the collection image corresponding to the cross graph card to obtain a translated collection image;

[0070] respectively acquire a horizontal edge extraction image of the translation acquisition image in a horizontal direction and a vertical edge extraction image of the translation acquisition image in a vertical direction;

[0071] respectively determine a horizontal outer contour point set of the horizontal edge extraction image and a vertical outer contour point set of the vertical edge extraction image;

[0072] respectively apply the transposed affine transformation matrix to the horizontal outer contour point set and the vertical outer contour point set to fit a first fitting line corresponding to the horizontal outer contour point set and a second fitting line corresponding to the vertical outer contour point set.

[0073] Further, the processor 1001 can invoke the cross graph card detection program stored in the memory 1005, and further perform the following operations:

[0074] determine an intersection point between the first fitting line and the second fitting line;

[0075] determine the intersection point as a sub-pixel center point of the cross graph card.

[0076] Further, the processor 1001 can invoke the cross graph card detection program stored in the memory 1005, and further perform the following operations:

[0077] respectively perform morphological operations on the horizontal edge extraction image and the vertical edge extraction image to obtain a horizontal double bright line and a vertical double bright line;

[0078] respectively determine a horizontal outer contour point set and a vertical outer contour point set according to the horizontal double bright line and the vertical double bright line.

[0079] Further, the processor 1001 can invoke the cross graph card detection program stored in the memory 1005, and further perform the following operations:

[0080] determine a coordinate deviation between the sub-pixel center point and the acquisition image center point;

[0081] determine a visual level of the optical imaging module based on the coordinate deviation.

[0082] In order to facilitate the understanding of the following various embodiments of the present application, the main technical solutions of the present application are briefly described here:

[0083] After an image acquisition device such as an industrial camera acquires a cross card displayed by an optical imaging module, a center point of the acquired image (acquired image center point) can be obtained, the acquired image is preprocessed according to certain connected domain conditions to obtain a (cross) connected domain center point, and then an affine transformation matrix between the two center points is obtained by subtraction, and then the connected domain center point is translated to the acquired image center point based on the affine transformation matrix, to obtain a translated acquired image. The translated acquired image is subjected to edge extraction in the x and y directions, and then subjected to morphological operations to obtain contour point sets in the x and y directions, respectively. The contour point sets in the x and y directions are regressed to the coordinate system before translation using the transpose matrix of the affine transformation matrix, and two intersecting lines are fitted using the regressed contour point sets, and the intersection point is the sub-pixel center point, that is, the more accurate center coordinate point of the cross card. The present application realizes more accurate optical axis detection of the optical imaging module, and does not need to use high-time-consuming algorithms such as template matching, thereby greatly improving the cross card detection efficiency.

[0084] Based on the hardware structure of the intelligent device, various embodiments of the cross card detection method of the present application are proposed.

[0085] The embodiment of the present application provides a cross card detection method.

[0086] Please refer to Figure 2 , Figure 2 is a flowchart of the first embodiment of the cross card detection method of the present application; in the first embodiment of the present application, the cross card detection method is applied to a head-mounted wearable device; the head-mounted wearable device includes an optical imaging module;

[0087] The cross card detection method includes the following steps:

[0088] Step S10, acquiring a cross card displayed in the optical imaging module;

[0089] In this embodiment, the head-mounted wearable device can be an AR or VR device, etc. Since the distance between the eyes and the screen of the user when using these devices is very close, the optical imaging module has very high imaging requirements, and the optical axis offset should be reduced as much as possible to improve the user's comfort, immersion, and other viewing experiences.

[0090] The cross card displayed by the optical imaging module of the head-mounted wearable device can be acquired by an image acquisition device such as an industrial camera (for the sake of convenience, the camera is used hereinafter). The optical imaging module at least includes a display screen and a lens arranged in front of the display screen.

[0091] The specific collection scene can be that the industrial camera is opposite to the display screen and photographs the cross chart displayed on the display screen through the lens in the optical imaging module when photographing the cross chart. The purpose of this is to simulate the actual viewing effect of the human eye when wearing the head-mounted wearable device. The cross chart can be considered as a cross pattern. It can be seen that the gray scale distribution of the cross chart is that the gray scale value increases from the cross end point to the inside.

[0092] For the concept of the cross chart, please refer to Figure 6 The cross chart can be considered as a cross pattern. It can be seen that the gray scale distribution of the cross chart is that the gray scale value increases from the cross end point to the inside.

[0093] Step S20, determining the collection image center point and the connected domain center point of the cross chart;

[0094] After the cross chart is collected, the collection image is obtained, as shown in Figure 6 The collection image center point of the cross chart refers to the center point (x0, y0) of the collection image, which is actually the center point of the camera coordinates. For determining the connected domain center point in the collection image, the collection image can be preprocessed for noise reduction and selection of connected domains according to actual needs, so as to obtain the connected domain center point (x i , y i ).

[0095] In an embodiment, the step S20 of determining the connected domain center point of the cross chart comprises:

[0096] Step a, determining the collection image corresponding to the cross chart, and determining the binary collection image corresponding to the collection image;

[0097] Step b, according to the preset filtering connected domain condition, performing noise reduction on the binary collection image to obtain a cross connected domain image after noise reduction;

[0098] Step c, determining the image center point of the cross connected domain image as the connected domain center point of the cross chart.

[0099] For the convenience of understanding this embodiment, refer to Figure 7 for description:

[0100] After the camera photographs the cross chart, the collection image of the cross chart is obtained, and the corresponding binary collection image is obtained by performing binary image processing on the collection image, so that the entire collection image presents obvious black and white effect, highlighting the contour of the target, and facilitating the detection of the cross chart.

[0101] Based on the binarized acquired image, a certain connected component algorithm (such as the two-pass scanning method, seed filling method, etc.) can be used to break the cross-shaped connected component and select the required cross-shaped connected regions based on the cross-shaped connected component features and setting the required connected component selection conditions. In other words, the cross-shaped connected regions are marked from the binarized acquired image, and other irrelevant image noise is removed to obtain the denoised cross-shaped connected component image. After obtaining the cross-shaped connected component image, its image center point, i.e., the coordinates of the connected component center (x, y), can be calculated. i y i ).

[0102] It should be noted that, as can be seen Figure 7 The center point of the connected domain in ( Figure 7 The white dot in the bottom right corner pattern is often not the center point coordinate of the crosshair, so the center point coordinates need to be further optimized to obtain a more accurate sub-pixel center point.

[0103] By performing the above-described preprocessing on the acquired image through this embodiment of the application, the consistency of the algorithm during subsequent image processing and the simplicity of coordinate transformation can be ensured, thereby improving the detection efficiency of the crosshair card.

[0104] Step S30: Determine the sub-pixel center point of the crosshair card based on the affine transformation matrix established between the center point of the acquired image and the center point of the connected region.

[0105] The affine transformation matrix established between the center point of the acquired image and the center point of the connected domain reflects how to move from one coordinate to another. In this embodiment, the affine transformation matrix can be obtained by moving from the center point of the connected domain to the center point of the acquired image. Based on the affine transformation matrix, a series of edge extraction and morphological operations are performed. Then, the transpose of the affine transformation matrix is ​​used to return to the original coordinate system before the translation, and sub-pixel line fitting is performed in the x and y directions to obtain two intersecting lines. The corresponding intersection point is the sub-pixel level center point.

[0106] Please refer to Figure 3 In one embodiment, step S30, which involves determining the sub-pixel center point of the crosshair card based on the affine transformation matrix established between the center point of the acquired image and the center point of the connected component, includes:

[0107] Step S31: Establish the affine transformation matrix from the center point of the connected component to the center point of the acquired image;

[0108] The affine transformation matrix between the center point of the connected component and the center point of the acquired image is obtained by subtracting the coordinates of the center point of the acquired image from the coordinates of the center point of the connected component.

[0109] The application can be further understood with reference to Figure 8 , for example, Figure 8 The two center points are the center point of the collected image (x0, y0) at the upper left corner and the center point of the connected domain (x i , y i ) at the lower right corner.

[0110] Step S32, determining a transposed affine transformation matrix corresponding to the affine transformation matrix;

[0111] The transposed affine transformation matrix is obtained by transposing the affine transformation matrix.

[0112] Step S33, performing sub-pixel line fitting on the collected image corresponding to the cross chart according to the affine transformation matrix and the transposed affine transformation matrix to determine a sub-pixel center point of the cross chart.

[0113] Under the action of the affine transformation matrix and the transposed affine transformation matrix, the collected image is first translated, and then edge extraction, morphological operation, and connected domain determination are performed to obtain the outer contour point sets in the x and y directions. These outer contour point sets are then regressed to the original coordinates for sub-pixel fitting, thereby detecting the center point (coordinates) of the cross chart with higher accuracy reaching the sub-pixel level.

[0114] Specifically, please refer to Figure 4 In an embodiment, the step S33 comprises:

[0115] Step S330, translating the collected image corresponding to the cross chart according to the affine transformation matrix to obtain a translated collected image;

[0116] After the affine transformation matrix is determined, the collected image is translated according to the affine transformation matrix reflecting the coordinate change to obtain a translated collected image after translation.

[0117] Step S331, respectively acquiring a horizontal axis direction edge extraction image of the translated collected image in the horizontal axis direction and a vertical axis direction edge extraction image in the vertical axis direction;

[0118] The horizontal axis is x, and the vertical axis is y, which correspond to two directions of a two-dimensional coordinate system.

[0119] Edge refers to a processing of a picture contour in digital image processing. For the boundary, the place where the gray value changes sharply is defined as the edge.

[0120] The translated collected image can be subjected to x direction edge detection and extraction and y direction edge detection and extraction through a preset edge detection algorithm (such as a Sobel edge detection operator), thereby obtaining the horizontal axis direction edge extraction image and the vertical axis direction edge extraction image.

[0121] Step S332, respectively determine the horizontal axis direction edge extraction image of the horizontal axis direction outer contour point set and the vertical axis direction edge extraction image of the vertical axis direction outer contour point set;

[0122] In the horizontal axis direction edge extraction image and the vertical axis direction edge extraction image, the corresponding horizontal axis direction outer contour point set and the vertical axis direction outer contour point set can be directly obtained. Each point set has a point coordinate.

[0123] In order to reduce the interference of other pixel points, and improve the accuracy of sub-pixel fitting, morphological operation can be performed on the horizontal axis direction edge extraction image and the vertical axis direction edge extraction image, which can be closed operation.

[0124] That is, in an embodiment, the step S332 comprises:

[0125] Step d, respectively on the horizontal axis direction edge extraction image and the vertical axis direction edge extraction image morphological operation, to get horizontal axis direction double bright line and vertical axis direction double bright line;

[0126] Please refer to Figure 9 and Figure 10 Some general morphological functions can be used to perform morphological operation on the horizontal axis direction edge extraction image and the vertical axis direction edge extraction image, such as by closed operation. After morphological operation, for the horizontal axis direction edge extraction image, as shown in Figure 9 , six bright lines (gray value larger pattern) are reserved. Similarly, for the vertical axis direction edge extraction image, as shown in Figure 10 , six bright lines are also reserved. Continue to refer to Figure 9 , according to the above embodiment, the middle four bright lines are connected vertically in pairs, and the horizontal axis direction double bright line, that is, the middle two vertical bright lines, are obtained. Similarly, refer to Figure 10 , the vertical axis direction double bright line is also obtained.

[0127] Step e, according to the horizontal axis direction double bright line and the vertical axis direction double bright line, respectively determine the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set.

[0128] The pixel outer contour point set of the horizontal axis direction double bright line is obtained, and the horizontal axis direction outer contour point set is obtained. The pixel outer contour point set of the vertical axis direction double bright line is obtained, and the vertical axis direction outer contour point set is obtained.

[0129] Step S333, the transpose affine transformation matrix is respectively applied to the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set to fit the first fitting line corresponding to the horizontal axis direction outer contour point set and the second fitting line corresponding to the vertical axis direction outer contour point set.

[0130] The transpose affine transformation matrix is respectively applied to the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set, so that the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set return to the original image coordinate system, and the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set are fitted respectively. The purpose of fitting is to obtain a straight line, a first fitting line and a second fitting straight line, that is, two straight lines are fitted in total, and the two straight lines intersect.

[0131] For the method of fitting a straight line based on a point set, the least square method straight line fitting can be performed through the fitline function, so that the first fitting line and the second fitting line are closer to the horizontal and vertical symmetry axes of the cross graph card, and the intersection point is more accurate.

[0132] In an embodiment, the step S30 of determining the sub-pixel center point of the cross graph card comprises:

[0133] Step f, determining the intersection point between the first fitting line and the second fitting line;

[0134] Step g, determining the intersection point as the sub-pixel center point of the cross graph card.

[0135] The first fitting line and the second fitting line are often perpendicular to each other, and the two lines are compared to a point, and the intersection point is more accurate than the pixel level, that is, the sub-pixel center point is at the sub-pixel level, which is more accurate than the pixel level of the traditional template matching traversal method for locating the cross graph card.

[0136] In an embodiment, after the step S30, the method further comprises:

[0137] Step h, determining the coordinate deviation between the sub-pixel center point and the center point of the collected image;

[0138] Step i, determining the visual level of the optical imaging module based on the coordinate deviation.

[0139] In the ideal state of the optical imaging module, the center point of the collected image should correspond to the center point of the cross chart displayed on the display screen. However, due to factors such as screen quality and assembly process, there is a deviation compared with the expectation. For example, the lens module of the head-mounted wearable device deviates during assembly, which will make the position of the image viewed by the human eye not the actual position of the image displayed on the display screen, causing poor viewing experience. When detecting the cross chart automatically, the center of the cross chart collected by shooting may not be the center of the cross chart displayed.

[0140] For the present application, after obtaining the accurate sub-pixel center point in the collected image, in order to evaluate the quality of the optical imaging module, the sub-pixel center point can be subtracted from the center point of the collected image to obtain a coordinate deviation. A certain visual level or quality level is preset, and each level corresponds to a deviation range. Then, based on the coordinate deviation, it is determined whether the coordinate deviation is within the deviation range, and the visual level or quality level is determined, which helps the detection personnel to evaluate the optical imaging module, improves the detection efficiency, and reduces the labor cost.

[0141] In addition, the coordinate deviation can also help the detection personnel to determine how to adjust the optical imaging module, so that the imaging effect is more ideal and the user experience is better.

[0142] The cross chart detection method in the technical scheme of the present application only needs to obtain the center point of the collected cross chart image, that is, the camera center coordinate of the image collection device, after the camera or other image collection device collects the cross chart displayed by the optical imaging module. A certain connectivity condition is set for simple preprocessing to obtain the connected domain center point of the cross chart. An affine transformation matrix is established between the center point of the collected image and the connected domain center point. Based on the affine transformation matrix, the collected image of the cross chart can be processed by sub-pixel fitting, so as to obtain the accurate sub-pixel center point of the cross chart. The present application is extremely simple in image processing of the cross chart, and does not require high hardware requirements and computing power requirements like template matching. Therefore, the efficiency of detecting the cross chart is greatly improved. The present application detects the cross chart from the perspective of sub-pixel, so its accuracy is much higher than that of the traditional detection method. Compared with the traditional template matching, the application scenario of the present application is more extensive, and it can achieve 100% detection of the optical equipment such as the head-mounted wearable device at a high speed and high accuracy, instead of selective sampling inspection, which greatly improves the quality control of the head-mounted wearable device.

[0143] In order to facilitate understanding of the implementation process of the main technical scheme of the present application, please refer to Figure 5 . As shown in Figure 5 ,

[0144] the center coordinate (xi, y i The affine transformation matrix M from the center coordinates (x0, y0) to the center coordinates (x0, y0);

[0145] Calculate the inverse transformation matrix M -1 ;

[0146] Apply the affine transformation matrix M to the image (acquired image);

[0147] x-direction:

[0148] Perform edge extraction in the x-direction;

[0149] Perform morphological operations in the x-direction;

[0150] Find the coordinate points of the edge contour in the x-direction;

[0151] M -1 Contour points acting in the x-direction;

[0152] Perform contour point-line fitting in the x-direction to form the L1 line equation;

[0153] y direction:

[0154] Perform edge extraction in the y-direction;

[0155] Perform morphological operations in the y-direction;

[0156] Find the coordinate points of the edge contour in the y-direction;

[0157] M -1 Contour points acting in the y-direction;

[0158] Perform contour point-line fitting in the y-direction to form the equation of the L2 line;

[0159] Solving by intersecting L1 and L2, we get (x) f ,y f ), the (x) f ,y f This refers to the sub-pixel center point mentioned in the above embodiments.

[0160] In addition, refer to Figure 11 , Figure 11 This is a schematic diagram of the frame structure of the crosshair card detection device of this application. This application also proposes a crosshair card detection device, which includes:

[0161] Image acquisition module A10 is used to acquire the crosshair chart displayed in the optical imaging module;

[0162] The image processing module A20 is configured to determine a center point of a collected image of the cross chart and a center point of a connected domain; determine a sub-pixel center point of the cross chart according to an affine transformation matrix established between the center point of the collected image and the center point of the connected domain; and determine a forehead temperature change compensation value corresponding to the environment temperature.

[0163] Optionally, the image processing module A20 is further configured to:

[0164] determine a collected image corresponding to the cross chart, and determine a binary collected image corresponding to the collected image;

[0165] perform noise reduction on the binary collected image according to a preset screening connected domain condition to obtain a cross connected domain image after noise reduction;

[0166] determine an image center point of the cross connected domain image as the center point of the connected domain of the cross chart.

[0167] Optionally, the image processing module A20 is further configured to:

[0168] establish an affine transformation matrix from the center point of the connected domain to the center point of the collected image;

[0169] determine a transposed affine transformation matrix corresponding to the affine transformation matrix;

[0170] perform sub-pixel line fitting on the collected image corresponding to the cross chart according to the affine transformation matrix and the transposed affine transformation matrix, to determine a sub-pixel center point of the cross chart.

[0171] Optionally, the image processing module A20 is further configured to:

[0172] perform translation on the collected image corresponding to the cross chart according to the affine transformation matrix to obtain a translated collected image;

[0173] obtain a horizontal axis direction edge extraction image of the translated collected image in a horizontal axis direction and a vertical axis direction edge extraction image of the translated collected image in a vertical axis direction, respectively;

[0174] determine a horizontal axis direction outer contour point set of the horizontal axis direction edge extraction image and a vertical axis direction outer contour point set of the vertical axis direction edge extraction image, respectively;

[0175] apply the transposed affine transformation matrix to the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set, respectively, to fit to obtain a first fitting line corresponding to the horizontal axis direction outer contour point set and a second fitting line corresponding to the vertical axis direction outer contour point set.

[0176] Optionally, the image processing module A20 is further configured to:

[0177] determining an intersection point between the first fitting line and the second fitting line;

[0178] determining the intersection point as a sub-pixel center point of the cross chart.

[0179] Optionally, the image processing module A20 is further configured to:

[0180] performing morphological operations on the horizontal-axis direction edge extraction image and the vertical-axis direction edge extraction image respectively to obtain horizontal-axis direction double bright lines and vertical-axis direction double bright lines;

[0181] determining a horizontal-axis direction outer contour point set and a vertical-axis direction outer contour point set respectively according to the horizontal-axis direction double bright lines and the vertical-axis direction double bright lines.

[0182] Optionally, the image processing module A20 is further configured to:

[0183] determining a coordinate deviation between the sub-pixel center point and the center point of the acquisition image;

[0184] determining a visual level of the optical imaging module based on the coordinate deviation.

[0185] The cross chart detection device of the present application has the same implementation as the above-mentioned cross chart detection method, and thus will not be described again.

[0186] In addition, the present application also provides a computer readable storage medium. The computer readable storage medium of the present application stores a cross chart detection program, wherein the cross chart detection program is executed by a processor to implement the steps of the above-mentioned cross chart detection method.

[0187] The method implemented by the cross chart detection program when executed can refer to the embodiments of the cross chart detection method of the present application, which will not be described again.

[0188] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0189] The present application is described in reference to the flowchart and / or block diagram of the method, apparatus (system) and computer program product according to an embodiment of present application. It shall be understood that each flow and / or block in the flowchart and / or block diagram, and a combination of flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart Figure 1 one or more flows and / or blocks. Figure 1 one or more blocks.

[0190] These computer program instructions can also be stored in a computer readable storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable storage medium produce an article of manufacture including instructions which implement the function specified in the flowchart Figure 1 one or more flows and / or blocks. Figure 1 one or more blocks.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more flows and / or blocks. Figure 1 one or more blocks.

[0192] It should be noted that in the claims the reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps not listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unitary claim, the features of the different embodiments can be combined with each other if this is explicit or implicit to a person skilled in the art. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these

[0193] Although preferred embodiments of the application have been described, a person of ordinary skill in the art can make additional changes and modifications to the embodiments once armed with the present disclosure. Therefore, the appended claims are intended to cover all such changes and modifications that are within the scope of the application.

[0194] The above merely describes the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application. Any equivalent structural changes made according to the content of the present application specification and drawings, or direct / indirect application in other related technical fields, are included in the patent protection scope of the present application.

Claims

1. A cross chart detection method, characterized by, The cross card detection method is applied to a head-mounted wearable device; the head-mounted wearable device comprises an optical imaging module; The method comprises the following steps: Collecting a cross card displayed in the optical imaging module; Determining a collection image center point and a connected domain center point of the cross card; According to an affine transformation matrix established between the collection image center point and the connected domain center point, determining a sub-pixel center point of the cross card; The step of determining the sub-pixel center point of the cross card according to the affine transformation matrix established between the collection image center point and the connected domain center point comprises: Establishing an affine transformation matrix from the connected domain center point to the collection image center point; Determining a transposed affine transformation matrix corresponding to the affine transformation matrix; According to the affine transformation matrix, translating the collection image corresponding to the cross card to obtain a translated collection image; Respectively acquiring a horizontal axis direction edge extraction image of the translated collection image in a horizontal axis direction and a vertical axis direction edge extraction image in a vertical axis direction; Respectively determining a horizontal axis direction outer contour point set of the horizontal axis direction edge extraction image and a vertical axis direction outer contour point set of the vertical axis direction edge extraction image; Respectively applying the transposed affine transformation matrix to the horizontal axis direction outer contour point set and the vertical axis direction outer contour point set to fit to obtain a first fitting line corresponding to the horizontal axis direction outer contour point set and a second fitting line corresponding to the vertical axis direction outer contour point set, so as to determine the sub-pixel center point of the cross card.

2. The cross chart detection method of claim 1, wherein, The step of determining the connected domain center point of the cross card comprises: Determining a collection image corresponding to the cross card and determining a binary collection image corresponding to the collection image; According to a preset screening connected domain condition, performing noise reduction on the binary collection image to obtain a cross connected domain image after noise reduction; Determining an image center point of the cross connected domain image as the connected domain center point of the cross card.

3. The cross chart detection method of claim 1, wherein, The step of determining the sub-pixel center point of the cross card comprises: Determining an intersection point between the first fitting line and the second fitting line; Determining the intersection point as the sub-pixel center point of the cross card.

4. The cross chart detection method of claim 1, wherein, The step of respectively determining the horizontal axis direction outer contour point set of the horizontal axis direction edge extraction image and the vertical axis direction outer contour point set of the vertical axis direction edge extraction image comprises: Respectively performing morphological operations on the horizontal axis direction edge extraction image and the vertical axis direction edge extraction image to obtain a horizontal axis direction double bright line and a vertical axis direction double bright line; Respectively determining a horizontal axis direction outer contour point set and a vertical axis direction outer contour point set according to the horizontal axis direction double bright line and the vertical axis direction double bright line.

5. The cross chart detection method of claim 1, wherein, After the step of determining the sub-pixel center point of the cross card, the method further comprises: Determining a coordinate deviation between the sub-pixel center point and the collection image center point; Based on the coordinate deviation, determining a visual level of the optical imaging module.

6. A cross chart detection device characterized by comprising: The cross card detection device comprises: An image collection module for collecting a cross card displayed in an optical imaging module; The image processing module is configured to determine a center point of a collection image of the cross chart and a center point of a connected domain, determine a sub-pixel center point of the cross chart according to an affine transformation matrix established between the center point of the collection image and the center point of the connected domain, establish the affine transformation matrix from the center point of the connected domain to the center point of the collection image, determine a transposed affine transformation matrix corresponding to the affine transformation matrix, perform translation on a collection image corresponding to the cross chart according to the affine transformation matrix to obtain a translated collection image, obtain a horizontal edge extraction image of the translated collection image in a horizontal direction and a vertical edge extraction image of the translated collection image in a vertical direction, respectively, determine a horizontal outer contour point set of the horizontal edge extraction image and a vertical outer contour point set of the vertical edge extraction image, respectively, and apply the transposed affine transformation matrix to the horizontal outer contour point set and the vertical outer contour point set, respectively, to fit to obtain a first fitting line corresponding to the horizontal outer contour point set and a second fitting line corresponding to the vertical outer contour point set, so as to determine the sub-pixel center point of the cross chart.

7. A smart device, comprising: The smart device includes a memory, a processor, and a cross chart detection program stored on the memory and executable on the processor, wherein the cross chart detection program, when executed by the processor, implements the steps of the cross chart detection method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a cross chart detection program, wherein the cross chart detection program, when executed by a processor, implements the steps of the cross chart detection method of any one of claims 1 to 5.

Images