Image geometric registration method, system, device and storage medium

Abstract

The present application discloses an image geometric registration method, system, device and storage medium, which can effectively avoid the stitching error caused by the algorithm, thereby improving the clarity of image stitching and the user experience. The method of the present application includes: determining a regional map in a VR display screen; using a camera to shoot a number of sub-region images of the VR display screen, wherein the images of adjacent regions in the several sub-region images have lattice points in the same row or column; extracting effective points from the lattice points of each sub-region image by binarization to generate a binarized region, calculating the centroid coordinates of the center point of the binarized region, setting a ROI envelope of a target size according to the centroid coordinates of the center point of the binarized region, and calculating the centroid coordinates of the midpoints of the ROI envelope.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image geometric registration method, system, device and storage medium. Background Art

[0002] With the rapid development of virtual reality (VR) technology, immersive experience has become one of the key technical indicators. In order to achieve this experience, VR devices usually adopt a helmet-type design, which is fully enclosed and has rubber or sponge products in the contact area with the face to make the face and the device fit tightly to prevent light leakage. VR devices are attached to the human face and the display screen is very close to the human eye, so the quality requirements for the display screen are relatively high.

[0003] In the prior art, sub-pixel defect detection is required to realize defect detection of VR devices. In sub-pixel defect detection, the focal length of the camera needs to be magnified several times, resulting in the camera field of view being unable to cover the entire screen, or even only being able to capture a small part of the screen. If you want to obtain an image of the entire screen, you need to shoot multiple times, and only after image stitching can you obtain a complete image for sub-pixel defect detection. However, when using the existing image stitching algorithm for image stitching, there will be a stitching deviation in the positional relationship of the overlapping areas of the images, and the deviation will be greater the farther the distance, resulting in poor clarity of the stitched image, affecting the user experience. Summary of the invention

[0004] The present application provides an image geometric registration method, system, device and storage medium, which can effectively avoid stitching errors caused by algorithms, thereby improving the clarity of image stitching and user experience.

[0005] The first aspect of the present application provides an image geometric registration method, comprising:

[0006] Determine the area map in the VR display;

[0007] Using a camera to shoot a plurality of sub-region images of the VR display screen, wherein images of adjacent regions in the plurality of sub-region images have dot matrix points in the same row or column;

[0008] Extracting effective points from the lattice points of each of the sub-region images by binarization to generate a binarized region, calculating the centroid coordinates of the center point of the binarized region, setting a ROI envelope of a target size according to the centroid coordinates of the center point of the binarized region, and calculating the centroid coordinates of the points in the ROI envelope;

[0009] The centroid coordinates of the points in each ROI envelope frame are sorted according to the target size, a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the region map is established, and a transformation matrix is ​​calculated, and the actual centroid coordinates of the dot matrix in the region map are calculated by the transformation matrix;

[0010] All the actual centroid coordinate sets are divided into several groups by row or column, and the coordinate sets of the corresponding rows or columns of the adjacent areas are rotated and translated so that the point sets of the adjacent areas are in the same row or column to obtain a distribution diagram of the complete dot matrix.

[0011] Optionally, extracting effective points from the lattice points of each of the sub-region images by binarization to generate a binarized region, calculating the centroid coordinates of the center point of the binarized region, setting a ROI envelope according to the centroid coordinates of the center point of the binarized region, and calculating the centroid coordinates of the points in the ROI envelope, includes:

[0012] Extracting the dot matrix points of each sub-region image by binarization;

[0013] Obtaining the ratio of the area to the length of the short side of the minimum circumscribed rectangle, and extracting effective points of the lattice points according to the ratio, wherein a plurality of the effective points constitute the binary area;

[0014] Calculating the centroid coordinates of the center point of the binary region according to the length and width of the binary region and the sub-region image;

[0015] Setting the ROI envelope of the target size according to the centroid coordinates of the center point of the binary region;

[0016] Calculate the centroid coordinates of the middle points of the ROI envelope frame according to the first formula;

[0017] The first formula:

[0018] ;

[0019] in, The horizontal coordinate represents the centroid coordinate of the point in the ROI envelope; The ordinate represents the centroid coordinates of the points in the ROI envelope; represents the abscissa of the middle point of the ROI envelope, represents the ordinate of the middle point of the ROI envelope, Indicates the grayscale value of the pixel in the ROI envelope.

[0020] Optionally, the centroid coordinates of the points in each ROI envelope frame are sorted according to the target layout, a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the area map is established, and a transformation matrix is ​​calculated, and the actual centroid coordinates of the dot matrix in the area map are calculated by the transformation matrix, including:

[0021] Sorting the centroid coordinates of the points in each ROI envelope frame according to the target size so that the centroid coordinates of the points in each ROI envelope frame correspond to the dot matrix layout of the region map in one-to-one order;

[0022] Establishing the conversion relationship between the centroid coordinates of the middle points of each ROI envelope frame and the row and column coordinates of the region map through perspective transformation, and calculating the transformation matrix;

[0023] The actual row and column coordinates on the area map are multiplied by the change matrix to obtain the actual centroid coordinates of the dot matrix in the area map.

[0024] Optionally, all the actual centroid coordinates are divided into several groups by row or column, including:

[0025] Dividing the area map into a plurality of sub-area maps, wherein the area map contains all the actual centroid coordinates, and the size of the sub-area map is the target size;

[0026] According to the plurality of sub-region diagrams, all the actual centroid coordinates are divided into a plurality of groups by rows or columns.

[0027] Optionally, the coordinate sets of the corresponding rows or columns of the adjacent regions are rotated and translated so that the point sets of the adjacent regions are in the same row or column to obtain a distribution diagram of the complete dot matrix, including:

[0028] Calculating the relationship between the coordinate points in the plurality of sub-regional graphs in a vertical direction to obtain a plurality of groups of collinear line segments;

[0029] A group of collinear line segments among the plurality of groups of collinear line segments is used as a reference, and line segment fitting, rotation and translation operations are performed with a group of collinear line segments in an adjacent vertical direction, so that the tails and the beginnings of the two groups of collinear line segments overlap, until the distribution diagram of the complete dot matrix is ​​obtained.

[0030] Optionally, the coordinate points in the plurality of sub-regional graphs are mutually calculated in a vertical direction to obtain a plurality of groups of collinear line segments, including:

[0031] Fitting two sets of vertically adjacent coordinate points in the plurality of sub-region maps into two line segments respectively according to the least square method, and calculating a first inclination angle and a second inclination angle of the two line segments relative to a horizontal line;

[0032] The least square method is: ,in are the coordinates of the fitted line, is the actual point coordinate, L is the calculation error;

[0033] Rotating the two line segments using a rotation matrix according to the first tilt angle and the second tilt angle so that the two line segments are parallel, and moving the two line segments using a translation matrix to obtain a set of collinear line segments, wherein the collinear line segments are line segments formed by connecting the two line segments end to end;

[0034] The above-mentioned rotation and translation operations are repeated to obtain a plurality of groups of collinear line segments corresponding to the coordinate points in the plurality of sub-region maps.

[0035] Optionally, the rotation matrix is:

[0036] ; wherein θ is the first tilt angle or the second tilt angle;

[0037] The translation matrix is:

[0038] ; Where M is the translation matrix, T x is the translation distance along the x direction, T y is the translation distance along the y direction.

[0039] A second aspect of the present application provides an image geometric registration system, comprising:

[0040] A determination unit, used to determine a region map in a VR display screen;

[0041] A shooting unit, used for shooting a plurality of sub-region images of the VR display screen using a camera, wherein images of adjacent regions in the plurality of sub-region images have dot matrix points in the same row or column;

[0042] A calculation unit, used for extracting effective points from the lattice points of each of the sub-region images by binarization to generate a binarized region, calculating the centroid coordinates of the center point of the binarized region, setting an ROI envelope of a target size according to the centroid coordinates of the center point of the binarized region, and calculating the centroid coordinates of the points in the ROI envelope;

[0043] An establishing unit is used to sort the centroid coordinates of the points in each ROI envelope frame according to the target size, establish a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the area map, and calculate a transformation matrix, and calculate the actual centroid coordinates of the dot matrix in the area map through the transformation matrix;

[0044] The operation unit is used to divide all the sets of actual centroid coordinates into several groups by row or column, and obtain a distribution diagram of the complete dot matrix by rotating and translating the coordinate sets of the corresponding rows or columns of the adjacent areas so that the point sets of the adjacent areas are in the same row or column.

[0045] A third aspect of the present application provides an image geometric registration device, the device comprising:

[0046] Processor, memory, input-output unit, and bus;

[0047] The processor is connected to the memory, the input and output unit, and the bus;

[0048] The memory stores a program, and the processor calls the program to execute the first aspect and any optional image geometric registration method in the first aspect.

[0049] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional image geometric registration method in the first aspect.

[0050] It can be seen from the above technical solutions that this application has the following advantages:

[0051] (1) The image geometric registration method of the present application uses multiple linear transformations to transform the dot coordinates in each local image to a state where they correspond one-to-one with the dot positions in the original image. At the same time, the overlapping areas of the images can be guaranteed to completely overlap, and the logic of the stitching algorithm can be simplified.

[0052] (2) The image geometric registration method of the present application can adapt to the splicing combination of any rows and columns and has good robustness.

[0053] (3) The algorithm logic in the image geometric registration method of the present application is concise and clear, which can avoid stitching errors caused by the algorithm. The image stitching error of the algorithm is extremely small and the stitching effect is good, which can effectively avoid the stitching error caused by the algorithm, thereby improving the clarity of image stitching and user experience.

[0054] (4) The stitching logic between images in the image geometric registration method of this application is almost the same, and the parallel processing method is adopted, which greatly saves the running time of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 A schematic diagram of an embodiment of the image geometric registration method of the present application;

[0057] Figure 2 This is a flow chart of another embodiment of the image geometric registration method of the present application;

[0058] Figure 3 A schematic diagram of the dot size layout on a region map in the image geometric registration method of this application;

[0059] Figure 4 This is a schematic diagram of the dot matrix layout of the ROI envelope in the image geometric registration method of this application;

[0060] Figure 5 A distribution diagram of the collinearity of line segments formed by the lattice points in the left area diagram and the right area diagram in the image geometric registration method of the present application;

[0061] Figure 6 for Figure 5 Schematic diagram of the distribution of the complete lattice points formed after the line segments in are translated;

[0062] Figure 7 A schematic diagram of the positional relationship between the centroid coordinates of the lattice points of the region map and the actual centroid coordinates of the lattice points in the image geometric registration method of multiple combined linear changes of the present application;

[0063] Figure 8 A schematic diagram of the dot layout of each sub-region map in the image geometric registration method of this application;

[0064] Fig. 9 for Figure 8 A schematic diagram of the dot matrix layout of each region in the image is formed by splicing the dot matrix layout of the complete region map;

[0065] Fig.10 This is a schematic diagram of an embodiment of the image geometric registration system of the present application;

[0066] Fig.11 This is a schematic diagram of an embodiment of the image geometric registration device of the present application. DETAILED DESCRIPTION

[0067] The present application provides an image geometric registration method, system, device and storage medium, which can effectively avoid stitching errors caused by algorithms, thereby improving the clarity of image stitching and user experience.

[0068] It should be noted that the image geometric registration method provided in this application can be applied to a terminal or a server. For example, the terminal can be a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, this application uses a terminal as an example for the execution subject.

[0069] See also Figure 1 , Figure 1 An embodiment of the image geometric registration method provided by the present application comprises:

[0070] 101. Determine a region map in a VR display screen;

[0071] In the embodiment of the present application, it is necessary to first clarify the physical size and resolution of the VR display screen. The physical size of the VR display screen can be initially determined to be 13*13. For the size layout of the dot matrix on the area map, please refer to Figure 3 , Figure 3 The dots in the figure represent dot matrix points. This size is used as an example to explain the contents of the subsequent specification. Further, the display screen is divided into a plurality of sub-areas, which may be rectangular, square or other shapes. Preferably, the shapes of the plurality of sub-areas are all rectangular. After determining the area map in the VR display screen, step 102 is performed.

[0072] 102. Use a camera to shoot a plurality of sub-region images of the VR display screen, wherein images of adjacent regions in the plurality of sub-region images have dot matrix points in the same row or column;

[0073] In the embodiment of the present application, a high-resolution camera or a professional screen capture device is used to shoot the VR display screen. When shooting, interference factors such as reflections and shadows should be eliminated or reduced to ensure that the captured image is clear and accurate. Since the display screen is divided into multiple areas, it is necessary to use a camera to shoot each area one by one. During the shooting process, it is necessary to ensure that the camera or device is stable to avoid image blur caused by shaking or movement, and it is necessary to ensure that the images of adjacent areas have overlapping parts, and the overlapping parts are dot matrix points in the same row or column for subsequent dot matrix matching and splicing.

[0074] Among them, after using the camera to take images of all the sub-areas, it is necessary to check the images to ensure that each image is clear, accurate and contains sufficient dot matrix details. If it is found that some images are of poor quality or lack dot matrix information, they can be re-shot or supplemented.

[0075] 103. Extract effective points from the lattice points of each of the sub-region images by binarization to generate a binarized region, calculate the centroid coordinates of the center point of the binarized region, set a ROI envelope of a target size according to the centroid coordinates of the center point of the binarized region, and calculate the centroid coordinates of the points in the ROI envelope;

[0076] In an embodiment of the present application, for each sub-region image, a binarization process is first performed to separate the lattice points in the image from the background, wherein the lattice points are usually bright colors and the background is usually dark colors. Then, valid lattice points are further extracted to form a binary region. Further, the centroid coordinates of the center point of the binary region are calculated. The centroid coordinates represent the center position of the lattice points in the binary region. According to the centroid coordinates, a ROI envelope frame of 5 rows and 7 columns is set. The envelope frame of 5 rows and 7 columns can contain all valid lattice points in the binary region. Then, the centroid coordinates of all points in the ROI envelope frame are calculated. The centroid coordinates of all points calculated will be used for subsequent lattice matching and stitching.

[0077] 104. Sort the centroid coordinates of the points in each ROI envelope frame according to the target size, establish a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the region map, calculate a transformation matrix, and calculate the actual centroid coordinates of the dot matrix in the region map through the transformation matrix;

[0078] In an embodiment of the present application, due to factors such as the camera shooting angle and lens distortion, the centroid coordinates in the ROI envelope frame may deviate from the dot matrix position on the area map. Therefore, the centroid coordinates of all points in the ROI envelope frame are converted according to the distribution of the sub-region image in the area map, that is, the actual coordinates of the coarse positioning dot matrix points are sorted according to a layout of 5 rows and 7 columns to ensure that the order of the centroid coordinates corresponds one-to-one to the dot matrix layout of the area map.

[0079] Specifically, by establishing a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the regional map, a transformation matrix can be calculated through this conversion relationship. The transformation matrix can be solved by comparing the known point positions and the corresponding centroid coordinates in the ROI envelope frame. After obtaining the transformation matrix, the actual centroid coordinates of the regional point array can be calculated through the change matrix.

[0080] 105. Divide all the actual centroid coordinate sets into several groups by row or column, and obtain a distribution diagram of a complete dot matrix by rotating and translating the coordinate sets of the corresponding rows or columns of the adjacent areas so that the point sets of the adjacent areas are in the same row or column.

[0081] In an embodiment of the present application, the above-mentioned operation steps are repeated. After the actual dot matrix centroid coordinates of all sub-regions are calculated, the point set of each region is divided into several groups by rows or columns. The coordinate sets of the corresponding rows or columns of the adjacent regions are rotated and translated so that these point sets can be in the same row or column, so that a distribution map of a complete dot matrix can be finally obtained.

[0082] In this process, the calculation is performed in the principle of "columns first and rows later", that is, the vertical relationship between each sub-region is calculated first, and then the dot matrix at the new vertical position is taken as a whole and the position relationship with the adjacent vertical overall dot matrix is ​​calculated in the horizontal direction, so that a stitching relationship with a very small algorithm error can be obtained, so that the stitched image can avoid the error caused by the algorithm and improve the stitching accuracy. The specific description of this step will be explained in the subsequent instructions.

[0083] See also Figure 2 , Figure 2 Another embodiment of the image geometric registration method provided by the present application includes:

[0084] 201. Determine a region map in a VR display screen;

[0085] 202. Use a camera to shoot a plurality of sub-region images of the VR display screen, wherein images of adjacent regions in the plurality of sub-region images have dot matrix points in the same row or column;

[0086] In the embodiment of the present application, step 201 to step 202 are similar to the aforementioned step 101 and step 102, and are not described in detail here.

[0087] 203. Extract effective points from the lattice points of each of the sub-region images by binarization to generate a binarized region, calculate the centroid coordinates of the center point of the binarized region, set a ROI envelope of a target size according to the centroid coordinates of the center point of the binarized region, and calculate the centroid coordinates of the points in the ROI envelope;

[0088] In the embodiment of the present application, specifically, the lattice points of each of the sub-region images are extracted by binarization, and then the ratio of the area to the long and short sides of the minimum circumscribed rectangle is obtained, and the effective points of the lattice points are extracted according to the ratio, and a number of effective points form a binary region. Further, the centroid coordinates of the center point of the binary region are calculated according to the length and width of the binary region and the sub-region image, and further according to the centroid coordinates of the center point of the binary region, a ROI envelope frame of 5 rows and 7 columns is set, and then the centroid coordinates of the midpoints in the ROI envelope frame are calculated according to the first formula;

[0089] First formula:

[0090] ;

[0091] in, The horizontal coordinate represents the centroid coordinate of the point in the ROI envelope; The ordinate represents the centroid coordinates of the points in the ROI envelope; represents the abscissa of the middle point of the ROI envelope, represents the ordinate of the middle point of the ROI envelope, Indicates the grayscale value of the pixel in the ROI envelope.

[0092] It should be noted that each sub-region map will generate an ROI envelope, in which there is a dot matrix arrangement in the ROI envelope. Figure 4 , Figure 4 The dot shown in the figure represents a dot matrix point. Therefore, when processing, it is necessary to calculate the centroid coordinates of the points in the envelope of all the sub-regional maps.

[0093] 204. Sort the centroid coordinates of the points in each ROI envelope frame according to the target size, establish a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the region map, calculate a transformation matrix, and calculate the actual centroid coordinates of the dot matrix in the region map through the transformation matrix;

[0094] In an embodiment of the present application, the centroid coordinates of the points in each ROI envelope frame are sorted according to a layout of 5 rows and 7 columns, so that the order of the centroid coordinates of the points in each ROI envelope frame corresponds to the dot matrix layout of the regional map one by one; then, a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the regional map is established through perspective transformation, and a transformation matrix is ​​calculated; the actual row and column coordinates on the regional map are multiplied by the change matrix to obtain the actual centroid coordinates of the dot matrix in the regional map, wherein the schematic diagram of the positional relationship between the centroid coordinates of the dot matrix points in the regional map and the actual centroid coordinates of the dot matrix points, please refer to Figure 7 , Figure 7 The dot shown in the figure represents a dot matrix point, and the rectangular border represents the ROI envelope.

[0095] 205. Divide the region map into a plurality of sub-region maps, wherein the region map contains all the actual centroid coordinates, and the size of the sub-region map is the target size;

[0096] In an embodiment of the present application, the area map is set to a size of 13 rows and 13 columns, and is divided into 6 sub-area maps. In the 6 sub-area maps, it is ensured that there is 1 row or 1 column overlap between adjacent areas. Each sub-area map is 5 rows and 7 columns in size, and the area map will contain all actual centroid coordinates of the 6 5-row and 7-column sub-area maps.

[0097] 206. Divide all the actual centroid coordinates into a plurality of groups by row or column according to the plurality of sub-region maps;

[0098] In an embodiment of the present application, according to the size layout of 13 rows and 13 columns of the area diagram, all actual center of mass coordinates are divided into several groups by row or by column. Specifically, the three area diagrams on the left side of the area diagram are marked with serial numbers 1, 3, and 5 from top to bottom, and the three areas on the right side of the area diagram are marked with serial numbers 2, 4, and 6 from top to bottom. Among them, there is one row overlap between area 1 and area 3, one row overlap between area 3 and area 5, one column overlap between area 1 and area 2, one column overlap between area 3 and area 4, and one column overlap between area 5 and area 6. According to the above method, all actual center of mass coordinates can be divided into 6 groups by row or by column.

[0099] 207. Calculate the relationship between the coordinate points in the plurality of sub-regional graphs in a vertical direction to obtain a plurality of groups of collinear line segments;

[0100] In the embodiment of the present application, the coordinate points in the six sub-regional maps are calculated in relation to each other in the vertical direction to obtain several groups of collinear line segments. Specifically, the coordinate sets of sequence numbers 1 and 3 in the three regional maps on the left are divided into 7 groups by column, and the corresponding coordinate points are taken according to the column numbers. According to the least squares method, the two sets of point sets are respectively fitted into two line segments, and the inclination angles θ1 and θ2 relative to the horizontal line are calculated. According to the rotation matrix of the affine transformation, the line segments in zone 3 are rotated to be parallel to the line segments in zone 1. The last point in zone 1 is the same as the first point in zone 3, so it is necessary to move the line segments in zone 3 through the translation matrix so that the starting point coincides with the end point of zone 1 to form a collinear line.

[0101] The other columns are processed according to the above steps, and finally the new coordinate points of the dot matrix of zone 3 are obtained. Then, the new coordinate points of zone 3 and the coordinates of zone 5 are calculated according to the processing method of zone 1 and zone 3 to obtain the new coordinates of zone 5. In this way, the coordinates of zones 1, 3, and 5 in the vertical direction can meet the co-column condition. Similarly, the three areas on the right vertical direction, namely areas numbered 2, 4, and 6, are also linearly transformed according to the same process so that they are in the same column. In the new coordinate set, the coordinates of areas 1 and 2 have not changed. The new coordinates of areas 3 and 5 can be in the same column with area 1, and areas 4 and 6 can be in the same column with area 2. Through the above operations, the coordinates of the 6 areas can be sorted into two areas on the left and right. Among them, the distribution diagram of the collinearity of the line segments composed of the dot matrix points in the left area diagram and the right area diagram can be referred to. Figure 5 ,exist Figure 5 In the figure, the white dots represent the lattice points, and the entire line segment represents the line segment formed between the lattice points.

[0102] That is, according to the least squares method, two sets of vertically adjacent coordinate points in several sub-region maps are respectively fitted into two line segments, and the first inclination angle and the second inclination angle of the two line segments relative to the horizontal line are calculated; then, the two line segments are rotated according to the first inclination angle and the second inclination angle through the rotation matrix to make the two line segments parallel, and the two line segments are moved through the translation matrix to obtain a set of collinear line segments, and the collinear line segments are line segments composed of two line segments connected end to end.

[0103] in,

[0104] The least squares method is: ,in are the coordinates of the fitted line, is the actual point coordinate, L is the calculation error;

[0105] in,

[0106] The rotation matrix is:

[0107] ; Wherein, θ is the first tilt angle or the second tilt angle;

[0108] in,

[0109] The translation matrix is:

[0110] ; Where M is the translation matrix, T x is the translation distance along the x direction, T y is the translation distance along the y direction;

[0111] Repeat the above rotation and translation operations to obtain several groups of collinear line segments corresponding to the coordinate points in the sub-region maps.

[0112] 208. Using one group of collinear line segments among the plurality of groups of collinear line segments as a reference, perform line segment fitting, rotation and translation operations with a group of collinear line segments in an adjacent vertical direction, so that the two groups of collinear line segments overlap in the end and the beginning, until a distribution diagram of the complete dot matrix is ​​obtained.

[0113] In the embodiment of the present application, in the new left and right side area map dot coordinates, the rows are divided into 13 groups, and the corresponding coordinate points are selected according to the row numbers, and the dot coordinate position on the left side is kept unchanged. The right side coordinates are fitted by line segments, rotated, and translated to be collinear with the point coordinates of the corresponding row on the left side, and the last point on the left side coincides with the first point on the right side. In this way, a complete dot matrix distribution diagram of the area map can be obtained. In the schematic diagram of the complete dot matrix distribution of the area map, please refer to Figure 6 , Figure 6 The distribution diagram of the complete dot matrix of the area map formed by the translation of the line segment is shown in Figure 6 The intersection in represents a lattice point.

[0114] Specifically, the coordinates of area 1 are consistent with the dot coordinates in the original area map, and the other areas are arranged in sequence with area 1 through several spatial position transformations in the order of the area map. The evolution of their position transformation relationship is specific: after sorting and co-row processing in area 2, T2' and T2" are processed by homography to obtain the transformation matrix A2 of area 2 image; after sorting and co-column processing in area 3 and area 5, T3' and T3" and T5' and T5" are processed by homography to obtain the transformation matrices A3 and A5 of area 3 and area 5 images; after sorting, co-column and co-row processing in area 4 and area 6, T4' and T4''' and T6' and T6''' are processed by homography to obtain the transformation matrices A4 and A6 of area 4 and area 6 images. Finally, T2", T3", T5", T4''', T6''' are respectively transformed with T1 through homography to calculate the transformation matrices A12, A13, A14, A15, A16.

[0115] area Figure 2 , 3, 4, 5, 6. For schematic diagrams of dot layout in each sub-area map, please refer to Figure 8 , Figure 8 The dots in the figure represent lattice points. The above transformation matrix is ​​used to obtain the new position of the area map through perspective transformation in the following order: {[A2, A12], [A3, A13], [A4, A14], [A5, A15], [A6, A16]}, and then compared with the area Figure 1 Splice together to obtain a complete dot matrix. For the complete dot matrix formed after splicing, please refer to Fig. 9 ,exist Fig. 9 The dots in the figure represent the dot matrix points after stitching.

[0116] See also Fig.10 , Fig.10 An embodiment of the image geometric registration system provided in the present application includes:

[0117] A determination unit 301, used to determine a region map in a VR display screen;

[0118] A shooting unit 302 is used to shoot a plurality of sub-region images of the VR display screen using a camera, wherein images of adjacent regions in the plurality of sub-region images have dot matrix points in the same row or column;

[0119] The calculation unit 303 is used to extract effective points from the lattice points of each of the sub-region images by binarization to generate a binary region, calculate the centroid coordinates of the center point of the binary region, set the ROI envelope of the target size according to the centroid coordinates of the center point of the binary region, and calculate the centroid coordinates of the points in the ROI envelope;

[0120] An establishing unit 304 is used to sort the centroid coordinates of the points in each ROI envelope frame according to the target size, establish a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the region map, and calculate a transformation matrix, and calculate the actual centroid coordinates of the dot matrix in the region map through the transformation matrix;

[0121] The operation unit 305 is used to divide all the sets of actual centroid coordinates into several groups by rows or columns, and obtain a distribution diagram of the complete dot matrix by rotating and translating the coordinate sets of the corresponding rows or columns of the adjacent areas so that the point sets of the adjacent areas are in the same row or column.

[0122] See also Fig.11 The present application also provides an image geometric registration device, the device comprising:

[0123] Processor 401, memory 402, input and output unit 403, bus 404;

[0124] The processor 401 is connected to the memory 402, the input and output unit 403 and the bus 404;

[0125] The memory 402 stores a program, and the processor 401 calls the program to execute any of the above image geometric registration methods.

[0126] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above-mentioned image geometric registration methods.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0128] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, etc. Various media that can store program codes.

Claims

1. A method for geometric registration of images, characterized in that: The method comprises: Determine the area map in the VR display; Using a camera to shoot a plurality of sub-region images of the VR display screen, wherein images of adjacent regions in the plurality of sub-region images have dot matrix points in the same row or column; Extracting effective points from the lattice points of each of the sub-region images by binarization to generate a binarized region, calculating the centroid coordinates of the center point of the binarized region, setting a ROI envelope of a target size according to the centroid coordinates of the center point of the binarized region, and calculating the centroid coordinates of the points in the ROI envelope; The centroid coordinates of the points in each ROI envelope frame are sorted according to the target size, a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the region map is established, and a transformation matrix is ​​calculated, and the actual centroid coordinates of the dot matrix in the region map are calculated by the transformation matrix; Dividing the area map into a plurality of sub-area maps, wherein the area map contains all the actual centroid coordinates, and the size of the sub-area map is the target size; Dividing all the actual centroid coordinates into several groups by rows or columns according to the several sub-region maps; Calculating the relationship between the coordinate points in the plurality of sub-regional graphs in a vertical direction to obtain a plurality of groups of collinear line segments; A group of collinear line segments among the plurality of groups of collinear line segments is used as a reference, and line segment fitting, rotation and translation operations are performed with a group of collinear line segments in an adjacent vertical direction, so that the tails and the beginnings of the two groups of collinear line segments overlap, until a distribution diagram of a complete dot matrix is ​​obtained.

2. The image geometric registration method according to claim 1, characterized in that: Extracting effective points from the dot matrix points of each of the sub-region images by binarization to generate a binarized region, calculating the centroid coordinates of the center point of the binarized region, setting a ROI envelope of a target size according to the centroid coordinates of the center point of the binarized region, and calculating the centroid coordinates of the points in the ROI envelope, including: Extracting the dot matrix points of each sub-region image by binarization; Obtaining the ratio of the area to the length of the short side of the minimum circumscribed rectangle, and extracting effective points of the lattice points according to the ratio, wherein a plurality of the effective points constitute the binary area; Calculating the centroid coordinates of the center point of the binary region according to the length and width of the binary region and the sub-region image; Setting the ROI envelope of the target size according to the centroid coordinates of the center point of the binary region; Calculate the centroid coordinates of the middle points of the ROI envelope frame according to the first formula; The first formula: ； in, The horizontal coordinate represents the centroid coordinate of the point in the ROI envelope; The ordinate represents the centroid coordinates of the points in the ROI envelope; represents the abscissa of the middle point of the ROI envelope, represents the ordinate of the middle point of the ROI envelope, Indicates the grayscale value of the pixel in the ROI envelope.

3. The image geometric registration method according to claim 1, characterized in that: The centroid coordinates of the points in each ROI envelope frame are sorted according to the target size, a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the region map is established, and a transformation matrix is ​​calculated. The actual centroid coordinates of the dot matrix in the region map are calculated by the transformation matrix, including: Sorting the centroid coordinates of the points in each ROI envelope frame according to the target size so that the centroid coordinates of the points in each ROI envelope frame correspond to the dot matrix layout of the region map in one-to-one order; Establishing the conversion relationship between the centroid coordinates of the middle points of each ROI envelope frame and the row and column coordinates of the region map through perspective transformation, and calculating the transformation matrix; The actual row and column coordinates on the area map are multiplied by the transformation matrix to obtain the actual centroid coordinates of the dot matrix in the area map.

4. The image geometric registration method according to claim 1, characterized in that: The coordinate points in the plurality of sub-regional graphs are calculated in a vertical direction to obtain a plurality of groups of collinear line segments, including: Fitting two sets of vertically adjacent coordinate points in the plurality of sub-region maps into two line segments respectively according to the least square method, and calculating a first inclination angle and a second inclination angle of the two line segments relative to a horizontal line; The least square method is: ,in are the coordinates of the fitted line, is the actual point coordinate, L is the calculation error; Rotating the two line segments using a rotation matrix according to the first tilt angle and the second tilt angle so that the two line segments are parallel, and moving the two line segments using a translation matrix to obtain a set of collinear line segments, wherein the collinear line segments are line segments formed by connecting the two line segments end to end; The above-mentioned rotation and translation operations are repeated to obtain a plurality of groups of collinear line segments corresponding to the coordinate points in the plurality of sub-region maps.

5. The image geometric registration method according to claim 4, characterized in that: The rotation matrix is: ; wherein θ is the first tilt angle or the second tilt angle; The translation matrix is: ; Where P is the translation matrix, T x is the translation distance along the x direction, T y is the translation distance along the y direction.

6. An image geometric registration system, characterized in that: include: A determination unit, used to determine a region map in a VR display screen; A shooting unit, used for shooting a plurality of sub-region images of the VR display screen using a camera, wherein images of adjacent regions in the plurality of sub-region images have dot matrix points in the same row or column; A first calculation unit is used to extract effective points from the lattice points of each of the sub-region images by binarization to generate a binary region, calculate the centroid coordinates of the center point of the binary region, set an ROI envelope of a target size according to the centroid coordinates of the center point of the binary region, and calculate the centroid coordinates of the points in the ROI envelope; An establishing unit is used to sort the centroid coordinates of the points in each ROI envelope frame according to the target size, establish a conversion relationship between the centroid coordinates of the points in each ROI envelope frame and the row and column coordinates in the area map, and calculate a transformation matrix, and calculate the actual centroid coordinates of the dot matrix in the area map through the transformation matrix; A first division unit is used to divide the region map into a plurality of sub-region maps, wherein the region map contains all the actual centroid coordinates, and the size of the sub-region map is the target size; A second division unit is configured to divide all the actual centroid coordinates into a plurality of groups by row or column according to the plurality of sub-region diagrams; A second calculation unit calculates the relationship between the coordinate points in the plurality of sub-regional maps in a vertical direction to obtain a plurality of groups of collinear line segments; The operation unit is used to use one group of collinear line segments among the plurality of groups of collinear line segments as a reference, and perform line segment fitting, rotation and translation operations with a group of collinear line segments in an adjacent vertical direction, so that the tails and the beginnings of the two groups of collinear line segments overlap, until a distribution diagram of a complete dot matrix is ​​obtained.

7. An image geometric registration device, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 5.

Images