Image rotation matrix estimation method and device, electronic equipment and storage medium

Abstract

The invention provides an image rotation matrix estimation method and device, electronic equipment and a storage medium. The method comprises the following steps: converting an acquired single-scene image into a pure-color scene image; mapping the pure-color scene image to a specific position of the panoramic image template, and generating an identification image of the pure-color scene image; determining a first corner point coordinate of each first image corner point of the pure color scene image under an image coordinate system of the identification image; and determining an image rotation matrix based on a first corner point coordinate of each first image corner point, a second corner point coordinate of each second image corner point of the pure color scene image, camera built-in parameters and distortion correction parameters in response to the identification image satisfying the panoramic mapping condition.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an estimation method, device, electronic device and storage medium for an image rotation matrix. Background Art

[0002] Image stitching is an important direction in the field of image processing. Stitching two or more single-scene images with overlapping areas collected from the same scene, different acquisition positions, and different acquisition angles to generate a panoramic image has become the main means of image application.

[0003] Image stitching mainly relies on third-party image processing software; since third-party image processing software can only directly output panoramic images and cannot obtain information such as the rotation and translation relationship between the panoramic image and the single-scene image (i.e., the image rotation matrix) and camera parameters, it is impossible to perform extended applications on the generated panoramic images, such as image measurement and other functions. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide an image rotation matrix estimation method, device, electronic device and storage medium, which can accurately determine the image rotation matrix between the single scene image and the panoramic image by combining the corner point coordinates of the identification image of the pure color scene image and the correspondence between the single scene image and the pure color scene image.

[0005] The present application embodiment provides a method for estimating an image rotation matrix, the method comprising:

[0006] Convert the acquired single scene image into a pure color scene image with a single color;

[0007] Mapping the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image;

[0008] By identifying the identification image, determining the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image;

[0009] In response to the identification image satisfying a panoramic mapping condition, an image rotation matrix is ​​determined based on first corner point coordinates of each first image corner point, second corner point coordinates of each second image corner point of the pure color scene image, camera built-in parameters, and distortion correction parameters.

[0010] In a possible implementation manner, mapping the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image includes:

[0011] The pixel values ​​at specific positions of the panoramic image template are filled with pixel values ​​corresponding to the colors of the pure color scene image, and the pixel values ​​at other positions of the panoramic image template are filled with preset pixel values ​​to generate an identification image of the pure color scene image.

[0012] In a possible implementation manner, determining the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image includes:

[0013] Recognize the identification image using a contour recognition function, and determine image contour information of the pure color scene image in an image coordinate system of the identification image;

[0014] A plurality of contour boundary points included in the image contour information are screened using a contour point screening function to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image.

[0015] In a possible implementation manner, the following steps are performed to determine whether the identification image satisfies the panoramic mapping condition:

[0016] For each first image corner point of the pure color scene image in the image coordinate system of the identification image, determine whether the first image corner point is located at an image boundary of the identification image based on the first corner point coordinates of the first image corner point;

[0017] If all the first image corner points are not located at the image boundary of the identification image, it is determined that the identification image meets the panoramic mapping condition.

[0018] In a possible implementation, determining the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, camera built-in parameters, and distortion correction parameters includes:

[0019] Sort the first corner point coordinates of the plurality of first image corner points and the second corner point coordinates of the plurality of second image corner points in a preset order to obtain a first coordinate matrix of the identification image and a second coordinate matrix of the pure color scene image;

[0020] The first coordinate matrix, the second coordinate matrix, the camera built-in parameters and the distortion correction parameters are input into a coordinate conversion function to determine an image rotation matrix.

[0021] In a possible implementation, the estimation method further includes:

[0022] Using the image rotation matrix, converting a first pixel coordinate of each pixel point of the single scene image to a second pixel coordinate in the panoramic image;

[0023] According to the second pixel point coordinates, the pixel point is displayed in the panoramic image.

[0024] In a possible implementation, the single color is a color obtained by any combination of three primary colors.

[0025] The embodiment of the present application further provides a device for estimating an image rotation matrix, the device comprising:

[0026] A color conversion module, used for converting the acquired single scene image into a pure color scene image with a single color;

[0027] An image generation module, used for mapping the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image;

[0028] A coordinate recognition module, configured to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by recognizing the identification image;

[0029] The matrix determination module is used to determine the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters and the distortion correction parameters in response to the identification image satisfying the panoramic mapping condition.

[0030] In a possible implementation manner, when the image generation module generates an identification image of the pure color scene image at a specific position for mapping the pure color scene image to a panoramic image template, the image generation module is configured to:

[0031] The pixel values ​​at specific positions of the panoramic image template are filled with pixel values ​​corresponding to the colors of the pure color scene image, and the pixel values ​​at other positions of the panoramic image template are filled with preset pixel values ​​to generate an identification image of the pure color scene image.

[0032] In a possible implementation, when the coordinate recognition module is used to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image, the coordinate recognition module is used to:

[0033] Recognize the identification image using a contour recognition function, and determine image contour information of the pure color scene image in an image coordinate system of the identification image;

[0034] A plurality of contour boundary points included in the image contour information are screened using a contour point screening function to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image.

[0035] In a possible implementation manner, the matrix determination module is used to determine whether the identification image satisfies the panoramic mapping condition through the following steps:

[0036] For each first image corner point of the pure color scene image in the image coordinate system of the identification image, determine whether the first image corner point is located at an image boundary of the identification image based on the first corner point coordinates of the first image corner point;

[0037] If all the first image corner points are not located at the image boundary of the identification image, it is determined that the identification image meets the panoramic mapping condition.

[0038] In a possible implementation, the matrix determination module is used to determine the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters, and the distortion correction parameters, and the matrix determination module is used to:

[0039] Sort the first corner point coordinates of the plurality of first image corner points and the second corner point coordinates of the plurality of second image corner points in a preset order to obtain a first coordinate matrix of the identification image and a second coordinate matrix of the pure color scene image;

[0040] The first coordinate matrix, the second coordinate matrix, the camera built-in parameters and the distortion correction parameters are input into a coordinate conversion function to determine an image rotation matrix.

[0041] In a possible implementation manner, the estimation device further includes an image mapping module, and the image mapping module is used to:

[0042] Using the image rotation matrix, converting a first pixel coordinate of each pixel point of the single scene image to a second pixel coordinate in the panoramic image;

[0043] According to the second pixel point coordinates, the pixel point is displayed in the panoramic image.

[0044] In a possible implementation, the single color is a color obtained by any combination of three primary colors.

[0045] An embodiment of the present application also provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the image rotation matrix estimation method as described above are performed.

[0046] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for estimating the image rotation matrix as described above are executed.

[0047] The image rotation matrix estimation method, device, electronic device and storage medium provided in the embodiments of the present application convert an acquired single scene image into a pure color scene image with a single color; map the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image; determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image; in response to the identification image satisfying the panoramic mapping condition, determine the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters and the distortion correction parameters.

[0048] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of a method for estimating an image rotation matrix provided in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a pure color scene image provided in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of a logo image provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of the structure of an image rotation matrix estimation device provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0056] Research has found that image stitching is an important direction in the field of image processing. Stitching two or more single-scene images with overlapping areas collected from the same scene, different acquisition positions, and different acquisition angles to generate a panoramic image has become the main means of image application.

[0057] Image stitching mainly relies on third-party image processing software; since third-party image processing software can only directly output panoramic images and cannot obtain information such as the rotation and translation relationship between the panoramic image and the single-scene image (i.e., the image rotation matrix) and camera parameters, it is impossible to perform extended applications on the generated panoramic images, such as image measurement and other functions.

[0058] Based on this, an embodiment of the present application provides a method for estimating an image rotation matrix, which can accurately determine the image rotation matrix between the single scene image and the panoramic image by combining the corner point coordinates of the identification image of the pure color scene image and the correspondence between the single scene image and the pure color scene image.

[0059] See also Figure 1 , Figure 1 The following is a flow chart of a method for estimating an image rotation matrix provided in an embodiment of the present application. Figure 1 As shown in , the image rotation matrix estimation method provided by the embodiment of the present application includes:

[0060] S101: Convert the acquired single scene image into a pure color scene image with a single color.

[0061] S102: Map the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image.

[0062] S103 . Determine first corner point coordinates of each first image corner point of the pure color scene image in an image coordinate system of the identification image by identifying the identification image.

[0063] S104. In response to the identification image satisfying a panoramic mapping condition, determine an image rotation matrix based on first corner point coordinates of each first image corner point, second corner point coordinates of each second image corner point of the pure color scene image, camera built-in parameters, and distortion correction parameters.

[0064] In step S101, a single scene image in a specific scene is captured by a camera, and the single scene image can be converted into a pure color scene image with a single color by third-party software; that is, the acquired single scene image (such as Figure 2 As shown, Figure 2 A schematic diagram of a pure color scene image provided in an embodiment of the present application).

[0065] The single color is a color obtained by any combination of three primary colors (RGB three primary colors); the three primary colors refer to three basic colors that can be mixed in different proportions to obtain all other colors. In additive color mixing, the above three colors refer to red, green and blue.

[0066] In step S102, a panoramic image template is obtained; the panoramic image is generated by the image stitching software (e.g., PTGui), and the mapping relationship between the single scene image and the full scene image cannot be directly obtained; however, the image stitching software (e.g., PTGui) can generate a panoramic image template; with the help of the image stitching software (e.g., PTGui) and the panoramic image template, the pure color scene image corresponding to the single scene image can be mapped to a specific position of the panoramic image template to generate an identification image of the pure color scene image (e.g., Figure 3 As shown, Figure 3 A schematic diagram of a logo image provided in an embodiment of the present application).

[0067] The specific position of the panoramic image template refers to the pixel area where the single scene image is mapped to the display position in the panoramic image;

[0068] Here, the identification image is an image that indicates the position of the single scene image in the panoramic image; the position of the single scene image in the identification image will be filled with the pixel value corresponding to the color of the solid color scene image, while other positions will be filled with preset pixel values ​​(for example, 0, i.e. black).

[0069] In one implementation, step S102 includes:

[0070] The pixel values ​​at specific positions of the panoramic image template are filled with pixel values ​​corresponding to the colors of the pure color scene image, and the pixel values ​​at other positions of the panoramic image template are filled with preset pixel values ​​to generate an identification image of the pure color scene image.

[0071] In this step, in the process of mapping the pure color scene image to a specific position of the panoramic image template, the pixel value of the color presented by the pure color scene image is determined, and the pixel value at the specific position of the panoramic image template is filled with the pixel value corresponding to the color; the pixel values ​​at other positions in the panoramic image template except the specific position are filled with preset pixel values ​​(for example, filled with 0) to generate an identification image of the pure color scene image.

[0072] In step S103, after mapping the single scene image to the panoramic image template to generate an identification image of the pure color scene image, the identification image is read through the opencv-read function, the image coordinate system of the identification image is constructed, the image data of the identification image is converted into a digital array form and the identification image size is obtained (for example, w1=9027, h1=9027).

[0073] Among them, the read function of OpenCV is a commonly used function used to read images from files. The basic syntax of this function is cv2.imread(filename, flags), where filename is the path of the image file and flags is an optional parameter used to specify the way to read the image, such as whether to read in grayscale mode.

[0074] In one implementation, step S103 includes:

[0075] S1031 . Identify the identification image using a contour recognition function, and determine image contour information of the pure color scene image in an image coordinate system of the identification image.

[0076] In this step, the read identification image is identified using a contour recognition function (opencv-contours) to determine the image contour information after the pure color scene image is mapped in the image coordinate system corresponding to the identification image; wherein the image contour information includes a plurality of contour boundary points, that is, the plurality of contour boundary points are fitted in the identification image to obtain the contour boundary of the pure color scene image.

[0077] Here, contours (contour recognition function) in OpenCV is used to find and draw a function with continuous boundary points in the identification image, which is often used in tasks such as shape analysis, object recognition, and image segmentation.

[0078] S1032: Filter multiple contour boundary points included in the image contour information using a contour point screening function to determine first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image.

[0079] In this step, after obtaining the contour boundary of the pure color scene image, the contour point screening function (opencv-approxPolyDP) is used to screen the multiple contour boundary points included in the image contour information to screen out the first image corner points of the pure color scene image, and determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image.

[0080] Here, approxPolyDP is a function in OpenCV for polygon approximation, which is a good way to simplify contours, especially when the contours are noisy or have too many details.

[0081] Considering that in the process of image stitching, if "incomplete" single scene images are used to stitch panoramic images, the panoramic image cannot be stitched accurately, resulting in the loss of some image details; therefore, when performing image stitching, it is necessary to use "complete" single scene images to stitch panoramic images; therefore, before determining the image rotation matrix, it is also necessary to judge whether the pure color scene image meets the panoramic mapping conditions, so as to ensure that the "complete" single scene image can be used for stitching panoramic images.

[0082] In step S104, in response to the identification image satisfying the panoramic mapping conditions, the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters and the distortion correction parameters are substituted into the coordinate conversion function (opencv-solvePnp) to determine the image rotation matrix.

[0083] The solvePnp function is a function in the OpenCV library, which is used for monocular camera pose estimation. It calculates the pose of a three-dimensional object from two-dimensional image points and returns it in the form of a rotation and translation matrix.

[0084] Here, if the first image corner point of the pure color scene image is located at the image boundary of the identification image, it means that when the single scene image corresponding to the pure color scene image is used for panoramic image stitching, the single scene image is at risk of being "cropped". At this time, in order to avoid using "incomplete" single scene images for panoramic image stitching, it is necessary to judge whether the pure color scene image is located at the image boundary of the identification image based on the first corner point coordinates of the first image corner point of the pure color scene image, so as to determine whether the identification image meets the mapping conditions.

[0085] The second image corner points of the pure color scene image are the default image corner points determined according to the default size of the pure color scene image. That is, the image size of the pure color image (for example, w2=1920, h2=1080); the second image corner points are [[0,0], [w2,0], [w2,h2], [0,h2]] (for example, [[0,0], [1920,0], [1920,1080], [0,1080]]).

[0086] In one implementation, the following steps are performed to determine whether the identification image satisfies the panoramic mapping condition:

[0087] S41. For each first image corner point of the pure color scene image in the image coordinate system of the identification image, determine whether the first image corner point is located at an image boundary of the identification image based on the first corner point coordinates of the first image corner point.

[0088] In this step, for each first image corner point of the pure color scene image in the image coordinate system of the identification image, determine whether the X-axis coordinate of the first image corner point coordinate is equal to 0 or the width w1 of the identification image (for example, 9027) or whether the Y-axis coordinate is equal to 0 or the height h1 of the identification image (for example, 9027). If the X-axis coordinate or the Y-axis coordinate meets the above judgment conditions, it is determined that the first image corner point is located at the image boundary of the identification image.

[0089] S42: If all the first image corner points are not located at the image boundary of the identification image, it is determined that the identification image meets the panoramic mapping condition.

[0090] Here, when determining the image rotation matrix, it is necessary to ensure that the first image corner point and the second image corner point are input into the coordinate conversion function in a corresponding form, so as to ensure the accuracy of the determined image rotation matrix.

[0091] In one implementation, determining the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, camera built-in parameters, and distortion correction parameters includes:

[0092] S43. Sort the first corner point coordinates of the plurality of first image corner points and the second corner point coordinates of the plurality of second image corner points in a preset order to obtain a first coordinate matrix of the identification image and a second coordinate matrix of the pure color scene image.

[0093] In this step, the first corner point coordinates of multiple first image corner points are sorted in a preset order to obtain a first coordinate matrix of the identification image, and the second corner point coordinates of multiple second image corner points are sorted in a preset order to obtain a second coordinate matrix of the pure color scene image; thereby, a one-to-one correspondence between each first element in the first coordinate matrix and each second element in the second coordinate matrix is ​​ensured.

[0094] Among them, the above-mentioned sorting according to a preset order refers to sorting according to the coordinate system quadrant of the image coordinate system to which the corner point coordinates belong, and / or in a clockwise order; for example, multiple first image corner points include an upper right first image corner point, a lower right first image corner point, a lower left first image corner point and a lower left first image corner point, and the first corner point coordinates of the multiple first image corner points are sorted in a clockwise order, and the obtained first coordinate matrix is ​​[the first corner point coordinates of the upper right first image corner point, the first corner point coordinates of the lower right first image corner point, the first corner point coordinates of the lower left first image corner point, and the first corner point coordinates of the upper left first image corner point]; accordingly, the second corner point coordinates of the multiple second image corner points are sorted in a clockwise order, and the obtained second coordinate matrix is ​​[the second corner point coordinates of the upper right second image corner point, the second corner point coordinates of the lower right second image corner point, the second corner point coordinates of the lower left second image corner point, and the second corner point coordinates of the upper left second image corner point]. Exemplarily, the first corner point coordinates of the upper right image corner point, the lower right image corner point, the lower left image corner point, and the upper left image corner point of the pure color scene image are [(x 11 ,y 11 )(x 12 ,y 12 )(x 13 ,y 13 )(x 14 ,y 14 )], the coordinates of the first corner points of the upper right corner point, lower right corner point, upper left corner point and lower left corner point of the identified image are [(x 21 ,y 21 )(x 22 ,y 22 )(x 23 ,y 23 )(x 24 ,y 24 )]; where (x 11 ,y 11 ) and (x 21 ,y 21 ) correspond to the upper right corner of the image, (x 12 ,y 12 ) and (x 22 ,y 22 ) correspond to the lower right corner of the image, (x 13 ,y 13 ) and (x23 ,y 23 ) corresponds to the lower left corner of the image, (x 14 ,y 14 ) and (x 24 ,y 24 ) correspond to the upper left corner of the image.

[0095] S44, inputting the first coordinate matrix, the second coordinate matrix, the camera built-in parameters and the distortion correction parameters into a coordinate conversion function to determine an image rotation matrix.

[0096] In this step, the first coordinate matrix, the second coordinate matrix, the camera built-in parameters and the distortion correction parameters are input into the coordinate conversion function (ie, the opencv-solvePnp function) to determine the image rotation matrix.

[0097] Among them, the camera built-in parameters (M value) can be defaulted to [1000, 0, 960, 0, 1000, 540, 0, 0, 1]; the distortion correction parameters (D value) can be defaulted to [0, 0, 0, 0, 0].

[0098] In one embodiment, the estimation method further includes:

[0099] S105 . Using the image rotation matrix, convert the first pixel coordinates of each pixel point of the single scene image into second pixel coordinates in the panoramic image.

[0100] In this step, in the process of mapping the single scene image into the panoramic image, the image rotation matrix is ​​used to convert the first pixel coordinates of each pixel in the single scene image into the second pixel coordinates of the pixel in the panoramic image.

[0101] S106: Display the pixel point in the panoramic image according to the second pixel point coordinates.

[0102] In this step, the pixel point is displayed at the corresponding position of the panoramic image according to the second pixel point coordinates.

[0103] The image rotation matrix estimation method provided in the embodiment of the present application converts the acquired single scene image into a pure color scene image with a single color; maps the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image; determines the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image; and determines the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters, and the distortion correction parameters in response to the identification image satisfying the panoramic mapping condition.

[0104] See also Figure 4 , Figure 4 The following is a schematic diagram of the structure of an image rotation matrix estimation device provided in an embodiment of the present application. Figure 4 As shown in , the estimation device 400 includes:

[0105] A color conversion module 410 is used to convert the acquired single scene image into a pure color scene image with a single color;

[0106] An image generation module 420, configured to map the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image;

[0107] A coordinate recognition module 430 is used to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image;

[0108] The matrix determination module 440 is used to determine the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters and the distortion correction parameters in response to the identification image satisfying the panoramic mapping condition.

[0109] In a possible implementation manner, when the image generation module 420 generates an identification image of the pure color scene image at a specific position for mapping the pure color scene image to a panoramic image template, the image generation module 420 is used to:

[0110] The pixel values ​​at specific positions of the panoramic image template are filled with pixel values ​​corresponding to the colors of the pure color scene image, and the pixel values ​​at other positions of the panoramic image template are filled with preset pixel values ​​to generate an identification image of the pure color scene image.

[0111] In a possible implementation manner, when the coordinate recognition module 430 is used to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image, the coordinate recognition module 430 is used to:

[0112] Recognize the identification image using a contour recognition function, and determine image contour information of the pure color scene image in an image coordinate system of the identification image;

[0113] A plurality of contour boundary points included in the image contour information are screened using a contour point screening function to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image.

[0114] In a possible implementation manner, the matrix determination module 440 is used to determine whether the identification image meets the panoramic mapping condition through the following steps:

[0115] For each first image corner point of the pure color scene image in the image coordinate system of the identification image, determine whether the first image corner point is located at an image boundary of the identification image based on the first corner point coordinates of the first image corner point;

[0116] If all the first image corner points are not located at the image boundary of the identification image, it is determined that the identification image meets the panoramic mapping condition.

[0117] In a possible implementation, when the matrix determination module 440 is used to determine the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters, and the distortion correction parameters, the matrix determination module 440 is used to:

[0118] Sort the first corner point coordinates of the plurality of first image corner points and the second corner point coordinates of the plurality of second image corner points in a preset order to obtain a first coordinate matrix of the identification scene image and a second coordinate matrix of the pure color scene image;

[0119] The first coordinate matrix, the second coordinate matrix, the camera built-in parameters and the distortion correction parameters are input into a coordinate conversion function to determine an image rotation matrix.

[0120] In a possible implementation manner, the estimation device 400 further includes an image mapping module (not shown in the figure), and the image mapping module is used to:

[0121] Using the image rotation matrix, converting a first pixel coordinate of each pixel point of the single scene image to a second pixel coordinate in the panoramic image;

[0122] According to the second pixel point coordinates, the pixel point is displayed in the panoramic image.

[0123] In a possible implementation, the single color is a color obtained by any combination of three primary colors.

[0124] The image rotation matrix estimation device provided in the embodiment of the present application converts the acquired single scene image into a pure color scene image with a single color; maps the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image; determines the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image; and determines the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters, and the distortion correction parameters in response to the identification image satisfying the panoramic mapping condition.

[0125] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in , the electronic device 500 includes a processor 510 , a memory 520 and a bus 530 .

[0126] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 The steps of the method for estimating the image rotation matrix in the method embodiment shown, and the specific implementation manner can be found in the method embodiment, which will not be repeated here.

[0127] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the method for estimating the image rotation matrix in the method embodiment shown, and the specific implementation manner can be found in the method embodiment, which will not be repeated here.

[0128] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0129] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0130] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0131] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0132] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0133] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for estimating an image rotation matrix, characterized in that: The estimation method includes: Convert the acquired single scene image into a pure color scene image with a single color; Mapping the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image; By identifying the identification image, determining the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image; In response to the identification image satisfying a panoramic mapping condition, an image rotation matrix is ​​determined based on first corner point coordinates of each first image corner point, second corner point coordinates of each second image corner point of the pure color scene image, camera built-in parameters, and distortion correction parameters.

2. The estimation method according to claim 1, characterized in that Mapping the pure color scene image to a specific position of a panoramic image template to generate a logo image of the pure color scene image includes: The pixel values ​​at specific positions of the panoramic image template are filled with pixel values ​​corresponding to the colors of the pure color scene image, and the pixel values ​​at other positions of the panoramic image template are filled with preset pixel values ​​to generate an identification image of the pure color scene image.

3. The estimation method according to claim 1, characterized in that The step of determining the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by identifying the identification image comprises: Recognize the identification image using a contour recognition function, and determine image contour information of the pure color scene image in an image coordinate system of the identification image; A plurality of contour boundary points included in the image contour information are screened using a contour point screening function to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image.

4. The estimation method according to claim 1, characterized in that: The following steps are used to determine whether the identification image meets the panoramic mapping condition: For each first image corner point of the pure color scene image in the image coordinate system of the identification image, determine whether the first image corner point is located at an image boundary of the identification image based on the first corner point coordinates of the first image corner point; If all the first image corner points are not located at the image boundary of the identification image, it is determined that the identification image meets the panoramic mapping condition.

5. The estimation method according to claim 1, characterized in that: The determining of the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters and the distortion correction parameters includes: Sort the first corner point coordinates of the plurality of first image corner points and the second corner point coordinates of the plurality of second image corner points in a preset order to obtain a first coordinate matrix of the identification image and a second coordinate matrix of the pure color scene image; The first coordinate matrix, the second coordinate matrix, the camera built-in parameters and the distortion correction parameters are input into a coordinate conversion function to determine an image rotation matrix.

6. The estimation method according to claim 1, characterized in that: The estimation method further comprises: Using the image rotation matrix, converting a first pixel coordinate of each pixel point of the single scene image to a second pixel coordinate in the panoramic image; According to the second pixel point coordinates, the pixel point is displayed in the panoramic image.

7. The estimation method according to claim 1, characterized in that: The single color is a color obtained by any combination of the three primary colors.

8. A device for estimating an image rotation matrix, characterized in that: The estimation device comprises: A color conversion module, used for converting the acquired single scene image into a pure color scene image with a single color; An image generation module, used for mapping the pure color scene image to a specific position of a panoramic image template to generate an identification image of the pure color scene image; A coordinate recognition module, configured to determine the first corner point coordinates of each first image corner point of the pure color scene image in the image coordinate system of the identification image by recognizing the identification image; A matrix determination module is used to determine the image rotation matrix based on the first corner point coordinates of each first image corner point, the second corner point coordinates of each second image corner point of the pure color scene image, the camera built-in parameters and the distortion correction parameters in response to the identification image satisfying the panoramic mapping condition.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the steps of the image rotation matrix estimation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for estimating an image rotation matrix according to any one of claims 1 to 7 are executed.

Images