Panoramic image stitching method, electronic equipment and computer readable storage medium

Abstract

The invention provides a panoramic image stitching method, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of to-be-stitched images, and determining the position relation between the to-be-stitched images; determining feature pixel points in the image to be spliced; aiming at two adjacent images to be spliced, matching the feature pixel points so as to determine a matched pixel point pair in the feature pixel points; and splicing the adjacent images to be spliced based on the matched pixel point pairs to obtain a target panoramic image. By matching the feature pixel points in the adjacent to-be-spliced images, the matching pixel point pairs representing the same position in the adjacent to-be-spliced images can be determined, then splicing of the to-be-spliced images is achieved based on the matching pixel point pairs, the obtained panoramic image is matched with the environment, and the splicing accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a panoramic image stitching method, an electronic device and a computer-readable storage medium. Background Art

[0002] In scenarios such as smart transportation and smart security, it is usually necessary to collect 360° panoramic environments in order to observe scene information clearly and comprehensively. Most existing panoramic acquisition devices calibrate the image according to the acquisition position and stitch the panoramic images based on the calibration results. This method ignores the strong correlation between the stitching effect and the scene, which will lead to obvious stitching seams and stitching errors. Summary of the invention

[0003] The main purpose of the present invention is to propose a panoramic image stitching method, an electronic device and a computer-readable storage medium, aiming to solve the problem of poor stitching results of panoramic images in the prior art.

[0004] To achieve the above object, the present invention provides a panoramic image stitching method, the method comprising the steps of:

[0005] Acquire a plurality of images to be stitched, and determine the positional relationship between the images to be stitched;

[0006] Determining characteristic pixel points in the images to be stitched;

[0007] For two adjacent images to be stitched, matching the characteristic pixel points to determine matching pixel point pairs among the characteristic pixel points;

[0008] Adjacent images to be stitched are stitched based on the matching pixel pairs to obtain a target panoramic image.

[0009] Optionally, determining characteristic pixel points in the images to be stitched includes:

[0010] For each unit pixel in the image to be stitched, determine a neighboring pixel whose distance to the unit pixel is less than a preset distance threshold;

[0011] Determine the difference between the neighborhood pixel and the unit pixel;

[0012] Determining whether the difference is greater than a preset difference threshold;

[0013] If the difference is greater than the preset difference threshold, the unit pixel is taken as a feature pixel.

[0014] Optionally, the determining the difference between the neighborhood pixel and the unit pixel includes:

[0015] Sampling the neighborhood pixel points to obtain sampled pixel points;

[0016] For each of the sampling pixel points, calculating a pixel difference between the sampling pixel point and the unit pixel point;

[0017] The sampling pixel points whose pixel difference is greater than a preset difference threshold are taken as valid sampling points;

[0018] Calculating the ratio of the number of the effective sampling points to the number of the sampling pixel points;

[0019] The quantitative ratio is taken as the degree of difference.

[0020] Optionally, if the difference is greater than the preset difference threshold, taking the unit pixel as a feature pixel includes:

[0021] If the difference is greater than the preset difference threshold, the unit pixel is used as the pixel before suppression;

[0022] Setting a sliding window to traverse the image to be stitched, wherein the sliding window includes a plurality of the unit pixels;

[0023] In each of the sliding windows, the pixel point before suppression with the largest difference is determined as the feature pixel point.

[0024] Optionally, matching the feature pixel points to determine matching pixel point pairs among the feature pixel points includes:

[0025] Determine a first characteristic pixel point in the front stitched image and a second characteristic pixel point in the back stitched image;

[0026] Matching the first characteristic pixel points with the second characteristic pixel points to determine a first pixel point in the first characteristic pixel points and a second pixel point in the second characteristic pixel points, wherein the first pixel points correspond to the second pixel points one by one;

[0027] The corresponding first pixel point and the second pixel point are used as the matching pixel point pair.

[0028] Optionally, the respectively stitching of adjacent images to be stitched based on the matching pixel pairs comprises:

[0029] Determine a first coordinate corresponding to a first pixel point and a second coordinate corresponding to a second pixel point in the matching pixel point pair, wherein the first coordinate is a coordinate of the first pixel point in a previous stitched image, and the second coordinate is a coordinate of the second pixel point in a subsequent stitched image;

[0030] determining a pixel transformation matrix according to the first coordinate and the second coordinate;

[0031] The adjacent images to be stitched are stitched according to the pixel transformation matrix.

[0032] Optionally, after obtaining the target panoramic image, the method includes:

[0033] Acquire image information of the image to be stitched;

[0034] The image information is associated with the target panoramic image.

[0035] To achieve the above object, the present invention further provides a panoramic image stitching device, the panoramic image stitching device comprising:

[0036] A first acquisition module is used to acquire a plurality of images to be stitched and determine the positional relationship between the images to be stitched;

[0037] A first determining module, used to determine characteristic pixel points in the image to be stitched;

[0038] A first matching module, used for matching the characteristic pixel points of two adjacent images to be stitched, so as to determine a matching pixel point pair among the characteristic pixel points;

[0039] The first stitching module is used to stitch adjacent images to be stitched based on the matching pixel pairs to obtain a target panoramic image.

[0040] To achieve the above objectives, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the panoramic image stitching method as described above.

[0041] To achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the panoramic image stitching method described above are implemented.

[0042] The present invention proposes a panoramic image stitching method, electronic device and computer-readable storage medium, which obtains multiple images to be stitched, and determines the positional relationship between the images to be stitched; determines characteristic pixel points in the images to be stitched; matches the characteristic pixel points for two adjacent images to be stitched to determine matching pixel point pairs in the characteristic pixel points; stitches the adjacent images to be stitched based on the matching pixel point pairs to obtain a target panoramic image. By matching the characteristic pixel points in the adjacent images to be stitched, it is possible to determine the matching pixel point pairs representing the same position in the adjacent images to be stitched, and then stitch the images to be stitched based on the matching pixel point pairs, so that the obtained panoramic image matches the environment, and the stitching accuracy is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of a first embodiment of a panoramic image stitching method of the present invention;

[0044] Figure 2 It is a schematic diagram of the module structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0046] The present invention provides a panoramic image stitching method, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a panoramic image stitching method according to the present invention, wherein the method comprises the following steps:

[0047] Step S10, acquiring a plurality of images to be stitched, and determining the positional relationship between the images to be stitched;

[0048] The image to be stitched is the source image for generating a panoramic image; multiple images to be stitched corresponding to the environment are obtained through different viewing angles; the specific acquisition method can be set based on the actual application scenario. For example, in this embodiment, multiple IP (Internet Protocol) cameras are evenly arranged on a cylindrical device, and the viewing angles of multiple IP cameras cover a 360° scene to achieve the acquisition of a 360° panoramic environment image. The IP camera and the computer are connected to the same local area network, and the images collected by the camera are transmitted to the computer based on the IP protocol to ensure that the computer obtains the images collected by the IP camera in real time. Each IP camera outputs an image to be stitched, and multiple images to be stitched are stitched to obtain a target panoramic image; the number of IP cameras can be determined based on the horizontal viewing angle of the IP camera; it can be understood that the more IP cameras there are, the greater the overlap ratio of adjacent images to be stitched, the better the stitching effect of the panoramic image, and the longer the stitching time. Therefore, the number of IP cameras can be set based on the overlap ratio requirements, panoramic image quality requirements, stitching time requirements, etc.; in order to ensure the basic effect, the overlap ratio of adjacent images to be stitched can be set to be greater than 50%

[0049] The positional relationship indicates the position of the IP camera corresponding to the image to be stitched. Specifically, an IP camera is set as the starting camera, and the set IP cameras are sorted in a clockwise or counterclockwise order starting from the starting camera to obtain the serial number corresponding to each IP camera; it can be understood that, except that the serial numbers of the starting camera and the end camera are not continuous, the serial numbers of the other adjacent IP cameras are continuous.

[0050] After the images to be stitched are generated, they will be associated with the serial numbers of the corresponding IP cameras, and the positional relationship between the images to be stitched can be determined according to the needs of each image to be stitched.

[0051] Step S20, determining characteristic pixel points in the image to be stitched;

[0052] A characteristic pixel point refers to a point where the gray value of an image changes dramatically or a pixel point with a large curvature on the edge of an image; the method for determining the characteristic pixel point can be set based on actual application needs.

[0053] Step S30, matching the characteristic pixel points of two adjacent images to be stitched, so as to determine a matching pixel point pair among the characteristic pixel points;

[0054] The matching pixel pair contains two matching pixels, and the two matching pixels are respectively located in adjacent images to be stitched; the matching pixel pair indicates that the two matching pixels contained therein correspond to the same environmental target, that is, when stitching, the two matching pixels in the matching pixel pair can be stitched as one pixel; the specific matching method can be set based on the actual application scenario.

[0055] Step S40 , stitching the adjacent images to be stitched together based on the matching pixel pairs to obtain a target panoramic image.

[0056] By stitching adjacent images to be stitched based on matching pixels in sequence, a target panoramic image can be obtained after all images to be stitched are stitched. Specifically, stitching is performed in sequence based on sequence numbers. The specific stitching order can be set based on actual needs.

[0057] This embodiment matches feature pixels in adjacent images to be stitched, so that matching pixel pairs representing the same position in adjacent images to be stitched can be determined, and then stitching of the images to be stitched is achieved based on the matching pixel pairs. The obtained panoramic image matches the environment, thereby improving stitching accuracy.

[0058] Furthermore, in a second embodiment of the panoramic image stitching method of the present invention proposed based on the first embodiment of the present invention, step S20 includes the following steps:

[0059] Step S21, for each unit pixel in the image to be stitched, determining a neighboring pixel whose distance to the unit pixel is less than a preset distance threshold;

[0060] Step S22, determining the difference between the neighborhood pixel and the unit pixel;

[0061] Step S23, determining whether the difference is greater than a preset difference threshold;

[0062] Step S24: if the difference is greater than the preset difference threshold, the unit pixel is taken as a feature pixel.

[0063] If the difference is less than or equal to the preset difference threshold, the unit pixel is not considered a feature pixel.

[0064] The image to be stitched is composed of multiple unit pixels.

[0065] The preset distance threshold is used to set the selection range of neighborhood pixel points; the neighborhood pixel point refers to the unit pixel point whose distance from the current unit pixel point is within the preset distance threshold; the specific value of the preset distance threshold can be set based on actual application needs; it should be noted that the distance referred to in this embodiment refers to the position distance of the unit pixel point, and it can be understood that based on the arrangement of the unit pixel points, the position of the unit pixel point can be represented by row coordinates and column coordinates; the calculation method of the distance between unit pixels can be selected based on actual needs, such as Euclidean distance.

[0066] In this embodiment, steps S21 to S24 are respectively executed for each unit pixel to determine whether it is a feature pixel.

[0067] The difference degree is used to indicate the degree of difference between a unit pixel and its corresponding neighborhood pixel. The larger the difference degree, the greater the difference degree between the unit pixel and the neighborhood pixel. The smaller the difference degree, the smaller the difference degree between the unit pixel and the neighborhood pixel. Based on the definition of a feature pixel, it can be seen that a feature pixel undergoes a mutation relative to its neighboring pixels. Therefore, the degree of difference between the feature pixel and the corresponding neighborhood pixel should be larger. In this embodiment, a preset difference threshold is set to determine the minimum degree of difference between the feature pixel. When the difference degree corresponding to the unit pixel is greater than the preset difference threshold, it is considered that the degree of difference between the unit pixel and the neighborhood pixel meets the characteristics of the feature pixel. Therefore, the unit pixel can be used as a feature pixel.

[0068] Furthermore, the step S22 comprises the steps of:

[0069] Step S221, sampling the neighborhood pixel points to obtain sampled pixel points;

[0070] In practical applications, if the difference is calculated based on all neighborhood pixels, the amount of calculation will be too large. Therefore, in this embodiment, the neighborhood pixels are sampled to obtain sampled pixels, and the difference is calculated based on the sampled pixels, thereby reducing the amount of calculation.

[0071] The sampling of the neighborhood pixel points to obtain the sampled pixel points comprises:

[0072] Constructing a unit coordinate system with the unit pixel point as the origin, and converting the coordinates corresponding to the neighborhood pixel point into the unit coordinate system;

[0073] Calculate the distance between each of the neighborhood pixel points and the unit pixel point, and set the distances whose coordinates are located in the third and fourth quadrants of the unit coordinate system to negative values;

[0074] Construct a probability density function based on the distance:

[0075]

[0076] Where l is the distance;

[0077] For each of the neighborhood pixels, calculating a sampling probability based on the probability density function;

[0078] The neighborhood pixel points are sampled based on the sampling probability to obtain the sampled pixel points.

[0079] The calculation formula for sampling probability is:

[0080]

[0081] Among them, Z is a constant, P i is the sampling probability of the i-th neighborhood pixel, l i is the distance between the i-th neighborhood pixel and the unit pixel.

[0082] It can be understood that the distances corresponding to the neighboring pixels in the first and second quadrants of the unit coordinate system are positive values, and the distances corresponding to the neighboring pixels in the third and fourth quadrants are negative values. Therefore, the distances corresponding to all the neighboring pixels corresponding to the unit pixel have a mean of 0 and a variance of σ. 2 Gaussian distribution, that is, l~N(0,σ 2 ), thus obtaining the aforementioned probability density function.

[0083] It can be seen from the sampling probability calculation formula that the sampling probability of the neighboring pixel points closer to the unit pixel point is greater, and the sampling probability of the neighboring pixel points farther away from the unit pixel point is smaller, that is, the sampling pixel points finally obtained by sampling are more distributed near the unit pixel point. It can be understood that the neighboring pixel points closer to the unit pixel point can better reflect the mutation of the unit pixel point, and the neighboring pixel points farther away from the unit pixel point are more likely to have a greater difference between the spatial distance and the unit pixel point, so the degree of representation of the mutation of the unit pixel point is lower. Therefore, in this embodiment, the sampling probability of the neighboring pixel points closer to the unit pixel point is set to be greater.

[0084] Step S222, for each of the sampling pixel points, calculating a pixel difference between the sampling pixel point and the unit pixel point;

[0085] Step S223, taking the sampling pixel points whose pixel difference is greater than a preset difference threshold as valid sampling points;

[0086] Step S224, calculating the ratio of the number of the effective sampling points to the number of the sampling pixels;

[0087] Step S225: taking the quantity ratio as the difference degree.

[0088] The sampling pixel points whose pixel difference is less than or equal to the preset difference threshold are invalid sampling points.

[0089] The pixel difference is used to indicate the difference in pixel value between the sampled pixel and the unit pixel; the specific type of the pixel value can be set based on the actual application scenario, such as grayscale value, color value corresponding to color mode, etc.

[0090] When the difference between the sampling pixel and the unit pixel is large, that is, the pixel difference is greater than the preset difference threshold, it is considered that the sampling pixel represents the mutation of the unit pixel and the sampling pixel is used as a valid sampling point. Otherwise, it represents the non-mutation of the unit pixel. Therefore, the proportion of valid sampling points to all sampling points, that is, the quantity ratio, can reflect the degree of difference between the unit pixel and the neighboring pixel. Therefore, the quantity ratio is used as the difference degree. When the difference degree is a quantity ratio, the preset difference threshold can be set based on actual application needs, such as 50%. The value of the preset difference threshold can be set based on actual application needs.

[0091] Furthermore, the step S24 includes the following steps:

[0092] Step S241, if the difference is greater than the preset difference threshold, the unit pixel is used as a pixel before suppression;

[0093] Step S242, setting a sliding window to traverse the image to be stitched, wherein the sliding window includes a plurality of the unit pixels;

[0094] Step S243: In each of the sliding windows, determine the pixel before suppression with the largest difference as the feature pixel.

[0095] It is understandable that when the number of feature pixels is too large, the amount of calculation required for subsequent matching is too large. Therefore, in this embodiment, a non-maximum suppression method is used to optimize the number of feature pixels.

[0096] The pixel before suppression is the feature pixel determined before non-maximum suppression is performed.

[0097] In this embodiment, the sliding window is a rectangular window, and the length, width, and sliding step of the sliding window can be set based on actual needs.

[0098] In one case, the sliding window takes the upper left corner of the image to be stitched as a starting point, and at the starting point or after each sliding, non-maximum suppression is performed on the suppressed front pixel points in the sliding window until the sliding window traverses the image to be stitched.

[0099] Non-maximum suppression can be specifically:

[0100] When the sliding window does not contain the suppressed front pixel, non-maximum suppression is not performed; when the sliding window contains only one suppressed front pixel, the suppressed front pixel is used as the feature pixel; when the sliding window contains multiple suppressed front pixels, the suppressed front pixel with the largest difference is used as the feature pixel.

[0101] It is understandable that in each sliding window, the number of determined feature pixels can be set based on needs;

[0102] In other embodiments, an elimination method may also be used to determine the characteristic pixel points, such as:

[0103] When the sliding window does not contain the suppressed front pixel, the sliding window is slid; when the sliding window contains only one suppressed front pixel, the sliding window is slid; when the sliding window contains multiple suppressed front pixels, the suppressed front pixels except the suppressed front pixel with the largest difference are removed from the suppressed front pixels. After the sliding window traverses the image to be spliced, the remaining suppressed front pixels are used as feature pixels.

[0104] It can be understood that in each sliding window, the number of pixels before suppression retained can be set based on needs;

[0105] This embodiment can accurately determine the characteristic pixel points.

[0106] Further, in a third embodiment of the panoramic image stitching method of the present invention proposed based on the first embodiment of the present invention, step S30 includes the steps of:

[0107] Step S31, determining a first characteristic pixel point in the front stitching image and a second characteristic pixel point in the rear stitching image;

[0108] Step S32, matching the first characteristic pixel points with the second characteristic pixel points to determine a first pixel point in the first characteristic pixel points and a second pixel point in the second characteristic pixel points, wherein the first pixel points correspond to the second pixel points one by one;

[0109] Step S33: taking the corresponding first pixel point and the second pixel point as the matching pixel point pair.

[0110] The front stitching image is the basic image in the adjacent images to be stitched, and the rear stitching image is the supplementary image in the adjacent images to be stitched; it can be understood that the image obtained after stitching the two images to be stitched is the front stitching image in the next stitching operation, and the corresponding adjacent image to be stitched is the rear stitching image.

[0111] The first pixel point and the second pixel point indicate the same environmental target; after splicing, the first pixel point and the second pixel point indicate the same pixel point.

[0112] The specific matching method can be selected based on actual needs. For example, in this embodiment, the first feature pixel point and the second feature pixel point are matched by the LK sparse optical flow algorithm to determine the corresponding first pixel point and the second pixel point, and then obtain a matching pixel point pair.

[0113] It can be understood that the first pixel point is included in the first characteristic pixel point set, and the second pixel point is included in the second characteristic pixel point set.

[0114] This embodiment can accurately determine matching pixel pairs.

[0115] Further, in a fourth embodiment of the panoramic image stitching method of the present invention proposed based on the first embodiment of the present invention, step S40 includes the steps of:

[0116] Step S41, determining a first coordinate corresponding to a first pixel and a second coordinate corresponding to a second pixel in the matching pixel pair, wherein the first coordinate is a coordinate of the first pixel in a previous stitched image, and the second coordinate is a coordinate of the second pixel in a subsequent stitched image;

[0117] Step S42, determining a pixel transformation matrix according to the first coordinate and the second coordinate;

[0118] Step S43: stitching the adjacent images to be stitched together according to the pixel transformation matrix.

[0119] The first coordinate of the first pixel is based on the previous stitched image, the second coordinate of the second pixel is based on the next stitched image, and the first pixel and the second pixel represent the same pixel after stitching. Therefore, the conversion relationship between the coordinates in the previous stitched image and the next stitched image can be determined by the corresponding multiple first coordinates and second coordinates. Specifically:

[0120] Q′=HP′

[0121] Where Q' is the coordinate of the pixel in the previous stitching image, and P' is the coordinate of the pixel in the subsequent stitching image. H is the pixel transformation matrix, specifically:

[0122]

[0123] Where H is a 3×3 homography matrix, (x qi ,y qi ) is the coordinate of the pixel in the previous spliced ​​image, (x pi ,y pi ) are the coordinates of the pixel in the post-stitched image;

[0124] Substituting the coordinates of the matching pixel pair into the above formula, the specific value of H can be calculated;

[0125] After the transformation matrix is ​​determined, the unit pixel points in the subsequent stitching image are converted to the previous stitching image based on the transformation matrix to complete the stitching operation; the specific stitching method can be adjusted based on actual application needs.

[0126] This embodiment can accurately realize the stitching of the images to be stitched.

[0127] Furthermore, in a fifth embodiment of the panoramic image stitching method of the present invention proposed based on the first embodiment of the present invention, the following steps are included after step S40:

[0128] Step S50, obtaining image information of the image to be stitched;

[0129] Step S60: Associating the image information with the target panoramic image.

[0130] The image information indicates the environment, equipment, parameters and other information corresponding to the image to be stitched when it was collected; the image information includes but is not limited to image EXIF ​​data, GPS positioning information and the like.

[0131] The image information of the images to be stitched is associated with the target panoramic image, so that the target panoramic image contains richer information and can be applied to more scenes.

[0132] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0134] The present application also provides a panoramic image stitching device for implementing the above-mentioned panoramic image stitching method, the panoramic image stitching device comprising:

[0135] A first acquisition module is used to acquire a plurality of images to be stitched and determine the positional relationship between the images to be stitched;

[0136] A first determining module, used to determine characteristic pixel points in the image to be stitched;

[0137] A first matching module, used for matching the characteristic pixel points of two adjacent images to be stitched, so as to determine a matching pixel point pair among the characteristic pixel points;

[0138] The first stitching module is used to stitch adjacent images to be stitched based on the matching pixel pairs to obtain a target panoramic image.

[0139] The initialization device of the visual inertial system matches the feature pixels in adjacent images to be stitched, so that the matching pixel pairs representing the same position in the adjacent images to be stitched can be determined, and then the images to be stitched are stitched based on the matching pixel pairs, so that the obtained panoramic image matches the environment, thereby improving the stitching accuracy.

[0140] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the first determination module in this embodiment can be used to execute step S20 in the embodiment of the present application, the first matching module in this embodiment can be used to execute step S30 in the embodiment of the present application, and the first splicing module in this embodiment can be used to execute step S40 in the embodiment of the present application.

[0141] Furthermore, the first determining module includes:

[0142] A first determining unit, configured to determine, for each unit pixel in the image to be stitched, a neighboring pixel whose distance from the unit pixel is less than a preset distance threshold;

[0143] A second determining unit, configured to determine a difference between the neighborhood pixel and the unit pixel;

[0144] A first judging unit, configured to judge whether the difference is greater than a preset difference threshold;

[0145] The first execution unit is configured to use the unit pixel as a feature pixel if the difference is greater than the preset difference threshold.

[0146] Further, the second determining unit includes:

[0147] A first sampling subunit, used for sampling the neighborhood pixel points to obtain sampling pixel points;

[0148] A first calculation subunit, configured to calculate, for each of the sampling pixel points, a pixel difference between the sampling pixel point and the unit pixel point;

[0149] A first execution subunit, configured to take the sampling pixel points whose pixel difference is greater than a preset difference threshold as valid sampling points;

[0150] A second calculation subunit, used to calculate the ratio of the number of the effective sampling points to the number of the sampling pixel points;

[0151] The second execution subunit is configured to use the quantity ratio as the difference degree.

[0152] Furthermore, the first execution unit includes:

[0153] A third execution subunit, configured to use the unit pixel as a pixel before suppression if the difference is greater than the preset difference threshold;

[0154] A first setting subunit is used to set a sliding window to traverse the image to be stitched, wherein the sliding window includes a plurality of the unit pixels;

[0155] The second determining subunit is used to determine, in each of the sliding windows, the pre-suppression pixel point with the largest difference as the feature pixel point.

[0156] Furthermore, the first matching module includes:

[0157] A third determining unit, used to determine a first characteristic pixel point in the front stitched image and a second characteristic pixel point in the rear stitched image;

[0158] A first matching unit is used to match the first characteristic pixel points with the second characteristic pixel points to determine a first pixel point in the first characteristic pixel points and a second pixel point in the second characteristic pixel points, wherein the first pixel points correspond to the second pixel points one by one;

[0159] The second execution unit is used to use the corresponding first pixel point and the second pixel point as the matching pixel point pair.

[0160] Furthermore, the first splicing module includes:

[0161] a fourth determining unit, configured to determine a first coordinate corresponding to a first pixel point and a second coordinate corresponding to a second pixel point in the matching pixel point pair, wherein the first coordinate is a coordinate of the first pixel point in a previous stitched image, and the second coordinate is a coordinate of the second pixel point in a subsequent stitched image;

[0162] a fifth determining unit, configured to determine a pixel transformation matrix according to the first coordinates and the second coordinates;

[0163] The first stitching unit is used to stitch the adjacent images to be stitched together according to the pixel transformation matrix.

[0164] Furthermore, the device also includes:

[0165] A second acquisition module, used to acquire image information of the image to be stitched;

[0166] The first associating module is used to associate the image information with the target panoramic image.

[0167] Reference Figure 2 In terms of hardware structure, the electronic device may include components such as a communication module 10, a memory 20, and a processor 30. In the electronic device, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.

[0168] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other electronic devices, servers or Internet of Things devices, such as televisions, etc.

[0169] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as obtaining multiple images to be stitched), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0170] The processor 30 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, so as to monitor the electronic device as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.

[0171] although Figure 2 Although not shown, the electronic device may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 2The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0172] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 2 The memory 20 in the electronic device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a television, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0173] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0174] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0175] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A panoramic image stitching method, characterized in that: The panoramic image stitching method comprises: Acquire a plurality of images to be stitched, and determine the positional relationship between the images to be stitched; Determining characteristic pixel points in the images to be stitched; For two adjacent images to be stitched, matching the characteristic pixel points to determine matching pixel point pairs among the characteristic pixel points; Adjacent images to be stitched are stitched based on the matching pixel pairs to obtain a target panoramic image.

2. The panoramic image stitching method according to claim 1, characterized in that: Determining characteristic pixel points in the image to be stitched includes: For each unit pixel in the image to be stitched, determine a neighboring pixel whose distance to the unit pixel is less than a preset distance threshold; Determine the difference between the neighborhood pixel and the unit pixel; Determining whether the difference is greater than a preset difference threshold; If the difference is greater than the preset difference threshold, the unit pixel is taken as a feature pixel.

3. The panoramic image stitching method according to claim 2, characterized in that: The determining the difference between the neighborhood pixel point and the unit pixel point comprises: Sampling the neighborhood pixel points to obtain sampled pixel points; For each of the sampling pixel points, calculating a pixel difference between the sampling pixel point and the unit pixel point; The sampling pixel points whose pixel difference is greater than a preset difference threshold are taken as valid sampling points; Calculating the ratio of the number of the effective sampling points to the number of the sampling pixel points; The quantitative ratio is taken as the degree of difference.

4. The panoramic image stitching method according to claim 2, characterized in that: If the difference is greater than the preset difference threshold, taking the unit pixel as a feature pixel includes: If the difference is greater than the preset difference threshold, the unit pixel is used as the pixel before suppression; Setting a sliding window to traverse the image to be stitched, wherein the sliding window includes a plurality of the unit pixels; In each of the sliding windows, the pixel point before suppression with the largest difference is determined as the feature pixel point.

5. The panoramic image stitching method according to claim 1, characterized in that: The matching the characteristic pixel points to determine matching pixel point pairs among the characteristic pixel points includes: Determine a first characteristic pixel point in the front stitched image and a second characteristic pixel point in the back stitched image; Matching the first characteristic pixel points with the second characteristic pixel points to determine a first pixel point in the first characteristic pixel points and a second pixel point in the second characteristic pixel points, wherein the first pixel points correspond to the second pixel points one by one; The corresponding first pixel point and the second pixel point are used as the matching pixel point pair.

6. The panoramic image stitching method according to claim 1, characterized in that: The respectively stitching of the adjacent images to be stitched based on the matching pixel pairs comprises: Determine a first coordinate corresponding to a first pixel point and a second coordinate corresponding to a second pixel point in the matching pixel point pair, wherein the first coordinate is a coordinate of the first pixel point in a previous stitched image, and the second coordinate is a coordinate of the second pixel point in a subsequent stitched image; determining a pixel transformation matrix according to the first coordinate and the second coordinate; The adjacent images to be stitched are stitched according to the pixel transformation matrix.

7. The panoramic image stitching method according to any one of claims 1 to 6, characterized in that: After obtaining the target panoramic image, it includes: Acquire image information of the image to be stitched; The image information is associated with the target panoramic image.

8. A panoramic image stitching device, characterized in that: The panoramic image stitching device comprises: A first acquisition module is used to acquire a plurality of images to be stitched and determine the positional relationship between the images to be stitched; A first determining module, used to determine characteristic pixel points in the image to be stitched; A first matching module, used for matching the characteristic pixel points of two adjacent images to be stitched, so as to determine a matching pixel point pair among the characteristic pixel points; The first stitching module is used to stitch adjacent images to be stitched based on the matching pixel pairs to obtain a target panoramic image.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the panoramic image stitching method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the panoramic image stitching method according to any one of claims 1 to 7 are implemented.

Images