Panoramic image processing method, system and image processing device based on smart computer room
By processing panoramic images in the smart computer room, identifying and correcting distortion areas, the problem of image distortion in the refraction reflection panoramic imaging system is solved, and the image quality is improved.
Patent Information
- Application Number
- CN202411303809.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-19
AI Technical Summary
The circular panoramic images obtained by the folding reflection panoramic imaging system are prone to deformation, resulting in distortion of the cylindrical panoramic image, affecting the image quality, and distortion correction is required to be suitable for the visual habits of the human eye.
Using a panoramic image processing method based on the smart computer room, the distortion area is determined by collecting the sub-images of the panoramic image and the reference image, the position and angle distribution relationship between feature points and adjacent points is analyzed, and the registration features are obtained by combining the flexible template matching algorithm. Point pairs, establish a distortion correction function, and correct pixel points in the distortion area.
It effectively avoids image matching errors, improves the accuracy of feature point matching and the accuracy of distortion area correction, and improves the quality of panoramic images.
Smart Images

Figure CN118822915B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of panoramic image processing, and in particular to a panoramic image processing method, system and image processing device based on a smart computer room. Background Art
[0002] In recent years, panoramic cameras have been widely used and received attention in robotics, visual surveillance and commercial fields. The catadioptric panoramic imaging system combines a reflector with a traditional camera to expand the camera's field of view, making the horizontal field of view 360°.
[0003] However, the circular panoramic images obtained by the catadioptric panoramic imaging system are prone to deformation and are not suitable for direct observation by the human eye. Therefore, it is necessary to transform the original catadioptric panoramic image into a cylindrical projection image that conforms to the visual habits of the human eye. However, in the process of stretching the spherical panoramic image to the cylindrical panoramic image, the cylindrical panoramic image still has distortion, which greatly affects and interferes with the application of the panoramic image. Therefore, the obtained cylindrical panoramic image needs to be corrected for distortion to improve the image quality. Summary of the invention
[0004] In order to solve the above technical problems, the purpose of this application is to provide a panoramic image processing method, system and image processing device based on a smart computer room. The technical solutions adopted are as follows:
[0005] In a first aspect, an embodiment of the present application provides a panoramic image processing method based on a smart computer room, the method comprising the following steps:
[0006] Collecting a panoramic image of the smart computer room, and using the standard panoramic image as a reference image;
[0007] Dividing the panoramic image and the reference image into sub-images respectively, selecting reference points in the sub-images, dividing the sub-images into rectangular areas, and determining the distortion area in each sub-image of the panoramic image based on the distance relationship between the vertices of each rectangular area and the corresponding reference points;
[0008] Based on the distorted area and the feature points in the corresponding rectangular area in the reference image; analyzing the position distribution relationship and the angle distribution relationship between the feature point and its neighboring points, determining the position measurement factor and the angle measurement factor of the feature point and its neighboring points; forming a feature point pair with any feature point in the distorted area and any feature point in the corresponding rectangular area, analyzing the difference in the position measurement factor and the difference in the angle measurement factor between the feature point pairs, and determining the degree of local position difference between the feature point pairs;
[0009] Analyze the differences in the changing trends of the corresponding coordinates of the edge pixel points in the neighborhood between the feature point pairs to determine the degree of local change difference between the feature point pairs; combine the degree of local position difference with the degree of local change difference, and the flexible template matching algorithm to obtain the registered feature point pairs in the distorted area and its corresponding rectangular area in the reference image; based on the mapping relationship between the registered feature point pairs, determine the distortion correction function of the distorted area, and correct each pixel point in the distorted area.
[0010] In one embodiment, determining the reference point includes:
[0011] Line detection is performed on the sub-images of the panoramic image, and the pixel points that appear most frequently on different straight lines are used as reference points. If there are multiple pixel points that appear most frequently, one of them is selected as the reference point.
[0012] In one embodiment, determining the distortion area includes:
[0013] Calculate the metric distance between each vertex of any rectangular area in any sub-image of the panoramic image and the reference point of the any sub-image, record it as the first distance, record the sub-image corresponding to the any sub-image in the reference image as the reference sub-image, take the pixel point at the same position of the reference point of the any sub-image in the reference sub-image as the corresponding reference point, calculate the metric distance between each vertex of the rectangular area at the same position of the any rectangular area in the reference sub-image and the corresponding reference point, record it as the second distance;
[0014] The difference between the first distance and the second distance is calculated, and the ratio of the difference to the second distance is used as the local distortion value of each vertex of any rectangular area. If the minimum value of the local distortion values of all vertices of any rectangular area is greater than or equal to a preset threshold, the any rectangular area is determined to be a distorted area.
[0015] In one embodiment, the position measurement factor and the angle measurement factor are determined as follows:
[0016] A feature point extraction algorithm is used to extract feature points from the distorted area and its corresponding rectangular area in the sub-image of the reference image, and the position measurement factor of each feature point and its neighboring points is determined according to the measurement distance between each feature point and its neighboring points.
[0017] The angle measurement factor is the slope angle between each feature point and its adjacent points.
[0018] In one embodiment, the local position difference degree is calculated as follows: ; Represents a pair of feature points ( , ), Indicates the number of neighboring points of the feature point. Feature points representing the distorted area The position metric factor of its mth neighbor point, For feature points The position metric factor of its mth neighbor point, is the feature point of the distorted area The result of the angle measure factor of its mth neighbor in radians, For feature points The result in radians of the angle measure factor to its mth neighbor.
[0019] In one embodiment, the determination of the degree of difference of the local changes includes:
[0020] Obtain each edge pixel point within a preset neighborhood range of each feature point in the distorted area and its corresponding rectangular area in a sub-image of the reference image, perform curve fitting on all the edge pixel points of each feature point, calculate the curvature of each edge pixel point, calculate the extreme value and the discrete degree of all the curvatures of each feature point, and fuse the difference in the extreme value and the difference in the discrete degree of each feature point in the distorted area and each feature point in the corresponding rectangular area as the degree of local change difference.
[0021] In one embodiment, the determination of each pair of registration feature points includes:
[0022] The product of the local position difference degree and the local change difference degree of the feature point pair is used as the weight of the decision function in the BBS flexible matching algorithm, and the BBS flexible matching algorithm is used to obtain the above-mentioned registration feature point pairs.
[0023] In one embodiment, the determining of the distortion correction function of the distorted area to correct each pixel in the distorted area includes:
[0024] Taking the two x-coordinates between each pair of registration feature points as a coordinate point, performing curve fitting on the coordinate points of all the pairs of registration feature points, taking the obtained fitting function as the distortion correction function of the x-coordinate, and using the same calculation method as the distortion correction function of the x-coordinate to obtain the distortion correction function of the y-coordinate;
[0025] The x-coordinate of each pixel in the distorted area is used as the independent variable of the distortion correction function of the x-coordinate to obtain the corrected x-coordinate of each pixel in the distorted area, and the corrected y-coordinate of each pixel in the distorted area is obtained by using the same calculation method as the corrected x-coordinate.
[0026] In a second aspect, the embodiment of the present application provides a panoramic image processing system based on a smart computer room, including:
[0027] Image acquisition module: used to acquire a panoramic image of the smart computer room and use the standard panoramic image as a reference image;
[0028] Distortion region determination module: used to divide the panoramic image and the reference image into sub-images respectively, select reference points in the sub-images, divide the sub-images into rectangular regions, and determine the distortion region in each sub-image of the panoramic image based on the distance relationship between the vertices of each rectangular region and the corresponding reference points;
[0029] Distortion feature analysis module: based on the distorted area and the feature points in the corresponding rectangular area in the reference image; analyzing the position distribution relationship and angle distribution relationship between the feature points and their neighboring points, determining the position measurement factor and angle measurement factor between the feature points and their neighboring points; forming a feature point pair with any feature point in the distorted area and any feature point in the corresponding rectangular area, analyzing the difference in the position measurement factor and the difference in the angle measurement factor between the feature point pairs, and determining the degree of local position difference between the feature point pairs;
[0030] Distorted area correction module: used to analyze the difference in the change trend of the corresponding coordinates of the edge pixel points in the neighborhood between the feature point pairs, and determine the degree of local change difference between the feature point pairs; combine the local position difference degree with the local change difference degree, and the flexible template matching algorithm to obtain the distorted area and each registered feature point pair in the corresponding rectangular area in the reference image; based on the mapping relationship between the registered feature point pairs, determine the distortion correction function of the distorted area, and correct each pixel point in the distorted area.
[0031] In a third aspect, an embodiment of the present application also provides a panoramic image processing device based on a smart computer room, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.
[0032] This application has at least the following beneficial effects:
[0033] The present application collects a panoramic image of a smart computer room, takes the standard panoramic image as a reference image, divides the panoramic image and the reference image into sub-images, selects reference points in the sub-images, divides the sub-images into rectangular areas, and determines the distorted areas in each sub-image of the panoramic image based on the distance relationship between the vertices of each rectangular area and the corresponding reference points. The beneficial effect is that the distorted image is corrected in a targeted manner according to the distorted position of the panoramic image, thereby reducing the complexity of image processing and improving the efficiency of image processing. Furthermore, based on the distorted area and its feature points in the corresponding rectangular area in the reference image, the position distribution relationship and angle distribution relationship between the feature point and its neighboring points are analyzed to determine the position measurement factor and angle measurement factor of the feature point. Any feature point in the distorted area and any feature point in the corresponding rectangular area are combined into a feature point pair, and the differences in the position measurement factors and angle measurement factors between the feature point pairs are analyzed to determine the distorted area. The degree of local position difference between feature point pairs; reflects the degree of position difference and angle difference between the distorted area and the reference image, and reflects the degree of distortion of the distorted area; analyzes the difference in the change trend of the corresponding coordinates of the edge pixel points in the neighborhood between the feature point pairs, and determines the degree of local change difference between the feature point pairs; combines the local position difference degree with the local change difference degree, and the flexible template matching algorithm to obtain the registered feature point pairs in the distorted area and its corresponding rectangular area in the reference image; based on the mapping relationship between the registered feature point pairs, determines the distortion correction function of the distorted area, and corrects each pixel point in the distorted area; The beneficial effect is to avoid the problem of large image matching error and low correction accuracy of the distorted area caused by inaccurate feature point matching during the image matching process, and improves the accuracy of feature point matching by analyzing the corresponding relationship between the distorted area and the reference image, thereby improving the accuracy of distortion area correction and the quality of the panoramic image. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0035] Figure 1 A flowchart of a panoramic image processing method based on a smart computer room provided in one embodiment of the present application;
[0036] Figure 2 Construct a flow graph for the distortion correction function. DETAILED DESCRIPTION
[0037] In order to further explain the technical means and effects adopted by this application to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, describes in detail the specific implementation methods, structures, features and effects of the panoramic image processing method, system and image processing device based on the smart computer room proposed in this application. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0038] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
[0039] The following is a detailed description of the specific solutions of the panoramic image processing method, system and image processing device based on the smart computer room provided by the present application in conjunction with the accompanying drawings.
[0040] See also Figure 1 , which shows a flowchart of a panoramic image processing method based on a smart computer room provided by an embodiment of the present application, the method comprising the following steps:
[0041] S1, collecting a panoramic image of the smart computer room, and using the standard panoramic image as a reference image.
[0042] Catadioptric panoramic imaging uses a curved mirror to reflect light from objects within a 360° horizontal range around it to the imaging sensor, and obtains the scene within the horizontal all-round field of view through a one-time shot. The conventional catadioptric panoramic imaging system mainly consists of three parts: a lens, a mirror and a photosensitive element.
[0043] This embodiment selects a refractive panoramic camera with a wide imaging range, low manufacturing cost, and good technical scalability to obtain all-round high-definition images in the smart computer room. Adjust the camera parameters according to the ambient light conditions of the smart computer room. The camera parameters include but are not limited to exposure and white balance. At the same time, set a suitable shooting height and shooting angle to avoid unnecessary occlusion or blind spots in the image. And ensure that the light in the smart computer room is uniform, without strong light directly hitting the lens to reduce reflections and shadows, turn off the device indicator lights that may affect the image quality, and perform panoramic shooting according to the preset parameters. Ensure that the camera is stable to avoid shaking during shooting. The image obtained by the refractive panoramic camera is an RGB circular ring panoramic image.
[0044] The cylindrical expansion method performs reverse mapping based on the mirror parameters and optical principles of catadioptric panoramic imaging, and has the advantages of good visual effect of the expanded image and small distortion, and is suitable for occasions with high requirements for the visual effect of the expanded image. In this embodiment, the cylindrical expansion method is selected to transform the RGB annular panoramic image into a cylindrical panoramic image that conforms to the visual habits of the human eye.
[0045] At the same time, a rotating central panoramic camera is used to obtain all-round, high-quality, distortion-free images of the smart computer room as reference images at the same position and under the same conditions.
[0046] S2, dividing the panoramic image and the reference image into sub-images respectively, selecting reference points in the sub-images, dividing the sub-images into rectangular areas, and determining the distortion areas in each sub-image of the panoramic image based on the distance relationship between the vertices of each rectangular area and the corresponding reference points.
[0047] The panoramic image obtained by the cylindrical expansion method may still have some distortion at the edge, and it is necessary to identify the distorted area to obtain the distorted area in the panoramic image.
[0048] The viewing angle of the existing standard lens is similar to that of the human eye. The perspective relationship between the scenes in the captured picture is very similar to the scene perceived by the human eye, without perspective distortion. It is the most basic type of photographic lens among all lenses, with a viewing angle range of 40°~53°. However, the panoramic image has the characteristic of horizontal 360° imaging. Therefore, in order to facilitate the subsequent identification of the distorted area, this embodiment divides the cylindrical panoramic image into 8 sub-images in the vertical direction to reduce the complexity of image processing. The horizontal viewing angle range of each sub-image is 45°, where the number of sub-image divisions can be set by the implementer according to the actual situation, and this application does not impose any restrictions on this.
[0049] At this point, the collected cylindrical panoramic image and the reference image can be divided into 8 sub-images respectively.
[0050] In the cylindrical projection image, the pixels that still maintain a straight line shape are most likely to be pixels that have not been distorted. Therefore, the reference points in the image can be obtained by using a straight line detection method on the image.
[0051] Specifically: This embodiment uses the LSD straight line detection algorithm to perform straight line detection on each sub-image of the cylindrical panoramic image. Based on each sub-image, the pixel point that appears most frequently on different straight lines is used as the reference point of the corresponding sub-image. If there are multiple pixel points that appear most frequently, one of them is selected as the reference point.
[0052] Taking any sub-image A in the cylindrical panoramic image as an example, for the reference point W of sub-image A in the cylindrical panoramic image, obtain the sub-image A1 corresponding to sub-image A in the reference image, recorded as the reference sub-image, and obtain the pixel point W1 at the same position as the reference point W in sub-image A1 as the reference point of sub-image A1, recorded as the corresponding reference point.
[0053] Further, a rectangular grid is constructed in each sub-image, including a sub-image of the cylindrical panoramic image and a sub-image of the reference image. Specifically, since the vertical viewing angle of the catadioptric panoramic image can reach up to 180° at most, and the viewing angle range of the segmented sub-image is 45°, in order to reduce the workload to a greater extent while ensuring accurate identification of the distorted area. Vertical and horizontal lines are constructed in the sub-image according to the viewing angle range of the image. The spacing between vertical lines is 9°, and the spacing between horizontal lines is 10°. The implementer can determine the spacing between vertical lines and the spacing between horizontal lines according to the actual situation, and this application does not impose any restrictions on this.
[0054] Each rectangular grid is recorded as each rectangular area. If the rectangular area in the sub-image of the cylindrical panoramic image is distorted, the vertex position of the rectangular area in the sub-image of the cylindrical panoramic image will change compared with the corresponding rectangular area in the reference image, and may shrink inward or expand outward. Taking the rectangular area B in the sub-image A of the cylindrical panoramic image as an example, the Euclidean distance between each vertex of the rectangular area B and the reference point in the sub-image A is calculated, which is recorded as the first distance , calculate the Euclidean distance between each vertex of the rectangular area B1 at the same position in the sub-image A1 and the corresponding reference point W1, recorded as the second distance .
[0055] Calculate the first distance The second distance The difference between the second distance The ratio of is used as the local distortion value of each vertex of the rectangular area B.
[0056] It should be understood that the difference represents the degree of difference between two variables, and can be calculated specifically by using the absolute value of the difference, a ratio, etc. This embodiment uses the absolute value of the difference as the calculation method for the difference.
[0057] For the local distortion values of all vertices of the rectangular area B, the larger the local distortion value, the greater the degree to which the vertices of the rectangular area B shrink inward or expand outward, and the greater the possibility of distortion in the rectangular area B. Therefore, if the minimum value of the local distortion values of all vertices of the rectangular area B is greater than or equal to the preset threshold, the rectangular area B is determined to be a distorted area, otherwise, the rectangular area B is determined to be a normal area.
[0058] In this embodiment, the preset threshold is artificially set to 0.03, and the implementer can set it according to the actual situation, and this application does not impose any restrictions on this.
[0059] The same determination method as that of the rectangular area B is adopted to determine all the rectangular areas in all the sub-images of the cylindrical panoramic image, thereby obtaining the distorted areas in the sub-images of the cylindrical panoramic image.
[0060] S3, based on the distorted area and the feature points in the corresponding rectangular area in the reference image; analyzing the position distribution relationship and the angle distribution relationship between the feature point and its neighboring points, and determining the position measurement factor and the angle measurement factor of the feature point; forming a feature point pair with any feature point in the distorted area and any feature point in the corresponding rectangular area, analyzing the difference in the position measurement factor and the difference in the angle measurement factor between the feature point pairs, and determining the degree of local position difference between the feature point pairs.
[0061] After obtaining the distorted region in each sub-image, this embodiment only needs to perform correction processing on the distorted region, thereby reducing the complexity of image correction and improving the efficiency of image correction.
[0062] In the process of correcting the distorted area, it is necessary to obtain the feature point pairs of the distorted area and its corresponding rectangular area in the reference sub-image, match the feature point pairs, and complete the correction of the distorted area.
[0063] When using the BBS (Best-Buddies-Similarity) matching algorithm for image matching, due to the deformation error of the distorted image, the feature point pairs obtained by using the Euclidean distance alone have large errors and are small in number. In the distorted area of the panoramic image of the smart computer room, when searching for feature point pairs, the point with the smallest distance is judged to be the corresponding point. However, in the actual process, due to the influence of image distortion, the Euclidean distance of the corresponding point may not be the minimum value, so it is easy to produce wrong feature point pairs. In addition, when the image is distorted, the originally corresponding feature points cannot be matched due to image distortion. This application mainly considers the impact of image distortion on feature point registration. Its main drawback is that the selection of the best feature point pairs uses the minimum value of the Euclidean distance as the standard, which has low accuracy.
[0064] Therefore, this embodiment takes the distorted area C in the sub-image A of the cylindrical panoramic image and the rectangular area C1 at the same position of the distorted area C in the sub-image A1 as examples, and uses the SIFT (Scale Invariant Feature Transform) algorithm to detect the distorted area C and the rectangular area C1 respectively, obtains each feature point of the distorted area C and each feature point of the rectangular area C1, and records all feature points of the distorted area C as a feature point set P, and records all feature points of the rectangular area C1 as a feature point set Q. Among them, the SIFT algorithm is an existing well-known technology, and this application will not be described in detail here.
[0065] The KD-Tree algorithm is used to perform k-nearest neighbor search to obtain the neighboring points of each feature point of the distorted area C and the neighboring points of each feature point of the rectangular area C1. In this embodiment, k=8, and the implementer can set it according to the actual situation, and this application does not limit this. The KD-Tree algorithm is an existing well-known technology, and this application will not be described in detail here, and the implementer can choose other neighboring point selection algorithms according to the actual situation, and this application does not limit this.
[0066] Randomly select a feature point from the feature point set P , randomly select a feature point from the feature point set Q , forming a pair of feature points ( , ), if the feature point and is a pair of registration feature points, i.e., feature points With feature points For the corresponding relationship, then the feature point The surrounding k neighboring points and feature points The surrounding k neighboring points can also correspond to each other, and the positional relationship should be consistent.
[0067] Based on the above analysis, the position measurement factor and angle measurement factor of each feature point and its neighboring points are calculated. The specific calculation method is:
[0068] ; In the formula, is the position measurement factor between the i-th feature point and its m-th neighboring point, is the horizontal coordinate of the i-th feature point, is the ordinate of the i-th feature point, is the horizontal coordinate of the mth neighboring point of the ith feature point, is the ordinate of the mth neighboring point of the ith feature point.
[0069] ; is the angle measurement factor between the i-th feature point and its m-th neighboring point, arctan() is the inverse trigonometric tangent function.
[0070] Combine the position measurement factor and angle measurement factor of each feature point and its neighboring points to calculate the degree of local position difference between feature point pairs. The specific calculation method is:
[0071] ; Represents a pair of feature points ( , ), Indicates the number of neighboring points of the feature point. Feature points representing the distorted area The position metric factor of its mth neighbor point, For feature points The position metric factor of its mth neighbor point, is the feature point of the distorted area The result of the angle measure factor of its mth neighbor in radians, For feature points The result in radians of the angle measure factor to its mth neighbor.
[0072] When the feature points in the distorted area When distortion occurs, the feature points calculated are The position measurement factor and angle measurement factor of its k neighboring points will also change; that is, the relative distance difference and relative angle difference between the feature point in the distorted area and the corresponding feature point in the reference image will be larger; the feature point pair finally calculated ( , ) will have a greater difference in local position, indicating that the feature point Affected by image distortion, there is a large deviation between the position information and the actual information.
[0073] S4, analyzing the difference in the change trend of the corresponding coordinates of the edge pixel points in the neighborhood between the feature point pairs, and determining the degree of local change difference between the feature point pairs; combining the local position difference degree with the local change difference degree, and the flexible template matching algorithm, obtaining the registered feature point pairs in the distorted area and its corresponding rectangular area in the reference image; based on the mapping relationship between the registered feature point pairs, determining the distortion correction function of the distorted area, and correcting each pixel point in the distorted area.
[0074] In addition, the edge contour in the distorted area is also prone to large deformation. With each feature point as the center, a neighborhood range of 9*9 is constructed. The Canny edge detection algorithm is used to obtain the edge pixels in the neighborhood range of each feature point. The least squares method is used to perform curve fitting on the position coordinates of all edge pixels in the neighborhood range in the image, and the curvature of each edge pixel on the fitting curve is calculated.
[0075] It should be noted that the implementer can set the size of the neighborhood range according to actual conditions, and this application does not impose any restrictions on this. The Canny edge detection algorithm and the least squares method are both existing well-known technologies, and this embodiment will not be described in detail here. The implementer can choose other edge detection algorithms or curve fitting algorithms according to actual conditions, and this application does not impose any restrictions on this.
[0076] Calculate the extreme value and discrete degree of all the curvatures in the neighborhood of each feature point, and take the feature point pair ( , ) as an example, calculate the feature point pair ( , ) With feature points The difference of the extreme value is recorded as the first difference, and the characteristic point With feature points The difference in the degree of discreteness is recorded as the second difference, and the fusion result of the first difference and the second difference is used as the feature point pair ( , ) between the local changes.
[0077] It should be understood that the degree of discreteness can be specifically calculated by means of variance, standard deviation, information entropy, etc. The present embodiment uses variance as the method for calculating the degree of discreteness. In another embodiment, variance can be replaced by standard deviation as the method for calculating the degree of discreteness. In addition, fusion means combining multiple variables in a manner to enhance the overall effect. Specifically, multiplication, addition, or a mixture of addition and multiplication can be used for calculation. The present embodiment uses multiplication as the method for calculating fusion. In another embodiment, multiplication can be replaced by addition as the method for calculating fusion.
[0078] When the feature points in the distorted area When distortion occurs, the larger the value of the calculated second difference is, the larger the feature point The range of the curvature of the reference image will also change, that is, the value of the first difference will also be larger, so that the feature point pair finally calculated ( , ) has a greater difference in local changes, indicating that the feature points The greater the impact of image distortion.
[0079] Using the feature point pair ( , ), the local position difference degree and local change difference degree between each distorted area and any feature point pair in the corresponding rectangular area in the reference image are obtained.
[0080] In combination with the local position difference degree and the local change difference degree, the weight of the decision function in the BBS matching algorithm is adjusted. The specific adjustment method is:
[0081] ; In the formula, P represents the feature point set of the distorted area, Q represents the feature point set in the rectangular area corresponding to the distorted area in the reference image, BBS() represents the similarity measurement result, min{} represents the minimum value function, M represents the number of feature points in the feature point set Q, and N represents the number of feature points in the feature point set P. Represents a pair of feature points ( , ), Feature point pairs ( , ), bb() represents the decision function in the BBS matching algorithm.
[0082] The decision function and the BBS matching algorithm in the BBS matching algorithm are both existing well-known technologies, and will not be described in detail in this embodiment.
[0083] The output of the BBS matching algorithm is the distorted area and its corresponding rectangular area in the reference image.
[0084] To register feature point pairs For example, The position coordinates in the distortion area are (x1, y1), The position coordinates in the reference image are (x2, y2). Taking the x coordinate as an example, x1 is used as the horizontal coordinate and x2 as the vertical coordinate to obtain the mapping relationship from x1 to x2, that is, (x1, x2) is used as a coordinate point, and the coordinate points of all the registered feature point pairs in the distortion area are curve fitted using the least squares method, and the obtained fitting function is used as the distortion correction function of the x coordinate.
[0085] Similarly, the distortion correction function of the y coordinate is obtained by the same calculation method as the distortion correction function of the x coordinate. The distortion correction function construction flow chart is as follows: Figure 2 shown.
[0086] The x-coordinate of each pixel in the distorted area is used as the independent variable of the distortion correction function of the x-coordinate to obtain the corrected x-coordinate of each pixel in the distorted area. The corrected y-coordinate of each pixel in the distorted area is obtained by using the same calculation method as the corrected x-coordinate to complete the position correction of all pixels in the distorted area.
[0087] The panoramic image processing system based on the smart computer room includes:
[0088] Image acquisition module: used to acquire a panoramic image of the smart computer room and use the standard panoramic image as a reference image;
[0089] Distortion region determination module: used to divide the panoramic image and the reference image into sub-images respectively, select reference points in the sub-images, divide the sub-images into rectangular regions, and determine the distortion region in each sub-image of the panoramic image based on the distance relationship between the vertices of each rectangular region and the corresponding reference points;
[0090] Distortion feature analysis module: based on the distorted area and the feature points in the corresponding rectangular area in the reference image; analyzing the position distribution relationship and angle distribution relationship between the feature points and their neighboring points, determining the position measurement factor and angle measurement factor between the feature points and their neighboring points; forming a feature point pair with any feature point in the distorted area and any feature point in the corresponding rectangular area, analyzing the difference in the position measurement factor and the difference in the angle measurement factor between the feature point pairs, and determining the degree of local position difference between the feature point pairs;
[0091] Distorted area correction module: used to analyze the difference in the change trend of the corresponding coordinates of the edge pixel points in the neighborhood between the feature point pairs, and determine the degree of local change difference between the feature point pairs; combine the local position difference degree with the local change difference degree, and the flexible template matching algorithm to obtain the distorted area and each registered feature point pair in the corresponding rectangular area in the reference image; based on the mapping relationship between the registered feature point pairs, determine the distortion correction function of the distorted area, and correct each pixel point in the distorted area.
[0092] Based on the same inventive concept as the above method, an embodiment of the present application also provides a panoramic image processing device based on a smart computer room, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned panoramic image processing methods based on a smart computer room are implemented.
[0093] It should be noted that the above sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0094] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0095] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application should be included in the protection scope of the present application.
Claims
1. A panoramic image processing method based on a smart computer room, characterized in that: The method comprises the following steps: Collecting a panoramic image of the smart computer room, and using the standard panoramic image as a reference image; Dividing the panoramic image and the reference image into sub-images respectively, selecting reference points in the sub-images, dividing the sub-images into rectangular areas, and determining the distortion area in each sub-image of the panoramic image based on the distance relationship between the vertices of each rectangular area and the corresponding reference points; Based on the distorted area and the feature points in the corresponding rectangular area in the reference image; analyzing the position distribution relationship and the angle distribution relationship between the feature point and its neighboring points, determining the position measurement factor and the angle measurement factor of the feature point and its neighboring points; forming a feature point pair with any feature point in the distorted area and any feature point in the corresponding rectangular area, analyzing the difference in the position measurement factor and the difference in the angle measurement factor between the feature point pairs, and determining the degree of local position difference between the feature point pairs; Analyze the differences in the changing trends of the corresponding coordinates of the edge pixel points in the neighborhood between the feature point pairs to determine the degree of local change difference between the feature point pairs; combine the degree of local position difference with the degree of local change difference, and the flexible template matching algorithm to obtain the registered feature point pairs in the distorted area and its corresponding rectangular area in the reference image; based on the mapping relationship between the registered feature point pairs, determine the distortion correction function of the distorted area, and correct each pixel point in the distorted area.
2. The panoramic image processing method based on the smart computer room according to claim 1, characterized in that: Determination of the reference point includes: Line detection is performed on the sub-images of the panoramic image, and the pixel points that appear most frequently on different straight lines are used as reference points. If there are multiple pixel points that appear most frequently, one of them is selected as the reference point.
3. The panoramic image processing method based on the smart computer room according to claim 1, characterized in that: The determination of the distortion area includes: Calculate the metric distance between each vertex of any rectangular area in any sub-image of the panoramic image and the reference point of the any sub-image, record it as the first distance, record the sub-image corresponding to the any sub-image in the reference image as the reference sub-image, take the pixel point at the same position of the reference point of the any sub-image in the reference sub-image as the corresponding reference point, calculate the metric distance between each vertex of the rectangular area at the same position of the any rectangular area in the reference sub-image and the corresponding reference point, record it as the second distance; The difference between the first distance and the second distance is calculated, and the ratio of the difference to the second distance is used as the local distortion value of each vertex of any rectangular area. If the minimum value of the local distortion values of all vertices of any rectangular area is greater than or equal to a preset threshold, the any rectangular area is determined to be a distorted area.
4. The panoramic image processing method based on the smart computer room according to claim 1, characterized in that: The position measurement factor and angle measurement factor are determined as follows: A feature point extraction algorithm is used to extract feature points from the distorted area and its corresponding rectangular area in the sub-image of the reference image, and the position measurement factor of each feature point and its neighboring points is determined according to the measurement distance between each feature point and its neighboring points. The angle measurement factor is the slope angle between each feature point and its adjacent points.
5. The panoramic image processing method based on the smart computer room according to claim 1, characterized in that: The local position difference degree is calculated as follows: ; Represents a pair of feature points ( , ), Indicates the number of neighboring points of the feature point. Feature points representing the distorted area The position metric factor of its mth neighbor point, For feature points The position metric factor of its mth neighbor point, is the feature point of the distorted area The result of the angle measure factor of its mth neighbor in radians, For feature points The result in radians of the angle measure factor to its mth neighbor.
6. The panoramic image processing method based on the smart computer room according to claim 1, characterized in that: The determination of the difference degree of the local changes includes: Obtain each edge pixel point within a preset neighborhood range of each feature point in the distorted area and its corresponding rectangular area in a sub-image of the reference image, perform curve fitting on all the edge pixel points of each feature point, calculate the curvature of each edge pixel point, calculate the extreme value and the discrete degree of all the curvatures of each feature point, and fuse the difference in the extreme value and the difference in the discrete degree of each feature point in the distorted area and each feature point in the corresponding rectangular area as the degree of local change difference.
7. The panoramic image processing method based on the smart computer room according to claim 1, characterized in that: The determination of each pair of registration feature points comprises: The product of the local position difference degree and the local change difference degree of the feature point pair is used as the weight of the decision function in the BBS flexible matching algorithm, and the BBS flexible matching algorithm is used to obtain the above-mentioned registration feature point pairs.
8. The panoramic image processing method based on the smart computer room according to claim 1, characterized in that: The distortion correction function of the distorted area is determined, and each pixel point in the distorted area is corrected, including: Taking the two x-coordinates between each pair of registration feature points as a coordinate point, performing curve fitting on the coordinate points of all the pairs of registration feature points, taking the obtained fitting function as the distortion correction function of the x-coordinate, and using the same calculation method as the distortion correction function of the x-coordinate to obtain the distortion correction function of the y-coordinate; The x-coordinate of each pixel in the distorted area is used as the independent variable of the distortion correction function of the x-coordinate to obtain the corrected x-coordinate of each pixel in the distorted area, and the corrected y-coordinate of each pixel in the distorted area is obtained by using the same calculation method as the corrected x-coordinate.
9. Panoramic image processing system based on smart computer room, including: Image acquisition module: used to acquire a panoramic image of the smart computer room and use the standard panoramic image as a reference image; Distortion region determination module: used to divide the panoramic image and the reference image into sub-images respectively, select reference points in the sub-images, divide the sub-images into rectangular regions, and determine the distortion region in each sub-image of the panoramic image based on the distance relationship between the vertices of each rectangular region and the corresponding reference points; Distortion feature analysis module: based on the distorted area and the feature points in the corresponding rectangular area in the reference image; analyzing the position distribution relationship and angle distribution relationship between the feature points and their neighboring points, determining the position measurement factor and angle measurement factor between the feature points and their neighboring points; forming a feature point pair with any feature point in the distorted area and any feature point in the corresponding rectangular area, analyzing the difference in the position measurement factor and the difference in the angle measurement factor between the feature point pairs, and determining the degree of local position difference between the feature point pairs; Distorted area correction module: used to analyze the difference in the change trend of the corresponding coordinates of the edge pixel points in the neighborhood between the feature point pairs, and determine the degree of local change difference between the feature point pairs; combine the local position difference degree with the local change difference degree, and the flexible template matching algorithm to obtain the distorted area and each registered feature point pair in the corresponding rectangular area in the reference image; based on the mapping relationship between the registered feature point pairs, determine the distortion correction function of the distorted area, and correct each pixel point in the distorted area.
10. A panoramic image processing device based on a smart computer room, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Patent Citations
Real-time electronic image stabilization method for video image under random jitter interference
CN111539872A
Three-dimensional position calibration method, device and equipment for continuous crystal gamma detector
CN111736203A
