Histogram-based augmented reality registration method, system and device and medium
By performing multi-channel equalization and adaptive dynamic threshold calculation on real images, combined with probability Hough transform detection, the problem of augmented reality registration method being sensitive to light changes is solved, and registration accuracy and smoothness are improved.
Patent Information
- Application Number
- CN202411212802.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-31
AI Technical Summary
Existing augmented reality registration methods are sensitive to light changes, resulting in a reduced registration accuracy.
Using a histogram-based method, multi-channel equalization of the real image, adaptive dynamic thresholds are calculated, edges are extracted and probability Hough transform detection is performed, initial line segments are filtered to calculate camera external parameters, and virtual objects are registered with target objects.
Improve registration accuracy, reduce the impact of lighting on registration results, reduce the repeated segment statistics caused by multiple edge points, and obtain smoother segments.
Smart Images
Figure CN119228654A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of augmented reality registration technology, and in particular to a histogram-based augmented reality registration method, system, device and medium. Background Art
[0002] Augmented reality (AR) is a technology that overlays virtual information onto the real world. AR systems typically use cameras to track the real world and overlay virtual information onto the camera view. In order to properly align virtual information with the real world, a registration process is required. The registration process is a key step in aligning virtual models with the real world.
[0003] Factors such as real-world lighting intensity, lighting type and angle will cause drastic changes in image values. Existing registration methods are sensitive to changes in lighting, which will affect the registration results and reduce registration accuracy. Summary of the invention
[0004] In order to solve one of the above problems, an object of the present invention is to provide a histogram-based augmented reality registration method, system, device and medium, which can improve registration accuracy.
[0005] In one aspect, the present invention provides a histogram-based augmented reality registration method, system, device, and medium, comprising the following steps:
[0006] Acquire a real image of the real world, and perform a plurality of channel histogram equalization on the real image to obtain an equalized image;
[0007] Calculating an adaptive dynamic threshold according to the equalized image, and binarizing the equalized image according to the adaptive dynamic threshold to obtain a binarized image;
[0008] Extracting the edge of the binary image, obtaining a voting matrix according to a Gaussian function, and performing a probabilistic Hough transform detection on the edge according to the voting matrix to obtain an initial line segment;
[0009] The initial line segments are screened to obtain a bounding box of a target object, a camera extrinsic parameter is calculated based on the bounding box, and a virtual object is registered to the target object based on the camera extrinsic parameter; the target object is located in the real image.
[0010] Optionally, performing a plurality of channel histogram equalization on the real image to obtain an equalized image specifically includes:
[0011] Determine the grayscale cumulative distribution function of at least three channels of the real image, and calculate the balanced upper bound and the balanced lower bound of each channel according to the preset boundary parameters and the grayscale cumulative distribution function of each channel;
[0012] The equalization upper bound and the equalization lower bound of each channel are substituted into the linear stretching function to obtain the equalized image.
[0013] Optionally, calculating the adaptive dynamic threshold according to the equalized image specifically includes:
[0014] Determine the maximum bounding rectangle of the target object in the equalized image;
[0015] The adaptive dynamic threshold is calculated according to the image grayscale value of the maximum circumscribed rectangle.
[0016] Optionally, obtaining a voting matrix according to a Gaussian function, and performing probabilistic Hough transform detection on the edge according to the voting matrix to obtain an initial line segment specifically includes:
[0017] Taking the edge as a foreground point of probabilistic Hough transform, calculating a parameter space curve of each foreground point;
[0018] A voting matrix of the foreground point to the parameter space curve is obtained according to a Gaussian function, and an initial line segment of the edge is obtained according to the voting matrix and the parameter space curve.
[0019] Optionally, the screening of the initial line segments to obtain the border of the target object specifically includes:
[0020] Extracting the border line segments of the target object according to the initial line segments;
[0021] The border of the target object is restored according to the border line segments.
[0022] Optionally, calculating camera extrinsic parameters according to the frame specifically includes:
[0023] Extracting image feature points of the frame, and obtaining a homography matrix according to the image feature points and preset template feature points;
[0024] The camera extrinsic parameters are calculated according to the homography matrix.
[0025] Optionally, registering the virtual object to the target object according to the camera extrinsic parameter specifically includes:
[0026] Determine a two-dimensional projection of a virtual object according to the camera extrinsic parameters;
[0027] The two-dimensional projection is rendered onto a target object in the real image.
[0028] On the other hand, the present invention provides an augmented reality registration device based on a histogram, comprising:
[0029] at least one processor;
[0030] at least one memory for storing at least one program;
[0031] When the at least one program is executed by the at least one processor, the at least one processor implements any of the methods described above.
[0032] In another aspect, the present invention provides a computer-readable storage medium storing a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to perform the method as described above.
[0033] On the other hand, the present invention provides an augmented reality registration system based on a histogram, comprising a camera and a computer device connected to the camera device; wherein,
[0034] The camera is used to collect real video to obtain continuous real images, and upload the real images to the computer device;
[0035] The computer device comprises:
[0036] at least one processor;
[0037] at least one memory for storing at least one program;
[0038] When the at least one program is executed by the at least one processor, the at least one processor implements the method described above.
[0039] The implementation of the present invention includes the following beneficial effects: the present invention obtains a real image of the real world, performs a plurality of channel histogram equalization on the real image to obtain a balanced image, calculates an adaptive dynamic threshold according to the balanced image, binarizes the balanced image according to the adaptive dynamic threshold to obtain a binarized image, extracts the edge of the binarized image, obtains a voting matrix according to a Gaussian function, performs a probabilistic Hough transform detection on the edge according to the voting matrix to obtain an initial line segment, screens the initial line segment to obtain a frame of a target object, calculates a camera extrinsic parameter according to the frame, registers a virtual object to the target object according to the camera extrinsic parameter, and the target object is located in the real world. In real images; by introducing the three-channel histogram equalization method, the utilization rate of the image color space is improved, the influence of lighting on the actual registration results is reduced, and the registration accuracy is improved; the equalized image is then used to calculate the adaptive dynamic threshold, so that the adaptive dynamic threshold can be applied to various lighting scenes; the Gaussian transform is used to improve the Hough transform, and the improved Hough transform is used to detect line segments, which reduces the repeated line segment statistics in a small range caused by multiple edge points, reduces the over-extraction problem caused by this rigid statistical division, and makes the multiple peaks in small areas of the image integrated into a more reasonable central position, obtains smoother line segments, and improves the registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is a flowchart of the steps of a histogram-based augmented reality registration method provided by the present invention;
[0041] Figure 2 It is a stage flow chart of augmented reality registration provided by the present invention;
[0042] Figure 3 It is an example diagram of a Hough transform provided by the present invention;
[0043] Figure 4 This is a schematic diagram of a registration result of a square target object with one corner blocked provided by the present invention;
[0044] Figure 5 It is a structural schematic diagram of a histogram-based augmented reality registration device provided by the present invention;
[0045] Figure 6 It is a structural schematic diagram of a histogram-based augmented reality registration system provided by the present invention. DETAILED DESCRIPTION
[0046] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.
[0047] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0048] When the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it is necessary to understand that the terms "first / S100", "second / S200", "third / S300", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.
[0049] like Figure 1 As shown, Figure 1 The present invention provides a method for augmented reality registration based on a histogram, comprising the following steps:
[0050] S100, obtaining a real image of the real world, performing several channel histogram equalization on the real image, and obtaining an equalized image.
[0051] Herein, several refers to more than two.
[0052] Specifically, one or more frames of real images can be recovered from the video. For the real images, the grayscale images of the three channels of R, G, and B are respectively subjected to histogram equalization to obtain the equalized images.
[0053] Different lighting changes in the same scene are manifested as left and right drifts of the grayscale peak on the histogram. For example, red light will cause the R channel histogram to gather in the highlight area. In general, this fluctuation phenomenon results in insufficient utilization of the image RGB feature space. Therefore, performing histogram equalization on the R, G, and B channels separately can make good use of the image RGB feature space.
[0054] S200 , calculating an adaptive dynamic threshold according to the equalized image, and binarizing the equalized image according to the adaptive dynamic threshold to obtain a binarized image.
[0055] Specifically, the maximum bounding rectangle of the target object in the equalized image is determined, and the adaptive dynamic threshold is calculated according to the image grayscale value of the maximum bounding rectangle, wherein the target object is a landmark of the equalized image and a registered position of the virtual object in the equalized image.
[0056] After the adaptive dynamic threshold is obtained, the pixel points whose grayscale values in the equalized image are greater than the adaptive dynamic threshold are assigned a value of 255, and the pixel points whose grayscale values in the equalized image are less than the adaptive dynamic threshold are assigned a value of 0 to obtain a binary image.
[0057] S300, extracting the edge of the binary image, obtaining a voting matrix according to a Gaussian function, and performing a probabilistic Hough transform detection on the edge according to the voting matrix to obtain an initial line segment.
[0058] Specifically, the Canny edge of the binary image is extracted, and the Canny edge is used as the foreground point of the probabilistic Hough transform. The parameter space curve of each foreground point is calculated, and the voting matrix of the foreground point on the parameter space curve is obtained according to the Gaussian function. The initial line segment of the edge is obtained according to the voting matrix and the parameter space curve. Among them, the parameter space curve refers to the curve obtained by using the coordinates of the foreground point as coefficients and the curve parameters of the curve where the foreground point is located as variables.
[0059] S400, screening the initial line segments to obtain a bounding box of a target object, calculating camera extrinsics according to the bounding box, and registering a virtual object to the target object according to the camera extrinsics.
[0060] The target object is located in the real image, which is the same as explained in S200.
[0061] Specifically, the frame line segments of the target object are extracted based on the initial line segments, and the frame of the target object is restored based on the frame line segments. The image feature points of the frame are extracted, and the homography matrix is obtained based on the image feature points and the preset template feature points. The camera extrinsic parameters are calculated based on the homography matrix. The two-dimensional projection of the virtual object is determined based on the camera extrinsic parameters, and the two-dimensional projection is rendered to the target object of the real image.
[0062] like Figure 2 As shown, Figure 2 is a stage flow chart of augmented reality registration, Figure 2 (a) is a real image under dark light. Figure 2 (b) is a multi-channel grayscale equalized image. Figure 2 (c) is a bounding box of the target object. Figure 2 (d) is a schematic diagram of a registration result. Figure 2 In (d), the yellow part is the virtual object.
[0063] In some embodiments, step S100 performs a plurality of channel histogram equalization on the real image to obtain an equalized image, specifically including:
[0064] S110, determining the grayscale cumulative distribution function of at least three channels of the real image, and calculating the balanced upper bound and the balanced lower bound of each channel according to preset boundary parameters and the grayscale cumulative distribution function of each channel.
[0065] Specifically, for real images, the grayscale images of the three channels R, G, and B are calculated respectively, and the grayscale cumulative distribution function F of each channel is calculated. R (a) = P(|x < = a), F G (a) = P(|x < = a), F B (a) = P(|x <= a), where X is the gray value of each pixel and a is the gray value threshold. For each channel, a uniform boundary parameter f is given uniformly. p =0.05, for any channel A, A belongs to any one of the three channels R, G, B, substitute the balanced upper bound and balanced lower bound of channel A into the grayscale cumulative distribution function of channel A, which satisfies formula (1):
[0066]
[0067] Among them, E Amin is the lower limit of the balance of channel A, E Amax It is the equalization upper limit of channel A.
[0068] By solving formula (1), we can obtain the lower bound and upper bound of channel A.
[0069] S120 , substituting the equalization upper bound and the equalization lower bound of each channel into a linear stretching function respectively to obtain the equalized image.
[0070] Specifically, after obtaining the upper and lower bounds of each channel, the upper and lower bounds are substituted into the linear stretching function, as shown in formula (2):
[0071]
[0072] Where k = 255 / (E Amax -E Amin ), d = 255 * E Amin / (E Amax -E Amin ), g(x) is the gray value after equalization.
[0073] After the grayscale values after three-channel equalization are obtained respectively, the grayscale values after three-channel equalization constitute a balanced image.
[0074] In some embodiments, calculating the adaptive dynamic threshold according to the equalized image in S200 specifically includes:
[0075] S210: Determine the maximum bounding rectangle of the target object in the equalized image.
[0076] Specifically, it can be but not limited to using OpenCV3.findContours() function to find the outer frame of the target object, and according to the four corner points of the outer frame, the maximum value in the X direction of the equalized image and the minimum value in the Y direction are taken to obtain the maximum circumscribed rectangle of the target object.
[0077] S220 , calculating the adaptive dynamic threshold according to the image grayscale value of the maximum circumscribed rectangle.
[0078] Specifically, the area within the maximum circumscribed rectangle is taken as the region of interest, and the average value of the image grayscale value of the region of interest is calculated to obtain the adaptive dynamic threshold.
[0079] In some embodiments, the step S300 of obtaining a voting matrix according to a Gaussian function and performing a probabilistic Hough transform detection on the edge according to the voting matrix to obtain an initial line segment specifically includes:
[0080] S310, taking the edge as a foreground point of probabilistic Hough transform, and calculating a parameter space curve of each foreground point.
[0081] Specifically, the Canny edge is set as the foreground point, the coordinates of the foreground point are used as the coefficients of the parameter space curve, and the coefficients of the curve where the foreground point is located are used as the variables of the parameter space curve to obtain the parameter space curve of the foreground point. For example, for a point (x0, y0), its curve is y0=k1x0+b1, then the parameter space curve of the foreground point is b1=y0-k1x0. At this time, the coefficients k1 and b1 of the curve where the foreground point is located are the independent variable and dependent variable of the parameter space curve respectively.
[0082] The curve where the foreground point is located forms an image coordinate system with (x0, y0), and the parameter space curve forms a parameter coordinate system with (b1, k1).
[0083] S320 , obtaining a voting matrix of the foreground point to the parameter space curve according to a Gaussian function, and obtaining an initial line segment of the edge according to the voting matrix and the parameter space curve.
[0084] Specifically, according to the above theory of probabilistic Hough transform, it can be known that in a real image, a curve has only one set of coefficients, and the parameters of the foreground points on the same curve are the same. Then the parameter space curves of the foreground points on the same curve will inevitably intersect at a certain point, and the coordinates of the intersection point are the coefficients of a curve in the image.
[0085] like Figure 3 As shown, Figure 3 This is an example of a Hough transform. It is the Hough transform result of three foreground points in the parameter space. When the three foreground points are on the same line segment, the parameter space curves corresponding to the three foreground points will also intersect at one point. The coordinates corresponding to this point are the intercept parameters of the same line segment where the three foreground points are located.
[0086] According to the parameter space curve passing through the intersection point, the intersection point is voted. In the general probabilistic Hough transform, only the parameter space curve passing through the intersection point is considered when voting (that is, only the foreground point where the parameter space curve intersects is considered). With this method, when detecting line segments of real images, even if the adjacent line segments are tilted by 0.1° or even smaller, the algorithm will still detect them as two line segments. At the same time, when the edge detection result is rough, multiple edge points may cause repeated line segment statistics within a small range. In order to solve this problem, the present invention obtains the voting matrix of the foreground point to the parameter space curve according to the Gaussian function, and the Gaussian function is shown in formula (3):
[0087]
[0088] Among them, i represents the X-direction coordinate of the pixel point, j represents the Y-direction coordinate of the pixel point, and δ 2 is the variance of the Gaussian function, is an adjustable parameter, l represents the mean value of the X-direction coordinate of the pixel point, It represents the mean value of the Y coordinate of the pixel points, and π represents the circumference of a circle.
[0089] According to formula (3), the voting contribution (voting score) G(i,j) of a certain intersection point of point (i,j) can be calculated. Assuming that the foreground point set corresponding to a certain intersection point is B, in order to detect the foreground points adjacent to foreground point set B in the real image as the same straight line as foreground point set B, a voting matrix of foreground points adjacent to foreground point set B to a certain intersection point can be constructed according to the Gaussian function. The elements of the voting matrix are the contributions of foreground points adjacent to foreground point set B to a certain intersection point. Taking a normalized 3×3 voting matrix as an example, for a certain foreground point b in foreground point set B, the coordinates are (i,j), and the voting matrix of the foreground points adjacent to foreground point b is shown in formula (4):
[0090]
[0091] Voting is performed on the parameter space curve according to the voting matrix, that is, the cumulative coefficients are counted. When the number of votes for a certain intersection point reaches the preset value, the coordinates of the intersection point in the parameter coordinate system are used as the coefficients of the line segment. The foreground points that meet the coefficients are connected into line segments and recorded to obtain the initial line segment of the edge. This process is repeated until all foreground points are counted.
[0092] In some embodiments, the screening of the initial line segments to obtain the border of the target object in step S400 specifically includes:
[0093] S410: extracting the border line segments of the target image according to the initial line segments.
[0094] Specifically, after step S300, the present invention can extract several initial line segments, and according to the color features on both sides of the initial line segments, extract the initial line segments at the color junction as the border line segments of the target object, for example, extract the initial line segments in the black and white junction area as the border line segments of the target object.
[0095] S420: Restoring the border of the target object according to the border line segments.
[0096] Specifically, according to the proximity relationship of the frame line segments, the straight lines surrounding the target object are restored, and the frame of the complete target object is restored using the intersection points of the straight lines.
[0097] In some embodiments, the step S00 of calculating the camera extrinsic parameters according to the frame specifically includes:
[0098] S430, extracting image feature points of the frame, and obtaining a homography matrix according to the image feature points and preset template feature points.
[0099] Specifically, a number of image feature points of the extracted border, generally the four corner points of the border, are used to construct a matching relationship between the image feature points and the preset template feature points to obtain a homography matrix, as shown in formula (5):
[0100]
[0101] Among them, (x′, y′) is the coordinate of the image feature point, (x, y) is the coordinate of the preset template feature point, and H is the homography matrix of the real image transformation.
[0102] According to the above formula (5), we can know that the degree of freedom of H is 8. The four image feature points of the border are extracted to complete the solution. That is, for each frame of the real image, the coordinates of the four corner points of the target object pair are found and matched one-to-one with the coordinates of the initial four corner points to solve the equation group and obtain H.
[0103] S440, calculating camera extrinsic parameters according to the homography matrix.
[0104] Specifically, calculating the camera extrinsic parameters according to the homography matrix is a related prior art and will not be described in detail here.
[0105] After obtaining the camera external parameters R and T, for the three-dimensional point (x O ,y o , z o ), the three-dimensional point (x o ,y o , z O ) is transformed into a two-dimensional projection point (x2, y2) on the display image, as shown in formula (6):
[0106]
[0107] According to formula (6), n three-dimensional control points with known coordinates on the virtual object are projected into two-dimensional points on the real image, and the virtual object is registered to the target object.
[0108] The implementation of the present invention includes the following beneficial effects: the present invention improves the utilization rate of the image color space by introducing a three-channel histogram equalization method; then uses the equalized image to calculate the adaptive dynamic threshold, so that the adaptive dynamic threshold can be applicable to various lighting scenes; then uses the Gaussian transform to improve the Hough transform, and uses the improved Hough transform to detect line segments, thereby reducing the repeated line segment statistics in a small range that may be caused by multiple edge points, reducing the over-extraction problem caused by such hard statistical division, and making the multiple peaks in the small area of the image integrated into a more reasonable central position, obtaining smoother line segments, and improving the registration accuracy.
[0109] like Figure 4 As shown, Figure 4 It is a schematic diagram of the registration result of a square target object with one corner blocked. In the figure, Figure 4 (a) is the image after multi-channel equalization. Figure 4 (b) is the result of the probabilistic Hough transform. Figure 4 (c) is the result of the probability Hough transform after the voting matrix is introduced. Figure 4 (d) is a schematic diagram of the result of extracting the border line segments of the target image. Figure 4 (e) is the extracted bounding box of the target object. Figure 4 (f) is a registration result, and the yellow part in the figure is a virtual object.
[0110] from Figure 4 In (b), we can see that when the probabilistic Hough transform is used directly to extract line segments, there are line breaks at the long edges, and multiple scattered line segments appear at the thick edges. The overly sensitive detection result is not conducive to the next step of effective line segment screening, i.e., border recovery. Figure 4 In (c), due to the introduction of Gaussian blur function in parameter statistics, the multiple peaks in the small area are integrated into a more reasonable central position, which is reflected in the image that there is generally only one detection result in the small area, and the long edges are also unified into the same line segment. After the edge of the target object is represented by a complete line segment, the four outer straight lines of the target object can be extracted based on the approximate parallel relationship of the border, and then the quadrilateral result representing the target object can be restored.
[0111] like Figure 5 As shown, Figure 5 The present invention is a structural schematic diagram of an augmented reality registration device based on a histogram. The present invention also provides an augmented reality registration device based on a histogram, comprising:
[0112] at least one processor;
[0113] at least one memory for storing at least one program;
[0114] When the at least one program is executed by the at least one processor, the at least one processor implements the steps of the histogram-based augmented reality registration method.
[0115] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0116] like Figure 6 As shown, Figure 6 The present invention is a structural schematic diagram of an augmented reality registration system based on a histogram. The present invention also provides an augmented reality registration system based on a histogram, comprising a camera and a computer device connected to the camera device; wherein,
[0117] The camera is used to collect real video to obtain continuous real images, and upload the real images to the computer device;
[0118] The computer device comprises:
[0119] at least one processor;
[0120] at least one memory for storing at least one program;
[0121] When the at least one program is executed by the at least one processor, the at least one processor implements the steps described in the histogram-based augmented reality registration method.
[0122] Specifically, for cameras, at least one camera may be included; and for computer equipment, it may be different types of electronic equipment, including but not limited to terminals such as desktop computers and laptops.
[0123] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0124] The present invention also provides a computer-readable storage medium, which stores a program executable by a processor. The program executable by the processor is used to implement the above method when executed by the processor.
[0125] It is understood that all or some steps and systems in the disclosed method above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0126] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0127] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.
Claims
1. A histogram-based augmented reality registration method, characterized in that: The following steps are involved: Acquire a real image of the real world, and perform a plurality of channel histogram equalization on the real image to obtain an equalized image; Calculating an adaptive dynamic threshold according to the equalized image, and binarizing the equalized image according to the adaptive dynamic threshold to obtain a binarized image; Extracting the edge of the binary image, obtaining a voting matrix according to a Gaussian function, and performing a probabilistic Hough transform detection on the edge according to the voting matrix to obtain an initial line segment; The initial line segments are screened to obtain a bounding box of a target object, a camera extrinsic parameter is calculated according to the bounding box, and a virtual object is registered to the target object according to the camera extrinsic parameter; The target object is located in the real image.
2. The method according to claim 1, characterized in that The performing of a plurality of channel histogram equalization on the real image to obtain an equalized image specifically includes: Determine the grayscale cumulative distribution function of at least three channels of the real image, and calculate the balanced upper bound and the balanced lower bound of each channel according to the preset boundary parameters and the grayscale cumulative distribution function of each channel; Substitute the equalization upper bound and the equalization lower bound of each channel into the linear stretching function respectively to obtain the equalized image.
3. The method according to claim 1, characterized in that The step of calculating the adaptive dynamic threshold according to the equalized image specifically includes: Determine the maximum bounding rectangle of the target object in the equalized image; The adaptive dynamic threshold is calculated according to the image grayscale value of the maximum circumscribed rectangle.
4. The method according to claim 1, characterized in that: The step of obtaining a voting matrix according to the Gaussian function and performing probabilistic Hough transform detection on the edge according to the voting matrix to obtain an initial line segment specifically includes: Taking the edge as a foreground point of probabilistic Hough transform, calculating a parameter space curve of each foreground point; A voting matrix of the foreground point to the parameter space curve is obtained according to a Gaussian function, and an initial line segment of the edge is obtained according to the voting matrix and the parameter space curve.
5. The method according to any one of claims 1 to 4, characterized in that: The screening of the initial line segments to obtain the border of the target object specifically includes: Extracting the border line segments of the target object according to the initial line segments; The border of the target object is restored according to the border line segments.
6. The method according to any one of claims 1 to 4, characterized in that: The calculating of the camera extrinsic parameters according to the frame specifically includes: Extracting image feature points of the frame, and obtaining a homography matrix according to the image feature points and preset template feature points; The camera extrinsic parameters are calculated according to the homography matrix.
7. The method according to any one of claims 1 to 4, characterized in that: The step of registering the virtual object to the target object according to the camera external parameter specifically includes: Determine a two-dimensional projection of a virtual object according to the camera extrinsic parameters; The two-dimensional projection is rendered onto a target object in the real image.
8. A histogram-based augmented reality registration device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.
9. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 7 when executed by the processor.
10. A histogram-based augmented reality registration system, characterized in that: It includes a camera and a computer device connected to the camera device; wherein, The camera is used to collect real video to obtain continuous real images, and upload the real images to the computer device; The computer device comprises: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.