An image rendering method based on ARG game

By obtaining the contrasting pixel points based on the grayscale and gradient distribution of each pixel point and the pixel points in the neighborhood in the ARG game, and calculating the difference in reference texture and corner point possibilities, the FAST corner point detection algorithm is solved in the corner point in ARG game. The problem of low detection accuracy improves the rendering effect.

CN118505869BActive Publication Date: 2024-11-15SHANDONG SHUNNENG NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410519462.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-28
Publication Date
2024-11-15
Estimated Expiration
2044-04-28

AI Technical Summary

Technical Problem

In the prior art, the FAST corner point detection algorithm has a low accuracy in detecting corner points of the real scene images collected by ARG game devices, resulting in poor rendering of ARG game images.

Method used

By obtaining the contrasting pixel points in the real scene image based on the grayscale deviation and gradient distribution of each pixel point with each pixel point in the neighborhood, calculate the reference texture difference and corner point possibility of each pixel point, and filter out the final corner point. To improve rendering effect.

Benefits of technology

Improve the accuracy of corner point detection and the rendering effect of ARG game images, making the rendered scene more in line with the user's immersive experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118505869B_ABST
    Figure CN118505869B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of virtual reality augmented technology, and in particular to an image rendering method based on an ARG game. The method first obtains different numbers of contrasting pixels according to grayscale distribution performance and gradient distribution performance within a neighborhood range of different pixels in a real scene image; further obtains reference texture differences according to grayscale gradient differences and color differences between each pixel and the contrasting pixel; and obtains more accurate corner point possibilities corresponding to each pixel based on the reference texture differences in combination with grayscale gradient amplitudes and edge significance, thereby making final corner points subsequently screened out according to corner point possibilities and a corner point detection method more accurate, that is, making the effect of ARG game image rendering based on the final corner points better.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of virtual reality enhancement technology, and in particular to an image rendering method based on ARG games. Background Art

[0002] Augmented Reality Game (ARG) is a type of game that integrates virtual elements with the real world. It combines virtual elements with real-world scenes based on virtual reality technology, so that users can get an immersive virtual game experience. In order to improve the accuracy of combining virtual elements with real-world scenes, corner detection algorithms are usually used to detect and locate key corner points in images corresponding to real-world scenes, so as to select appropriate materials for rendering virtual images, making the rendered scenes more suitable for the user's immersive experience.

[0003] ARG combines virtual elements with real-time reality scenes. In order to ensure the user's immersion, a higher rendering speed is usually required. Therefore, the existing technology usually uses the FAST corner detection algorithm with a faster calculation speed to detect corner points in the real scene image; when performing detection, the FAST corner detection algorithm usually compares the grayscale value of each pixel with each pixel in the neighborhood to determine whether it is the required corner point; however, the real scene image collected by the ARG game device is an image corresponding to the three-dimensional space, and there may be spatial distance scaling or changes in light brightness in the real space, resulting in the feature points in the real scene image, that is, the features of the real corner points are not obvious enough, that is, the prior art usually uses the FAST corner detection algorithm to detect the final corner points of the real scene image collected by the ARG game device, and the accuracy of the final corner points detected is low, resulting in poor effect of ARG game image rendering based on the final corner points. Summary of the invention

[0004] In order to solve the technical problem that the accuracy of the final corner points detected by the FAST corner point detection algorithm in the prior art for the real scene image collected by the ARG game device is low, resulting in poor effect of ARG game image rendering based on the final corner points, the purpose of the present invention is to provide an image rendering method based on the ARG game, and the technical solution adopted is as follows:

[0005] The present invention proposes an image rendering method based on an ARG game, the method comprising:

[0006] Collect real scene images through ARG game equipment;

[0007] In the real scene image, all contrasting pixels corresponding to each pixel are obtained according to the grayscale deviation between each pixel and each pixel in the neighborhood, and the gradient distribution in the neighborhood of each pixel;

[0008] According to the grayscale gradient difference and color difference between each pixel and the contrasting pixel in the real scene image, the reference texture difference between each pixel and each contrasting pixel is obtained; according to all the reference texture differences, grayscale gradient amplitude and edge significance corresponding to each pixel, the corner point possibility of each pixel is obtained;

[0009] According to the corner point possibility and the corner point detection method, all final corner points in the real scene image are screened out; and the ARG game image is rendered according to the final corner points.

[0010] Furthermore, the method for obtaining the comparison pixel points includes:

[0011] The preset number of neighboring pixels closest to each pixel is used as the reference pixel of each pixel; the gray importance of each pixel is obtained according to the gray deviation and gray gradient distribution between each pixel and the corresponding reference pixel;

[0012] The product of the positive correlation mapping value of the grayscale importance and the preset neighborhood number is rounded up to obtain the comparison neighborhood number of each pixel point; the comparison neighborhood number of other pixel points closest to each pixel point are used as all the comparison pixel points corresponding to each pixel point.

[0013] Furthermore, the calculation formula of the gray importance degree includes:

[0014]

[0015] Among them, W k is the gray importance of the kth pixel in the real scene image; n k is the number of reference pixels corresponding to the k-th pixel in the real scene image; G k is the gray value of the kth pixel in the real scene image; C k,i is the gray value of the i-th reference pixel corresponding to the k-th pixel in the real scene image; S k is the number of pixels whose grayscale gradient is not 0 in the k-th pixel and all corresponding reference pixels; exp() is an exponential function with a natural constant as the base; Norm() is a normalization function.

[0016] Furthermore, the method for obtaining the reference texture difference includes:

[0017] Get the gradient angles of all pixels in the real scene image;

[0018] The difference between the grayscale value of each pixel and the grayscale value of each corresponding contrasting pixel is used as the reference grayscale difference of each contrasting pixel; the difference between the cosine value of the gradient angle of each pixel and the cosine value of the gradient angle of each corresponding contrasting pixel is used as the gradient direction difference of each contrasting pixel;

[0019] The real scene image is converted into the LAB space, and the distance between each pixel and each corresponding contrast pixel in the LAB space is used as the reference color difference of each contrast pixel; the distance between each pixel and each corresponding contrast pixel in the real scene image is used as the actual distance of each contrast pixel;

[0020] According to the reference grayscale difference, the gradient direction difference, the reference color difference and the actual distance, a reference texture difference between each pixel and each contrasting pixel is obtained; the reference grayscale difference, the gradient direction difference and the reference color difference are all positively correlated with the reference texture difference, and the actual distance is negatively correlated with the reference texture difference.

[0021] Furthermore, the method for obtaining a reference texture difference between each pixel point and each comparison pixel point according to the reference grayscale difference, the gradient direction difference, the reference color difference and the actual distance includes:

[0022] The sum of the reference grayscale difference and the gradient direction difference is used as a reference sum value; the ratio of the reference color difference and the actual distance is used as a reference ratio; and the product of the reference sum value and the reference ratio is used as a reference texture difference between each pixel point and each contrasting pixel point.

[0023] Furthermore, the method for obtaining the possibility of the corner point includes:

[0024] The average value of all reference texture differences corresponding to each pixel is used as the texture importance of each pixel; the gradient change characteristic value of each pixel is obtained according to the gray gradient deviation of each pixel compared with all corresponding contrast pixels;

[0025] The minimum distance between each pixel and all edge pixels in the real scene image is taken as the edge proximity of each pixel;

[0026] According to the texture importance, the gradient change characteristic value and the edge proximity, the corner point possibility of each pixel in the real scene image is obtained; the texture importance and the gradient change characteristic value are both positively correlated with the corner point possibility; the edge proximity is negatively correlated with the corner point possibility.

[0027] Furthermore, the method for obtaining the gradient change characteristic value includes:

[0028] The mean value of the grayscale gradient amplitude of all contrasting pixel points corresponding to each pixel point and the sum of the preset adjustment parameters are used as the contrast gradient amplitude of each pixel point; the ratio of the grayscale gradient amplitude of each pixel point to the contrast gradient amplitude is used as the gradient change characteristic value of each pixel point.

[0029] Furthermore, the method for obtaining the corner point possibility of each pixel in the real scene image according to the texture importance, the gradient change characteristic value and the edge proximity includes:

[0030] The normalized value of the product of the negative correlation mapping value of the edge proximity, the texture importance and the gradient change feature value is used as the corner point possibility of each pixel point in the real scene image.

[0031] Furthermore, the method for obtaining the final corner point includes:

[0032] Taking the mean of the corner point possibilities of all pixels in the real scene image as the initial segmentation threshold, performing iterative threshold segmentation on the corner point possibilities of all pixels in combination with an iterative threshold segmentation algorithm to obtain all iterative segmentation thresholds;

[0033] The average of the distances between each pixel and a preset reference number of edge pixels closest to each pixel in the real scene image in the LAB space is used as the color proximity of each pixel;

[0034] According to the edge proximity and color proximity of the pixel points whose corner point probability is greater than each iterative segmentation threshold, the optimal threshold probability of each iterative segmentation threshold is obtained; the iterative segmentation threshold corresponding to the maximum optimal threshold probability is used as the final segmentation threshold;

[0035] All pixel points in the real scene image that are greater than the final division threshold are taken as candidate corner points; all candidate corner points are detected using the FAST corner point detection algorithm to obtain all final corner points in the real scene image.

[0036] Furthermore, the method for obtaining the optimal threshold possibility includes:

[0037] For any iterative segmentation threshold:

[0038] The negative correlation mapping value of the product of the mean of the edge proximity and the mean of the color proximity of all pixels whose corner point likelihood is greater than the iterative segmentation threshold is used as the optimal threshold likelihood of the iterative segmentation threshold.

[0039] The present invention has the following beneficial effects:

[0040] In the process of corner point detection, the FAST corner point detection algorithm compares the grayscale of each pixel with a fixed number of pixels in its neighborhood, so as to determine the possibility of each pixel being a corner point, so as to screen the final corner points in the future. However, since the real scene image collected by the ARG game device is a display image in three-dimensional space, the feature performance of some pixels or feature points is weak. For example, the pixel that should be a corner point is affected by light or space scaling, resulting in its grayscale value being close to that of the neighboring pixel points, so that it is classified as a non-real corner point when the corner point is judged. Therefore, in order to make the central pixel point select the accuracy of its analysis and judgment with the neighboring pixel points, it is necessary to appropriately increase the number of contrasting pixels corresponding to each pixel point according to the local feature performance of the neighborhood of different pixels, so that the corner point that may perform weakly is compared with the contrasting pixels in its neighborhood. The calculated feature difference is greater, thereby improving the accuracy of the final corner points screened out in the future; therefore, the present invention obtains all contrasting pixels corresponding to each pixel point according to the grayscale deviation between each pixel point and each pixel point in the neighborhood, and the gradient distribution in the neighborhood of each pixel point.

[0041] The basis for the final corner point screening of the FAST corner point detection algorithm is the grayscale information difference between each pixel and the contrasting pixel in its corresponding neighborhood; however, for objects in three-dimensional space, the color information of their surfaces is relatively rich, and simply using grayscale values ​​for calculation may result in certain deviations; therefore, the present invention analyzes the difference in grayscale performance between each pixel and the contrasting pixel based on the difference in grayscale performance, combined with the difference in corresponding color performance, so as to obtain a reference texture difference that is more accurate in characterizing the feature difference; and on the basis of the reference texture difference, the closer the pixel is to the edge, the greater the possibility of being observed, the more significant its feature is, and the greater the corresponding corner point possibility; and the grayscale gradient amplitude can also characterize the grayscale significance at the current pixel position, so the present invention obtains a more accurate corner point possibility corresponding to each pixel according to all reference texture differences, grayscale gradient amplitudes and edge significance corresponding to each pixel, thereby making the final corner point screened out based on the corner point possibility and the corner point detection method more accurate, that is, making the effect of ARG game image rendering based on the final corner point better. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A flowchart of an image rendering method based on an ARG game provided by an embodiment of the present invention;

[0044] Figure 2 A flow chart of a method for obtaining corner point possibility provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the image rendering method based on the ARG game proposed by the present invention, its specific implementation, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0046] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0047] The following is a detailed description of a specific solution of an image rendering method based on an ARG game provided by the present invention in conjunction with the accompanying drawings.

[0048] See also Figure 1 , which shows a flow chart of an image rendering method based on an ARG game provided by an embodiment of the present invention, the method comprising:

[0049] Step S1: Capture real scene images through the ARG game device.

[0050] The embodiment of the present invention aims to provide an image rendering method based on an ARG game, which aims to analyze the performance of real scene images collected by an ARG game device in color space and in grayscale space, and calculate the corner point possibility of each pixel point in combination with its edge significance, and further combine the corner point detection method to screen out more accurate final corner points, so that the effect of ARG game image rendering based on the final corner points is better.

[0051] The embodiment of the present invention collects real scene images through the ARG game device. Specifically: when the user uses the ARG game device for experience, the real scene image is collected by the ARG game device in real time; and in order to make the user's viewing experience better, the real scene image collected by the embodiment of the present invention is an RGB image. At the same time, in order to facilitate subsequent analysis, the real scene image is grayed to obtain the gray value corresponding to each pixel in the real scene image; the real scene image is edge detected to obtain the edge pixel points in the real scene image; and the gradient amplitude and gradient angle of each pixel in the real scene image are calculated by the Scharr operator. In the embodiment of the present invention, the edge detection method adopts canny edge detection, and the implementer can also calculate the gradient amplitude and gradient angle of each pixel by other operators other than the Scharr operator, and the Scharr operator, the canny edge detection method and the graying are the prior art known to those skilled in the art, and are not further defined or elaborated here.

[0052] Step S2: In the real scene image, all contrast pixels corresponding to each pixel are obtained according to the grayscale deviation between each pixel and each pixel in the neighborhood, and the gradient distribution in the neighborhood of each pixel.

[0053] ARG combines virtual elements with real-time reality scenes. In order to ensure the user's immersion, a high rendering speed is usually required. Therefore, the existing technology usually uses the FAST corner detection algorithm with a faster calculation speed to detect corner points in real scene images. In the process of corner point detection, the FAST corner detection algorithm compares the grayscale of each pixel with a fixed number of pixels in its neighborhood to determine the possibility of each pixel as a corner point for subsequent final corner point screening. However, since the real scene image collected by the ARG game device is a display image in three-dimensional space, the characteristics of some pixels or feature points are weak. For example, the pixel that should be a corner point is affected by light or space scaling, resulting in its grayscale value being closer to the neighboring pixel point, and is therefore classified as a non-real corner point when the corner point is judged. Therefore, in order to ensure the accuracy of the central pixel in the selection of the analysis and judgment of the neighboring pixels, it is necessary to appropriately increase the number of contrast pixels corresponding to each pixel according to the local feature performance of the neighborhood of different pixels, so that the corner points that may perform weaker have a larger feature difference when compared with the contrast pixels in their neighborhood, thereby improving the accuracy of the final corner points screened out later. Therefore, in the embodiment of the present invention, all contrast pixels corresponding to each pixel are obtained according to the grayscale deviation between each pixel and each pixel in the neighborhood, and the gradient distribution in the neighborhood of each pixel, in the real scene image, so as to adaptively calculate the number of contrast pixels corresponding to each pixel, thereby improving the robustness of corner point detection at different positions in the real scene image.

[0054] Preferably, the method for obtaining the comparison pixel points includes:

[0055] The preset number of neighboring pixels closest to each pixel is used as the reference pixel of each pixel; the grayscale importance of each pixel is obtained according to the grayscale deviation and grayscale gradient distribution between each pixel and the corresponding reference pixel. In the embodiment of the present invention, the preset number of neighbors is set to 16, which is also the fixed number of neighbors corresponding to each pixel in the FAST corner detection algorithm, and the implementer can adjust it according to the specific implementation environment.

[0056] Preferably, each pixel point is taken as the k-th pixel point in turn, and the calculation formula of the gray importance of the k-th pixel point includes:

[0057]

[0058] Among them, W k is the gray importance of the kth pixel in the real scene image; n k is the number of reference pixels corresponding to the k-th pixel in the real scene image; G k is the gray value of the kth pixel in the real scene image; G k,i is the gray value of the i-th reference pixel corresponding to the k-th pixel in the real scene image; S k is the number of pixels whose grayscale gradient is not 0 in the k-th pixel and all corresponding reference pixels; exp() is an exponential function with a natural constant as the base; Norm() is a normalization function.

[0059] For each pixel in the real scene image, the closer the grayscale value is to the overall reference pixel in the neighborhood, the more similar the grayscale performance of the pixel is to the pixels in the neighborhood, that is, the weaker the feature performance is. Therefore, in order to further reflect the local feature performance of its neighborhood and better distinguish it from the reference pixel, it is necessary to appropriately increase the number of contrast pixels corresponding to each pixel, so a greater grayscale importance is required; therefore The smaller the overall value, the more similar the grayscale performance of the pixel is to the pixels in the neighborhood, and the greater the corresponding grayscale importance, so negative correlation mapping is further performed; it should be noted that in addition to negative correlation mapping through the exp(-) function, the implementer also performs negative correlation mapping through other methods, such as inverse, etc., which will not be further elaborated here.

[0060] In addition, corner points are usually feature points where the edges of an object surface intersect or where the color of an object surface changes greatly. Therefore, the more pixels with non-zero gradients in the reference pixels in the neighborhood corresponding to the pixel point, the more likely the pixel point is to be at a location where the edge of the object surface intersects or where the color of the object surface changes greatly, and the greater the possibility of the corresponding corner point. Therefore, when analyzing and comparing pixel points, more pixels need to be selected to facilitate the subsequent acquisition of more comparison pixel information, so that the subsequent corner point analysis is more accurate and the corresponding grayscale importance is greater. k The larger the value, the more pixels with non-zero gradients in the reference pixels in the neighborhood corresponding to the pixel point, and the greater the grayscale importance. Therefore, the grayscale importance is further obtained by multiplication. k +1 as S k The denominator of makes the corresponding value range more reasonable and has better robustness; further and The product of is normalized to obtain the grayscale importance value range limited to between 0 and 1, which is convenient for the subsequent calculation of the number of comparison pixels.

[0061] The product of the positive correlation mapping value of the grayscale importance and the preset number of neighbors is rounded up to obtain the number of comparison neighbors for each pixel; the number of other pixels closest to each pixel as all the comparison pixels corresponding to each pixel. Since the grayscale importance calculated by the embodiment of the present invention has a value range of 0 to 1, if the grayscale importance is used as a weight to directly multiply the preset number of neighbors, it will only result in a reduction in the number of comparison pixels. Therefore, it is necessary to perform positive correlation mapping on the grayscale importance so that the mapping value is greater than or equal to 1, thereby achieving the purpose of increasing the number of comparison pixels for pixels with greater grayscale importance.

[0062] In the embodiment of the present invention, the method for obtaining the number of comparison neighborhoods of the k-th pixel point is expressed in the formula as follows:

[0063] Y k =(1+W k )×X

[0064] Among them, Y k is the number of comparison neighborhoods of the kth pixel in the real scene image; W k is the gray importance of the kth pixel in the real scene image; X is the number of preset neighborhoods. (1+W k) is the positive correlation mapping value of the grayscale importance of the kth pixel in the real scene image. The positive correlation mapping value obtained by adding the positive number 1 to the grayscale importance ranges from 1 to 2. The preset number of neighborhoods is weighted using the positive correlation mapping as a weight, so that the number of contrast pixels corresponding to pixels with greater grayscale importance can be higher, but within a reasonable value range. The implementer can adjust the positive correlation mapping method according to the specific implementation environment, but ensure that the positive correlation mapping value of the grayscale importance is greater than or equal to 1.

[0065] Step S3: According to the grayscale gradient difference and color difference between each pixel and the contrasting pixel in the real scene image, the reference texture difference between each pixel and each contrasting pixel is obtained; according to all reference texture differences, grayscale gradient amplitudes and edge significance corresponding to each pixel, the corner point possibility of each pixel is obtained.

[0066] The basis for the final corner point screening by the FAST corner detection algorithm is the grayscale information difference between each pixel point and the contrasting pixel point in its corresponding neighborhood; however, for objects in three-dimensional space, the color information of their surfaces is relatively rich, and simply using grayscale values ​​for calculation may result in certain deviations; therefore, the embodiment of the present invention obtains the reference texture difference between each pixel point and each contrasting pixel point based on the grayscale gradient difference and color difference between each pixel point and the contrasting pixel point in the real scene image.

[0067] Preferably, the method for obtaining the reference texture difference includes:

[0068] When the FAST corner detection algorithm calculates the possibility of a corner point for each pixel, it is necessary to compare the grayscale value between each pixel and the contrasting pixel in the neighborhood, so as to judge the possibility of a corner point for each pixel based on the grayscale value difference. Therefore, in combination with the corner point possibility analysis principle of the FAST corner detection algorithm, the embodiment of the present invention analyzes the difference between the grayscale value between each pixel and the contrasting pixel in combination with the difference between the grayscale gradient angle or the grayscale gradient direction. The gradient angle of all pixels in the real scene image is obtained; the difference between the grayscale value of each pixel and the grayscale value of each corresponding contrasting pixel is used as the reference grayscale difference of each contrasting pixel; the difference between the cosine value of the gradient angle of each pixel and the cosine value of the gradient angle of each corresponding contrasting pixel is used as the gradient direction difference of each contrasting pixel. Since the features of corner points are usually more significant, the greater the corresponding reference grayscale difference and the greater the gradient direction difference, the more likely the corresponding pixel is to be a corner point. It should be noted that, according to the specific implementation environment, the implementer may directly use the normalized value of the difference between the gradient angle of each pixel point and the gradient angle of each corresponding comparison pixel point as the gradient direction difference of each comparison pixel point.

[0069] Furthermore, since the color information of the surface of the object in the three-dimensional space is relatively rich, the difference between the pixel point and each corresponding contrast pixel point in the color space is further analyzed; the embodiment of the present invention converts the real scene image into the LAB space, and takes the distance between each pixel point and each corresponding contrast pixel point in the LAB space as the reference color difference of each contrast pixel point. Considering that the size of the calculated reference color difference will be affected by the distance between the contrast pixel point and the corresponding pixel point to a certain extent, the distance between each pixel point and each corresponding contrast pixel point in the real scene image is further taken as the actual distance of each contrast pixel point; further considering the size of the actual distance on the basis of the reference color difference, the calculated color difference feature is more accurate, that is, the calculation of the feature significance of each pixel point is more accurate; the larger the corresponding reference color difference and the smaller the actual distance, the greater the difference between the contrast pixel point and the corresponding pixel point in the color space, and the greater the possibility of the corresponding corner point. In the embodiment of the present invention, the distance calculation method adopts the Euclidean distance, and the implementer can also adopt other distance calculation methods according to the specific implementation environment, which will not be further elaborated here. It should be noted that calculating the distance between two pixel points in the LAB space is an existing technology well known to those skilled in the art, and will not be further defined or elaborated herein.

[0070] Further combine the differences between the pixel point and each contrasting pixel point in grayscale, grayscale gradient direction, and color space, and obtain the reference texture difference between each pixel point and each contrasting pixel point based on the reference grayscale difference, gradient direction difference, reference color difference, and actual distance; the reference grayscale difference, gradient direction difference, and reference color difference are all positively correlated with the reference texture difference, and the actual distance is negatively correlated with the reference texture difference. The comprehensive feature difference between each pixel point and each corresponding contrasting pixel point is comprehensively characterized by the reference texture difference, so that the possibility of the corner point obtained subsequently is more accurate. The larger the corresponding reference texture difference, the more significant the corresponding pixel point is in the neighborhood, and the greater the possibility of the corresponding corner point. Preferably, the method for obtaining the reference texture difference between each pixel point and each contrasting pixel point based on the reference grayscale difference, gradient direction difference, reference color difference, and actual distance includes:

[0071] The sum of the reference grayscale difference and the gradient direction difference is used as the reference sum; the product of the reference sum and the reference ratio is used as the reference texture difference between each pixel and each contrasting pixel. The reference sum is used to comprehensively represent the difference between each pixel and each contrasting pixel in the grayscale space; and in order to eliminate the influence of the actual distance in the calculated reference color difference, the ratio of the reference color difference and the actual distance is used as the reference ratio; so that the calculated reference ratio can weaken the influence of the actual distance, making the represented difference in the LAB color space more accurate; further, through the combination in the form of product, the calculated reference texture difference can comprehensively represent the difference between each pixel and each corresponding contrasting pixel.

[0072] In the embodiment of the present invention, each comparison pixel corresponding to the k-th pixel is sequentially used as the h-th comparison pixel corresponding to the k-th pixel. Then, the method for obtaining the reference texture difference between the k-th pixel and the corresponding h-th comparison pixel is expressed in the formula as follows:

[0073]

[0074] Among them, T k,h is the reference texture difference between the kth pixel and the corresponding hth contrast pixel, G k is the gray value of the kth pixel; G k,h is the gray value of the hth contrast pixel corresponding to the kth pixel; C k is the cosine value of the gradient angle of the kth pixel; C k,h is the cosine value of the gradient angle of the hth contrast pixel corresponding to the kth pixel; ΔLAB k,h is the reference color difference of the hth contrast pixel corresponding to the kth pixel; d k,h is the actual distance between the kth pixel and the hth contrast pixel; |G k -G k,h | is the reference grayscale difference of the hth contrast pixel corresponding to the kth pixel; |C k -C k,h | is the gradient direction difference of the hth contrast pixel corresponding to the kth pixel; || is the absolute value sign.

[0075] On the basis of the reference texture difference, the closer the pixel point is to the edge, the greater the possibility of being observed, and the more significant its feature is, then the greater the corresponding corner point possibility; and the grayscale gradient amplitude can also characterize the grayscale significance at the current pixel position, so the embodiment of the present invention obtains the corner point possibility of each pixel point according to all reference texture differences, grayscale gradient amplitudes and edge significance corresponding to each pixel point.

[0076] Preferably, the method for obtaining the possibility of corner points includes:

[0077] See also Figure 2 , which shows a flow chart of a method for obtaining corner point possibility provided by an embodiment of the present invention, the method comprising:

[0078] Step S301: Calculate the texture importance of each pixel.

[0079] The average of all reference texture differences corresponding to each pixel is used as the texture importance of each pixel. For each pixel, the greater the overall reference texture difference between it and all corresponding contrasting pixels, the more prominent or significant the local feature of the pixel is, and the greater the possibility that the pixel is a corner point; that is, the greater the texture importance, the greater the possibility of a corner point.

[0080] Step S302: Calculate the gradient change characteristic value of each pixel;

[0081] According to the grayscale gradient deviation of each pixel point compared with all corresponding comparison pixels, a gradient change characteristic value of each pixel point is obtained; preferably, a method for obtaining the gradient change characteristic value includes:

[0082] The mean value of the grayscale gradient amplitude of all contrasting pixels corresponding to each pixel and the sum of the preset adjustment parameters are taken as the contrast gradient amplitude of each pixel; the ratio of the grayscale gradient amplitude of each pixel to the contrast gradient amplitude is taken as the gradient change characteristic value of each pixel.

[0083] Since the contrast gradient amplitude represents the overall grayscale gradient amplitude of all contrast pixels corresponding to each pixel, for each pixel, the larger the corresponding grayscale gradient amplitude is relative to the contrast gradient amplitude, the larger the overall deviation of the corresponding pixel relative to the contrast pixels in the neighborhood in the dimension of grayscale gradient amplitude, the more significant the feature of the pixel, and the greater the possibility of the corresponding corner point.

[0084] Step S303: Calculate the edge proximity of each pixel.

[0085] The minimum distance between each pixel and all edge pixels in the real scene image is taken as the edge proximity of each pixel. Since the closer the pixel is to the edge, the more likely it is to be observed, and the more significant its features are, the more likely it is to be a corner point; therefore, the smaller the edge proximity, the greater the possibility of the corresponding corner point.

[0086] Step S304: Obtain the possibility of a corner point according to the texture importance, gradient change characteristic value and edge proximity of each pixel point.

[0087] The possibility of corner points is further comprehensively characterized by combining the importance of texture, the characteristic value of gradient change and the degree of edge proximity. The embodiment of the present invention obtains the possibility of corner points for each pixel point in the real scene image according to the importance of texture, the characteristic value of gradient change and the degree of edge proximity; the importance of texture and the characteristic value of gradient change are both positively correlated with the possibility of corner points; the degree of edge proximity is negatively correlated with the possibility of corner points. Preferably, the method of obtaining the possibility of corner points for each pixel point in the real scene image according to the importance of texture, the characteristic value of gradient change and the degree of edge proximity includes:

[0088] The normalized value of the product of the negative correlation mapping value of the edge proximity, the texture importance and the gradient change feature value is used as the corner point possibility of each pixel in the real scene image.

[0089] In the embodiment of the present invention, the method for obtaining the corner point possibility of the k-th pixel point in the real scene image is expressed in the formula as follows:

[0090]

[0091] Among them, P k is the corner point probability of the kth pixel in the real scene image; F k is the grayscale gradient amplitude of the kth pixel in the real scene image; is the mean value of the grayscale gradient amplitude of all contrasting pixels corresponding to the k-th pixel in the real scene image, that is, the gradient change characteristic value corresponding to the k-th pixel in the real scene image; b k is the edge proximity degree corresponding to the kth pixel point in the real scene image; M k is the number of contrast pixels corresponding to the k-th pixel in the real scene image; T k,m is the reference texture difference between the kth pixel and the corresponding mth comparison pixel in the real scene image; is the texture importance corresponding to the k-th pixel in the real scene image; exp() is an exponential function with a natural constant as the base; Norm() is a normalization function. In the embodiment of the present invention, the normalization method adopts linear normalization. The implementer can adjust the normalization method according to the specific implementation environment, which will not be further elaborated here.

[0092] Step S4: According to the corner point possibility and the corner point detection method, all the final corner points in the real scene image are screened out; and the ARG game image is rendered according to the final corner points.

[0093] After obtaining the corner point possibility of each pixel in the real scene image, the final corner points are further screened by a corner point detection algorithm to obtain more accurate final corner points. The embodiment of the present invention screens out all the final corner points in the real scene image according to the corner point possibility and the corner point detection method.

[0094] Preferably, the method for obtaining the final corner point includes:

[0095] The mean of the corner point possibilities of all pixels in the real scene image is used as the initial segmentation threshold, and the corner point possibilities of all pixels are iteratively thresholded and segmented in combination with the iterative threshold segmentation algorithm to obtain all iterative segmentation thresholds; the mean of the distances between the preset reference number of edge pixels closest to each pixel in the real scene image and each pixel in the LAB space is used as the color proximity of each pixel; the best threshold possibility of each iterative segmentation threshold is obtained according to the edge proximity and color proximity of the pixel whose corner point possibility is greater than each iterative segmentation threshold. Since iterative threshold segmentation can obtain multiple iterative segmentation thresholds, it is further necessary to analyze the segmentation effect of each iterative segmentation threshold, so as to screen out the iterative segmentation threshold with the best segmentation effect. In the embodiment of the present invention, the preset reference number is set to 5, and the implementer can adjust it according to the specific implementation environment. It should be noted that the condition for stopping the iterative threshold segmentation corresponding to the embodiment of the present invention is to perform 10 iterations, and the implementer can adjust the number of stopping iterations according to the specific implementation environment, which will not be further described here.

[0096] Preferably, the method for obtaining the optimal threshold possibility includes:

[0097] For any iterative segmentation threshold:

[0098] The negative correlation mapping value of the product of the mean of the edge proximity of all pixels whose corner point probability is greater than the iteration segmentation threshold and the mean of the color proximity is used as the optimal threshold probability of the iteration segmentation threshold. Pixels whose corner point probability is greater than the iteration segmentation threshold are pixels with greater corner point probability; therefore, when the overall edge proximity of all pixels whose corner point probability is greater than the iteration segmentation threshold is smaller, it means that the overall position of these pixels is closer to the edge, then the classification effect of these pixels as candidate corner points is more accurate, and their features are more significant, that is, the better the effect of division with the iteration segmentation threshold, the greater the corresponding optimal threshold probability; similarly, when the overall color proximity of all pixels whose corner point probability is greater than the iteration segmentation threshold is smaller, it means that the color features between these pixels and edge pixels are more similar, then the classification effect of these pixels as candidate corner points is more accurate, and the division effect of the corresponding iteration segmentation threshold is better. Therefore, the mean of the edge proximity and the mean of the color proximity of all pixels greater than the iteration segmentation threshold are both negatively correlated with the optimal threshold probability.

[0099] In the embodiment of the present invention, each iterative segmentation threshold is sequentially used as the vth iterative segmentation threshold, and the method for obtaining the optimal threshold possibility of the vth iterative segmentation threshold is expressed in the formula as follows:

[0100]

[0101] Among them, Z v is the optimal threshold likelihood for the vth iteration segmentation threshold; A is the average of the edge proximity of all pixels whose corner point probability is greater than the vth iterative segmentation threshold; v is the number of pixels in the real scene image whose corner probability is greater than the vth iterative segmentation threshold; LAB v,u It is the color proximity of the u-th pixel whose corner probability in the real scene image is greater than the v-th iterative segmentation threshold.

[0102] Further, the iterative segmentation threshold corresponding to the maximum optimal threshold possibility is used as the final segmentation threshold. All pixel points in the real scene image that are greater than the final segmentation threshold are used as candidate corner points; in another embodiment of the present invention, the normalized value of the corner point possibility of each pixel point in the real scene image is used as the reference possibility of each pixel point; the pixel points whose reference possibility is greater than the preset segmentation threshold are used as candidate corner points; the preset segmentation threshold is set to 0.8, and the implementer can adjust it according to the specific implementation environment. Further, all candidate corner points screened out according to the embodiment of the present invention are detected by the FAST corner detection algorithm to obtain all final corner points in the real scene image. It should be noted that the FAST corner detection algorithm is a prior art well known to those skilled in the art, and will not be further defined or elaborated here.

[0103] ARG game image rendering is performed based on the final corner point. Specifically, the game system recognizes the final corner point position in the real scene image to select appropriate materials to render the virtual image, and locates it to the appropriate position according to the content of the material to merge with the real image.

[0104] To summarize, the present invention first obtains different numbers of comparison pixels according to the grayscale distribution and gradient distribution in the neighborhood range of different pixels in the real scene image; further obtains the reference texture difference according to the grayscale gradient difference and color difference between each pixel and the comparison pixel; on the basis of the reference texture difference, the grayscale gradient amplitude and edge significance are combined to obtain a more accurate corner point possibility corresponding to each pixel, so that the final corner point selected according to the corner point possibility and the corner point detection method is more accurate, that is, the effect of ARG game image rendering based on the final corner point is better.

[0105] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. An image rendering method based on ARG game, It is characterized in that The method comprises: Collect real scene images through ARG game equipment; In the real scene image, all contrasting pixels corresponding to each pixel are obtained according to the grayscale deviation between each pixel and each pixel in the neighborhood, and the gradient distribution in the neighborhood of each pixel; According to the grayscale gradient difference and color difference between each pixel and the contrasting pixel in the real scene image, the reference texture difference between each pixel and each contrasting pixel is obtained; according to all the reference texture differences, grayscale gradient amplitude and edge significance corresponding to each pixel, the corner point possibility of each pixel is obtained; According to the corner point possibility and the corner point detection method, all final corner points in the real scene image are screened out; and the ARG game image is rendered according to the final corner points; The method for obtaining the reference texture difference includes: Get the gradient angles of all pixels in the real scene image; The difference between the grayscale value of each pixel and the grayscale value of each corresponding contrasting pixel is used as the reference grayscale difference of each contrasting pixel; the difference between the cosine value of the gradient angle of each pixel and the cosine value of the gradient angle of each corresponding contrasting pixel is used as the gradient direction difference of each contrasting pixel; The real scene image is converted into the LAB space, and the distance between each pixel and each corresponding contrast pixel in the LAB space is used as the reference color difference of each contrast pixel; the distance between each pixel and each corresponding contrast pixel in the real scene image is used as the actual distance of each contrast pixel; A reference texture difference between each pixel and each contrasting pixel is obtained according to the reference grayscale difference, the gradient direction difference, the reference color difference and the actual distance; the reference grayscale difference, the gradient direction difference and the reference color difference are all positively correlated with the reference texture difference, and the actual distance is negatively correlated with the reference texture difference; The method for obtaining the possibility of the corner point includes: The average value of all reference texture differences corresponding to each pixel is used as the texture importance of each pixel; According to the gray gradient deviation of each pixel point compared with all corresponding contrast pixels, the gradient change characteristic value of each pixel point is obtained; The minimum distance between each pixel and all edge pixels in the real scene image is taken as the edge proximity of each pixel; According to the texture importance, the gradient change characteristic value and the edge proximity, the corner point possibility of each pixel in the real scene image is obtained; the texture importance and the gradient change characteristic value are both positively correlated with the corner point possibility; the edge proximity is negatively correlated with the corner point possibility.

2. The image rendering method based on ARG game according to claim 1, It is characterized in that The method for obtaining the comparison pixel points includes: The preset number of neighboring pixels closest to each pixel is used as the reference pixel of each pixel; the gray importance of each pixel is obtained according to the gray deviation and gray gradient distribution between each pixel and the corresponding reference pixel; The product of the positive correlation mapping value of the grayscale importance and the preset neighborhood number is rounded up to obtain the comparison neighborhood number of each pixel point; the comparison neighborhood number of other pixel points closest to each pixel point are used as all the comparison pixel points corresponding to each pixel point.

3. The image rendering method based on ARG game according to claim 2, It is characterized in that The calculation formula of the gray importance degree includes: in, For the real scene image The gray importance of each pixel; For the real scene image The number of reference pixels corresponding to each pixel; For the real scene image The gray value of each pixel; For the real scene image The pixel corresponding to Gray value of reference pixel; For the pixels and the number of pixels whose grayscale gradient is not 0 among all corresponding reference pixels; is an exponential function with a natural constant as base; is the normalization function.

4. The image rendering method based on ARG game according to claim 1, It is characterized in that The method for obtaining the reference texture difference between each pixel point and each comparison pixel point according to the reference grayscale difference, the gradient direction difference, the reference color difference and the actual distance comprises: The sum of the reference grayscale difference and the gradient direction difference is used as a reference sum value; the ratio of the reference color difference and the actual distance is used as a reference ratio; and the product of the reference sum value and the reference ratio is used as a reference texture difference between each pixel point and each contrasting pixel point.

5. The image rendering method based on ARG game according to claim 1, It is characterized in that The method for obtaining the gradient change characteristic value includes: The mean value of the grayscale gradient amplitude of all contrasting pixel points corresponding to each pixel point and the sum of the preset adjustment parameters are used as the contrast gradient amplitude of each pixel point; the ratio of the grayscale gradient amplitude of each pixel point to the contrast gradient amplitude is used as the gradient change characteristic value of each pixel point.

6. The image rendering method based on ARG game according to claim 1, It is characterized in that The method for obtaining the possibility of a corner point of each pixel in the real scene image according to the texture importance, the gradient change characteristic value and the edge proximity comprises: The normalized value of the product of the negative correlation mapping value of the edge proximity, the texture importance and the gradient change feature value is used as the corner point possibility of each pixel point in the real scene image.

7. The image rendering method based on ARG game according to claim 1, It is characterized in that The method for obtaining the final corner point includes: Taking the mean of the corner point possibilities of all pixels in the real scene image as the initial segmentation threshold, performing iterative threshold segmentation on the corner point possibilities of all pixels in combination with an iterative threshold segmentation algorithm to obtain all iterative segmentation thresholds; The average of the distances between each pixel and a preset reference number of edge pixels closest to each pixel in the real scene image in the LAB space is used as the color proximity of each pixel; According to the edge proximity and color proximity of the pixel points whose corner point probability is greater than each iterative segmentation threshold, the optimal threshold probability of each iterative segmentation threshold is obtained; the iterative segmentation threshold corresponding to the maximum optimal threshold probability is used as the final segmentation threshold; All pixel points in the real scene image that are greater than the final division threshold are taken as candidate corner points; all candidate corner points are detected using the FAST corner point detection algorithm to obtain all final corner points in the real scene image.

8. The image rendering method based on ARG game according to claim 7, It is characterized in that The method for obtaining the optimal threshold possibility includes: For any iterative segmentation threshold: The negative correlation mapping value of the product of the mean of the edge proximity and the mean of the color proximity of all pixels whose corner point likelihood is greater than the iterative segmentation threshold is used as the optimal threshold likelihood of the iterative segmentation threshold.