A quantitative evaluation method for illumination consistency in virtual-reality fusion scenes

Abstract

The present invention provides a quantitative evaluation method for lighting consistency in a virtual-reality fusion scene. The method constructs a real scene, then constructs and unifies the coordinate systems of the real and virtual worlds. The method then captures images of the real scene and performs lighting estimation based on shadows. Based on the results of the shadow estimation, the method calculates the angular error between the real light source and the virtual light source. The method then removes the cuboid in the real scene, generates a virtual cuboid with the same attributes, generates a virtual light source at the estimated position, and renders the corresponding shadow. The method then captures an image of the virtual-reality fusion scene based on the corresponding shadows generated after rendering and the captured real scene image. After shadow separation of the virtual-reality fusion scene image, the shadow image overlap is calculated. This method can objectively and quantitatively evaluate lighting consistency in a virtual-reality fusion scene under a single light source.

Description

Technical Field

[0001] The present invention belongs to the field of virtual-reality fusion scenes, and more specifically, relates to a method for quantitatively evaluating illumination consistency in virtual-reality fusion scenes. Background Art

[0002] Augmented reality (AR) is a current research hotspot in fields such as computer graphics and computer vision. Its applications are widespread, including maintenance and assembly, healthcare, military affairs, architectural design, and lifestyle entertainment. AR technology renders virtual objects into real-world scenes, creating a blend of virtual and real, providing users with a sense of immersion. This natural blend of virtual and real is key to creating a sense of realism in the scene.

[0003] Research on illumination consistency in augmented reality is relatively limited and still in its early stages. Illumination consistency reflects the lighting matching relationship between virtual objects and real scenes, such as brightness and shadows, when integrating virtual and real objects, thereby ensuring that users' immersion in augmented reality scenes is not affected.

[0004] The key to achieving lighting consistency lies in how to perform accurate lighting estimation. Current research on this topic falls into the following four categories:

[0005] (1) Illumination estimation based on shadows of landmarks: Analyze the shadows of the cube landmark and the real object, detect the shadow edges, and use spatial geometry to infer the position of the point light source;

[0006] (2) Method for estimating illumination based on the surface image of the marker: Diffuse reflection and highlight are separated from the calibration sphere, where the highlight is used to estimate the position of the light source and the diffuse reflection is used to estimate the intensity of the light source;

[0007] (3) Estimating illumination with the help of auxiliary equipment: Using a depth camera to create an HDR environment map in real time, and using a clustering algorithm to extract the position of point light sources from the environment map;

[0008] (4) Lighting estimation based on deep learning: Using a large amount of image or video data to drive and train a deep network to estimate the light source.

[0009] Most existing research on lighting consistency in augmented reality scenes uses certain methods to obtain the position and direction of light sources in the real scene in a relatively fixed environment, and then renders the fused virtual objects to achieve lighting effects consistent with the real scene. The shortcomings of existing technologies are as follows:

[0010] (1) Various existing technologies require complex algorithm support, among which deep learning-based illumination estimation requires a lot of computing power; (2) The virtual-reality fusion effect in illumination consistency research can only be evaluated based on subjective feelings, lacking quantitative analysis methods and objective evaluation standards. To compare illumination consistency in virtual-reality fusion scenarios, an objective quantitative evaluation method is needed. Summary of the Invention

[0011] To address the shortcomings of existing technologies, this paper provides a method for quantitatively evaluating lighting consistency in virtual-reality fusion scenarios. This method provides an objective, quantitative evaluation of lighting consistency in virtual-reality fusion scenarios under a single light source, as lighting consistency in virtual-reality fusion scenarios currently lacks objective standards and can only be assessed subjectively.

[0012] The present invention provides a method for quantitatively evaluating illumination consistency in a virtual-reality fusion scene, the method comprising the following steps:

[0013] (1) Place two cuboids (denoted as M and N) of known shape, size, and color and a marker image in a real scene with a single point light source L as a real light source. Construct a global coordinate system of the real world based on the position of the marker image. Start the augmented reality application and unify the real world and virtual world coordinate systems based on the transformation relationship between the real world global coordinate system and the virtual world global coordinate system.

[0014] (2) Estimate the light source position in the real scene using the shadow areas of the cuboids M and N, record the estimated result as the virtual light source position, and simultaneously capture a two-dimensional image I1 of the real scene using a camera;

[0015] (3) Calculate the angular error E between the real light source position and the virtual light source position estimated by step 2 θ , as the quantitative evaluation index of this method1;

[0016] (4) Remove the cuboid from the real scene and use an augmented reality application to overlay a virtual scene on the real scene, including a virtual light source, a cuboid with the same attributes as the real scene, and its shadow. Capture a two-dimensional image I2 of the virtual-reality fusion scene. Calculate the shadow similarity between the real scene image I1 and the virtual-reality fusion scene image I2 as the quantitative evaluation metric 2 of this method.

[0017] Furthermore, step (1) can be divided into the following steps:

[0018] 1.1) Take the center position O of the placed marker image I0 as the origin and its posture as the coordinate axis to construct the global coordinate system Oxyz of the real world, and measure and record the position L(x L ,yL ,z L ), the center position of the bottom surface of the rectangular parallelepiped M and the rectangular parallelepiped N (x M ,y M ,z M ) and (x N ,y N ,z N ) and posture;

[0019] 1.2) Set and adjust the camera position and posture, and start the augmented reality application. The augmented reality application will initialize and establish the virtual world global coordinate system based on the camera position and posture;

[0020] 1.3) Use the image tracking and recognition function of the augmented reality application to identify and locate the landmark image. Combined with the real-world position and posture of the landmark image recorded in step 1.1, the transformation relationship between the virtual world global coordinate system and the real-world coordinate system is calculated to achieve the unification of the real-world and virtual world coordinate systems.

[0021] Furthermore, step (2) can be divided into the following steps:

[0022] 2.1) Measure in a real scene, recording the position A(x1, y1, z1) of a vertex in rectangular parallelepiped M and the position B(x2, y2, z2) of the corresponding point in its shadow. Also record the position C(x3, y3, z3) of a vertex in rectangular parallelepiped N and the position D(x4, y4, z4) of the corresponding point in its shadow.

[0023] 2.2) Using the two pairs of vertices and the spatial positions of their corresponding shadows recorded in step 2.1, calculate the vector v1 from A to B and the vector v2 from C to D using the following formula:

[0024]

[0025] 2.3) Combined with step 2.2, take point A and point C as the base points respectively, and according to the definition of a straight line in space, derive the equation of the line l1 where vector v1 lies:

[0026] l1(t)=A+tv1,

[0027] And the equation of the line l2 where the vector v2 lies

[0028] l2(s)=C+sv2.

[0029] 2.4) Find the points P1(x5,y5,z5) and P2(x6,y6,z6) on lines l1 and l2 that are the shortest in distance.

[0030]

[0031] Solving a system of equations

[0032]

[0033]

[0034] Obtain the specific values of parameters t and s and substitute them into formula 1 to obtain the coordinates of P1 and P2;

[0035] 2.5) Combine the points P1 and P2 obtained in step 2.4 and calculate the midpoint coordinates P(x P ,y P ,z P ) is used as the estimated virtual light source position, and the specific formula is as follows:

[0036] x P =(x5+x6) / 2,y P =(y5+y6) / 2, z P =(z5+z6) / 2;

[0037] 2.6) Capture a two-dimensional image I1 of the real scene through camera information.

[0038] Furthermore, step (3) can be divided into the following steps:

[0039] 3.1) According to the light source position L(x L ,y L ,z L ) and the estimated virtual light source position P(x P ,y P ,z P ), respectively calculate the vectors pointing from the origin O to the real light source L and the virtual light source P in the real world global coordinate system as follows:

[0040]

[0041] 3.2) Based on the vector obtained in step 3.1 and Calculate the cosine of the angle θ between them:

[0042]

[0043] Since the maximum angle between vectors is 180°, i.e., π radians, the relative error E between the real light source position and the virtual light source position is calculated. θ :

[0044]

[0045] Furthermore, step (4) can be divided into the following steps:

[0046] 4.1) Keeping the camera position and posture unchanged, remove the cuboids M and N in the real scene, generate virtual cuboids M' and N' of the same shape, size, and color according to the position and posture of the cuboids recorded by the augmented reality application in step (1), generate a virtual light source at the virtual light source position estimated in step (2), render the shadows corresponding to the virtual cuboids M' and N', and simultaneously capture a two-dimensional image I2 of the virtual-reality fusion scene through the camera;

[0047] 4.2) Perform shadow segmentation on the real scene image I1 and the virtual-real fusion scene image I2. The shadow segmentation process is as follows: first, read the scene image, then traverse the image pixel by pixel. For pixels whose pixel values are lower than a preset threshold, the pixel value is reassigned to the variance of the RGB three-channel pixel values at that location. Finally, histogram equalization, binarization, and Gaussian filtering are performed in sequence to obtain the segmented shadow images. Based on this shadow segmentation process, the real scene shadow image S1 and the virtual-real fusion scene shadow image S2 are obtained.

[0048] 4.3) Calculate the similarity between the real scene shadow image S1 and the virtual-reality fusion scene shadow image S2. The similarity calculation process is as follows: first, calculate the mean of the RGB three-channel pixel values of the input shadow image and assign the mean value, then perform pixel-by-pixel numerical normalization using the two-norm, and finally calculate the dot product of the two standardized two-dimensional image arrays to calculate the similarity of the shadow images.

[0049] The beneficial effects of the present invention are as follows: (1) the angle error between the real light source and the virtual light source, and the similarity between the real shadow and the virtual shadow are used as quantitative evaluation indicators, so that the lighting consistency of the image can be quantitatively evaluated instead of relying solely on human subjective feelings, thereby comparing the rendering effects of various lighting consistency algorithms; (2) the light source position estimation relies on the straight line relationship in mathematical space geometry, with low solution cost and fast speed; (3) the shadow segmentation method is simple and intuitive, with low development and calculation costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of the method for quantitatively evaluating illumination consistency in a virtual-reality fusion scene of the present invention.

[0051] Figure 2 This is a schematic diagram of a real scene construction method for quantitatively evaluating illumination consistency in a virtual-reality fusion scene of the present invention.

[0052] Figure 3 Schematic diagram of light source position estimation in the method for quantitatively evaluating illumination consistency in a virtual-reality fusion scene of the present invention.

[0053] Figure 4This is a schematic diagram of light source error calculation for the quantitative evaluation method of illumination consistency in a virtual-reality fusion scene of the present invention.

[0054] Figure 5 Schematic diagram of virtual object rendering in the method for quantitatively evaluating illumination consistency in a virtual-reality fusion scene according to the present invention.

[0055] Figure 6 This is a real scene shadow segmentation schematic diagram of the quantitative evaluation method of illumination consistency in the virtual-reality fusion scene of the present invention.

[0056] Figure 7 It is a schematic diagram of shadow segmentation of a virtual-real fusion scene according to the quantitative evaluation method of illumination consistency in the virtual-real fusion scene of the present invention.

[0057] The names corresponding to the numerical symbols in the accompanying drawings are as follows:

[0058] 1. Real light source, whose light source position is L(x L ,y L ,z L );

[0059] 2. Cuboid M; 21. The shadow of Cuboid M; 22. A vertex A of Cuboid M, whose position is A(x1, y1, z1); 23. The corresponding point B of Cuboid M in the shadow of Cuboid M, whose position is B(x2, y2, z2);

[0060] 3. Cuboid N; 31. The shadow of Cuboid N; 32. A vertex C of Cuboid N, whose position is C(x3, y3, z3); 33. The corresponding point D of Cuboid N in the shadow of Cuboid N, whose position is D(x4, y4, z4);

[0061] 4. Mark the image; 41. Mark the center of the image as the origin of the coordinate system O(0,0,0);

[0062] 5. Virtual light source, whose light source position is P(x P ,y P ,z P );

[0063] 6. Virtual cuboid M'; 61. Shadow of virtual cuboid M';

[0064] 7. Virtual cuboid N'; 71. Shadow of virtual cuboid N'. DETAILED DESCRIPTION

[0065] The description of the following embodiments is only intended to help understand the method of the present invention and its core ideas. It should be noted that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications may be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention. The following description of the disclosed embodiments enables professionals in this field to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but may be applied to a wider range consistent with the principles and novel features disclosed herein.

[0066] The “augmented reality application” mentioned in the present invention refers to an application with functions such as image recognition (recognizing a marker image and displaying the position and rotation angle of the marker image), virtual point light source generation (generating a virtual point light source, displaying it on demand, and setting the position of the point light source according to input), and virtual cuboid generation (generating a virtual cuboid, displaying it on demand, and setting the position and rotation angle according to input). The application can be implemented by importing ARFoundation, a platform-independent AR development toolkit developed by Unity Technologies, into Unity3D, a professional game engine developed by the company. The application can also be obtained by directly downloading an existing toolkit, such as downloading an application called AR Toolbox from Google Play. The “augmented reality application” mentioned in the present invention can be run on a device with a camera, such as a mobile phone.

[0067] In addition, the "real world" mentioned in the present invention is interchangeable with the "real scene", which refers to objectively existing scenes and objects with entities that are independent of applications and devices. The "virtual world" mentioned in the present invention is interchangeable with the "virtual scene", which refers to the non-physical scenes and objects rendered when the augmented reality application is running. The "virtual-reality fusion scene" mentioned in the present invention refers to the augmented reality application rendering a "virtual scene" on the screen of the "real scene" and overlapping the two screens so that they visually exist in the same screen and space at the same time.

[0068] The technical solution adopted by the present invention is a quantitative evaluation method for illumination consistency in a virtual-real fusion scene, the flow chart of which is as follows: Figure 1As shown in the figure, the steps are as follows: after starting the evaluation, construct a real scene, then construct and unify the coordinate systems of the real world and the virtual world, then respectively capture the real scene image and perform lighting estimation based on the shadows, and record the lighting estimation result as the virtual light source position. At this time, based on the result of the lighting estimation based on the shadows, calculate the angular error between the real light source and the virtual light source (evaluation index 1), then remove the cuboid of the real scene, generate a virtual cuboid with the same attributes, generate a virtual light source at the estimated position, and render the corresponding shadow. Then, based on the corresponding shadows generated after rendering and the captured real scene image, capture the virtual and real fusion scene image. After shadow separation of the virtual and real fusion scene image, calculate the shadow image overlap (evaluation index 2).

[0069] The steps of the method flow are as follows:

[0070] Step 1: If Figure 2 As shown, two cuboids (M and N) of known shape, size, and color and a marker image are placed in a real scene with a single point light source L. The global coordinate system of the real world is constructed based on the position of the marker image. According to the transformation relationship between the global coordinate system of the real world and the global coordinate system of the virtual world, the coordinate systems of the real world and the virtual world are unified.

[0071] Step 2: If Figure 3 As shown, the light source position in the real scene is estimated by the shadow area of the rectangular blocks M and N, and the estimated result is used as the virtual light source position and recorded, while a two-dimensional image I1 of the real scene is captured by a camera;

[0072] It should be noted that Figure 3 The diagram in the figure is an ideal case with no errors. In reality, due to the measurement errors caused by the blurred edges of soft shadows in real scenes and the precision of floating-point numbers in practice, it is necessary to solve the foot point of the co-perpendicular lines of non-planar lines when estimating the light source. For the two-dimensional image I1, it should be noted that the two-dimensional image I1 will be used for shadow segmentation in the subsequent step 4. Therefore, the two-dimensional image I1 should contain the complete rectangular blocks M and N and their shadows, while other objects in the scene (such as marker images) are not required.

[0073] Step 3: If Figure 4 As shown, the angular error E between the real light source position and the virtual light source position estimated by step 2 is calculated. θ , as the quantitative evaluation index of this method1;

[0074] Step 4: Keep the camera position and posture unchanged, remove the cuboids M and N in the real scene, generate virtual cuboids M' and N' of the same shape, size and color according to the position and posture of the cuboids recorded by the augmented reality application in step 1, and generate a virtual light source at the virtual light source position estimated in step 2, render the shadows corresponding to the virtual cuboids M' and N', and at the same time capture the two-dimensional image I2 of the virtual-reality fusion scene through the camera. The virtual-reality fusion scene is as follows: Figure 5 As shown;

[0075] Regarding the two-dimensional image I2, it should be noted that the two-dimensional image I2 will be used for shadow segmentation in the subsequent step 5. Therefore, the two-dimensional image I2 should contain the complete rectangular blocks M' and N' and their shadows, while other objects in the scene (such as marker images) are not required. For ease of comparison, the camera position and posture should be consistent when shooting the two-dimensional images I1 and I2.

[0076] Finally, shadow segmentation is performed on the real scene image I1 and the virtual-real fusion scene image I2 respectively. The shadow segmentation effects are as follows: Figure 6 and Figure 7 As shown, the similarity of the shadow images is calculated as the quantitative evaluation index of this method2.

[0077] (1) Specifically, step 1 includes the following steps:

[0078] Step 1.1: The present invention estimates the light source position by using the shadows of objects in the real scene. First, two cuboids (M and N) of known shape, size, and color and a marker image are placed in the real scene with a single point light source (real light source);

[0079] Step 1.2: Based on the center position O of the marker image I0 placed in step 1.1 as the origin, and using its posture as the coordinate axis to construct the global coordinate system of the real world, measure and record the position L (x L ,y L ,z L ), the center position of the bottom surface of the rectangular parallelepiped M and the rectangular parallelepiped N (x M ,y M ,z M ) and (x N ,y N ,z N The purpose of recording the position and posture is to accurately set the position and posture of the virtual cuboid M 'and N 'in the subsequent steps so that the real scene and the virtual and real fusion scene are arranged in the same way;

[0080] Step 1.3: Set and adjust the camera position and posture, and start the augmented reality application. The augmented reality application will initialize and establish the virtual world global coordinate system based on the camera position and posture;

[0081] Step 1.4: Use the image tracking and recognition function of the augmented reality application to identify and locate the landmark image. Combined with the real-world position and posture of the landmark image recorded in step 1.2, the transformation relationship between the virtual world global coordinate system and the real-world coordinate system is calculated to achieve the unification of the real-world and virtual world coordinate systems.

[0082] (2) Specifically, step 2 includes the following steps:

[0083] Step 2.1: Measure in the real scene, record the position of a vertex A (x1, y1, z1) in the cuboid M and the position of the corresponding point B (x2, y2, z2) in its shadow, and record the position of a vertex C (x3, y3, z3) in the cuboid N and the position of the corresponding point D (x4, y4, z4) in its shadow. The vertex positions selected for recording in this step should have corresponding visible shadow projections. When selecting recording points, you can also record any point in the geometric body and the position of its corresponding shadow. The reason for selecting vertices is that their positions are obvious and more convenient to record.

[0084] Step 2.2: Using the two pairs of vertices and the spatial positions of the corresponding shadows recorded in step 2.1, calculate the vector v1 from A to B and the vector v2 from C to D. The calculation formula is as follows:

[0085]

[0086] Step 2.3: Combined with step 2.2, take point A and point C as the base points respectively, and according to the definition of a straight line in space, derive the equation of the straight line l1 where vector v1 lies.

[0087] l1(t)=A+tv1,

[0088] And the equation of the line l2 where the vector v2 lies

[0089] l2(s)=C+sv2.

[0090] Step 2.4: In theory, both line l1 and line l2 are rays emitted by the light source and should intersect at the light source. However, due to the measurement error caused by the blurred edges of soft shadows in real scenes and the precision of floating-point numbers in practice, lines l1 and l2 rarely intersect exactly and actually become skew lines. Therefore, it is necessary to solve the foot points P1 (x5, y5, z5) and P2 (x6, y6, z6) of the common perpendicular line of lines l1 and l2. These are the points on lines l1 and l2 with the shortest distance between them. According to the line equations in step 2.3, since P1 and P2 are on lines l1 and l2 respectively and are exactly the foot points of their respective lines, the common perpendicular line Perpendicular to the vectors v1 and v2 on the lines l1 and l2, that is, v1 and v2 are perpendicular to the common perpendicular line The dot product of is 0.

[0091] P1=l1(t)=A+tv1

[0092] P2=l2(s)=C+sv2

[0093] The following set of equations can be listed

[0094]

[0095] By solving, we can get the specific values of parameters t and s, and then get the coordinates of points P1 and P2.

[0096] Step 2.5: Combine the points P1 and P2 obtained in step 2.4 and calculate the midpoint coordinates P(x P ,y P ,z P ) is used as the estimated virtual light source position, and the specific formula is as follows:

[0097] x P =(x5+x6) / 2,y P =(y5+y6) / 2, z P =(z5+z6) / 2;

[0098] Step 2.6: Capture a two-dimensional image I1 of the real scene through the camera information.

[0099] (3) Specifically, step 3 includes the following steps:

[0100] Step 3.1: Combine the light source position L(x L ,y L ,z L ) and the virtual light source position P(x P ,y P ,z P), respectively calculate the vectors pointing from the origin O to the real light source L and the virtual light source P in the real world global coordinate system as follows:

[0101]

[0102] Step 3.2: Based on the vector obtained in step 3.1 and Calculate the angle θ between them and use the angle θ to establish the relative error E between the real light source and the virtual light source θ , as the quantitative evaluation index of this method1. The cosine formula of the angle between the two vectors is:

[0103]

[0104] Since the maximum angle between vectors is 180°, that is, π radians, the relative error E can be solved according to the above formula θ :

[0105]

[0106] (4) Specifically, step 4 includes the following steps:

[0107] Step 4.1: Keeping the camera position and posture unchanged, remove the cuboids M and N in the real scene, generate virtual cuboids M' and N' of the same shape, size, and color according to the position and posture of the cuboids recorded by the augmented reality application in step (1), and generate a virtual light source at the virtual light source position estimated in step (2), render the shadows corresponding to the virtual cuboids M' and N', and at the same time capture a two-dimensional image I2 of the virtual-reality fusion scene through the camera;

[0108] Step 4.2: Perform shadow segmentation on the real scene image I1 and the virtual-real fusion scene image I2. The shadow segmentation process is as follows: first read the scene image, then traverse the image pixel by pixel. For pixels whose pixel values are lower than the preset threshold, the pixel value is reassigned to the variance of the RGB three-channel pixel values at that location. Finally, histogram equalization, binarization, and Gaussian filtering are performed in sequence to finally obtain the segmented shadow image. According to this shadow segmentation process, the real scene shadow image S1 and the virtual-real fusion scene shadow image S2 are obtained;

[0109] Step 4.3: Calculate the similarity between the real scene shadow image S1 and the virtual-real fusion scene shadow image S2. The similarity calculation process is as follows: First, the pixel values of the three RGB channels of the input shadow image are averaged and assigned values using the average value. Then, the pixel values are normalized pixel by pixel using the bi-norm. Finally, the dot product of the two normalized two-dimensional image arrays is calculated as the similarity of the shadow images, resulting in the quantitative evaluation indicator 2 of this method.

Claims

1. A quantitative evaluation method for illumination consistency in a virtual-reality fusion scene, characterized by: The steps of the method are: (1) Place two cuboids (denoted as M and N) of known shape, size, and color and a marker image in a real scene with a single point light source L as a real light source. Construct a global coordinate system of the real world based on the position of the marker image. Unify the coordinate systems of the real world and the virtual world based on the transformation relationship between the global coordinate system of the real world and the global coordinate system of the virtual world. (2) Estimate the light source position in the real scene using the shadow areas of the cuboids M and N, record the estimated result as the virtual light source position, and simultaneously capture a two-dimensional image I1 of the real scene using a camera; (3) Calculate the angular error E between the real light source position and the virtual light source position estimated by step 2 θ , as the quantitative evaluation index of this method1; (4) Keeping the camera position and posture unchanged, remove the cuboids M and N in the real scene, generate virtual cuboids M' and N' of the same shape, size and color according to the position and posture of the cuboids recorded by the augmented reality application in step (1), generate a virtual light source at the virtual light source position estimated in step (2), render the shadows corresponding to the virtual cuboids M' and N', and simultaneously capture a two-dimensional image I2 of the virtual-reality fusion scene through the camera; (5) Perform shadow segmentation on the real scene image I1 and the virtual-real fusion scene image I2 respectively, and calculate the similarity of the shadow images as the quantitative evaluation index 2 of this method.

2. The method according to claim 1, wherein Step (1) can be divided into the following steps: 1.1) Construct a global coordinate system of the real world based on the position of the placed marker image, and measure and record the position of the real light source L(x L ,y L ,z L ), mark the center position (x0, y0, z0) and posture of image I0, the center position of the bottom surface of rectangular parallelepiped M and rectangular parallelepiped N (x M ,y M ,z M ) and (x N ,y N ,z N ) and posture; 1.2): Set and adjust the camera position and posture, and start the augmented reality application. The augmented reality application will initialize and establish the virtual world global coordinate system based on the camera position and posture; 1.3): Use the image tracking and recognition function of the augmented reality application to identify and locate the landmark image. Combined with the real-world position and posture of the landmark image recorded in step 1), the transformation relationship between the virtual world global coordinate system and the real-world coordinate system is calculated to achieve the unification of the real-world and virtual world coordinate systems.

3. The method according to claim 2, wherein Step (2) can be divided into the following steps: 2.1) Measure in a real scene, recording the position A(x1, y1, z1) of a vertex in rectangular parallelepiped M and the position B(x2, y2, z2) of the corresponding point in its shadow. Also record the position C(x3, y3, z3) of a vertex in rectangular parallelepiped N and the position D(x4, y4, z4) of the corresponding point in its shadow. 2.2) Using the two pairs of vertices and the spatial positions of the corresponding shadows (A and B, C and D) recorded in step 2.1), calculate their respective direction vectors and The calculation formula is as follows: 2.3) Combined with step 2.2), using points B and D as the base points, and using the definition of a line in space, derive the equation of the line L1 between points A and B. Similarly, derive the equation of the line L2 between points C and D. 2.4) Calculate the shortest points P1(x5,y5,z5) and P2(x6,y6,z6) on lines L1 and L2; 2.5) Combine the positions of points P1 and P2 calculated in step 2.4) and solve the midpoint coordinate P0 (x P ,y P ,z P ) is used as the estimated virtual light source position, and the specific formula is as follows: x P =(x5+x6) / 2 <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> P <h2 style=";text-align:left;direction:ltr"> (y5+y6) / 2 z P =(z5+z6) / 2; 2.6) Capture a two-dimensional image I1 of the real scene through camera information.

4. The method according to claim 3, wherein Step (3) can be divided into the following steps: 3.1) According to the light source position L(x L ,y L ,z L ) and the estimated virtual light source position P0(x P ,y P ,z P ), respectively calculate the vectors pointing from the origin O to the real light source L and the virtual light source P0 in the real world global coordinate system; 3.2) Based on the vectors obtained in step 3.1), calculate the angle θ between the vectors according to formula 1, and calculate the relative error E between the real light source position and the virtual light source position according to formula 2 θ .

5. The method according to any one of claims 1 to 4, characterized in that Step (5) can be divided into the following steps: 5.1) Perform shadow segmentation on the real scene image I1 and the virtual-real fusion scene image I2. The shadow segmentation process is as follows: first, read the scene image, then traverse the image pixel by pixel. For pixels with a value below a preset threshold, the pixel value is reassigned to the variance of the RGB three-channel pixel values at that location. Finally, histogram equalization, binarization, and Gaussian filtering are performed in sequence to obtain the segmented shadow images. Based on this shadow segmentation process, the real scene shadow image S1 and the virtual-real fusion scene shadow image S2 are obtained. 5.2) Calculate the similarity between the real scene shadow image S1 and the virtual-reality fusion scene shadow image S2. The similarity calculation process is as follows: first, the average of the RGB three-channel pixel values of the input shadow image is calculated and assigned using the average value, then the pixel-by-pixel numerical normalization is performed using the two-norm, and finally the dot product of the two standardized two-dimensional image arrays is calculated as the similarity of the shadow images.

Images